Lately, nanoparticles (NPs) have shown the potential to improve the performance of oil well fluids significantly. Several studies have reported the ability of NPs to produce improved properties of both water and oil-based drilling fluids. In this study, hydrophobic iron oxide NPs were synthesized by thermal decomposition of iron pentacarbonyl in an inert atmosphere, and its performance was tested in the oil-based drilling fluid with 90/10 oil-to-water ratio (base fluid). Oil-based drilling fluids treated with nanofluids were formulated by adding 0.5 wt% and 1.0 wt% iron oxide NPs in hexane solution to the base drilling fluid. The base fluid and the nanofluid-treated drilling fluids were evaluated by characterizing their rheological properties at different temperatures, viscoelastic properties, lubricity, filtrate loss, static and dynamic settling, and separation properties. Results showed that 0.5 wt% iron oxide dispersed in hexane reduced the high pressure high temperature (HPHT) filtrate loss by 70%, filter cake thickness by 55%, and the coefficient of friction by 39%. Moreover, the nanofluid based drilling fluid reduced the free oil layer caused by syneresis during aging at high temperature by 16.3% compared to the base fluid. This study has shown that hydrophobic iron oxide NPs have the potential to improve the properties of oil-based drilling fluid.