To improve the accuracy of the machined surface produced by an ordinary milling machine, a system called workpiece-referred form accuracy control (WORFAC) was developed and confirmed in diamond turning. However, non-rotational symmetric surface structures, such as V-grooves, pyramid structures, F-theta lenses, and other free form surface cannot be machined by diamond turning. We proposed to improve the form accuracy of a machined surface produced by an ordinary milling machine by diamond fly cutting using controlled cutting with reference surface (CCRS), an in-process measurement and control method. Fly cutting is usually used to manufacture ultra-precision microstructures with nanometric surface roughness and submicrometric form accuracy, without the need for subsequent polishing. Nevertheless, a high level of accuracy has only recently been achieved on ultra-precision milling machines. In this study, we verified the effectiveness of fly cutting with CCRS on an ordinary milling machine. CCRS improves machined surface accuracy by controlling the relative displacement between the tool and workpiece. Diamond fly cutting using CCRS was demonstrated to reduce the table motion error on an ordinary milling machine. The experiments of curved surface machining by uncontrolled machining and control machining were conducted, and the effectiveness of improving the circular are machining accuracy of the general-purpose milling machine was confirmed.