Recent studies have shown that implicit neural representation can be effectively applied to geometric surface reconstruction. Existing methods have achieved impressive results. However, they often struggle to recover geometric details, or require normal vectors as supervisory information for surface points, which is often unavailable in actual scanned data. In this paper, we propose a coarse-to-fine approach to enhance the geometric details of the reconstructed results without relying on normal vectors as supervision, and able to fill holes caused by missing scanned data. In the coarse stage, a local spatial normal consistent term is presented to estimate a stable but coarse implicit neural representation. In the fine stage, a local fitting penalty is proposed to locally modify the reconstruction results obtained in the previous stage to better fit the original input data and recover more geometric details. Experimental results on three widely used datasets (ShapeNet, SRB and ABC) indicate that our method is very competitive when compared with current state-of-the-art methods, especially for restoring the geometric details.