The growing interest in lunar and deep-space exploration has underscored the need for fast, cost-effective, accurate, and efficient orbit determination (OD) methods for spacecraft. We present Visual Only Onboard Determination Of Orbit (Voodoo), a novel solution to the “lost-in-space” problem deployable on satellite hardware in real-time using lunar surface landmarks. Our approach utilizes visual data from an onboard camera and onboard computing to perform OD autonomously on orbit. Machine learning algorithms are employed to detect pre-selected surface landmarks on the lunar surface. These landmarks are then used by a Perspective-n-Point (PnP) solver to estimate the orbiter’s position in each frame. The estimated positions are subsequently refined by a constrained nonlinear batch least-squares state estimator, which reconstructs the full trajectory of the spacecraft. We validate Voodoo through comprehensive simulations of a lunar orbiter using real Moon image data, achieving 1.35km average position accuracy within a minute and 100 meter accuracy within 13 hours. This represents a significant improvement in both accuracy and time to first fix compared to traditional methods that rely on terrestrial infrastructure.