Advanced driver assistant systems are supposed to assist the driver and ensure their safety while at the same time providing a fulfilling driving experience that suits their individual driving styles. What a driver will do in any given traffic situation depends on the driver’s mental model which describes how the driver perceives the observable aspects of the environment, interprets these aspects, and on the driver’s goals and beliefs of applicable actions for the current situation. Understanding the driver’s mental model has hence received great attention from researchers, where defining the driver’s beliefs and goals is one of the greatest challenges. In this paper we present an approach to establish individual drivers’ temporal-spatial mental models by considering driving to be a continuous Partially Observable Markov Decision Processes (POMDPs) wherein the driver’s mental model can be represented as a graph structure following the Bayesian Theory of Mind (BToM). The individual’s mental model can then be automatically obtained through deep reinforcement learning. Using the driving simulator CARLA and deep Q-learning, we demonstrate our approach through the scenario of keeping the optimal time gap between the own vehicle and the vehicle in front.