Drone observations on freeways offer exceptional insights into spatio-temporal microscopic traffic behavior, as the drone video captures a 600 m freeway segment over 20 minutes per drone flight in parallel with a data resolution of 0.1 seconds. Especially in dense and congested traffic the data analysis of lane changing on the different freeway lanes offers extended insights into the driver behavior. The approach defines the criticality of the vehicle maneuvers in the different traffic phases as defined in Kerner’s three phase traffic theory based on freeway segments in Germany, China and the United States. The analysis demonstrates the high-risk human drivers take while overtaking and passing activities which are and will be almost impossible for assisted or automated driving systems. This study shows that around 40 % of cars drive with a net gap smaller than 1.8 s and are, therefore, classified as risky. In addition, we analyze lane changing maneuvers and show that only 25 % of lane changes are classified as safe for surrounding vehicles. Due to our suggested hypothetical model for a lane change assistant, we can increase 50 % of all lane-changing maneuvers into a risk-free zone.