The automotive controller area network (CAN) bus functions as the communications backbone of automobiles around the globe. Unfortunately, the CAN bus was developed for the closed-system vehicles of the 1980s, not the inter-connected---even autonomous---vehicles hitting the roads today, meaning that the CAN bus is extraordinarily insecure. Much of the literature points to automotive intrusion detection as the solution; it is lightweight and does not involve re-engineering the CAN bus. That said, in safety-critical automotive environments, we can expect to see millions of messages every ten minutes; as such, accuracy is paramount. A false positive rate (FPR) of 0.00001 corresponds to about ten false positives every ten minutes.

Therefore, in this paper, we investigate false positives generated by the machine learning models that constitute automotive intrusion detection systems (IDSs). In particular, we explore the timestamp and time delta features to determine if they have a positive or negative impact on the FPR. Then, we look into the patterns of false positives, and we discover that many false positives are produced during actual attacks. Essentially, when legitimate messages are interlaced with attack messages, anomalous patterns are produced, and legitimate messages are flagged as anomalous. The IDS has done its job and detected an attack---it simply misidentified legitimate messages as part of the attack. When it comes to automotive attacks, many of the mitigation strategies do not require precise identification of the attack messages; that is, it is enough to know that an attack is ongoing. As such, we exclude false positives that occur during attack conditions from the FPR, and we examine the results. We find that, for a number of machine learning models, discounting attack-related false positives can significantly improve the FPR.