After an extensive, 1000-mile evaluation, AMCI Testing issues Warnings and Limited Encouragement, for Tesla’s latest version of “Full Self Driving (Supervised)”
Weeks out from Robotaxi unveiling, overall performance of Tesla’s camera-enabled, autonomous-driving software is suspect.
Los Angeles, CA September 24. 2024
After what is likely the most extensive real-world test of Tesla’s FSD ever conducted by an independent third party – covering more than 1000 miles (1610 km) – AMCI Testing’s result demonstrates just how far Tesla must go before Robotaxi operations can be safely undertaken. AMCI Testing’s evaluation was based exclusively on Tesla’s latest software iterations, 12.5.1 and 12.5.3, with time spent across four distinct driving environments: city streets, rural two-lane highways, mountain roads and freeways.
While impressive for a uniquely camera-based system, AMCI Testing’s evaluation of Tesla FSD exposed how often human intervention was required for safe operation. In fact, our drivers had to intervene over 75 times during the evaluation; an average of once every 13 miles. “ With all hands-free augmented driving systems, and even more so with driverless autonomous vehicles, there is a compact of trust between the technology and the public. When this technology is offered the public is largely unaware of the caveats (such as monitor or supervise) and the tech considered empirically foolproof. Getting close to foolproof, yet falling short, creates an insidious and unsafe operator complacency issue as proven in the test results,” said David Stokols, CEO of AMCI Testing’s parent company, AMCI Global. “ Although it positively impresses in some circumstances, you simply cannot reliably rely on the accuracy or reasoning behind its responses.”
AMCI Testing has just dropped the first round in a series of test-video releases intended to demonstrate the complex issues of trust and performance that FSD poses to drivers and the public. These and future videos can be seen at www.amcitesting.com/teslafsd
“It’s undeniable that FSD 12.5.1 is impressive, for the vast array of human-like responses it does achieve, especially for a camera-based system,” said Guy Mangiamele, Director of AMCI Testing. “ But its seeming infallibility in anyone’s first five minutes of FSD operation breeds a sense of awe that unavoidably leads to dangerous complacency. When drivers are operating with FSD engaged, driving with their hands in their laps or away from the steering wheel is incredibly dangerous. As you will see in the videos, the most critical moments of FSD miscalculation are split-second events that even professional drivers, operating with a test mindset, must focus on catching.”
AMCI Testing found that the more miles you drive with FSD engaged, the more likely you are to encounter these failure modes.
“What’s most disconcerting and unpredictable is that you may watch FSD successfully negotiate a specific scenario many times – often on the same stretch of road or intersection – only to have it inexplicably fail the next time,” Mangiamele continued. “Whether it’s a lack of computing power, an issue with buffering as the car gets ”behind” on calculations, or some small detail of surrounding assessment, it’s impossible to know. These failures are the most insidious. But there are also continuous failures of simple programming inadequacy, such as only starting lane changes toward a freeway exit a scant tenth of a mile before the exit itself, that handicaps the system, and casts doubt on the overall quality of its base programming.”
Next week, we will release another round of videos to further illustrate the real-world capabilities of FSD (Supervised). AMCI Testing will continue to evaluate subsequent iterations of Tesla’s FSD as they become available. Go to www.amcitesting.com to sign up to receive updates as they occur.
For further information please contact:
Ian Beavis, Chief Strategy officer at AMCI Global; Ph 714 813 6300 or Email ibeavis@amciglobal.com
About AMCI Testing
AMCI Testing is an independent automotive research firm, specializing in unbiased, exclusive, comparative evaluations of automotive products since 1984. The breadth of our testing includes ICE, HEV, PHEV, BEV, FCEV powertrains and every facet of measurement and product category. AMCI Testing Certification is recognized globally as an industry gold standard