Vision-only Self-driving Cars Will Never, Ever, Ever, Ever Happen!

Lessons from the Military

“First of all, I want to thank the conference organizers for inviting me back after these past years. Now, let’s get to the heart of the matter. Are we there yet? The short answer is ‘No’& - ‘not yet.’ You might as well head straight to the bar (laughter). But let me explain why.
As you’ve heard, I was a U.S. Navy fighter pilot. Yes, I flew F-18s and landed on aircraft carriers. Why is this relevant today? During my three years as a pilot, I witnessed, on average, one death per month due to poorly designed autonomous systems. Aviation has had decades of painful lessons that gradually improved safety. Unfortunately, those lessons have not yet carried over into transportation.
Afterward, I earned my Ph.D. and spent years researching drones before moving into automotive. This photo shows my 2013 work with Waymo, back when Google X had just begun looking seriously at self-driving cars. At the same time, I worked for the U.S. government on remote operations of the Predator, Reaper, and Army Shadow drones.
So why does this matter? Because even though we already learned many lessons from remote operations, those lessons are still not being applied in transportation.”
Progress and Persistent Challenges
“Now, let’s move to something positive.
The progress in autonomous vehicles has been remarkable. When I started with Google X in 2013, it was still early days and full of difficulties. Fast-forward 12 years: Waymo now operates in multiple cities, and Zoox has developed a multi-passenger shuttle, launching commercial operations in Las Vegas. Multi-passenger shuttles may actually unlock the economies of scale necessary for profitability.
So yes, AVs are real. They are operating in the U.S. and generating revenue. But let’s be clear - making money is not the same as making profit. I’ll come back to that point. For now, I want to applaud the companies that have worked tirelessly to reach this stage.
Autonomous vehicle R&D has made incredible strides. If it had remained in the military sector, it would never have advanced this quickly. Civilian industry has moved far faster. But still, major challenges remain.”







The ‘Hallucination’ Problem

“The first -  and perhaps deadliest - problem is something you may not want to hear.
Most of you have used ChatGPT or other LLMs and have seen ‘hallucinations.’ Strictly speaking, these are not hallucinations but statistical inference errors. But autonomous cars really do hallucinate.
This photo shows a Tesla crash that illustrates the danger better than anything else. A few Thanksgivings ago, a Tesla on San Francisco’s Bay Bridge tunnel ‘saw’ something that wasn’t there. At 65 miles per hour it slammed on the brakes, triggering an eight-car pileup. Thankfully no one died that day, but people have died in similar phantom braking accidents. The U.S. NHTSA continues to investigate Tesla for this very issue.
As a researcher in computer vision, I can tell you - we do not know how to solve this yet. It remains a mystery. And because it remains unsolved, accidents continue. Every company I investigated during my time at NHTSA - Tesla, Waymo, Zoox, and others - struggled with this. Shuttles, passenger cars, trucks  anywhere computer vision is used, hallucinations appear. If we do not solve this, large-scale deployment will never happen.”







Highways and the Limits of Vision Sensors

“Let me share another shocking case. In 2023, a fatal accident involving Cruise in California forced the company to suspend operations. A jaywalker was first struck by a human-driven vehicle, then thrown into the path of a Cruise AV. The AV braked correctly, but when the pedestrian was pinned underneath, the system no longer classified it as ‘human-involved.’ The car began moving sideways with the woman trapped beneath. This was no edge case - jaywalking in San Francisco is routine.
And yet another alarming fact: nine seconds before impact, the car had already detected the pedestrian but accelerated anyway. Once again, these systems do not think, do not imagine, and do not understand. End-to-end learning will not fix this.
Now let’s talk highways. Shockingly, no one has solved this yet - not trucks, not Waymo, no one. No AV company in the U.S. can drive safely and consistently on highways. Demos may look impressive, but proof of safe operation at speed does not exist. The hallucination problem is the main reason.
And one more point - since I’m in Europe, I can say this without fear of getting shot: Tesla already knows this, Mercedes has told them countless times - vision-only self-driving cars will never, ever, ever, ever happen! In robotics 101, we teach students that no single sensor can ensure safe navigation. It is impossible. Period.”







Human Babysitters and Remote Operations

“Back to remote operations. There are two types: what Waymo does is ‘remote assist,’ not tele-operation. If the vehicle gets stuck at an intersection, a remote human gives guidance like ‘yes, go ahead’ or ‘turn right instead of left.’
But let’s be honest: there is no such thing as a truly autonomous vehicle. Every company needs human babysitters. No exceptions.
This is why we say remote assist is safer than tele-driving, especially above 30 km/h. Still, accidents have occurred due to latency. Waymo even moved some remote operators overseas to the Philippines. And unsurprisingly, signal delays caused crashes in California intersections.
We need to be honest. Companies must stop claiming to be ‘autonomous vehicle companies.’ At best, they are human babysitter-dependent AV companies.”







The Swiss Cheese Model and Unsafe Testing

“Now, about safety. The model you see is the old aviation safety model known as the ‘Swiss Cheese Model.’ It shows how accidents occur when holes in multiple layers - human error, poor supervision, organizational pressure - line up.
I’ve reimagined this model for AI. Because the ultimate outcome is still the same: accidents and deaths. But instead of ‘unsafe acts,’ the leading cause is ‘unsafe testing.’
How do I know? Because I was the lead expert in Tesla’s massive $243 million punitive damages case last July. Tesla skipped critical testing in the name of cost savings. And they’re not alone. Many companies cut corners, neglect retraining neural networks, or avoid expensive end-to-end learning updates. But skipping this process can cost hundreds of millions in court.
Poor AI design is another risk. Vision-only autonomy is a terrible idea. And poor oversight is equally dangerous. Boasting that technology always works is unacceptable.”



Cooperation or Collapse

“I know I may sound negative, but I am deeply invested in robotics. There are valuable AV applications - robotic shuttles are my favorite example. But we must treat safety challenges seriously. Companies need frameworks for assessment, and most importantly, they must start cooperating.
The U.S. market is fiercely competitive. Companies avoid cooperation, and regulators are often kept at arm’s length. But this is the wrong path. The only way forward is to recognize that we are one team, share results, and jointly solve critical challenges like hallucination. Only then can we bring truly safe systems to market faster.”




 

---

Q&A
What are the most critical technical or operational challenges that have caused setbacks? Are there differences between trucks and passenger cars?
Missy: There is no fundamental difference between trucks and passenger cars - the difference lies in the consequences. A jackknifing truck is inevitable, and when it happens, it could kill many and potentially collapse the industry. To operate in such high-risk environments, equivalent risk mitigations must be in place. But so far, I have not seen safe and reliable highway AV operation - neither in trucks nor in cars. The hallucination problem must be solved first.

Should vehicles process complex datasets in the cloud or locally?
Missy: This is critical. Safety-critical functions can never be offloaded to the cloud. Response must be immediate. I have not seen any connectivity environment that reliably achieves the necessary sub-10 millisecond latency. Maybe someday, but not yet.

What are AI’s biggest limitations in handling rare or unusual scenarios?
Missy: Let me repeat: AI does not think, imagine, or know. Do not use generative AI for safety validation. Neural nets regress to the mean; they are not built for edge cases. No amount of interpolation will help. This is why human–machine collaboration is essential. You cannot fire your safety teams. Humans will always be needed, because they keep finding new ways to trick the machines.

[AEM] Automotive Electronics Magazine

---