I’m not sure how much you keep up with bridge-related holiday car crashes, but there was a huge one this past Thanksgiving on the San Francisco Bay Bridge. This was a genuine pileup, with eight vehicles involved and nine people injured. That’s already big news, but what makes this bigger big news is that the pileup seems to have been caused by a Tesla that was operating under the misleadingly-named Full Self-Driving beta software, according to the driver. As you likely already know, the inclusion of the nouns “Tesla” and the string of words “full self-driving” is internet click-catnip, but that’s not really what I want to focus on here. What this crash really demonstrates are the inherent conceptual – not technological – problems of all Level 2 semi-automated driving systems. Looking at what happened in this crash, it’s hard not to see it as an expensive, inconvenient demonstration of something called “the vigilance problem.” I’ll explain.
First, let’s go over just what happened. Thanks to a California Public Records Act request from the website The Intercept, video and photographs of the crash are available, as is the full police report of the incident. The crash happened on I-80 eastbound, in the lower level of the Bay Bridge. There’s five lanes of traffic there, and cars were moving steadily at around 55 mph; there appeared to be no obstructions and good visibility. Nothing unusual at all.
A Tesla was driving in the second lane from the left, and had its left turn signal on. The Tesla began to slow, despite no traffic anywhere ahead of it, then pulled into the leftmost lane and came to a complete stop — on the lower level of a bridge, with traffic all around it going between 50 and 60 mph or so. The results were grimly predictable, with cars stopping suddenly behind the now-immobile Tesla, leading to the eight-car crash.
Here’s what it looked like from the surveillance cameras:
…and here’s the diagram from the police report:
According to the police report, here’s what the Tesla (referred to in the report as V-1) said of what happened
“He was driving V-1 on I-80 eastbound traveling at 50 miles per hour in the #1 lane. V-1 was in Full Auto mode when V-1 slowed to 20 miles per hour when he felt a rear impact… He was driving V-1 on I-80 eastbound in Full Self Driving Mode Beta Version traveling at approximately 55 miles per hour…When V-1 was in the tunnel, V-1 moved from the #2 lane into the #1 lane and started slowing down unaccountably.”
So, the driver’s testimony was that the car was in Full Self-Driving (FSD) mode, and it would be easy to simply blame all of this on the demonstrated technological deficiencies of FSD Beta. This could be an example of “phantom braking,” where the system becomes confused and attempts to stop the car even when there are no obstacles in its path. It could have been the system disengaged for some reason and attempted to get the driver to take over, or it could be caused by any number of technological issues, but that’s not really what the underlying problem is.
This is the sort of wreck that, it appears, would be extremely unlikely to happen to a normal, unimpaired driver (unless, say, the car depleted its battery, though the police report states that the Tesla was driven away, so it wasn’t that) because there was really no reason for it to happen at all. It’s about the simplest driving situation possible: full visibility, moderate speed, straight line, light traffic. And, of course, if the driver was using this Level 2 system as intended – remember, even though the system is called Full Self-Driving, it is still only a semi-automated system that requires a driver’s full, nonstop attention and a readiness to take over at any moment, which is something the “driver” of this Tesla clearly did not do.
Of course, Tesla knows this and we all technically know this and the police even included a screengrab from Tesla’s site that states this in its report:
We all know this basic fact about L2 systems, that they must be watched nonstop, but what we keep seeing is that people are just not good at doing this. This is a drum I’ve been banging for years and years, and sometimes I think to myself: “Enough already, people get it,” but then I’ll see a crash like this, where a car just does something patently idiotic and absurd and entirely, easily preventable if the dingus behind the wheel would just pay the slightest flapjacking bit of attention to the world outside, and I realize that, no, people still don’t get it.
So I’m going to say it again. While, yes, Tesla’s system was the particular one that appears to have failed here, and yes, the system is deceptively named in a way that encourages this idiotic behavior, this is not a problem unique to Tesla. It’s not a technical problem. You can’t program your way out of the problem with Level 2; in fact, the better the Level 2 system seems to be, the worse the problem gets. That problem is that human beings are simply no good at monitoring systems that do most of the work of a task and remaining ready to take over that task with minimal to no warning.
This isn’t news to people who pay attention. It’s been proven since 1948, when N.H. Mackworth published his study The Breakdown of Vigilance During Prolonged Visual Search which defined what has come to be known as the “vigilance problem.” Essentially, the problem is that people are just not great at paying close attention to monitoring tasks, and if a semi-automated driving system is doing most of the steering, speed control, and other aspects of the driving task, the human in the driver’s seat’s job changes from one of active control to one of monitoring for when the system may make an error. The results of the human not performing this task well are evidenced by the crash we’re talking about.
I think it’s not unreasonable to think of Level 2 driving as potentially impaired driving, because the mental focus of the driver when engaging with the driving task from a monitoring approach is impaired when compared to an active driver.
I know lots of people claim that systems like these make driving safer – and they certainly can, in a large number of contexts. But they also introduce significant and new points of failure that simply do not need to be introduced. The same safety benefits can be had if the Level 2 paradigm was flipped, where the driver was always in control, but the semi-automated driving system was doing the monitoring, and was ready to take over if it detected dangerous choices by the human driver. This would help in situations of a tired or distracted or impaired driver, but would be less sexy in that the act of driving wouldn’t feel any different than normal human driving.
If we take anything away from this wreck, it shouldn’t be that Tesla’s FSD Beta is the real problem here. It’s technically impressive in many ways though certainly by no means perfect; it’s also not the root of what’s wrong, which is Level 2 itself. We need to stop pretending this is a good approach, and start being realistic about the problems it introduces. Cars aren’t toys, and as much fun as it is to show off your car pretending to drive itself to your buddies, the truth is it can’t, and when you’re behind the wheel, you’re in charge — no question, no playing around.
If you want to read about this even more, for some reason, I might know of a book you could get. Just saying.