Incident 253: Cruise's Self-Driving Cars Allegedly Lost Connection to Their Server, Causing Traffic Blockages in San Francisco

Description: Cruise’s autonomous vehicles were shown on video stopping in the middle of the road and causing blockages in San Francisco, as they were disabled allegedly due to lost connection to their company’s server.
Alleged: Cruise developed and deployed an AI system, which harmed San Francisco traffic participants and San Francisco public.

Suggested citation format

Perkins, Kate. (2022-05-18) Incident Number 253. in Lam, K. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
253
Report Count
4
Incident Date
2022-05-18
Editors
Khoa Lam

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscover

Incident Reports

A very San Francisco Saturday night. A pack of 3 driverless @Cruise cars got confused and stopped and blocked two lanes of Masonic. The robots will win just by blocking traffic.

Tweet: @sgatz

Cruise’s driverless cars experienced serious issues Tuesday night with as many as 20 of its vehicles standing motionless for about two hours at the corner of Gough and Fulton streets, according to an eyewitness. The incident was only resolved when the robo-cars were manually moved out of the way by Cruise staff.

Sean Sinha, the bouncer at Smuggler’s Cove bar on Gough Street, told the Examiner that around 10 p.m. Tuesday, “Cruise cars started to flood down Gough and make a right on Fulton, only to stop. Pretty soon over twenty cars were stopped, blocking traffic on Gough.”

At one point, two of Gough’s three southbound lanes lanes were impassable due to the motionless cars, as photos shared by Sinha on Reddit show.

After about 20 minutes, the first of several Cruise employees arrived and began manually moving the cars. The ordeal wasn’t resolved until after midnight, Sinha said.

In the meantime, between the Cruise driverless cars and the double-parked cars of arriving Cruise staff, some travel lanes on Gough and Fulton remained impassable. The motionless cars inhibited regular street sweeping, Sinha said.

That same night, at about 12:30 a.m. Wednesday, Twitter user @mannsiii wrote a now-deleted tweet saying that four Cruise driverless cars were idling near Fell and Masonic over multiple light cycles, blocking traffic and causing other cars to reverse off of the block in order to get around them.

In response to specific questions about both incidents, a Cruise spokesperson provided the following statement: "We had an issue earlier this week that caused some of our vehicles to cluster together. While it was resolved and no passengers were impacted, we apologize to anyone who was inconvenienced.”

The San Francisco Police Department said it did not receive any reports about these events.

The issues come less than a month after the California Public Utilities Commission approved Cruise to begin a late-night, paid ride-hailing service in San Francisco without a backup driver, allowing the robo-taxis to directly compete with Uber and Lyft. Already, Cruise and autonomous vehicle competitor Waymo are providing free rides, sans driver, to a select group of passengers around San Francisco. Both companies are also testing their vehicles around The City during daylight hours with backup safety drivers.

San Francisco transportation, police and fire officials urged the CPUC to reject Cruise’s permit for commercial ride hailing services, in part due to concerns about Cruise vehicles blocking traffic. City officials claimed in a letter to the CPUC that they do not believe Cruise vehicles are technologically capable of pulling over to the curb to pick up and drop off passengers, instead double-parking in the travel lane. Cruise denies this.

City officials also cited three specific incidents where Cruise vehicles behaved unsafely, including one where a Cruise vehicle idled in a construction zone for five minutes blocking traffic. In another instance, an idling Cruise vehicle blocked the passage of a fire truck responding to a blaze for about 30 seconds.

“SFFD is extremely concerned about vehicles stopping in travel lanes and the potential negative impact of this driving behavior on fire department response times,” the letter reads.

The City’s letter to the CPUC also revealed that a loophole in state law makes it impossible for law enforcement to cite driverless vehicles with moving violations, like running a red light or an illegal turn. Driverless vehicles can, however, be cited for parking violations, like blocking the travel lane.

Cruise driverless cars blocked traffic for hours Tuesday night

Around midnight on June 28, Calvin Hu was driving with his girlfriend near San Francisco’s Golden Gate Park when he pulled up at an intersection behind two white and orange autonomous Chevrolet Bolts operated by Cruise, a subsidiary of General Motors. Another was stopped to his right in the adjacent lane. The light turned green but the cars, which operate in the city without drivers, didn’t move.

When Hu prepared to reverse and go around the frozen vehicles, he says, he noticed that several more Cruise vehicles had stopped in the lanes behind him. Hu, another driver, and a paratransit bus were trapped in a robotaxi sandwich.

This content can also be viewed on the site it originates from.

After a few minutes of bemused waiting, Hu says, he resorted to driving over the curbs of the street’s median to escape. When he returned on foot a few minutes later to see whether the situation had resolved, the Cruise vehicles hadn’t budged. A person who appeared to work for the company had parked in the intersection, Hu says, as if to indicate the street was closed, and was trying to direct traffic away from the immobile self-driving cars. Hu estimates that the robot car blockade, which has not previously been reported, lasted at least 15 minutes.

The Cruise vehicles that trapped Hu weren’t the only autonomous cars holding up traffic in San Francisco that night. Internal messages seen by WIRED show that nearly 60 vehicles were disabled across the city over a 90-minute period after they lost touch with a Cruise server. As many as 20 cars, some of them halted in crosswalks, created a jam in the city’s downtown in an incident first reported by the San Francisco Examiner and detailed in photos posted to Reddit. In a written statement the California Department of Motor Vehicles, which oversees the state's autonomous vehicle operations, said it was aware of the incident and would meet with Cruise to “gather additional information.”

The June 28 outage wasn’t Cruise’s first. On the evening of May 18, the company lost touch with its entire fleet for 20 minutes as its cars sat stopped in the street, according to internal documentation viewed by WIRED. Company staff were unable to see where the vehicles were located or communicate with riders inside. Worst of all, the company was unable to access its system which allows remote operators to safely steer stopped vehicles to the side of the road.

A letter sent anonymously by a Cruise employee to the California Public Utilities Commission that month, which was reviewed by WIRED, alleged that the company loses contact with its driverless vehicles “with regularity,” blocking traffic and potentially hindering emergency vehicles. The vehicles can sometimes only be recovered by tow truck, the letter said. Images and video posted on social media in May and June show Cruise vehicles stopped in San Francisco traffic lanes seemingly inexplicably, as the city’s pedestrians and motorists navigate around them.

Cruise spokesperson Tiffany Testo says that the cars stuck on May 18 “were able to move over as part of the suite of fallback systems Cruise has in place.” She provided a written statement that said the company’s vehicles are programmed to pull over and turn on their hazard lights when they encounter a technical problem or meet road conditions they can’t handle. “We’re working to minimize how often this happens, but it is and will remain one aspect of our overall safety operations,” the statement said. Testo did not respond to questions about multiple incidents in which Cruise vehicles stopped in traffic.

The outages come at a vital time for Cruise, which is accelerating its autonomous vehicle program on the tricky streets of San Francisco as it competes with well-capitalized rivals like Google’s sister company Waymo, Aurora, and Zoox, which is owned by Amazon. In the spring, General Motors bought out the SoftBank Vision Fund’s $2.1 billion stake in Cruise and invested another $1.35 billion into the self-driving unit. Just over two weeks after the May outage that froze Cruise’s fleet, the CPUC approved Cruise’s permit to charge money for Uber-like ride-hail rides—opening a path to a full commercial robotaxi service that could help the company start to recover the billions it has poured into building its technology.

Cruise began testing its autonomous technology in San Francisco in 2015, with safety drivers behind the wheel to intervene if something went wrong. Five years later, the DMV granted approval for the company to test its cars without humans onboard. Early this year, Cruise invited members of the public to apply to join a select group of testers in the city, who can summon completely driverless rides with an app. The service is available between 10 pm and 6 am, and covers 70 percent of the city, but isn’t permitted to operate in rain and fog.

Around midnight on June 21, nearly two weeks after Cruise won permission to charge for rides, San Francisco resident Stephen Merity was walking through the city’s Tenderloin neighborhood when he saw a driverless Cruise stopped in a crosswalk, blocking a right-hand turn lane. When he returned a few minutes later, he found two more Cruise vehicles stopped behind the first. When another driverless car appeared and started to navigate around its stuck brethren, an apparently inebriated bystander cheered it on: “You can do it!”

Merity and the robot’s cheerleader told a human driver patiently waiting between two of the motionless Cruise vehicles that she should steer her SUV around the robotaxis. The vehicles had been stuck for at least 10 minutes before he left to go home, Merity says.

The scene initially struck Merity, who works in machine learning, as pretty funny. But after more reflection, he turned anxious about the technology. When he saw a news report about the June 28 outage, he was dismayed. “I had assumed alarm bells were going off at Cruise HQ, and they were thinking of pulling the cars off the road and putting drivers back in,” Merity says.

Losing connection with its vehicles, and especially its backup safety systems, might violate Cruise’s permits to operate in California, says Bryant Walker Smith, an associate professor at the South Carolina School of Law who studies autonomous vehicles. The California DMV program that regulates driverless cars requires a vehicle’s operator to certify that it has a link allowing for “two-way communications” between a vehicle—including its passengers—and an employee remotely overseeing the robot’s movements. However, much like autonomous cars themselves, regulations drawn up to apply to the vehicles have not been tested in every possible scenario.

Cruise did not respond to specific questions about its permits. Neither the California DMV nor the CPUC would say how Cruise’s permits might be affected by the outage incidents. The CPUC did not say whether it had responded to the anonymous Cruise employee’s letter, or taken its contents into consideration before approving the company’s permit.

Regardless of Cruise’s legal obligations, Walker Smith says that self-driving companies should be open and transparent about what’s happening on public roads. “From the perspective of the public, when the vehicles do something wrong or weird, it’s on the company—the ‘driver’—to really explain it.”

Interior of Cruise car with screen displaying assistance message Screens inside autonomous vehicles stuck near an intersection in downtown San Francisco urged first responders to contact Cruise. PHOTOGRAPH: STEPHEN MERITY People who have seen Cruise’s vehicles on the streets of San Francisco, or have ridden in the cars, say the robotaxis generally avoid the busiest streets. Rodney Brooks, an MIT roboticist and entrepreneur, signed up for Cruise’s taxi service via the company's online public portal. In May, he and a friend took a series of nighttime rides throughout northern San Francisco. On one trip, Brooks noticed that their driverless taxi went at least 11 blocks farther than necessary, roughly half a mile, apparently to avoid the most crowded roads. The journeys were mostly smooth, Brooks says, though one car he summoned stopped alongside a construction site, forcing him and his friend to walk through traffic to get into the car.

For Brooks, robotaxis getting stuck in and blocking traffic is evidence of the challenges faced by Cruise and its competitors as they try to turn promising prototype autonomous vehicles into large-scale commercial services. “A lot of technologists think if you do a demo, then that’s it. But scaling is what kills you,” he says. “You run into all sorts of things that didn’t happen at a smaller scale.

Incidents of Cruise losing touch with its vehicles appear to have caused inconvenience, not injuries. Some have enjoyed the spectacle. One night in May, Scott Gatz was caught behind four stopped Cruise vehicles at the same intersection where Hu would be trapped weeks later. A city worker approached the vehicles, looking confused, Gatz says, and later a man with a tablet arrived who appeared to work for Cruise.

Gatz and other drivers escaped that night by squeaking through a small gap to the side of the robot vehicles. “Cruise really needs to fix its software, and we as a city need to figure out how we can coexist with these cars,” Gatz says—but he’s still glad to see the technology tested in San Francisco. His 12-year-old son, who was in the car with him that night, thought the whole thing was pretty fun.

Operating large, heavy robots around humans inevitably comes with some danger. On June 3, the day after the company received its permit to charge for rides in California, a Cruise vehicle making a left turn in front of traffic was struck by an oncoming Toyota Prius. A Cruise employee in the car and the Prius driver both had to seek medical treatment, according to a report filed by the company with the DMV, in line with a requirement to report all autonomous vehicle-related crashes. The report said the Cruise vehicle had stopped in the intersection before completing its turn, and that the Prius had sped straight through its right turn lane into the motionless robot.

In response to that crash, Cruise temporarily reprogrammed its vehicles to make fewer unprotected left turns, according to internal messages seen by WIRED. At an internal meeting Jeff Bleich, Cruise’s chief legal officer, said the company was investigating the incident, according to a recording reviewed by WIRED. He also warned employees not working on that investigation to try and tune out crashes or related news reports, saying they were unavoidable and would increase in frequency as the company scaled up its operations. “We just have to understand that at some point this is now going to be a part of the work that we do, and that means staying focused on the work ahead,” he said.

On Thursday, the National Highway Traffic Safety Agency said it would open a special investigation into the crash. In a statement, Testo, the Cruise spokesperson, said the company is “proud” of its safety record, “and it speaks for itself.”

Updated 7/8/2022 9:45 pm ET This article has been updated with additional comment from Cruise about an incident in May in which the company lost contact with its autonomous vehicles.

Cruise’s Robot Car Outages Are Jamming Up San Francisco

GM Cruise snafus have blocked SF streets with suddenly immobilized self-driving cars at least three times in the last two months, with one incident where almost 60 vehicles stopped, and another where Cruise “lost touch with its entire fleet.”

We wrote last week about an eerie cyber-snafu with GM Cruise self-driving robotaxis, where at least eight of the driverless cars ganged up and blocked Gough Street “for a couple of hours,” and would not move until GM employees showed up and moved the cars manually.

Turns out that wasn’t the only such incident of self-driving robot cars blocking the street and not allowing  traffic through. In fact, it wasn’t even the only such incident that night.

Wired has a new report detailing numerous incidents of GM Cruise vehicles suddenly freezing and stopping traffic in San Francisco. These include that night’s incident, but also a previously unreported incident that same night “near San Francisco’s Golden Gate Park,” where two normal human-driven cars “and a paratransit bus were trapped in a robotaxi sandwich.”

Terrifyingly, Wired adds that on that particular evening “Internal messages seen by WIRED show that nearly 60 vehicles were disabled across the city over a 90-minute period after they lost touch with a Cruise server.”

“Lost touch with a Cruise server.” Goin’ great!

A very San Francisco Saturday night. A pack of 3 driverless @Cruise cars got confused and stopped and blocked two lanes of Masonic. The robots will win just by blocking traffic. pic.twitter.com/63Xi8RG21I

— Scott Gatz (@sgatz) May 22, 2022

This had happened in previously unreported incidents, too. On the night of May 18, according to internal documents Wired acquired, “the company lost touch with its entire fleet for 20 minutes as its cars sat stopped in the street.” (This is documented by a passerby in the tweet above.) Wired adds that in that incident, “Company staff were unable to see where the vehicles were located or communicate with riders inside. Worst of all, the company was unable to access its fallback system, which allows remote operators to safely steer stopped vehicles to the side of the road.”

@Cruise walking past #GoldenGatePark thankfully night isnt it's busiest hour or this would have been a bigger accident. This is why they need a driver in the car incase something goes wrong! Like if somebody had hit that how are they going to get the insurance information? pic.twitter.com/bfQAWkzTme

— Be Awkward😎it's Ok! (@WonderG78) June 24, 2022

For good measure this appears to have happened again on June 24, as seen above, where it was apparently only one GM Cruse vehicle that froze, and other drivers were able to maneuver around it.

Cruise spokesperson Tiffany Testo said in a statement to Wired that “We’re working to minimize how often this happens, but it is and will remain one aspect of our overall safety operations,” That doesn’t really explain why this keeps happening, nor does it inspire much confidence that it won’t happen again and again, nor does it show any concern for potentially deadly consequences.

Wired’s report got some of its information from “A letter sent anonymously by a Cruise employee to the California Public Utilities Commission” sounding the alarm. So apparently someone on the inside at Cruise feels this is a showstopper problem. And the only reason we know about any of this is via that anonymous letter, or regular folks who just happen across these snafus and post their experience to social media.

Cruise does not seem keen to publicize the frequency of these incidents, and the California Public Utilities Commission that regulates these machines clearly knows of the trouble, and seems content to allow it to continue.

That may mean there will be no regulatory blowback, or operational changes, until an emergency vehicle gets blocked and/or something tragic happens. So for pedestrians, residents, and drivers in the City and County of San Francisco, we can only hope the roulette ball of that tragedy, or those tragedies, does not fall upon us.

GM Cruise Robotaxis Blocked SF Streets More Frequently Than We Knew, One Night ‘Nearly 60’ Cars Stopped