Incident 187: YouTuber Tested Tesla on Self Driving Mode, Colliding with Street Pylons

Description: A YouTuber who was a Tesla’s employee conducted an on-road review of Tesla's Full Self Driving (FSD) Beta, showing its navigation in various road environments in San Jose and collision with a bollards during Autopilot, allegedly causing his dismissal from the company.
Alleged: Tesla developed an AI system deployed by AI Addict, which harmed John Bernal and San Jose public.

Suggested citation format

AIAAIC. (2022-02-04) Incident Number 187. in McGregor, S. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
187
Report Count
3
Incident Date
2022-02-04
Editors
Sean McGregor, Khoa Lam

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscover

Incident Reports

Hey YouTube AI addict here welcome to another FSD Beta video. Today we are on 10.10 in downtown San Jose for another downtown stress test. I'm so sorry you guys haven't seen us here in a while; it's been maybe a month since we made a video I've just been very busy you know with work because i actually put the wrong navigation in ... "Now turn left onto [inaudible] Avenue" ... So it that the map is actually taking us the way I don't want to go so I'm going to wait for it to update.

Now we're here, I'm going to reactivate FSD and we'll hopefully get an update and it takes me down the road. So it looks like Tesla is purposely avoiding the route I always take right here on the left; that's very interesting. It shows two creatively longer ways to avoid this specific left turn for some reason, the one where it sometimes often struggles. It's very interesting; I wonder if that was, uh, if that was just organic or if that was like on purpose. Okay coming to a hard break here, I think that it thinks that this thing's at the stop sign; that's really bizarre. Going kind of wide, but it makes the turn. Interesting. As i was saying if, uh, if anyone knows anything about inverters, controllers, batteries, or just EV motors, I'm looking for some consultants you know here in our community who can help me on a project I'm working on for 2022 for this channel; we're doing an EV conversion. Email us at aiaddictofficial@gmail.com for more information, I would love to get in contact with some of you guys out there in our community. Um, right now we're trying to turn left across a busy road I ... it is a bit clear from our left, but now it's not going to be ... Here comes a truck and it's moving forward and I don't like this ... Holy ... okay ... oh wow ... Okay now it is clear to go and ... nope, never mind ... oh wow, we backed up, did you see that, going backwards.

And here we go made the left-hand turn, and again the right-hand lane, because our destination is here on the right. All right, so we've added a new destination; let's see how the car does. We actually have to turn the left of this light up here ... is it going to slow down to let us make this left? It is going to slow down, it is signaling that that car is blue ... I think that's waiting for it. And here, it does, it does get us all the way over so that was actually pretty good. That was good and it made the light! Oh this is ... no, it found it, it found it, but it almost took us the wrong way. We can go right at this light here ... is it just gonna run this light? Holy shit, it just ran that red light. All right, so they took out the, uh, rolling stop signs, but they did a rolling right turn on red. Yeah wow ... I guess ... I'll report that. Jesus. "Now turn left onto [inaudible] Boulevard" ... A little tricky there on that turn, but that's got us here ... changing lanes. Ooh ... oh ... oh fuck! Oh shit, we hit that we actually hit it! We hit it! Wow we were so close on the quarter panel ... I was, I think ... Oh i hit that too, I hit that for a fact! I hit the brakes to the floor ... yeah wow, I can't believe the car didn't stop!

So yeah so we did hit the pylon; you can see those marks right here on the bottom of my bumper as well as a couple marks right up

here, as you guys can see.

All right YouTube, it is confirmed, I have hit that pylon, um, it's the first for me to have actually hit an object on FSD. As we ... as you can see, we put a new navigation in, so let's try and see how it does. There's two cars on our left, um, driving next to us ... Oh, a little, little ... strong there coming out, but that's okay, I would have waited for both in the past if we're starting to roll forward ... doing a good job though, keeping us in our lanes, not going in the bus taxi pull-off area over there. Up here, in about three blocks, we will

have a right-hand turn across some railroad tracks, which you guys have seen in previous iterations, um, I'll run it right now, so you can see it.

And it tried to go down the actual railroad track on the right there. We've seen various improvements in this turn ... let's see how it does on this iteration ... It's coming to a hard slow down because of this person that's right here in the crosswalk. So it's being conservative, which is nice. "Now turn right onto ..." oh what's this, what's this ... where is it taking us? There's the right hand turn across the railroad tracks, there are pedestrians in it, and it did a decent job actually ... did a really good job there, and it's getting us out of the bus lane, and our navigation had us right here on the right, so that's actually pretty perfect.

So now we're back, um, new navigation ... we're gonna make a complete U-turn, as you can see on the navigation, going across two more railroad track paths. Let's see how the car does ... it did a good job just going around everyone who was in the bus lane

that was double parked. It is green for us but red for the railroad tracks for anyone on YouTube is curious about what those signs mean, and this lane is closed right here ... does a good job of going around it. "Now turn right onto South 1st Street." So this is a "Keep Clear" because there's a railroad ... the trolley has to be able to go right here, and it's doing a good job of keeping clear, for the first time ... this is the first time i've seen it keep clear right here. It's also a "no turn on red," and it's patiently waiting for the light which is

cool. I wonder that has anything to do with the fact that the right ... far right turn lane is closed. Yeah here it goes, it gets us in the far

right lane. Goes to make the right hand turn ... oh ... I'm gonna stop it. Yep, yep. That's very unfortunate, I'm gonna report that.

All right, right here we got some people on some lime scooters in front of the car in front of us, slowing down traffic

a bit. The car is trying to go in the right turn lane to go around the slow car, but realize that it can't, so it did fix itself which was kind of nice ... not sure why it tried in the first place though. "In 500 feet, turn right on to East ..." Now, looks like we're gonna be the people

that will see these ... it says ... shows us on this walk instead of on a lime scooter, which is interesting. Those little scooters are probably hard to see and identify. It's going to a full stop here, and it's always because the wall fire goes right I'm right here on the side, so i'm going to hit the accelerator to let it go through.

So we are allowed to turn right on right here, oh, but it goes green, and there are pedestrians who are crossing. It's doing a good job of patiently waiting for them. Now it proceeds ... oh god ... going towards those pylons again ... I really don't like that! Right here again, it's going towards the pylons "... to South Second Street" ... like, we're not parking. So this says right here it's a no right on red, so let's see how it handles it. All right, the light has turned green, and there it goes, it did it. Perfect, and it got us to our destination!

Well, thank you guys so much for joining us on 10.10 right here in downtown San Jose for our stress test. Um, I cannot believe that it actually hit a pylon and, um, it's ... i'm just ... just ... I'm in shock man! It's come so close so many times and I never thought it would actually happen, but uh ... But yeah, like i said if you know anything about inverters, controllers, batteries, EV motors, feel free to hit us up on aiaddictofficial@gmail, link down in the description, so that way you can help us out on future projects if you want, and, uh, make sure to join us on Patreon, link down in the description, so you can help support us, you know, do these stress tests and for your entertainment purposes. Alright, we'll see you guys next time. Thank you and goodbye!

Tesla Full Self Driving Crash

Tesla has fired a former Autopilot employee named John Bernal after he shared candid video reviews on his YouTube channel, AI Addict, showing how the company’s Full Self Driving Beta system worked in different locations around Silicon Valley.

Following Bernal’s dismissal, Tesla also cut off his access to the FSD Beta system in the vehicle he personally owns, a 2021 Tesla Model 3, despite having no safety “strikes” in the software. He still has FSD, Tesla’s premium driver assistance software. Tesla’s technology does not make its cars autonomous today.

The FSD Beta option can best be summarized as a set of new driver assistance features that are not finished or fully debugged. Chief among them is “autosteer on city streets,” which lets the car navigate around complex urban environments without the driver needing to move the steering wheel. Customers must first have FSD, which costs $12,000 up front or $199 per month in the U.S., and then obtain and maintain a high driver-safety score, as determined by Tesla software that monitors their driving habits.

Although Tesla did not put details into writing saying why he was fired, Tesla and other Silicon Valley companies often foster a culture of loyalty. Internal criticisms may be tolerated, but criticism in public is viewed as disloyal.

Tesla did not immediately respond to a request for comment about Bernal’s situation.

The origins of his channel

Bernal started working for Elon Musk’s electric vehicle maker as a data annotation specialist in August 2020 in an office in San Mateo, California. He was dismissed in the second week of February this year, after having moved into the role of advanced driver assistance systems test operator, according to records he shared with CNBC.

As a lifelong car enthusiast proud to be working at Tesla, Bernal put in an order to buy a 2021 Model 3 with a long-range battery a few months after he began working there. He took delivery of the car on December 26, 2020.

He says he bought the car in part because Tesla offered employees free access to FSD — then worth $8,000 — as a perk. Employees had to agree to give the company the right to collect internal and external vehicle data in exchange.

Amazed by what he saw as Tesla’s “potentially life-saving technology,” he started the AI Addict channel on YouTube in February 2021 to show what the public version of FSD Beta could do.

Most of the AI Addict videos show Bernal driving around Silicon Valley with a friend in his Tesla, using the newest released versions of the FSD Beta software.

Bernal was not alone in posting his experiences with Tesla’s experimental software. Tesla FSD Beta users like Dirty Tesla, Chuck Cook, Kim Paquette and many others rush to review each new release on their channels.

‘I still care about Tesla’

When the company fired Bernal last month, his written separation notice did not include the reason for his firing. It came after one of his videos depicted a drive in San Jose where his car knocked over bollards while FSD Beta was engaged.

Bernal says just before he was dismissed, managers verbally told him he “broke Tesla policy” and that his YouTube channel was a “conflict of interest.” They also admonished him for selling merchandise with the words “full self-driving (beta)” on it, and for using FSD Beta in an unsafe manner.

Bernal said he was always transparent about his YouTube channel, both with his managers at Tesla and with the public. His online resume on LinkedIn, for example, always listed his Tesla employment right alongside his YouTube channel name. Bernal said he had never seen a policy barring him from creating car tech reviews on his own time using his own property.

A copy of Tesla’s social media policy, provided by a current employee, makes no direct reference to criticizing the company’s products in public. The policy states, “Tesla relies on the common sense and good judgment of its employees to engage in responsible social media activity.” It lists social networks including Facebook, Twitter, Instagram, Reddit, Snapchat, LinkedIn, WeChat and personal blogs, but does not specifically mention YouTube.

Bernal said he never disclosed anything in his videos that Tesla had not released to the public. “The FSD Beta releases I was demonstrating were end-user consumer products,” he said.

But his videos did sometimes show problems with Tesla’s FSD Beta system.

In March 2021, for example, AI Addict posted a video entitled “FSD Beta 8.2 Oakland - Close Calls, Pedestrians, Bicycles!” that showed his car experiencing several “disengagements.” That’s where FSD Beta required Bernal to take over steering manually to avoid danger. At 11 minutes and 58 seconds into the video, the Tesla FSD Beta system begins to roll into an intersection just as a vehicle is crossing in front of Bernal’s Model 3. He narrowly avoided hitting the other car.

That video has since racked up around a quarter million views.

Tesla FSD Beta — an experiment on public roads

After it first ran, Bernal told CNBC, “A manager from my Autopilot team tried to dissuade me from posting any negative or critical content in the future that involved FSD Beta. They held a video conference with me but never put anything in writing.”

According to an analysis of his channel by CNBC, roughly ten of 60 videos he posted revealed flaws in FSD Beta. Three of the videos focused on other Tesla topics and didn’t discuss FSD Beta, while another three focused on other automakers’ electric vehicles and were not Tesla-related at all.

Bernal shared screenshots and photos that indicate his FSD Beta access was revoked by the company after he was terminated, even though he had not gotten any “strikes” for unsafe driving or improper use of the system. Generally, FSD Beta users are allowed several strikes before access is revoked.

Losing FSD Beta access in his own car has curtailed his ability to create reviews of the system. However, he has attained access to other vehicles with FSD Beta enabled, and plans to continue his independent research and reviews.

Bernal knew he might attract attention by posting honest FSD Beta reviews. But as long as he was truthful, he said, and given his generally favorable views of the technology, he thought Tesla would allow it or at least formally tell him if he needed to stop before it ever came to his losing his dream job.

He told CNBC, “I still care about Tesla, vehicle safety and finding and fixing bugs.”

Free speech with exceptions

Tesla CEO Elon Musk recently labeled himself a “free speech absolutist.” But his car company has a long history of asking customers and employees not to speak publicly about problems concerning their cars or the business.

For example, like many large companies, Tesla requires its employees to sign an arbitration agreement committing to resolve conflicts with the company without public lawsuits. Employees or temps can legally challenge and sometimes get released from the mandatory arbitration and go on to have their day in court, but those instances have been rare.

Tesla also used to require customers to sign non-disclosure agreements in exchange for service.

When FSD Beta first rolled out, as CNBC previously reported, the company asked drivers who enrolled in the early access program to be selective or refrain from posting to social media.

Federal vehicle safety regulators worried that the practice could have a chilling effect and hide critical safety complaints from the agency. They initiated a probe into the FSD Beta program as a result.

By September 2021, Musk said at a conference that the company shouldn’t have any restrictions like that at all. He said at the Code Conference during an interview with Kara Swisher that FSD Beta testers were “not really following it anyway.”

Clarification: After this story initially ran, Bernal said his managers at Tesla verbally confronted him about two other issues at the time they fired him, including selling merchandise and using FSD Beta in an improper manner on his YouTube channel.

Tesla fired an employee after he posted driverless tech reviews on YouTube

John Bernal shared clips of close calls and crashes on his channel, AI Addict

Tesla has a complicated relationship with customers who pay to test the beta version of its “Full Sell Driving” software. Often, these people are diehard fans, keen to promote the company’s prowess and show off its new driver assist features. Sometimes, though, they overshare — posting videos that show exactly when and how Tesla’s software gets it wrong.

When the person sharing these clips also works for Tesla, things are even more complicated. John Bernal, a former Tesla employee who reviewed the company’s FSD Beta software on his YouTube channel AI Addict says he was fired by the company last month after posting a video that showed his Tesla hitting a bollard.

As reported by CNBC, Bernal says that prior to his dismissal he was told verbally by his managers that he “broke Tesla policy” and that his YouTube channel was a “conflict of interest” (though his written separation notice did not specify a reason for his dismissal). Bernal said he’d previously been approached by managers after posting a video in March 2021 that showed a number of close calls with pedestrians and cyclists while using the FSD beta software. The video has more than 250,000 views and was shared widely on social networks like Twitter.

Bernal said that after posting the video, “A manager from my Autopilot team tried to dissuade me from posting any negative or critical content in the future that involved FSD Beta. They held a video conference with me but never put anything in writing.”

CNBC says Tesla’s social media policy for employees does not forbid criticism of the company’s products in public, but notes that the company “relies on the common sense and good judgment of its employees to engage in responsible social media activity.” Bernal says that after being fired, his access to the FSD Beta software was revoked.

The FSD Beta gives customers who pay for the company’s advanced driver assist software (branded by Tesla as “Full Self Driving,” to the annoyance of many industry groups) access to experimental updates. Tesla says some 60,000 customers are signed up to test FSD Beta on public roads, providing it with invaluable data to improve the software. The company says there have been zero accidents or injuries involving FSD Beta since its launch (though there have been a number of deaths where Tesla’s regular driver assist software was involved).

Tesla previously required customers who opted in to the FSD Beta to sign a non-disclosure agreement, which told signatories: “there are a lot of people that want Tesla to fail; Don’t let them mischaracterize your feedback and media posts.” Tesla CEO Elon Musk later said the company “probably” didn’t need the NDAs as people tended to ignore them anyway.

A number of FSD Beta testers post reviews of the company’s software on YouTube, and Bernal says he intends to maintain his channel (though he now needs to borrow friends’ cars to test FSD Beta). As Bernal told CNBC: “I still care about Tesla, vehicle safety and finding and fixing bugs.”

Tesla fired employee who reviewed its driver assist features on YouTube