Incident 77: Knightscope's Park Patrol Robot Ignored Bystander Pressing Emergency Button to Alert Police about Fight

Description: A Knightscope K5 autonomous "police" robot patrolling Huntington Park, California failed to respond to an onlooker who attempted to activate its emergency alert button when a nearby fight broke out.
Alleged: Knightscope developed and deployed an AI system, which harmed Cogo Guebara and unnamed woman injured in the fight.

Suggested citation format

Zhu, Helen. (2019-10-04) Incident Number 77. in McGregor, S. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
77
Report Count
5
Incident Date
2019-10-04
Editors
Sean McGregor, Khoa Lam

Tools

New ReportNew ReportDiscoverDiscover

CSET Taxonomy Classifications

Taxonomy Details

Full Description

A Knightscope K5 autonomous security robot was patrolling Salt Lake Park in Huntington Park, CA in fall 2018 when a fight broke out nearby. An onlooker pressed the emergency alert button on the K5, named HP RoboCop by the town, but the robot did not respond and returned to its patrol of the park. Knightscope says that the emergency alert feature was still under development and therefore the signal was not sent to local police.

Short Description

In Fall 2018, a Knightscope K5 autonomous security robot took no action to stop a nearby fight, despite an onlooker attempting to activate its emergency alert feature.

Severity

Unclear/unknown

AI System Description

Knightscope K5 autonomous security robot uses several environmental sensors and voice commands to conduct security operations.

System Developer

Knightscope

Sector of Deployment

Administrative and support service activities

Relevant AI functions

Perception, Cognition, Action

AI Techniques

unknown

AI Applications

Image classification, image recognition, facial recognition, speech recognition, self-driving, environment sensing

Location

Huntington Park, CA

Named Entities

Knightscope, Knightscope K5, HP RoboCop, Huntington Park, Salt Lake Park, Huntington Park Police Department, Cosme Lozano

Technology Purveyor

Huntington Park Police Department, Knightscope

Beginning Date

2018-01-01

Ending Date

2018-01-01

Near Miss

Unclear/unknown

Intent

Accident

Lives Lost

No

Data Inputs

LIDAR, sonar, video camera, vibration detection, thermal anomaly detection, automatic signal detection, audio

Incidents Reports

HUNTINGTON PARK, Calif. — When a fight broke out recently in the parking lot of Salt Lake Park, a few miles south of downtown Los Angeles, Cogo Guebara did what seemed the most practical thing at the time: she ran over to the park’s police robot to push its emergency alert button.

“I was pushing the button but it said, ‘step out of the way,’” Guebara said. “It just kept ringing and ringing, and I kept pushing and pushing.”

She thought maybe the robot, which stands about 5 feet tall and has “POLICE” emblazoned on its egg-shaped body, wanted a visual of her face, so she crouched down for the camera. It still didn’t work.

Without a response, Rudy Espericuta, who was with Guebara and her children at the time, dialed 911. About 15 minutes later, after the fight had ended, a woman was rolled out on a stretcher and into an ambulance, her head bleeding from a cut suffered during the altercation.

Amid the scene, the robot continued to glide along its pre-programmed route, humming an intergalactic tune that could have been ripped from any low-budget sci-fi film. The almost 400-pound robot followed the park’s winding concrete from the basketball courts to the children’s splash zone, pausing every so often to tell visitors to “please keep the park clean.”

The robot, officially named HP RoboCop, has been patrolling Salt Lake Park for the Huntington Park Police Department since June. NBC News visited the park numerous times to observe its operation and its interactions with people.

While people are beginning to more commonly encounter robots in everyday life, they can often fall short of expectations — much as HP RoboCop did. That gap can be exploited as a way to make a robot more effective than it actually is, but also runs the risk of creating situations in which people rely on robots in ways they’re unprepared for, as was the case for Guebara.

The robot’s alert button is not yet connected to the police department, said Cosme Lozano, chief of police of Huntington Park, a city just southeast of downtown Los Angeles. The calls are instead directed to Knightscope, the company that creates and leases the robots.

“That’s why we’re not advertising those features,” he said. “It’s a new program for us and were still developing some protocols… to be able to fully adopt the program.”

On patrol

HP RoboCop is one of more than 70 autonomous security robots developed by Knightscope. The Silicon Valley company says it has combined self-driving technology, robotics and artificial intelligence to create what it calls “crime-fighting autonomous data machines.” The robots are deployed across the U.S., serving everywhere from airports to gas stations.

Lozano said the department is facing some technical challenges incorporating the robot into the force, adding that it’s on “a trial basis for the city.” Once fully connected, the calls will go directly to the department’s dispatch center, he said.

HP RoboCop is a K5 model, specialized for outdoor use, and is one of the company’s first to wear a police moniker. Knightscope’s website promotes some of the K5’s abilities as including a 360-degree high-definition live video stream, a license plate reader that can scan 1,200 plates a minute, a two-way intercom and the ability to track cell phone use in the vicinity.

But as it is currently used, HP RoboCop is little more than a glorified security camera on wheels. The robot’s five cameras provide 24/7 live monitoring, with the ability to send footage directly to officers’ phones, but that’s currently only accessible to Knightscope. It’s another feature that the police are working on activating, Lozano said.

The robot is confined to the park’s cement path, which has been blocked by construction for a new aquatics center, curtailing its patrol of the north end of the park. Leasing the robot for a year costs the city between $60,000-$70,000, Lozano said. As of 2018, a Huntington Park police officer with a basic assignment makes an annual salary that falls in the same range.

Other Knightscope robots have not been without tribulations. One K5 robot patrolling around an office complex in Washington, D.C., ended up falling into a fountain. Another of the same model struck a toddler at a shopping mall in Silicon Valley.

Great expectations

The city’s official adopted name for the machine is HP RoboCop, but everyone in the park seems to have their own nicknames for it. “R2D2,” “Spy Machine,” and “Wall-E’’ are just some of the pet names that highlight the disconnect between people’s expectations of the robot and the reality of its capabilities.

Concession stand worker Orlando Enrique said he’s noticed a change in people’s behavior since the robot’s arrival. Just its presence seems to offer a sense of reassurance to many.

“A lot of people like it because they feel secure,” he said. “They can leave their kids.”

Enrique, 28, lives in Norwalk, California, and has worked at the snack bar for nine years. The store closes at 10:30 p.m. and now with the robot there, he feels more comfortable walking to his car after a shift.

People come from all over to see it, Enrique said, referring to the mini-tourist boom that HP RoboCop brought to the park. It is especially popular among kids.

On Labor Day, curious children trailed HP RoboCop as it patrolled. One little boy bumbled behind with a bag of Doritos in tow. He stared at the machine, doe-eyed, before lightly stroking its cylinder body ― until another boy asked, “Why are you petting it like a dog?”

This anthropomorphizing of the robot is totally normal, said Ross Knepper, an associate professor in the department of computer science at Cornell University.

There are a lot of unknowns when it comes to these machines, and humans just don’t have the right expectations to draw from, so they refer to “Hollywood magic,” he said. People recall what they’ve seen in movies and transfer those expectations onto the real thing.

“It’s something the brain automatically tries to do,” said Knepper, who is helping develop robots that can work with humans as peers rather than tools.

This can lead people to believe that a robot is much more capable and intelligent than it really is and can create a false sense of security, he said.

Violeta Alvaraz, who was visiting the park with her kids and mother, is from the neighboring township Bell, and learned about HP RoboCop through her son. His generation is more comfortable with that kind of technology, she said, commenting that to her it looked like a 1970s washing machine.

Though she said she appreciates the city’s investment in their safety, Alvaraz still has plenty of questions.

“Are we going to get in trouble if we touch it?” she asked. “Who’s guiding it? I don’t know how it works. Should I still call 911?”

‘Tipping’ point

Most people in the park who spoke with NBC News were fond of HP RoboCop, usually attaching a “he” pronoun when speaking about it, but all admitted they did not know its full functionality.

Despite HP RoboCop’s months-long tenure in the park, there is no signage describing what it does or why it is there. Lozano said that’s because the department does not want to falsely advertise the robot, but the information will be posted when the machine’s features are properly connected, he added.

Leveraging people’s uncertainty about the robot is core to its value as a security tool, Knepper said.

The fact that the public is unaware of all of the robot’s capabilities is integral to Knightscope robots’ mission to be a physical deterrent to crime, confirmed Stacey Stephens, the company’s executive vice president and chief client officer.

“They could have any kind of grand thought about what the robot might be able to do, which could lead them to say, ‘you know what, I’d rather not risk it. Let me go somewhere else,’” Stephens said.

Crime in the robot’s patrol zone has gone down, Lozano said. Though that’s hard to measure, the police have not had to contact Knightscope to review footage, except for the few times the robot itself was vandalized, he said. HP RoboCop has weathered a few minor personal assaults, including fielding a “wheelie” from a biker who purposefully rode into it. But the only significant damage was when the police first deployed the robot, Lozano said. Officers were able to track down the suspects of what he called the “robot tipping incident,” using HP RoboCop’s own footage.

Still, the police took the assault personally, said Lozano. The children in the park aren’t the only people who have developed something of an affinity for HP RoboCop. “How could you go mess with our robot?” Lozano said. “You would think the robot was murdered. We kind of jumped on it like we would have a significant crime.”

Talk the talk

Even if they are simple phrases, a robot’s ability to talk can change how people feel about it, Knepper said. As soon as it speaks, people’s expectations are dramatically altered.

“As researchers, we are very eager to employ all these new technologies, speech and acting in a socially competent way, but if we don’t have this other side of setting expectations of awareness of how people will interpret these behaviors, we run the risk of actually making an interaction with the robot worse and not better,” Knepper said.

Stephens prefers not to use the name HP RoboCop. He said it gives a false impression of the Knightscope’s mission, which is to gather intelligence for law enforcement.

“I think what people have in their head when they hear RoboCop is something that is going to shoot everybody up and intervene in a crime and that’s not what this is about,” he said.

Stephens was referencing the titular RoboCop from a series of movies that began in the late 1980s, offering a futuristic vision of a human-machine crime fighter and an explanation for why expectations about robotics in policing can quickly surpass reality. The character remains a relevant pop culture figure, with a new RoboCop movie released in 2014 and another currently in development.

But HP RoboCop seems to elicit the opposite reaction ― consistently followed by an entourage of kids high-fiving or holding its can-shaped body like a fifth-grade dance partner. Although a company can’t control how people project their imagination onto robots, said Knepper, it can encourage them to recall certain references through design.

Despite all of HP RoboCop’s capabilities, both real and perceived, it does not have the capacity to explain itself. The city, however, has an obligation to educate the public about the technology it’s using, Knepper said. This could also help the police achieve their goals of reducing crime.

“Once people really understand what’s at stake, I think they will modify their behavior in much more predictable ways,” he said.

A RoboCop, a park and a fight: How expectations about robots are clashing with reality

A police robot told a woman to go away after she tried to report a violent brawl breaking out nearby – then trundled off while singing a song. Cogo Guebara rushed over to the motorized police officer and pushed its emergency alert button on seeing the brawl break out in Salt Lake Park, Los Angeles, last month. But instead of offering assistance, the egg-shaped robot, whose official name is HP RoboCop, barked at Guebara to ‘Step out of the way’. To add insult to injury, the high-tech device then rolled away while humming an ‘intergalactic tune’, pausing periodically to say ‘Please keep the park clean.’ Guebara told NBC News: ‘I was pushing the button but it said, “Step out of the way.”

‘It just kept ringing and ringing, and I kept pushing and pushing.’ The concerned bystander thought the five-foot tall robot might have needed to see her face before it began to work, so she crouched down in front of its camera.

But the egg-shaped device still didn’t work, forcing Guebara to call 911 on her phone instead. Officers from the Huntington Park Police Department cops finally arrived 15 minutes later, after the row had ended. It left one woman with a bad head wound which saw her stretchered into an ambulance and taken to hospital for emergency treatment. Local Police Chief Cosme Lozano says the robots, which cost between $60,000 and $70,000 a year to lease, are still in a trial phase and that their alert buttons have not yet been activated. He said that law enforcement have not yet started advertising the robots crime-fighting activities. Any help requests are currently sent to a company called Knightscope, which creates and leases the robots. Lozano added that once the robot completes its trial, calls made using its alert button will be sent straight to dispatch. Other versions of the same model have previously hit the headlines after one fell into a fountain in Washington DC. And a third HP RoboCop struck a child while patrolling a mall in California’s Silicon Valley.

Police robot told woman to go away after she tried to report crime – then sang a song

As our world becomes more and more automated, with robotics taking over many human roles and offering assistance, we have to wonder just how useful some of these inventions truly are.

When you need the police, you still have to call 911

As the woman trying to help, Cogo Guebara, realized that pressing the emergency button on the RoboCop was not going to call for backup, she tried another method. 

Guebara then crouched down to the robot's camera lens height, thinking it needed a visual in order to assist her. 

That was not the case. The RoboCop kept spouting out 'get out of the way'. 

Given the police robot had the actual word 'Police' written in large letters on its front and back, it was quite a natural reaction to assume it may, in fact, offer help, just like the police would. 

However, these HP RoboCops, built and leased by Knightscope, are not yet linked up to police departments. Currently, their emergency button calls Knightscope, who then has to call the police. A bit of a long-ended mission when you're in need of responsive and immediate assistance. 

Cosme Lozano, chief of police at Hungtington Park said, "That’s why we’re not advertising those features. It’s a new program for us and we're still developing some protocols… to be able to fully adopt the program."

So, what is the purpose of these robots?

At the moment, Knightscope has developed and launched 70 of these RoboCops across the U.S. They are autonomous security robots that combine self-driving technology, robotics, and artificial intelligence. They're meant to be 'crime-fighting autonomous data machines'.

They've been deployed in a number of different U.S. locations, from gas stations to airports. That said, they're still on a trial basis according to Lozano, as they're not yet fully linked to the police force.

What can RoboCops do that is actually useful? They have a 360 degree high-definition live video stream, they can read license plates at a rate of 1,200 plates per minute, they have a two-way intercom (although we noticed that it didn't work very well during this recent park fiasco), as well as the ability to track mobile phone use in the vicinity. 

They're glorified 24/7 moving CCTV cameras. Which may be pretty useful when sending information directly to the police force. 

However, for the time being, a fair bit of work still needs to be done if the RoboCops are to become useful to people in need of police assistance. This was clearly demonstrated when Rudy Espericuta, with Guebara at the scene, had to dial 911.

Police Robot Ignores Woman's Call for Help in an American Park

HUNTINGTON PARK, Calif. – Police officers can’t be everywhere at once. Between budget cuts, staffing issues and an increase in violent crime across the country, cops are in huge demand.

So naturally we turn to technology to help solve the problem. 

But what do we do when that technology we depend on fails us miserably?

That’s the exact scenario police found themselves dealing with in California this week, after a woman attempting to get help from police was completely ignored by a security robot patrolling the area.

A report from NBC News said that Cogo Guebara was in Salt Lake Park recently when she noticed two people fighting in the parking lot. Thinking quickly, she ran over to a nearby roving LAPD police robot in an attempt to get police dispatched to the area.

But it didn’t exactly go as planned.

“I was pushing the button but it said, ‘step out of the way,’” Guebara said. “It just kept ringing and ringing, and I kept pushing and pushing.”

Guebara said she tried a number of things, even going so far as to crouch down in case the embedded camera wanted to get visual confirmation of her face.

But nothing worked.

Looks like this needs a little more work before going live ? https://t.co/kPeGWU7Y5h

— Greg Stevenson (@GT_Stevenson) October 10, 2019

A man nearby used his cellphone to call 911 and emergency crews got to the scene. By the time the fight was over, one woman had to be hospitalized for head trauma.

The frustrating experience has a number of people wondering what function the automated police bots serve if they can’t be used to get officers to the scene of an emergency. 

While the officers and EMS workers cleaned up the scene, the police bot reportedly continued to put through the park, stopping routinely to display messages about keeping the park clean.

Cosme Lozano, police chief of Huntington Park, said that the robot’s alert button was not yet hooked up to reach the department, despite it being on ‘patrol’ since June.

Lozano says they need more time to get the robots fully functioning with all features.

“That’s why we’re not advertising those features,” Lozano said. “It’s a new program for us and were still developing some protocols… to be able to fully adopt the program.”

The company who designed the bots is called Knightscope. They have over 70 autonomous security robots on the market. 

The K5 model, nicknamed HP RoboCop by the department, supposedly has abilities that include a 360-degree HD live video stream, a license plate reader that can scan 1,200 plates a minute, a two-way intercom and the ability to track cell phone use in the vicinity, according to Knightscope.

But right now, HP Robocop appears to be more of a glorified nanny cam on wheels… Essentially, the bot serves as a deterrent to criminals and allows parents to feel that their children are safe at play.

Knightscope’s patrol bots aren’t the only robotic technology being tested and implemented in the law enforcement field.

A robotics company known as SRI International has developed their own take on tech in policing, and if used, it could save lives.

SRI has developed a deployable robot that keeps officers in their cruisers during traffic stops. 

The new device was designed to keep members of law enforcement out of harm’s way. Keeping LEO’s in their cars protects against deadly motor vehicle accidents as well as suspects using guns or other weapons against the responding unit.

So far this year, 10 law enforcement officers have been struck and killed in the line of duty while working on the side of roadways. Another 15 were killed in vehicle crashes and 36 have been killed by gunfire. 

Might some of them still be with us if this technology was employed?

#MenloPark's SRI International presents the 'Robot for Making Traffic Stops'#FortressBayArea #SanMateoCounty #BoringDystopiahttps://t.co/SztCrEJT3B

— Fortress Bay Area (@FortressBayArea) May 2, 2019

While the idea seems pretty cool, it leaves a lot of questions unanswered. 

This assumes that everyone pulling over for a traffic stop is courteous and obeys commands given by the officer. What happens when the driver doesn’t comply?

Does a camera really allow for the same ability to observe the interior of the car to look for suspicious items like weapons or narcotics?

We know that minor infringements can escalate very quickly. What happens when something doesn’t go according to plan?

Since the ticket-printing robot extends out to the car on an arm, what happens when the suspect runs? Is the car still equipped to give chase?

Check out this video on how the new design would work.

Scott’s Law has been a trending topic this year, as well as the hashtag #MoveOver on social media, both issuing a call to remind drivers to give room to emergency vehicles that are stopped on the side of the road.

Could this initiative save lives?

Or does taking the human element out of the scenario change law enforcement all together?

Police robot completely ignores woman begging for help

Not everybody is a fan of law enforcement. Perhaps this is what inspired the Californian city of Huntington Park to take a more modern approach to police.

A mechanical enforcement officer, dubbed HP Robocop, has been patrolling Salt Lake Park, close to Los Angeles, for several months.

Let’s be clear, though – this particular RoboCop is nothing like Alex Murphy from the ultraviolent 80s action movie. In fact, he looks more like R2D2. You can watch HP Robocop at work in the video below.

HP Robocop intends to provide the local authorities with an extra pair of eyes and ears. In theory, he notifies the police force when something untoward is unfolding.

What’s more, members of the public who witness a crime can report it to HP Robocop and relax as justice is done. At least, again, that’s the theory.

There’s never a cop around when you need one

Cogo Guebara was minding her own business in Salt Lake Park when a fight broke out in the parking lot. Ms. Guebara did what anybody would do in such a situation. She looked around for the nearest police officer, so action could be taken before anybody was seriously hurt.

HP Robocop was making his rounds at this point, so Ms. Guebara raced over and attempted to notify him of the problem. A button on HP Robocop’s control panel supposedly connects people to the local police department. HP Robocop also sports a range of HD cameras, ensuring that he can record any crimes and present video footage as evidence in court if required.

Unfortunately, this particular cop appeared to be off-duty. When Ms. Guebara frantically attempted to report the incident, the mechanical peacekeeper told her to “step out of the way.” After a while, having continually refused to help, HP Robocop rolled on his merry way, humming what was described as “an intergalactic tune.” Way to protect and serve, buddy.

Ms. Guebara resorted to calling 911 on her cellphone. Fifteen minutes passed before officers arrived on the scene, by which time the fight was over, and one woman was left injured and needed hospital treatment. HP Robocop continued to make his rounds while this was taking place, occasionally stopping to remind local children to keep the park clean.

So was HP Robocop malfunctioning, or just unhelpful?

Neither is a good look for the Huntington Park PD, especially seeing as the robot costs in the region of $70,000 to lease. However, it was revealed that HP Robocop’s crime-reporting functionality is not yet active. Rather than the police department, the calls are directed to the company where the robots are designed.

Now, we’re not claiming anything untoward about the clearly intelligent men and women that created HP Robocop … but let’s be honest here. The only scientist that you want responding to a crime is one that gained superpowers from a lab accident.

This, it is claimed, is why HP Robocop is not yet being advertised as the crime-busting future of law enforcement. Perhaps it’s advisable to wait a while before doing so, too. This is not the first time that one of these droids has caused an issue.

Wait, there’s more than one Robocop out there?

Yep, there are actually about 70 – all of which are designed by the same company, Knightscope. Not all of these droids have been assigned to the police, though. Most act as security guards for gas stations, airports, and businesses.

Back in 2017, an earlier model had a mishap in an office complex in Washington, DC. The Knightscope K5 was making his rounds when he fell into a fountain. We’re not sure if he got back to work afterward, or this tragic accident ended his life just one week from retirement. We hope Mrs. K5 received a generous compensation payout if so.

A year earlier, a toddler was knocked over by a similar droid in Silicon Valley’s Stanford Shopping Center. It’s claimed that the machine collided with the child, then rolled away without a care in the world. It all sounds like the world’s slowest hit-and-run incident.

It seems that we’re still a way off robots policing the streets on a full-time basis, and we’ll need to place our faith in the boys and girls in blue for a while longer. In the meantime, be on your guard while in Salt Lake Park in Huntington Park. HP Robocop may not actually DO anything, but he’s always watching.

California’s First Robotic Police Officer Ignored a Woman Reporting a Crime, Then 'Sang her a Song'

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents