Incident 255: Gunshot Detection Audio Picked up by ShotSpotter’s Algorithms Previously Admitted as Evidence of Conviction in a Murder Case in Chicago Now Dismissed due to Reports of Unreliability

Description: ShotSpotter’s audios were dismissed by prosecutors as insufficient evidence due to reports showing unreliability and modifiability by employees and police, after previously being admitted to criminally convict an innocent grandfather in a murder case in Chicago, which resulted in his nearly-one-year-long arrest.
Alleged: ShotSpotter developed an AI system deployed by Chicago Police Department, which harmed Michael Williams.

Suggested citation format

Perkins, Kate. (2020-05-31) Incident Number 255. in Lam, K. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
255
Report Count
9
Incident Date
2020-05-31
Editors
Khoa Lam

Tools

New ReportNew ReportDiscoverDiscover

Incidents Reports

On May 31 last year, 25-year-old Safarain Herring was shot in the head and dropped off at St. Bernard Hospital in Chicago by a man named Michael Williams. He died two days later. 

Chicago police eventually arrested the 64-year-old Williams and charged him with murder (Williams maintains that Herring was hit in a drive-by shooting). A key piece of evidence in the case is video surveillance footage showing Williams’ car stopped on the 6300 block of South Stony Island Avenue at 11:46 p.m.—the time and location where police say they know Herring was shot.

How did they know that’s where the shooting happened? Police said ShotSpotter, a surveillance system that uses hidden microphone sensors to detect the sound and location of gunshots, generated an alert for that time and place.

Except that’s not entirely true, according to a motion filed by Williams’ public defender.. 

That night, 19 ShotSpotter sensors detected a percussive sound at 11:46 p.m. and determined the location to be 5700 South Lake Shore Drive—a mile away from the site where prosecutors say Williams committed the murder, according to a motion filed by Williams’ public defender. The company’s algorithms initially classified the sound as a firework. That weekend had seen widespread protests in Chicago in response to George Floyd’s murder, and some of those protesting lit fireworks. (While the address listed on the real-time alert was about a mile away from the where the shooting allegedly occurred, the alert also contained a map and GPS coordinates identifying the intersection where the shooting allegedly occurred.)

But after the 11:46 p.m. alert came in, a ShotSpotter analyst manually overrode the algorithms and “reclassified” the sound as a gunshot. Later, the company issued a forensic report with a map and GPS coordinates identifying the same intersection that ShotSpotter had initially identified in the real-time alert, but updated the address to be closer to the actual GPS coordinates it had initially identified.

A screenshot of the ShotSpotter alert from 11:46 PM, May 31, 2020 showing that the sound was manually reclassified from a firecracker to a gunshot.

“Through this human-involved method, the ShotSpotter output in this case was dramatically transformed from data that did not support criminal charges of any kind to data that now forms the centerpiece of the prosecution’s murder case against Mr. Williams,” the public defender wrote in the motion.

The document is what’s known as a Frye motion—a request for a judge to examine and rule on whether a particular forensic method is scientifically valid enough to be entered as evidence. Rather than defend ShotSpotter’s technology and its employees' actions in a Frye hearing, the prosecutors withdrew all ShotSpotter evidence against Williams. 

The case isn’t an anomaly, and the pattern it represents could have huge ramifications for ShotSpotter in Chicago, where the technology generates an average of 21,000 alerts each year. The technology is also currently in use in more than 100 cities.

Motherboard’s review of court documents from the Williams case and other trials in Chicago and New York State, including testimony from ShotSpotter’s favored expert witness, suggests that the company’s analysts frequently modify alerts at the request of police departments—some of which appear to be grasping for evidence that supports their narrative of events.

Untested evidence

Had the Cook County State’s Attorney’s office not withdrawn the evidence in the Williams case, it would likely have become the first time an Illinois court formally examined the science and source code behind ShotSpotter, Jonathan Manes, an attorney at the MacArthur Justice Center, told Motherboard.

“Rather than defend the evidence, [prosecutors] just ran away from it,” he said. “Right now, nobody outside of ShotSpotter has ever been able to look under the hood and audit this technology. We wouldn’t let forensic crime labs use a DNA test that hadn’t been vetted and audited.”

Sam Klepper, senior vice president for marketing and product strategy at ShotSpotter, told Motherboard in an email that the company has no reason to believe the prosecutor’s decision reflects a lack of faith in its technology.

ShotSpotter evidence and employee testimony has been admitted in 190 court cases, he wrote. “Whether ShotSpotter evidence is relevant to a case is a matter left to the discretion of a prosecutor and counsel for a defendant … ShotSpotter has no reason to believe that these decisions are based on a judgment about the ShotSpotter technology,” he said.

The Chicago Police Department, Cook County State’s Attorney’s Office, Mayor Lori Lightfoot’s office, and Alderman Chris Taliaferro, who chairs the city council’s public safety committee, did not respond to interview requests or questions.

A pattern of alterations

In 2016, Rochester, New York, police looking for a suspicious vehicle stopped the wrong car and shot the passenger, Silvon Simmons, in the back three times. They charged him with firing first at officers.

The only evidence against Simmons came from ShotSpotter. Initially, the company’s sensors didn’t detect any gunshots, and the algorithms ruled that the sounds came from helicopter rotors. After Rochester police contacted ShotSpotter, an analyst ruled that there had been four gunshots—the number of times police fired at Simmons, missing once.

Paul Greene, ShotSpotter’s expert witness and an employee of the company, testified at Simmons’ trial that “subsequently he was asked by the Rochester Police Department to essentially search and see if there were more shots fired than ShotSpotter picked up,” according to a civil lawsuit Simmons has filed against the city and the company. Greene found a fifth shot, despite there being no physical evidence at the scene that Simmons had fired. Rochester police had also refused his multiple requests for them to test his hands and clothing for gunshot residue.

The original ShotSpotter audio files of the fifth shot were deleted, leaving a “redact[ed]” copy of the recording with the five alleged gunshots.

Both the company and the Rochester Police Department “lost, deleted and/or destroyed the spool and/or other information containing sounds pertaining to the officer-involved shooting,” according to Simmons’ civil suit. “Greene acknowledged at plaintiff’s criminal trial that employees of ShotSpotter and law enforcement customers with an audio editor can alter any audio file that’s not been locked or encrypted.” 

A jury ultimately acquitted Simmons of attempted murder and a judge overturned his conviction for possession of a gun, citing ShotSpotter’s unreliability.

Excerpt from Silvon Simmons civil lawsuit against ShotSpotter and the Rochester Police Department.

Greene—who has testified as a government witness in dozens of criminal trials—was involved in another altered report in Chicago, in 2018, when Ernesto Godinez, then 27, was charged with shooting a federal agent in the city. 

The evidence against him included a report from ShotSpotter stating that seven shots had been fired at the scene, including five from the vicinity of a doorway where video surveillance showed Godinez to be standing and near where shell casings were later found. The video surveillance did not show any muzzle flashes from the doorway, and the shell casings could not be matched to the bullets that hit the agent, according to court records.

During the trial, Greene testified under cross-examination that the initial ShotSpotter alert only indicated two gunshots (those fired by an officer in response to the original shooting). But after Chicago police contacted ShotSpotter, Greene re-analyzed the audio files.

“An hour or so after the incident occurred, we were contacted by Chicago PD and asked to search for—essentially, search for additional audio clips. And this does happen on a semi-regular basis with all of our customers,” Greene told the court, according to a transcript of the trial. He later ruled that there were five additional gunshots that the company’s algorithms did not pick up.

Greene also acknowledged at trial that “we freely admit that anything and everything in the environment can affect location and detection accuracy.”

Excerpt from the transcript of Paul Greene's expert witness testimony during the trial of Ernesto Godinez.

ShotSpotter analysts “agree with the machine classification over 90% of the time,” Klepper, from ShotSpotter, wrote to Motherboard. “In a tiny number of cases, our customers request us to perform a location analysis to validate the accuracy of the location. If we find an error, we provide a more accurate location to the customer to assist the investigation.” 

Prior to the trial, the judge ruled that Godinez could not contest ShotSpotter’s accuracy or Greene’s qualifications as an expert witness. Godinez has appealed the conviction, in large part due to that ruling.

“The reliability of their technology has never been challenged in court and nobody is doing anything about it,” Gal Pissetzky, Godinez’s attorney, told Motherboard. “Chicago is paying millions of dollars for their technology and then, in a way, preventing anybody from challenging it.”

The evidence

At the core of the opposition to ShotSpotter is the lack of empirical evidence that it works—in terms of both its sensor accuracy and the system’s overall effect on gun crime. 

The company has not allowed any independent testing of its algorithms, and there’s evidence that the claims it makes in marketing materials about accuracy may not be entirely scientific.

Over the years, ShotSpotter’s claims about its accuracy have increased, from 80 percent accurate to 90 percent accurate to 97 percent accurate. According to Greene, those numbers aren’t actually calculated by engineers, though.

“Our guarantee was put together by our sales and marketing department, not our engineers,” Greene told a San Francisco court in 2017. “We need to give them [customers] a number … We have to tell them something. … It’s not perfect. The dot on the map is simply a starting point.”

In May, the MacArthur Justice Center analyzed ShotSpotter data and found that over a 21-month period 89 percent of the alerts the technology generated in Chicago led to no evidence of a gun crime and 86 percent of the alerts led to no evidence a crime had been committed at all.

Klepper disputed those findings to Motherboard, saying that “the data source used to draw their conclusions, on its own, results in an incomplete picture of an incident” because a gun may have been fired even if there is no documented police evidence that it was.

He also said that Greene’s testimony in the San Francisco trial “had nothing to do with the determination of our actual historical accuracy rate. While marketing and sales have appropriate input on our service level guarantees for our contracts, actual accuracy rates are based on detections that we record.”

Meanwhile, a growing body of research suggests that ShotSpotter has not led to any decrease in gun crime in cities where it’s deployed, and several customers have dropped the company, citing too many false alarms and the lack of return on investment.

One recent study of ShotSpotter in St. Louis found that ShotSpotter “has little deterrent impact on gun-related violent crime in St. Louis. [Automated gun detection systems] also do not provide consistent reductions in police response time, nor aid substantially in producing actionable results.”

Klepper contested those and other research findings, saying that “the studies’ conclusions do not reflect what we see.” 

He pointed to a 2021 study by New York University School of Law’s Policing Project that determined that assaults (which include some gun crime) decreased by 30 percent in some districts in St. Louis County after ShotSpotter was installed. The study authors disclosed that ShotSpotter has been providing the Policing Project unrestricted funding since 2018, that ShotSpotter’s CEO sits on the Policing Project’s advisory board, and that ShotSpotter has previously compensated Policing Project researchers.

Chicago pushes back

Chicago is one of the most important cities in ShotSpotter’s portfolio and is increasingly becoming a battleground over its use.

If a court ever agrees to examine the forensic viability of ShotSpotter, or if prosecutors continue to drop the evidence when challenged, it could have massive ramifications. From January 2017 through June 2021, ShotSpotter reported 94,313 gunfire incidents in the city, an average of 20,958 per year, according to data obtained by Motherboard through a public records request. 

Chicago is ShotSpotter’s second biggest client, after New York City, accounting for 13 percent of the company’s revenue during the first quarter of 2021. But Chicago’s $33 million contract with the company is coming to an end and city officials must decide this August whether or not to renew it.

Meanwhile, the city is grappling with new research, a rise in shootings, cases like the Williams and Godinez trials, and tragedies that have prompted renewed criticism of the technology. 

It was a ShotSpotter alert in the early-morning hours of March 29 that dispatched police to a street in Little Village where they eventually shot and killed 13-year-old Adam Toledo, who was unarmed at the time.

That and other recent events have sparked a new campaign by community and civil rights groups in Chicago calling on city officials to drop ShotSpotter.

“These tools are sending more police into Black and Latinx neighborhoods,” Alyx Goodwin, a Chicago organizer with the Action Center on Race and the Economy, one of the groups leading the campaign, told Motherboard. “Every ShotSpotter alert is putting Black and Latinx people at risk of interactions with police. That’s what happened to Adam Toledo.”’

Motherboard recently obtained data demonstrating the stark racial disparity in how Chicago has deployed ShotSpotter. The sensors have been placed almost exclusively in predominantly Black and brown communities, while the white enclaves in the north and northwest of the city have no sensors at all, despite Chicago police data that shows gun crime is spread throughout the city.

Community members say they’ve seen little benefit from the technology in the form of less gun violence—the number of shootings in 2021 is on pace to be the highest in four years—or better interactions with police officers.

“If you had relationships with any of the people on the block, you wouldn’t need the technology, ’cause we could tell you,” Asiaha Butler, president of the Resident Association of Greater Englewood, told Motherboard. Instead, the technology seems to have given police another excuse not to build relationships with residents. When shots ring out in the neighborhood, police may respond faster, but it’s an “over-militarized police presence. You see a lot of them. It’s not a friendly interaction,” she said.

Editor’s Note: Following the publication of this article, VICE received copies of court documents from the Michael Williams case, which show that ShotSpotter did not change the coordinates of the gunfire by a mile, but had identified the same intersection for the gunfire in both its initial real-time alert and in its later detailed forensic report. The article has also been updated to clarify that the original recording of the gunshots in the Silvon Simmons case were deleted, but that the jury heard a redacted copy of the recording with the five alleged gunshots.

Police Are Telling ShotSpotter to Alter Evidence From Gunshot-Detecting AI

CHICAGO (AP) — Michael Williams’ wife pleaded with him to remember their fishing trips with the grandchildren, how he used to braid her hair, anything to jar him back to his world outside the concrete walls of Cook County Jail.

His three daily calls to her had become a lifeline, but when they dwindled to two, then one, then only a few a week, the 65-year-old Williams felt he couldn’t go on. He made plans to take his life with a stash of pills he had stockpiled in his dormitory.

Williams was jailed last August, accused of killing a young man from the neighborhood who asked him for a ride during a night of unrest over police brutality in May. But the key evidence against Williams didn’t come from an eyewitness or an informant; it came from a clip of noiseless security video showing a car driving through an intersection, and a loud bang picked up by a network of surveillance microphones. Prosecutors said technology powered by a secret algorithm that analyzed noises detected by the sensors indicated Williams shot and killed the man.

“I kept trying to figure out, how can they get away with using the technology like that against me?” said Williams, speaking publicly for the first time about his ordeal. “That’s not fair.”

Williams sat behind bars for nearly a year before a judge dismissed the case against him last month at the request of prosecutors, who said they had insufficient evidence.

Williams’ experience highlights the real-world impacts of society’s growing reliance on algorithms to help make consequential decisions about many aspects of public life. Nowhere is this more apparent than in law enforcement, which has turned to technology companies like gunshot detection firm ShotSpotter to battle crime. ShotSpotter evidence has increasingly been admitted in court cases around the country, now totaling some 200. ShotSpotter’s website says it’s “a leader in precision policing technology solutions” that helps stop gun violence by using “sensors, algorithms and artificial intelligence” to classify 14 million sounds in its proprietary database as gunshots or something else.

But an Associated Press investigation, based on a review of thousands of internal documents, emails, presentations and confidential contracts, along with interviews with dozens of public defenders in communities where ShotSpotter has been deployed, has identified a number of serious flaws in using ShotSpotter as evidentiary support for prosecutors.

AP’s investigation found the system can miss live gunfire right under its microphones, or misclassify the sounds of fireworks or cars backfiring as gunshots. Forensic reports prepared by ShotSpotter’s employees have been used in court to improperly claim that a defendant shot at police, or provide questionable counts of the number of shots allegedly fired by defendants. Judges in a number of cases have thrown out the evidence.

ShotSpotter’s proprietary algorithms are the company’s primary selling point, and it frequently touts the technology in marketing materials as virtually foolproof. But the company guards how its closed system works as a trade secret, a black box largely inscrutable to the public, jurors and police oversight boards.

The company’s methods for identifying gunshots aren’t always guided solely by the technology. ShotSpotter employees can, and often do, change the source of sounds picked up by its sensors after listening to audio recordings, introducing the possibility of human bias into the gunshot detection algorithm. Employees can and do modify the location or number of shots fired at the request of police, according to court records. And in the past, city dispatchers or police themselves could also make some of these changes.

Amid a nationwide debate over racial bias in policing, privacy and civil rights advocates say ShotSpotter’s system and other algorithm-based technologies used to set everything from prison sentences to probation rules lack transparency and oversight and show why the criminal justice system shouldn’t outsource some of society’s weightiest decisions to computer code.

When pressed about potential errors from the company’s algorithm, ShotSpotter CEO Ralph Clark declined to discuss specifics about their use of artificial intelligence, saying it’s “not really relevant.”

“The point is anything that ultimately gets produced as a gunshot has to have eyes and ears on it,” said Clark in an interview. “Human eyes and ears, OK?”

This story, supported by the Pulitzer Center for Crisis Reporting, is part of an ongoing Associated Press series, “Tracked,” that investigates the power and consequences of decisions driven by algorithms on people’s everyday lives.

A GAME CHANGER

Police chiefs call ShotSpotter a game-changer. The technology, which has been installed in about 110 American cities, large and small, can cost up to $95,000 per square mile per year. The system is usually placed at the request of local officials in neighborhoods deemed to be the highest risk for gun violence, which are often disproportionately Black and Latino communities. Law enforcement officials say it helps get officers to crime scenes quicker and helps cash-strapped public safety agencies better deploy their resources.

“ShotSpotter has turned into one of the most important cogs in our wheel of addressing gun violence,” said Toledo, Ohio Police Chief George Kral during a 2019 International Association of Chiefs of Police conference in Chicago.

Researchers who took a look at ShotSpotter’s impacts in communities where it is used came to a different conclusion. One study published in April in the peer-reviewed Journal of Urban Health examined ShotSpotter in 68 large, metropolitan counties from 1999 to 2016, the largest review to date. It found that the technology didn’t reduce gun violence or increase community safety.

“The evidence that we’ve produced suggests that the technology does not reduce firearm violence in the long-term, and the implementation of the technology does not lead to increased murder or weapons related arrests,” said lead author Mitch Doucette.

ShotSpotter installs its acoustic sensors on buildings, telephone poles and street lights. Employees in a dark, restricted-access room study hundreds of thousands of gunfire alerts on multiple computer screens at the company’s headquarters about 35 miles (56 kilometers) south of San Francisco or a newer office in Washington.

Forensic tools such as DNA and ballistics evidence used by prosecutors have had their methodologies examined in painstaking detail for decades, but ShotSpotter claims its software is proprietary, and won’t release its algorithm. The company’s privacy policy says sensor locations aren’t divulged to police departments, although community members can see them on their street lamps. The company has shielded internal data and records revealing the system’s inner workings, leaving defense attorneys no way of interrogating the technology to understand the specifics of how it works.

“We have a constitutional right to confront all witnesses and evidence against us, but in this case the ShotSpotter system is the accuser, and there is no way to determine if it’s accurate, monitored, calibrated or if someone’s added something,” said Katie Higgins, a defense attorney who has successfully fought ShotSpotter evidence. “The most serious consequence is being convicted of a crime you didn’t commit using this as evidence.”

The Silicon Valley startup launched 25 years ago backed by venture capitalist Gary Lauder, heir to Estée Lauder’s makeup fortune. Today, the billionaire remains the company’s largest investor.

ShotSpotter’s profile has grown in recent years.

The U.S. government has spent more than $6.9 million on gunshot detection systems, including ShotSpotter, in discretionary grants and earmarked funds, the Justice Department said in response to questions from AP. States and local governments have spent millions more, from a separate pool of federal tax dollars, to purchase the system.

The company’s share price has more than doubled since it went public in 2017 and it posted revenue of nearly $30 million in the first half of 2021. It’s hardly ubiquitous, however. ShotSpotter’s website lists 119 communities in the U.S., the Caribbean and South Africa where it operates. The company says it has deployed 18,000 sensors covering 810 square miles (2,100 square kilometers).

In 2018, it acquired a predictive policing company called HunchLab, which integrates its AI models with ShotSpotter’s gunshot detection data to purportedly predict crime before it happens.

That system can “forecast when and where crimes are likely to emerge and recommends specific patrols and tactics that can deter these events,” according to the company’s 2020 annual report filed with the Securities and Exchange Commission. The company said it plans to expand in Latin America and other regions of the world. It recently appointed Roberta Jacobson, the former U.S. ambassador to Mexico, to its board.

Late last year, a Trump administration commission on law enforcement urged increased funding for systems like ShotSpotter to “combat firearm crime and violence.”

And amid rising homicides, this spring, the Biden administration nominated David Chipman, a former ShotSpotter executive, to head the Bureau of Alcohol, Tobacco, Firearms and Explosives.

In June, President Joe Biden encouraged mayors to use American Rescue Plan funds — aimed at speeding up the U.S. pandemic recovery — to buy gunshot detection systems, “to better see and stop gun violence in their communities.”

‘SOMETHING IN ME HAD JUST DIED’

On a balmy Sunday evening in May 2020, Williams and his wife Jacqueline Anderson settled in at their apartment building on Chicago’s South Side. They fed their Rottweiler Lily and German shepherd Shibey. Anderson fell asleep. Williams said he left the house to buy cigarettes at a gas station.

Looters had beaten him to it. Six days before in Minneapolis, George Floyd had been killed by police Officer Derek Chauvin. Four hundred miles away (640 kilometers), in Williams’ neighborhood, outrage boiled over. Shops were torn up, store windows broken, fires burned.

Williams found the gas station destroyed, so he said he made a U-turn to head home on South Stony Island Avenue. Before he reached East 63rd Street, Williams said Safarian Herring, a 25-year-old he said he had seen around the neighborhood, waved him down for a ride.

“I didn’t feel threatened or anything because I’ve seen him before, around. So, I said yes. And he got in the front seat, and we took off,” Williams said.

According to documents AP obtained through an open records request, Williams told police that as he approached an intersection another vehicle pulled up beside his car. A man in the front passenger seat fired a shot. The bullet missed Williams, but hit his passenger.

“It shocked me so badly, the only thing I can do was slump down in my car,” he said. As Herring bled all over the seat from wounds to the side of his head, Williams ran a red light to escape.

“I was hollering to my passenger ‘Are you ok?’” said Williams. “He didn’t respond.”

Williams drove his passenger to St. Bernard Hospital, where medical workers rushed Herring into the emergency room and doctors fought to save his life.

Two weeks before being picked up by Williams, Samona Nicholson, Herring’s mother, said the aspiring chef had survived a shooting at a bus stop. Nicholson, who called her son ‘Pook,’ arranged for him to stay with a relative where she thought he’d be safe.

Doctors pronounced Herring dead on June 2, 2020, at 2:53 p.m.

For days after the shooting Williams’ wife said he curled up on his bed, having flashbacks and praying for his passenger.

Three months after Herring’s death, the police showed up. Williams recalls officers told him they wanted to take him to the station to talk and assured him he did nothing wrong.

He had a criminal history and spent three different stints behind bars, for attempted murder, robbery and discharging a firearm, records show.

That was all when he was a younger man. Williams said he had moved on with life, avoiding legal trouble since his last release more than 15 years ago and working numerous jobs.

At the police station, detectives interrogated him about the night Herring was shot, then took him to a holding cell.

“They just said that they were charging me with first-degree murder,” Williams said. “When he told me that, it was just like something in me had just died.”

“IT’S NOT PERFECT”

On the night Williams stepped out for cigarettes, ShotSpotter sensors triangulated a loud noise the system initially assigned to 5700 S. Lake Shore Dr. near Chicago’s historic Museum of Science and Industry alongside Lake Michigan, according to an alert the company sent to police.

That material anchored the prosecutor’s theory that Williams shot Herring inside his car, even though the case supplementary report from police did not cite a motive, nor did it mention any eyewitnesses. There was no gun found at the scene of the crime.

Prosecutors also leaned on a surveillance video viewed by AP showing that Williams’ car ran a red light, as did another car that appeared to have its windows up. This detail ruled out the possibility that the shot came from the other car’s passenger window, they said.

Chicago police did not respond to AP’s request for comment. The Cook County State’s Attorney’s Office said in a statement that after careful review prosecutors “concluded that the totality of the evidence was insufficient to meet our burden of proof,” but did not answer specific questions about the case.

As ShotSpotter’s gunshot detection systems expand around the country, so has its use as forensic evidence in the courtroom — some 200 times in 20 states since 2010, with 91 of those cases in the past three years, the company said.

“Our data compiled with our expert analysis help prosecutors make convictions,” said a recent ShotSpotter press release. Even during the pandemic, ShotSpotter participated in 18 court cases, some over Zoom, according to a recent company presentation to investors.

But even as its use has expanded in court, ShotSpotter’s technology has drawn scrutiny.

For one, the algorithm that analyzes sounds to distinguish gunshots from other noises has never been peer reviewed by outside academics or experts.

“The concern about ShotSpotter being used as direct evidence is that there are simply no studies out there to establish the validity or the reliability of the technology. Nothing,” said Tania Brief, a staff attorney at The Innocence Project, a nonprofit that seeks to reverse wrongful convictions.

A 2011 study commissioned by the company found that dumpsters, trucks, motorcycles, helicopters, fireworks, construction, trash pickup and church bells have all triggered false positive alerts, mistaking these sounds for gunshots. Clark said the company is constantly improving its audio classifications, but the system still logs a small percentage of false positives.

In the past, these false alerts — and lack of alerts — have prompted cities from Charlotte, North Carolina, to San Antonio, Texas, to end their ShotSpotter contracts, the AP found.

In Fall River, Massachusetts, police said ShotSpotter worked less than 50% of the time and missed all seven shots in a downtown murder in 2018. The results didn’t improve over time, and later that year ShotSpotter turned off its system.

The public school district in Fresno, California, ended its ShotSpotter contract last year, after paying $1.25 million over four years and finding it too costly. Also, parents and board members were concerned that district funds meant to help high-needs students were used to pay for ShotSpotter, said school board trustee Genoveva Islas.

“We were at the point where George Floyd had been murdered and there was a lot of push around racism and discrimination in the district. There was this mounting questioning about that investment in particular,” Islas said.

Some courts, too, have been less than impressed with the ShotSpotter system. In 2014, a judge in Richmond, California, didn’t allow ShotSpotter evidence to be used during a gang murder conspiracy case, although the accused man, Todd Gillard, was still convicted of being involved in a drive-by shooting.

“The expert testimony that a gun was fired at a particular location at a given time, based on the ShotSpotter technology, is not presently admissible in court, because it has not, at this point, reached general acceptance in the relevant scientific community,” ruled Contra Costa Superior Court Judge John Kennedy. ShotSpotter later provided AP with additional court records showing that three years later, Kennedy reconsidered admissibility of the ShotSpotter evidence, and found that it could be admitted.

In a Chicago case, prosecutors had surveillance videos of gang member Ernesto Godinez in a neighborhood where an ATF agent was shot after dark — but none showing him actually shooting a gun. At a 2019 trial, they entered ShotSpotter data to show gunshots originated from the location where video evidence indicated Godinez was when shots rang out. This month, a federal appeals court ruled that a trial judge erred by not vetting the reliability of ShotSpotter data before letting jurors hear it. Nonetheless, the split three-judge panel concluded that other evidence prosecutors presented was enough to uphold Godinez’s conviction.

ShotSpotter says it’s constantly fine-tuning its machine learning model to recognize what is and isn’t a gunshot sound by getting detectives and investigators to add crime scene observations to its system. As a part of that process, which they call “ground truth,” ShotSpotter asks patrol officers to add and notate shell casings, bullet holes, gather witness testimony and other “evidence of gunfire” using its software.

“We have the opportunity to make the machine classification better and better and better because we get real-world feedback loops from humans,” Clark said.

Several experts warned that training an algorithm based on a set of observations submitted by police risks contaminating the model if harried officers — perhaps inadvertently — feed it incomplete or incorrect data.

“I’m kind of aghast,” said Clare Garvie, a senior associate with the Center on Privacy & Technology at Georgetown Law. “You are building an inherent uncertainty into that system, and you are telling that system it’s fine. You are contaminating the reliability of your system.”

ShotSpotter said the more data it receives from police, the more accurate its model becomes. The company says their system is accurate 97% of the time.

“In the small number of cases where ShotSpotter is incorrect, providing feedback to the algorithm can improve accuracy,” the company said.

Beyond the ShotSpotter algorithm, other questions have been raised about how the company operates.

Court records show that in some cases, employees have changed sounds detected by the system to say that they are gunshots.

During 2016 testimony in a Rochester, New York, officer-involved shooting trial, ShotSpotter’s engineer Paul Greene was pressed to explain why one of its employees reclassified sounds from a helicopter to a bullet. The reason? He said its customer, in this case the Rochester Police Department, told them to.

The prosecutor in that case replied: “Is that something that occurs in the regular course of business at ShotSpotter?”

“Yes, it is. It happens all the time,” said Greene. “Typically, you know, we trust our law enforcement customers to be really upfront and honest with us.”

Testifying in a 2017 San Francisco murder trial, Greene gave similar testimony that an analyst had moved the location of its initial alert a block away, suddenly matching the scene of the crime.

“It’s not perfect. The dot on the map is simply a starting point,” he said.

In the Williams case, evidence in pretrial hearings shows that ShotSpotter initially said the noise the sensor picked up was a firecracker, a classification the company’s algorithm made with 98% confidence. But a ShotSpotter employee relabeled the noise as a gunshot.

Later, ShotSpotter senior technical support engineer Walter Collier changed the reported Chicago address of the sound to the street where Williams was driving, about 1 mile (1.6 kilometers) away, according to court documents. ShotSpotter said Collier corrected the report to match the actual location that the sensors had identified. The company later provided AP with a copy of the full real-time alert, which contained a street address, location maps and latitude and longitude coordinates as did the second, detailed forensic analysis Collier prepared. The assigned street address changed from the first to the second report, but the location identified on the maps and GPS coordinates in both reports remained around the same intersection.

Collier worked for the Chicago Police Department for more than two decades before joining ShotSpotter, according to his LinkedIn profile. After Williams was sent to jail, his attorney requested more information about Collier’s training. The attorney, Brendan Max, said he was shocked by the company’s response.

In court filings, ShotSpotter acknowledged: “Our experts are trained using a variety of ‘on the job’ training sessions, and transfer of knowledge from our scientists and other experienced employees. As such no official or formal training materials exist for our forensic experts.”

Law enforcement officials in Chicago continue to stand by their use of ShotSpotter. Chicago’s three-year, $33 million contract, signed ​​in 2018, makes the city ShotSpotter’s largest customer. ShotSpotter has been at the heart of the police department’s “intelligence-action cycle” for predictive policing that uses gunshot alerts to “identify areas of risk,” according to a presentation obtained by AP.

Late last month, on July 22, Attorney General Merrick Garland flew to Chicago to announce a new initiative to combat gun violence and toured a police precinct, looking on as officials showed him how they use ShotSpotter.

INSUFFICIENT EVIDENCE

The next day, Williams hobbled into Courtroom 500 leaning on his wooden cane, dressed in tan jail garb and sandals, as a sheriff’s deputy towered over him. He had been locked up for 11 months.

Williams lifted his head to the famously irascible Judge Vincent Gaughan. The 79-year-old Vietnam veteran looked back from high on his bench and told Williams his case was dismissed. The reason: insufficient evidence.

ShotSpotter maintains it had warned prosecutors not to rely on its technology to detect gunshots fired inside vehicles or buildings. The company said the disclaimer can be found in the small print embedded in its contract with Chicago police.

But the company declined to say at what point during Williams’ nearly yearlong incarceration it got in touch with prosecutors, or why it prepared a forensic report for a gunshot that allegedly was fired in Williams’ vehicle, given the fact that the system had trouble identifying gunshots in enclosed spaces. The report itself contained contradictory information suggesting the technology did, in fact, work inside cars. Clark, the company’s CEO, declined to comment on the case, but in a follow-up statement, the company equivocated, telling AP that under “certain conditions,” the system can actually pick up gunshots inside vehicles.

Max, Williams’ attorney, said prosecutors never disclosed any of this information to him, and instead dropped charges two months after he subpoenaed ShotSpotter for the company’s correspondence with state’s attorneys.

The judge agreed to schedule a hearing in the coming weeks about whether to release ShotSpotter’s operating protocol and other documents the company wants to keep secret. Max, who requested it, said such material could be used to cast doubt on the validity and reliability of ShotSpotter evidence in cases nationwide.

“My client did not deserve to have his liberty taken away based on unscientific, unproven evidence,” Max said. “Given the history of flawed forensic evidence in our courts, we can’t let ShotSpotter be the next thing that racks up wrongful convictions.”

On the evening of July 23, Williams walked out of Cook County Jail into the hot Chicago night.

He was picked up by his attorney, and Anderson, his wife of 20 years, was waiting at home. When her husband stepped out of his attorney’s car, she took him in her arms and cried.

That first night at home, Anderson made ribs and chicken, cornbread and macaroni and cheese.

But Williams couldn’t eat on his own. He’d beat COVID-19 twice while in jail, but had developed an uncontrollable tremor in his hand that kept him from holding a spoon. So Anderson fed him. And as they sat together on the couch, she held onto his arm to try and stop the shaking.

For her part, Herring’s mother believes police had the right suspect in Williams. She blames ShotSpotter for botching the case by passing on, then withdrawing what she called flimsy data.

Williams remains shaken by his ordeal. He said he doesn’t feel safe in his hometown anymore. When he walks through the neighborhood he scans for the little microphones that almost sent him to prison for life.

“The only places these devices are installed are in poor Black communities, nowhere else,” he said. “How many of us will end up in this same situation?”

This story was originally published on August 19, 2021. The Associated Press reported that a ShotSpotter engineer changed the reported Chicago address of a sound the company labeled a gunshot to the street where Michael Williams was driving. The story included ShotSpotter’s explanation that the engineer had corrected the street address that was generated in its initial real-time alert to match the actual street address that the company’s sensors had identified. The company has now provided the AP with a copy of the full real-time alert. The two reports the company issued – the initial real-time alert and the detailed forensic analysis later filed in court – contained a street address, location maps and latitude and longitude coordinates. The assigned street address changed from the first to the second report, but the location identified on the maps and GPS coordinates in both reports remained around the same intersection. ShotSpotter says the street address in the initial real-time alert sent to police was wrong because the GPS coordinates fell within a large park for which the officially designated address was about a mile away from the actual location identified by the sensors. In addition, the AP story misstated the status of an attorney who pressed a ShotSpotter engineer testifying in a trial to explain why one of its employees reclassified sounds from a helicopter to a bullet. The article said the attorney was a defense attorney but he was actually a prosecutor. The story also reported that in 2014, a judge in Richmond, California, didn’t allow ShotSpotter evidence to be used during a gang murder conspiracy case. ShotSpotter has now provided AP with additional court records showing that, three years later, the judge reconsidered admissibility of the ShotSpotter evidence and found, based upon the new evidence, that it could be admitted.

How AI-powered tech landed man in jail with scant evidence

If you like your dystopia, you can keep your dystopia.

That’s where we are right now: dealing with a gunshot AI company that felt compelled to sue journalists for (accurately) reporting on things the company has done as well as offering their opinions on the company’s actions.

The company is ShotSpotter. Utilizing microphones and AI no defendant has been able to examine in court, ShotSpotter tells cops there have been shots in the (recorded) area and law enforcement stuff flows from there.

The problem with ShotSpotter is that it’s often inaccurate. And, according to some law enforcement agencies, it’s completely useless. On top of that, it has been caught altering gunshot records at the request of law enforcement investigators who perhaps find the original, unaltered reports aren’t helping them close cases or make arrests.

This seemingly accurate reporting has made ShotSpotter very angry. It has issued statements vehemently denying what’s been uncovered by public records requests and defense lawyers. But it did even more to Vice Media, home of tech reporting wing, Motherboard. It sued the website for defamation, claiming Vice’s reporting was full of lies, some of them actually and legally malicious-,Actual%20malice%20is%20the%20legal%20standard%20established%20by%20the%20Supreme,lawsuits%20against%20the%20news%20media.).

ShotSpotter’s presumably high-powered lawyers dumped a 413-page complaint [PDF] into a Delaware court, because that’s where you file lawsuits when you’re a corporation seeking to sue another corporation and don’t want federal precedent on defamation lawsuits to get your case dismissed post haste.

Four hundred and thirteen pages. I hope ShotSpotter wasn’t paying by the word. Taxpayers may be obligated to front money for local court action, but they’re going to be out much less than ShotSpotter. The loss [PDF], handed to ShotSpotter by the Delaware Superior Court, runs only 28 pages. (h/t Justia, an invaluable source of legal documents and the only site that actually included the ruling, despite several others reporting on it.)

It seems ShotSpotter and its lawyers hoped to intimidate the court into a win with a massive wall of text. For all the redundant and pointless agitating, ShotSpotter’s anger is limited to 15 separate statements/allegations made by Vice and its reporters.

ShotSpotter seems stung the most by Vice/Motherboard’s allegation that it will alter reports at the request of law enforcement customers — something that converts evidence into something far more questionable: convenient contributions to the official narrative.

ShotSpotter alleged all of this was defamatory. First, the title of the Motherboard article:

“Police Are Telling ShotSpotter to Alter Evidence from Gunshot-Detecting AI”

This section heading:

“A pattern of alterations”

And this section of the article:

“Motherboard’s review of court documents from the Williams case and other trials in Chicago and New York State, including testimony from ShotSpotter’s favored expert witness, suggests that the company’s analysts frequently modify alerts at the request of police departments – some of which appear to be grasping for evidence that supports their narrative of events.”

To which the court responds, “Where’s the lie?” It starts by quoting expert testimony offered in a criminal case by ShotSpotter’s Senior Forensic Engineer, Paul Green.

Forensic examination of an incident is always done at a customer’s request, only at a customer’s request. It’s not something we do on a regular basis. In this case, ShotSpotter only detected the final two shots that you heard in the audio clip. An hour or so after the incident occurred, we were contacted by Chicago PD and asked to search for — essentially, search for additional audio clips. And this does happen on a semi-regular basis with all of our customers

Here’s Greene again, being cross-examined in another criminal case:

Q. Mr. Greene, I want to stop you right there. This note here denotes some employee at [ShotSpotter] changed the classification per the instruction of the customer?A. Per the customer’s instruction, yes.Q. Is that something that occurs in the regular course of business at [ShotSpotter]?

A. Yes, it is. It happens all the time.

_Q. What happens if a customer calls and asks you to change a classification that has no link to the audio that you’re listening to?

A. We have refused customers [sic] to change classifications on incidents in the past. Typically, you know, we trust our law enforcement customers to be really upfront and honest with us . . .

_

ShotSpotter claimed it was defamatory to suggest the company “modified alerts at the request of the company.” The court says this obviously happens all the time, according to ShotSpotter’s own expert witnesses.

It is apparent, from Greene’s testimony, that there is a pattern of alterations, and that these alterations sometimes come by request of police departments.

That’s the biggest, most obvious problem with ShotSpotter’s lawsuit. Adding to its problems is the fact that neither ShotSpotter or its presumably expensive legal representation can discern the difference between actionable assertions of fact and the hyperbole often present in social media posts.

It is clear to the Court that certain words used by Mr. Koebler are opinion. In Statement 12, Koebler tweets, “This is horrifying and nuts.” In Statement 14, Koebler tweets, “Blatant corruption.” These words are not actionable. They are no worse than a plaintiff being accused of being “shockingly racist” or accused of “blackmail.” “[A] published statement that is ‘pointed, exaggerated, and heavily laden with emotional rhetoric and moral outrage’ is not defamatory.”

And, if you believe (as ShotSpotter appears to) that being called “shockingly racist” or involved in “blackmail” is actionable, footnotes appended to both of those phrases cite precedent (from Delaware’s top court as well as the US Supreme Court) ruling otherwise.

As for any deliberate misleading by Vice reporter Jason Koebler, the court has this to say:

There is no misplacement or mistake of hyperlink. There is no chance that a trier of fact could understand a link to apply to one Statement and not another. Koebler links to the entire Article in the first tweet, and links to specific screenshots of the Article and relevant testimony in the next two. It would be clear to a reader that these three tweets should be read in conjunction with the Article, the Greene testimony, and the excerpt about the Simmons case.

As for the rest of the allegedly defamatory statements buried in ShotSpotter’s 413-page complaint, the court says there’s nothing actionable about those either.

The Court finds that the remainder of the proffered Statements are not defamatory.

More specifically: they’re substantially true, even if somewhat carelessly deployed:

There is substantial truth in the Williams and Simmons Statements. As demonstrated in the Complaint, the prosecutors’ case and ensuing ShotSpotter evidence was withdrawn in Williams. While the Statement oversimplifies the sequence of events, it is admitted in the Complaint that prosecutors learned of the limitations of ShotSpotter technology, then “dropped the case.” Also, the location was in fact changed for the gunshots. The Complaint acknowledges that the location change was due to ShotSpotter providing police with the geolocation of the park entrance, rather than the specific gunshot location. Further, these Statements are supported by and derived from a motion filed by William’s public defender. The Article specifically states: “That night, 19 ShotSpotter sensors detected a percussive sound at 11:46 p.m. and determined the location to be 5700 South Lake Shore Drive—a mile away from the site where prosecutors say Williams committed the murder, according to a motion filed by Williams’ public defender.”

In Simmons, the Article states that a fifth shot disappeared. It bases this Statement on a New York court decision which overturned the defendant’s conviction; the judge called it “troubling” that ShotSpotter evidence had disappeared. The full context provides that this happened after the evidence was already heard by a jury, then was later deleted per company protocol. While these Statements may lack the sufficient journalistic context, they are substantially true in their conveyance.

Vice wins. Unfortunately, it will be out its own costs for defending itself from this bogus lawsuit. Delaware’s anti-SLAPP law is extremely limited and doesn’t cover actions arising from one private entity’s complaints about another private entity’s statements. Once again, for the people in the back of a whole bunch of states: FEDERAL. ANTI. SLAPP. NOW.

The good news is ShotSpotter wasn’t able to sue Vice into silence. Reporting on the company’s questionable tactics will continue. And until the company is actually willing to force its law enforcement customers to accept what’s been detected by ShotSpotter devices (rather than run more searches and/or alter data), it’s just going to keep suffering similar losses in the court of public opinion, not to mention the literal courts where it presents its dubious evidence.

Judge Tosses Defamation Suit Brought By ShotSpotter Against Vice Media For Reporting On Its Shady Tactics

CHICAGO (AP) — A federal lawsuit filed Thursday alleges Chicago police misused “unreliable” gunshot detection technology and failed to pursue other leads in investigating a grandfather from the city’s South Side who was charged with killing a neighbor.

Chicago prosecutors used audio picked up by a network of sensors installed by the gunshot detection company ShotSpotter as critical evidence in charging Michael Williams with murder in 2020 for allegedly shooting the man inside his car. Williams spent nearly a year in jail, and The Associated Press reported last year that a judge dismissed his case at the request of prosecutors, who said they had insufficient evidence.

The lawsuit filed by the MacArthur Justice Center at Northwestern University’s law school seeks damages from the city for mental anguish, loss of income and legal bills for the 65-year-old Williams, who said he still suffers from a tremor in his hand that developed while he was locked up. It also details the case of a second plaintiff Daniel Ortiz, a 36-year-old father who the lawsuit alleges was arbitrarily arrested and jailed by police who were responding to a ShotSpotter alert.

The suit seeks class-action status for any Chicago resident who was stopped on the basis of the alerts. It also seeks a court order barring the technology’s use in the nation’s third-largest city.

“Even though now I’m so-called free, I don’t think I will ever be free of the thought of what they have done and the impact that has on me now, like the shaking with my hand,” Williams said. “I constantly go back to the thought of being in that place. ... I just can’t get my mind to settle down.”

ShotSpotter isn’t named as a defendant in the 103-page filing though the lawsuit claims the company’s algorithm-powered technology is flawed. The suit also alleges the city’s decision to place most of its gunshot-detection sensors in predominantly Black and Latino neighborhoods is racially discriminatory.

Asked for a comment, the city’s law department, which represents police in such cases, said later Thursday that it had not yet been served with the complaint.

Chicago police have previously praised the ShotSpotter system, saying it puts officers on the scene of shootings far faster than if they wait for someone to call 911. Police have also said crime rates — not residents’ race — determine where the technology is deployed.

ShotSpotter said in a statement that the evidence it collects and its expert witnesses have been admitted in 200 court cases in 20 states, and have survived dozens of evidentiary challenges.

It described its system as “highly accurate with a 97% aggregate accuracy rate for real-time detections across all customers” and that an analytics company had verified the effectiveness of the technology.

ShotSpotter’s website says the company is “a leader in precision policing technology solutions” that help stop gun violence by using sensors, algorithms and artificial intelligence to classify 14 million sounds in its proprietary database as gunshots or something else.

But the AP investigation identified a number of flaws in using ShotSpotter as evidentiary support for prosecutors, and found the system can miss live gunfire right under its microphones, or misclassify the sounds of fireworks or cars backfiring as gunshots. Last year, Chicago’s nonpartisan watchdog agency concluded that actual evidence of a gun-related crime was found in about 9% of ShotSpotter alerts that were confirmed as probable gunshots.

The lawsuit says the investigating officers “put blind faith in ShotSpotter evidence they knew or should have known was unreliable” in order to charge Williams with killing 25-year-old Safarian Herring. The lawsuit alleges investigators used ShotSpotter material in a way that went beyond its intended use, quoting a disclaimer in one document related to Williams’ case that says the investigative lead summary “should only be used for initial investigative purposes.”

The suit also accuses investigators of not pursuing other leads that could have produced credible suspects, including reports that someone previously shot at Herring at a bus stop.

Police and prosecutors never established a motive for Williams to have shot Herring, never found witnesses to the shooting, and never recovered a weapon or physical evidence tying Williams to the killing, the suit alleges.

“The Defendant Officers engaged in tunnel vision to target Mr. Williams, arresting him for First-Degree murder, without probable cause,” the lawsuit said.

Lawsuit: Chicago police misused ShotSpotter in murder case

A new lawsuit aims to end the use of ShotSpotter technology by the city of Chicago, arguing the gunfire alert system is unreliable and leads to unconstitutional policing.

But the company itself claims a 97% accuracy rate, and it has challenged the idea that ShotSpotter alerts result in “hyped up” police arriving at a scene.

The Chicago Police Department’s use of ShotSpotter came under increased scrutiny after the March 2021 police-shooting death of 13-year-old Adam Toledo. In August 2021, the city’s Office of the Inspector General found it rarely leads to investigatory stops or evidence of gun crimes but can change the way officers interact with areas they patrol.

Now, Lucy Parsons Labs and others have filed a federal lawsuit alleging CPD has misused ShotSpotter to make “scores of illegal stops and arrests.” Lucy Parsons Labs, a nonprofit that investigates police surveillance, says on its website it has “worked tirelessly within the Coalition to Stop ShotSpotter to prevent the city of Chicago from furthering scientific racism and deploying harmful algorithms.”

It predicted it would “successfully cancel Chicago’s [contract] with ShotSpotter in 2022.”

The new lawsuit seeks class-action status and an order stopping the city from using or relying on ShotSpotter. It points to the experiences of two additional plaintiffs, Michael Williams and Daniel Ortiz, whose rights were allegedly violated through the use of ShotSpotter technology.

It also alleges CPD has covered “nearly the entire South and West sides” with ShotSpotter sensors, leading to discrimination “against Black and Brown Chicagoans.”

Named as defendants are the city, CPD Supt. David Brown and several other police officers. ShotSpotter is not named as a defendant. Still, the company defended itself in a statement Thursday, pointing to an independently verified 97% aggregate accuracy rate and survival in “dozens” of court challenges.

“ShotSpotter’s coverage areas are determined by police using objective historical data on shootings and homicides to identify areas most impacted by gun violence to provide residents who live in communities experiencing persistent gunfire a rapid police response,” the company said.

Williams, 65, spent 11 months in the Cook County Jail after Chicago police wrongly accused him of murder based on ShotSpotter data, according to the lawsuit. It said a young man flagged down Williams for a ride home on May 31, 2020, and Williams agreed to help him. Then, it said, that young man was fatally shot while in the car.

Williams took the young man to the hospital and gave his own name and information to hospital security, the lawsuit says. However, Williams was eventually arrested based upon a ShotSpotter report that led officers to believe the fatal shot was fired from inside the car, the lawsuit alleges.

It says officers “ignored the fact that ShotSpotter itself warns that its alerts cannot provide the exact location of a shot,” as well as the fact that ShotSpotter had given a “wildly inaccurate” address for the shooting. It also said investigators failed to look into a dark sedan in the next lane that crept slowly alongside Williams’ car the night of the shooting before peeling away through a red light.

Later, the lawsuit says, prosecutors “balked” at vouching for ShotSpotter, abandoned the evidence against Williams and dismissed the case.

The city of Chicago responded to a request for comment Thursday by stating it had not yet been served with the lawsuit. The Cook County State’s Attorney’s Office declined to comment.

As for Ortiz, the lawsuit says police arrested him April 19, 2021, on “false drug charges” after responding to a ShotSpotter alert in the Schorsch Village neighborhood. It said he was outside a laundromat that afternoon when officers arrived, “moved aggressively” toward him and immediately detained him for questioning about what turned out to be a false gunfire alert.

From there, the lawsuit says, police illegally searched Ortiz’s car, found a legal amount of marijuana and bottle of prescription drugs and arrested him. Ortiz spent a night in jail and the charges were dismissed the next day after he prevailed in a court hearing, according to the lawsuit.

New lawsuit aims to halt Chicago’s use of ShotSpotter

A 65-year-old man named Michael Williams spent almost a year in jail over the shooting of a man inside his car before prosecutors asked a judge to dismiss his case due to insufficient evidence. Now, the MacArthur Justice Center has sued the city of Chicago for using ShotSpotter, which it calls an "unreliable" gunshot detection technology, as critical evidence in charging him with first-degree murder. The human rights advocate group out of Northwestern University accuses the city's cops of relying on the technology and failing to pursue other leads in the investigation.

Williams was arrested in 2021 over the death of Safarian Herring, a young man from the neighborhood, who asked him for a ride in the middle of unrest over police brutality in May that year. According to an AP report from March, the key piece of evidence used for his arrest was a clip of noiseless security video showing a car driving through an intersection. That's coupled with a loud bang picked up by ShotSpotter's network of surveillance microphones. ShotSpotter uses a large network of audio sensors distributed through a specific area to pick up the sound of gunfire. The sensors work with each other to triangulate the shot's location, so perpetrators can't hide behind walls or other structures to mask their crime.

However, a study conducted by the MacArthur Justice Center in 2021 found that 89 percent of the alerts the system sends law enforcement turn up no evidence of any gun-related crime. "In less than two years, there were more than 40,000 dead-end ShotSpotter deployments," the report said. The group also pointed out that ShotSpotter alerts "should only be used for initial investigative purposes." San Francisco's surveillance technology policy (PDF), for instance, states that its police department must only use ShotSpotter information to find shell casing evidence on the scene and to further analyze the incident.

The lawsuit accuses Chicago's police department of failing to pursue other leads in investigating Williams, including reports that the victim was shot earlier at a bus stop. Authorities never established what's supposed to be Williams' motive, didn't find a firearm or any kind of physical evidence that proves that Williams shot Herring, the group said.

On its website, ShotSpotter posted a response to "false claims" about its technology, calling reports about its inaccuracy "absolutely false." The company claims its technology has a 97 percent accuracy rate, including a 0.5 percent false positive rate, and says those numbers were independently confirmed by Edgeworth Analytics, a data science firm in Washington, D.C. It also answers the part of the lawsuit that criticizes Chicago's decision to place most of it sensors in predominantly Black and Latino neighborhoods, which could lead to potentially dangerous clashes with the police. ShotSpotter said it's a false narrative that its coverage areas are biased and racially discriminatory and that it works with clients to determine coverage areas based on historical gunfire and homicide data .

As AP reports, the lawsuit is seeking class-action status for any Chicago resident who was stopped because of a ShotSpotter alert. The MacArthur Justice Center is also seeking damages from the city for the mental anguish and loss of income Williams had experienced throughout the whole ordeal, as well as for the legal fees he incurred. Further, the group is asking the court to ban the technology's use in the city altogether.

Lawsuit accuses Chicago authorities of misusing gunshot detection system in a murder case

CHICAGO (CBS) -- A Chicago man is taking issue with what he calls flaws with ShotSpotter – saying he was charged with murder and thrown in jail because of the gunshot detection technology.

"It does not have eyes. It has nothing," said Michael Williams. "It's just a device that picks up sound."

Williams said his is a cautionary tale.

"They want to detain you for as long as it takes," he said.

Williams said he gave a ride to a 25-year-old man named Safarian Herring back in May 2020.

"A car pulled up on the side of mine and fired a shot into my car," Williams said. "Next thing I knew, I'm charged with first-degree murder."

Prosecutors said an alert from a gunshot detection system known as ShotSpotter proved that it was Williams who shot Herring inside the car. 

Herring's father, Lajuane Herring, talked with Hickey back in November of last year. When Williams was charged, the Herring family thought the technology had delivered the justice they were praying for.

But after sitting in jail for nearly a year - and contracting COVID twice - Williams got some unexpected news.

"The State's Attorney, he told the judge, 'Your Honor, we're dropping all charges,'" Williams said.

The reliability of the gunshot detection report — coupled with data showing the number of times system gives off false alerts — left prosecutors without a leg to stand on. Williams was a free man.

Now, he and the victim's family question the technology's usefulness.

"I feel like ShotSpotters, they messed up really bad," Lajuane Herring said in November.

And what made the ordeal worse was that a few months after Williams was released, we discovered that the city quietly extended ShotSpotter's multimillion-dollar contract another two years without any public input.

"We as taxpayers - shouldn't we be allowed to have some say-so into whether or not there should have been an extension of ShotSpotter?" Williams said.

Alejandro Ruizesparza with the Lucy Parsons Labs points to a MacArthur Justice Center study and an Office of Inspector General investigation that found the majority of ShotSpotter-based deployments in Chicago turn up no evidence of any reportable incident or crime. The study also found that ShotSpotter-based deployments have an exacerbated impact on communities of color.

"So we filed a class-action lawsuit," Ruizesparza said.

The aforementioned are a few of the reasons why Lucy Parson Labs, Williams, and another plaintiff filed the lawsuit against the city on Thursday.

"The OIG raised operational questions to the city that we would argue were ignored, instead, in favor of renewing the contract again and in secret," Ruizesparza said.

Williams said the ShotSpotter evidence distracted investigators from finding the person who actually killed Safarian Herring.

"I don't want to see another family have to go through that," Williams said.

ShotSpotter released the following statement:

"The ShotSpotter system is highly accurate, with a 97% aggregate accuracy rate for real-time detections across all customers that has been independently verified by Edgeworth Analytics. ShotSpotter evidence and ShotSpotter expert witness testimony have been successfully admitted in 200 court cases in 20 states and ShotSpotter evidence has survived scrutiny in dozens of Frye and Daubert challenges, and counting. ShotSpotter's coverage areas are determined by police using objective historical data on shootings and homicides to identify areas most impacted by gun violence to provide residents who live in communities experiencing persistent gunfire a rapid police response."

Chicago man says flaws with ShotSpotter technology had him framed for murder, files lawsuit

A federal lawsuit filed last Thursday alleges Chicago police misused “unreliable” gunshot detection technology and failed to pursue other leads in investigating a grandfather from the city’s South Side who was charged with killing a neighbor.

Chicago prosecutors used audio picked up by a network of sensors installed by the gunshot detection company ShotSpotter as critical evidence in charging Michael Williams with murder in 2020 for allegedly shooting the man inside his car. Williams spent nearly a year in jail, and The Associated Press reported last year that a judge dismissed his case at the request of prosecutors, who said they had insufficient evidence.

The lawsuit filed by the MacArthur Justice Center at Northwestern University’s law school seeks damages from the city for mental anguish, loss of income and legal bills for the 65-year-old Williams, who said he still suffers from a tremor in his hand that developed while he was locked up. It also details the case of a second plaintiff Daniel Ortiz, a 36-year-old father who the lawsuit alleges was arbitrarily arrested and jailed by police who were responding to a ShotSpotter alert.

The suit seeks class-action status for any Chicago resident who was stopped on the basis of the alerts. It also seeks a court order barring the technology’s use in the nation’s third-largest city.

“Even though now I’m so-called free, I don’t think I will ever be free of the thought of what they have done and the impact that has on me now, like the shaking with my hand,” Williams said. “I constantly go back to the thought of being in that place. … I just can’t get my mind to settle down.”

ShotSpotter isn’t named as a defendant in the 103-page filing though the lawsuit claims the company’s algorithm-powered technology is flawed. The suit also alleges the city’s decision to place most of its gunshot-detection sensors in predominantly Black and Latino neighborhoods is racially discriminatory.

Asked for a comment, the city’s law department, which represents police in such cases, said later Thursday that it had not yet been served with the complaint.

Chicago police have previously praised the ShotSpotter system, saying it puts officers on the scene of shootings far faster than if they wait for someone to call 911. Police have also said crime rates _ not residents’ race _ determine where the technology is deployed.

ShotSpotter said in a statement that the evidence it collects and its expert witnesses have been admitted in 200 court cases in 20 states, and have survived dozens of evidentiary challenges.

It described its system as “highly accurate with a 97% aggregate accuracy rate for real-time detections across all customers” and that an analytics company had verified the effectiveness of the technology.

ShotSpotter’s website says the company is “a leader in precision policing technology solutions” that help stop gun violence by using sensors, algorithms and artificial intelligence to classify 14 million sounds in its proprietary database as gunshots or something else.

But the AP investigation identified a number of flaws in using ShotSpotter as evidentiary support for prosecutors, and found the system can miss live gunfire right under its microphones, or misclassify the sounds of fireworks or cars backfiring as gunshots. Last year, Chicago’s nonpartisan watchdog agency concluded that actual evidence of a gun-related crime was found in about 9% of ShotSpotter alerts that were confirmed as probable gunshots.

The lawsuit says the investigating officers “put blind faith in ShotSpotter evidence they knew or should have known was unreliable” in order to charge Williams with killing 25-year-old Safarian Herring. The lawsuit alleges investigators used ShotSpotter material in a way that went beyond its intended use, quoting a disclaimer in one document related to Williams’ case that says the investigative lead summary “should only be used for initial investigative purposes.”

The suit also accuses investigators of not pursuing other leads that could have produced credible suspects, including reports that someone previously shot at Herring at a bus stop.

Police and prosecutors never established a motive for Williams to have shot Herring, never found witnesses to the shooting, and never recovered a weapon or physical evidence tying Williams to the killing, the suit alleges.

“The Defendant Officers engaged in tunnel vision to target Mr. Williams, arresting him for First-Degree murder, without probable cause,” the lawsuit said.

Lawsuit: Chicago Police Misused Gunshot Detection Technology in Murder Case

A Chicago man is taking issue with what he calls flaws with ShotSpotter – saying he was charged with murder and thrown in jail because of the gunshot detection technology. CBS 2 Investigator Megan Hickey reports.

Chicago man says flaws with ShotSpotter technology had him framed for murder