Incident 139: Amazon’s Search and Recommendation Algorithms Found by Auditors to Have Boosted Products That Contained Vaccine Misinformation

Description: Evidence of the "filter-bubble effect" were found by vaccine-misinformation researchers in Amazon's recommendations, where its algorithms presented users who performed actions on misinformative products with more misinfomative products.
Alleged: Amazon developed and deployed an AI system, which harmed Amazon Customers.

Suggested citation format

McGregor, Sean. (2021-01-21) Incident Number 139. in McGregor, S. (ed.) Artificial Intelligence Incident Database. Responsible AI Collaborative.

Incident Stats

Incident ID
139
Report Count
2
Incident Date
2021-01-21
Editors
Sean McGregor, Khoa Lam

Tools

New ReportNew ReportNew ResponseNew ResponseDiscoverDiscover

Incident Reports

Abstract: There is a growing concern that e-commerce platforms are amplifying vaccine-misinformation. To investigate, we conduct two-sets of algorithmic audits for vaccine misinformation on the search and recommendation algorithms of Amazon -- world's leading e-retailer. First, we systematically audit search-results belonging to vaccine-related search-queries without logging into the platform -- unpersonalized audits. We find 10.47% of search-results promote misinformative health products. We also observe ranking-bias, with Amazon ranking misinformative search-results higher than debunking search-results. Next, we analyze the effects of personalization due to account-history, where history is built progressively by performing various real-world user-actions, such as clicking a product. We find evidence of filter-bubble effect in Amazon's recommendations; accounts performing actions on misinformative products are presented with more misinformation compared to accounts performing actions on neutral and debunking products. Interestingly, once user clicks on a misinformative product, homepage recommendations become more contaminated compared to when user shows an intention to buy that product.

Introduction: The recent onset of coronavirus pandemic has unleashed a barrage of online health misinformation [4, 22] and renewed focus on the anti-vaccine movement, with anti-vax social media accounts witnessing a 19% increase in their follower base [49]. As scientists work towards creating a vaccine for the disease, health experts worry that vaccine hesitancy could make it difficult to achieve herd immunity against the new virus [3]. Battling health misinformation, especially anti-vaccine misinformation has never been more important.

Statistics show that people increasingly rely on the internet [53], and specifically online search engines [8], for health information including information about medical treatments, immunizations, vaccinations and vaccine-related side effects [6, 23]. Yet, the algorithms powering search engines are not traditionally designed to take into account the credibility and trustworthiness of such information. Search platforms being the primary gateway and reportedly the most trusted source [19], persistent vaccine misinformation on them, can cause serious health ramifications [38]. Thus, there has been a growing interest in empirically investigating search engine results for health misinformation. While multiple studies have performed audits on commercial search engines to investigate problematic behaviour [35, 36, 56], e-commerce platforms have received little to no attention ([11, 59] are two exceptions), despite critics calling e-commerce platforms, like Amazon, a “dystopian” store for hosting anti-vaccine books [17]. Amazon specifically has faced criticism from several technology critics for not regulating health-related products on its platform [5, 55]. Consider the most recent instance. Several medically unverified products for coronavirus treatment, like prayer healing, herbal treatments and antiviral vitamin supplements proliferated Amazon [18, 28], so much so that the company had to remove 1 million fake products after several instances of such treatments were reported by the media [22]. The scale of the problematic content suggests that Amazon could be a great enabler of misinformation, especially health misinformation. It not only hosts problematic health-related content but its recommendation algorithms drive engagement by pushing potentially dubious health products to users of the system [27, 59]. Thus, in this paper we investigate Amazon—world’s leading e-retailer—for most critical form of health misinformation—vaccine misinformation.

What is the amount of misinformation present in Amazon’s search results and recommendations? How does personalization due to user history built progressively by performing real-world user actions, such as clicking or browsing certain products, impact the amount of misinformation returned in subsequent search results and recommendations? In this paper, we dabble into these questions. We conduct 2 sets of systematic audit experiments: Unpersonalized audit and Personalized audit. In the Unpersonalized audit, we adopt Information Retrieval metrics from prior work [42] to determine the amount of health misinformation users are exposed to when searching for vaccine-related queries. In particular, we examine search-results of 48 search queries belonging to 10 popular vaccinerelated topics like ‘hpv vaccine’, ‘immunization’, ‘MMR vaccine and autism’, etc. We collect search results without logging in to Amazon to eliminate the influence of personalization. To gain indepth insights about the platform’s searching and sorting algorithm, our Unpersonalized audits ran for 15 consecutive days, sorting the search results across 5 different Amazon filters each day: “featured”, “price low to high”, “price high to low”, “average customer review” and “newest arrivals”. The first audit resulted in 36,000 search results and 16,815 product page recommendations which we later annotated for their stance on health misinformation—promoting, neutral or debunking.

In our second set of audit—Personalized audit, we determine the impact of personalization due to user history on the amount of health misinformation returned in search results, recommendations and auto-complete suggestions. User history is built progressively over 7 days by performing several real-world actions, such as “search” , “search + click”, “search + click + add to cart”, “search + click + mark top-rated all positive review as helpful”, “follow contributor” and “search on third party website” ( Google.com in our case) . We collect several Amazon components in our Personalized audit, like homepages, product pages, pre-purchase pages, search results, etc. Our audits reveal that Amazon hosts a plethora of health misinformative products belonging to several categories, including Books, Kindle eBooks, Amazon Fashion (e.g. apparel, t-shirt, etc.) and Health & Personal care items (e.g. dietary supplements). We also establish the presence of a filter-bubble effect in Amazon’s recommendations, where recommendations of misinformative health products contain more health misinformation.

Below we present our formal research questions, key findings, contributions and implication of this study along with ethical considerations taken for conducting platform audits.

Auditing E-Commerce Platforms for Algorithmically Curated Vaccine Misinformation

New York - Amid growing concern that e-commerce platforms are amplifying vaccine-misinformation, a new study by researchers at University of Washington has found that Amazon hosts a large number of misinformative products belonging to categories such as books, ebooks, health, personal care and apparel. The results led to a dataset of 4,997 Amazon products annotated for health misinformation, said the study published at preprint repository arXiv.org.

For the study, Prerna Juneja and Tanushree Mitra, both from The Information School at the University of Washington, conducted two-sets of algorithmic audits for vaccine misinformation on the search and recommendation algorithms of Amazon. Their aim was to empirically determine the amount misinformation returned by its search and recommendation algorithm. They also investigated whether personalisation due to user history plays any role in amplifying misinformation.

So, they did a systematic audit search-results belonging to vaccine-related search-queries without logging into the platform -- unpersonalised audits. They found 10.47 per cent of search-results promote misinformative health products on the platform. "We found that search results returned for many vaccine-related queries contain large number of misinformative products leading to high misinformation bias," the researchers said.

Moreover, misinformative products are also ranked higher than debunking products, the results said. Next, they analysed the effects of personalisation due to account-history, where history is built progressively by performing various real-world user-actions, such as clicking a product. "Our study also suggests presence of a filter-bubble effect in recommendations, where users performing actions on misinformative products are presented with more misinformation in their homepages, product page recommendations and pre-purchase recommendations," the researchers said.

Amazon algorithms boost vaccine misinformation, says study

Similar Incidents

By textual similarity

Did our AI mess up? Flag the unrelated incidents