Report 5112

On April 26, moderators of r/ChangeMyView, a community on Reddit dedicated to understanding the perspectives of others, revealed that academic researchers from the University of Zürich conducted a large-scale, unauthorized AI experiment on their community. The researchers had used AI bots to secretly impersonate people for experiments in persuasion.

Crucially, this experiment was carried out without the Reddit community's knowledge or consent, and the bots were not labeled as AI. Reddit users were unknowingly exposed to sometimes deceptive AI-generated content designed to shape their opinions---raising significant ethical and transparency concerns.

When the researchers shared preliminary results with the community, r/ChangeMyView moderators contacted the researchers' ethics board out of concern. When told the project met institutional ethics standards, moderators disclosed the study to their shocked and outraged community. Reddit banned the associated accounts and issued a statement condemning the activity.

This case has drawn sharp criticism from the tech research and ethics community. It highlights a mistaken belief by some researchers that public-interest intentions justify ignoring the ethical responsibilities of researchers toward participants and the public.

A Call to Action on Digital Experiment Ethics

There is no question: this experiment was unethical. Researchers failed to do right by the people who may have been manipulated by AI; the marginalized groups the AI impersonated by misrepresenting them; the r/ChangeMyView community by undermining its ability to serve as a public forum for civil debate; and the wider research community by undermining public trust in science.

Researchers need to stand up for ethical practices because ethics failures are often used as an excuse to restrict public-interest work. Previous scandals, such as the Facebook emotional contagion study and work by Cambridge Analytica generated fears about manipulation through experimentation. In response, companies have cited fears about unethical research and the need to protect users' privacy when limiting public interest research and threatening scientists, despite experimenting on their users themselves. We believe it is likely that many more unethical, disruptive generative AI experiments are being conducted right now by industry actors who have even less oversight than scientists.

As a public forum where users engage in deep discussions on millions of topics, Reddit has long been an important source of public-interest research. And Reddit communities have long partnered with scientists in mutually-beneficial collaborations that grow understanding of our world. To protect the diverse and vibrant communities of Reddit (even its more controversial ones), it is imperative that researchers remember the human by anticipating how their research might harm individuals, groups, and communities.

As Coalition members, we call for the following actions:

1. We call on ethics review boards, journal editorial boards, and peer reviewers to be extra vigilant to uphold ethics at a time when generative AI research is still relatively recent. New technologies do not override enduring ethics principles. If you have questions or doubts, we have compiled a list of resources below. We also invite you to reach out to those of us working at the forefront of technology and research ethics: you don't need to do this alone.
2. We call on regulators and the technology industry to support transparency for digital experiments conducted on the public. People deserve to know how and when our lives have been systematically influenced, whether experiments are by scientists, companies, or some random person with a large language model. Transparency provisions in laws like Minnesota's Prohibiting Social Media Manipulation Act could more generally help safeguard the public from unethical research while supporting beneficial science.

About CITR

The Coalition for Independent Technology Research is a global network working to ensure that technology research serves the public good. The Coalition's mission is to advance, defend, and sustain the right to ethically study the impact of technology on society.

To learn more about the Coalition and its work, read our founding document and manifesto.

Resources from Coalition members about ethical research practices on Reddit:

• Background:

  • How scientists are making the most of Reddit
  • Remember the Human: A Systematic Review of Ethical Considerations in Reddit Research
  • Cross-Platform user expectations for social media data reuse
  • New approaches to platform data research

• Guides:

  • Association for Internet Researchers: Internet Research: Ethical Guidelines 3.0
  • PERVADE Data Ethics Tool
  • AskHistorians Policy for Researchers using r/AskHistorians in Research
  • Excavating awareness and power in data science: A manifesto for trustworthy pervasive data research

• Best practices and examples:

  • Towards intersectional moderation: An alternative model of moderation built on care and power
  • Procedural and Substantive Ethics in the Design of Research Ethics Systems
  • CivilServant: Community-Led Experiments in Platform Governance