Associated Incidents
The national policy document intended to shape South Africa's approach to artificial intelligence (AI) may have fallen victim to one of its most widely understood pitfalls.
News24 can exclusively reveal that some of the academic journal articles cited in the Draft National AI Policy are completely fictitious. The most likely explanation for how this happened, ironically, is that an AI tool hallucinated them.
Earlier this month, Communications and Digital Technologies Minister Solly Malatsi published the draft policy in the Government Gazette for public comment. The document states that its purpose is to establish national priorities and norms for AI, and to recognise "sector-specific dynamics".
The draft policy proposes creating a raft of new institutions to regulate AI. This includes an AI Ethics Board, an AI Safety Institute, and an AI Insurance Superfund responsible for compensating people or entities "harmed by AI-driven outcomes".
News24 reviewed the 67 references cited at the bottom of the draft policy. Several of them either cited an academic journal that does not exist or an article that does not appear in an established journal.
Here are five examples:
Babatunde, O., & Mnguni, P. (2023). "Challenges and Opportunities in Regulating AI: Perspectives from South Africa." AI Policy Journal, 2(3), 143-156.
- News24 can find no evidence that the AI Policy Journal exists. No academic journal article with that title appears in any major journal.
Burman, A., & Sewpersadh, K. (2022). "Legal Frameworks for AI in South Africa: Balancing Innovation and Accountability." South African Journal of Philosophy, 41(2), 207-217.
- The South African Journal of Philosophy (SAJP) does exist. However, its managing editor, Dominic Griffiths, told News24 that "no such article was published by our journal". Griffiths said no one with the surname "Burman" had ever published in the SAJP. "The reference is definitely an AI hallucination," he said.
Karr, V., & Smith, L. (2023). "Digital Rights and AI Governance in Africa." Journal of African Law, 67(1), 95--108.
- The Journal of African Law is real, but the article does not appear in Volume 67, Issue 1. An article with that title does not seem to appear in any other journal either.
Cavaliere, F., McGregor, R., & Hersh, M. (2022). "Artificial Intelligence and Ethics in Emerging Economies: The Case of South Africa." AI & Society, 37(4), 565-583.
- Karamjit Gill, the editor-in-chief of AI & Society, said he "can't locate the publication of this article" in the journal. News24 also could not find the article on Google Scholar.
Smith, M., & Mahomed, R. (2021). "The Impact of AI on Social Justice in South Africa." Journal of Ethics and Social Philosophy, 18(3), 313-329.
- Louise Simpson, the managing editor of the Journal of Ethics and Social Philosophy (JESP), confirmed that the article does not exist. "I can confirm that JESP has not published an article with that title or by that combination of authors in that, or any other issue," she said.
Some of the sources on the list were submissions as part of government's inquiry, or were not purporting to be academic journal articles.
News24 was unable to verify beyond any doubt exactly how many references on the list were fictitious, but at least six were made up.
Author and South African economic historian at Stellenbosch University, Johan Fourie, assisted in checking the references.
"I did two things: I checked three citations myself. I, too, concluded that they don't exist. I then tested them with an AI tool, and it also concluded that all three were 'not found in common academic databases,'" Fourie said.
"In short, they were made up."
News24 reached out to the Department of Communications and Digital Technologies to clarify whether it had used an AI tool to draft either the body or citations of the Draft National AI Policy, or both. We also asked the department to provide evidence that the journal articles existed.
The department indicated that it was reviewing the reference list to ensure accuracy, including how the "minor referencing discrepancies" may have happened.
The department also indicated that it kicked off the draft policy process in 2024, and the document had gone through many iterations. It said it incorporated research, expert input, and industry engagements.
The department said it needed time to establish where the discrepancies came from, but downplayed their importance.
"We can confirm that these technical referencing matters do not affect the substance, integrity or policy direction of the Draft National AI Policy Framework," it said.
Irresponsible
Anné Verhoef, director of the North-West University AI Hub and a professor in philosophy, said AI text often includes fabricated sources and references.
"This occurs because AI applications are programmed to always provide an answer. As prediction tools based on large language models, they anticipate what sources and citations would make a text appear credible, authoritative and scientific," he said.
Verhoef said including fictitious sources in academic and research work is "indicative of irresponsible use of AI".
"Instances such as the Draft National AI Policy containing inaccurate citations and sources, created without disclosure of AI involvement, reflect possible unethical and irresponsible use of AI," Verhoef said.
Fourie said AI model hallucinations were unsurprising, but it was surprising that there seemed to be no oversight.
"These tools are powerful. Firms, academics and governments already use them. Their value, however, depends on close human supervision.
"Fabricated references of this kind suggest that the tool was not used with sufficient care. The consequence, regrettable but understandable, is that doubts will now extend beyond the error itself to the credibility of the entire document," Fourie added.
- With Katharina Moser