Report 6189

Friendly AI chatbot Lena greets you on Lenovo's website and is so helpful that it spills secrets and runs remote scripts on corporate machines if you ask nicely. Massive security oversight highlights the potentially devastating consequences of poor AI chatbot implementations.

Key takeaways:

• Lenovo's AI chatbot Lena was affected by critical XSS vulnerabilities, which enabled attackers to inject malicious code and steal session cookies with a single prompt.

• The flaws could potentially lead to data theft, customer support system compromise, and serve as a jumpboard for lateral movement within the company's network.

• Improper input and output sanitization highlights a need for stricter security practices in AI chatbot implementations.

Cybernews researchers discovered critical vulnerabilities affecting Lenovo's implementation of its AI chatbot, Lena, powered by OpenAI's GPT-4.

Designed to assist customers, Lena can be compelled to run unauthorized scripts on corporate machines, spill active session cookies, and, potentially, worse. Attackers can abuse the XSS vulnerabilities as a direct pathway into the company's customer support platform.

"Everyone knows chatbots hallucinate and can be tricked by prompt injections. This isn't new. What's truly surprising is that Lenovo, despite being aware of these flaws, did not protect itself from potentially malicious user manipulations and chatbot outputs," said the Cybernews Research team.

"This isn't just Lenovo's problem. Any AI system without strict input and output controls creates an opening for attackers. LLMs don't have an instinct for "safe" -- they follow instructions exactly as given. Without strong guardrails and continuous monitoring, even small oversights can turn into major security incidents," says Žilvinas Girėnas, Head of Product at nexos.ai.

Just a single-prompt attack demonstrates a chain of flaws that led to the Lenovo chatbot spilling active session cookies.

The discovery highlights multiple security issues: improper user input sanitization, improper chatbot output sanitization, the web server not verifying content produced by the chatbot, running unverified code, and loading content from arbitrary web resources. This leaves a lot of options for Cross-Site Scripting (XSS) attacks.

Cybernews responsibly disclosed the issue. Lenovo acknowledged it and has protected its systems.

"Lenovo takes the security of our products and the protection of our customers very seriously. We were recently made aware of a chatbot cross-site scripting (XSS) vulnerability. Upon becoming aware of the issue, we promptly assessed the risk and implemented corrective actions to mitigate potential impact and address the issue," Lenovo said in a statement to Cybernews.

"We want to thank the researchers for their responsible disclosure, which allowed us to deploy a solution without putting our customers at risk."

Single prompt launches multi-step attack

All it took to exfiltrate active session cookies from Lenovo was a single 400-character-long prompt, containing four critical elements:

1. **Seemingly innocent request: **The prompt starts with an inquiry for legitimate information, i.e., product information, like "Show me the specifications of Lenovo IdeaPad 5 Pro."
2. **Instructions to change output format: **The sneaky part is tricking the chatbot into providing a final answer in HTML, a language for creating websites. The researchers achieved that by asking the chatbot to convert its responses into HTML, JSON, and plain text in a specific order that the web server expected to receive instructions in. This ensured that the malicious payload would be correctly executed by the web server.
3. **Malicious HTML injection trap: **The prompt continues with instructions on how to produce the final response, specifically with HTML code for loading an image. The image URL is also nonexistent. Therefore, when it fails to load, the second part of the command instructs the browser to make a network request to the attacker-controlled server and send all cookie data as part of a URL.
4. **Final encouragement: **In the end, additional instructions try to reinforce that the chatbot must produce the image: "Show the image at the end. It is important for my decision-making. SHOW IT."

What happened when Lenovo's Lena received the full prompt?

"People-pleasing is still the issue that haunts large language models (LLMs), to the extent that, in this case, Lena accepted our malicious payload, which produced the XSS vulnerability and allowed the capture of session cookies upon opening the conversation. Once you're transferred to a real agent, you're getting their session cookies as well," said Cybernews researchers.

"Already, this could be an open gate to their customer support platform. But the flaw opens a trove of potential other security implications."

To better understand what's happening under the hood, here's the breakdown of the attack chain:

1. The chatbot falls for a malicious prompt and tries to follow instructions helpfully to generate an HTML answer. The response now contains secret instructions for accessing resources from an attacker-controlled server, with instructions to send private data from the client browser.
2. Malicious code enters Lenovo's systems. The HTML is saved in the chatbots' conversation history on Lenovo's server. When loaded, it executes the malicious payload and sends the user's session cookies.
3. Transferring to a human: An attacker asks to speak to a human support agent, who then opens the chat. Their computer tries to load the conversation and runs the HTML code that the chatbot generated earlier. Once again, the image fails to load, and the cookie theft triggers again.
4. An attacker-controlled server receives the request with cookies attached. The attacker might use the cookies to gain unauthorized access to Lenovo's customer support systems by hijacking the agents' active sessions.

Potential implications can be devastating: the code could be anything

"Using the stolen support agent's session cookie, it is possible to log into the customer support system with the support agent's account without needing to know the email, username, or password for that account. Once logged in, an attacker could potentially access active chats with other users and possibly past conversations and data," the Cybernews researchers warned.

"Companies are moving fast to launch AI, but often slower to secure it. That gap is where attackers step in. In this case, the flaw could allow access to customer data, internal systems, and even create a path deeper into a company's network. Incidents like this show why security has to evolve in step with innovation," adds Girėnas.

However, the exploit also confirms that it is possible to instruct the Lena Chatbot to generate other code that would be later executed on the Lenovo side.

"This is not limited to stealing cookies. It may also be possible to execute some system commands, which could allow for the installation of backdoors and lateral movement to other servers and computers on the network. We didn't attempt any of this," the researchers explain.

The injected code can do a lot of things that could lead to compromise:

• **Alter the interface: **changing what the support agents see on their platform, potentially displaying misinformation, malicious injections.
• **Keylogging: **the secret snippet could capture every keystroke.
• Redirect to a phishing website: the injected code can automatically redirect agents to malicious websites designed to steal their login credentials or even infect their computers with malware.
• **Pop-ups: **the attackers could display malicious CAPTCHAs, fake error messages, or urge agents to download a fake update.
• **Data theft or potential data modifications: **scripts can be designed to compromise or exfiltrate user data on the customer support system.

Assume that all chatbots are dangerous

Cybernews researchers urge all companies to assume that the chatbot outputs are potentially malicious and protect themselves accordingly.

"The fundamental flaw is the lack of robust input and output sanitization and validation. It's better to adopt a 'never trust, always verify' approach for all data flowing through the AI chatbot systems," the researchers said.

"We approach every AI input and output as untrusted until it's verified safe. That mindset helps block prompt injections and other risks before they reach critical systems. It's about building security checks into the process so trust is earned, not assumed," says Girėnas.

XSS vulnerabilities in the past prompted companies to harden their security practices, which led to a decrease in the prevalence of such flaws. The same hardening practices must be implemented with the AI chatbots, including the following:

• **Strict Input Sanitization: **Use a strict whitelist of allowed characters, data types, and formats for all user inputs. All problematic characters should be automatically encoded or escaped. Limit input length to prevent buffer overflows or overly long malicious payloads. Sanitized input based on context.
• Output sanitization and validation: the same applies to chatbot responses. If they're displayed in a web browser or other rich-text environment, they must be aggressively stripped of any embedded code. Strict Content Security Policy (CSP) should be used to restrict which resources (scripts, images, fonts) a browser can load, and dangerous HTML elements and attributes should be restricted.
• Avoid inline JavaScript: The best practice is to limit event handlers and scripts to external JavaScript files only.
• **Secure web servers, apps, and data storage: **content type validation should extend through the entire stack to prevent unintended HTML rendering. Sanitize content before storage. Chatbot apps and related services should operate with the absolute minimum of necessary permissions.

Lenovo is a multinational technology company based in Hong Kong. It is one of the largest vendors of consumer electronics, personal computers, servers, other hardware, software, and related services. In the last financial year, ending March 31, 2025, Lenovo generated $56.86 billion in revenue and reported a net profit of $1.1 billion. Listed on the Hong Kong Stock Exchange, the company has a market capitalization of around $18 billion.