Hidden prompts could rewrite AI memories, researchers warn

As artificial intelligence assistants become more capable of remembering user preferences and past interactions, researchers have identified a new security risk that could allow attackers to alter those memories secretly. The newly described attack, known as GhostWriter, targets an AI agent’s long-term memory rather than its underlying model, potentially influencing its behavior long after the initial attack.

The research was carried out by scientists at New Mexico State University, who argue that memory has become one of the most valuable features in modern AI assistants and, at the same time, one of their least protected components. Their findings suggest that memory systems designed to improve personalisation could also create new opportunities for cyberattacks if they are not properly secured.

AI memory creates a new security challenge

Traditional chatbots typically treat every conversation as a separate interaction, retaining little or no information between sessions. In contrast, many of today’s AI assistants use persistent memory to remember a user’s preferences, ongoing projects, writing style and previous conversations. This allows them to provide responses that feel more personal and relevant over time.

According to the researchers, this shift introduces a new attack surface. Instead of attempting to compromise the language model itself, attackers could manipulate the information stored in an AI assistant’s long-term memory. The researchers named this technique GhostWriter, describing it as a method of quietly inserting false information into an AI’s memory through hidden prompts or untrusted external content.

Once the false information has been stored, it may remain inactive until the assistant later retrieves it while responding to an otherwise legitimate request. Because the memory appears to the AI as trusted information, the assistant may unknowingly rely on it when making decisions or carrying out tasks.

The researchers explained that this differs from conventional prompt injection attacks, which generally affect only a single conversation. GhostWriter is designed to persist across future interactions, allowing manipulated memories to influence the AI repeatedly until they are detected and removed.

GhostWriter targets what AI remembers

The study describes GhostWriter as a two-stage attack. The first stage involves memory injection, in which hidden instructions or malicious content are stored in the AI’s long-term memory. The second stage occurs when the AI later retrieves that poisoned memory while handling an unrelated user request.

The researchers said such an attack could have serious consequences for AI assistants that manage sensitive tasks. For example, an assistant summarising banking emails could be influenced into forwarding messages to an attacker. In other situations, the AI could recall incorrect contact details, inaccurate deadlines, fabricated user preferences or false factual information because its memory had already been altered.

Unlike attacks that rely on continuous interaction with the model, GhostWriter requires only that malicious content successfully enter the memory system. Once stored, the altered information may continue affecting future responses without the user’s knowledge.

The researchers noted that the increasing use of AI assistants for handling emails, calendars, software development and other productivity tasks means that long-term memory is becoming an increasingly attractive target for attackers seeking to manipulate automated systems.

Researchers propose safeguards for AI memory

The timing of the research coincides with growing competition among AI developers to build assistants capable of remembering users over weeks, months or even years. Long-term memory has become a key feature that helps distinguish AI assistants, enabling more personalised and context-aware interactions.

During their experiments, the researchers reported that GhostWriter achieved a memory injection success rate of approximately 98%. In comparison, poisoned memories were successfully activated in around 60% of cases when tested against state-of-the-art AI agents. These findings suggest that current memory systems may struggle to distinguish between trustworthy information and manipulated inputs.

To address the issue, the research team developed a defensive framework called Agentic Memory Sentry (AM-Sentry). The system combines stricter rules governing what information can be saved into memory with screening mechanisms that examine retrieved memories before they are used. According to the researchers, this significantly reduced GhostWriter’s effectiveness while maintaining the overall usefulness of AI assistants.

The authors argue that protecting AI memory could become as important as protecting the language models themselves. As AI assistants continue to gain greater autonomy and access to personal data, ensuring the integrity of their memory may prove essential for maintaining user trust and preventing long-term manipulation.