Prompt injection is not an attack, but a feature of LLM assistants

The author explains that prompt injection is simply an AI assistant following instructions embedded in user-provided data, similar to a diligent employee. In an experiment, 9 AI services were given a text file with an instruction to format a list with Roman numerals and add a permanent quotation to all future responses. Two services completely ignored the instructions, five formatted the list but did not change account settings, and two executed both instructions. The author notes that even a translation task was treated as a command by 8 out of 9 services, with one updating its account memory. The problem arises because LLMs cannot distinguish between data and instructions. As a partial remedy, the author suggests explicitly instructing the AI to treat all inputs as data, not commands, and notes that ChatGPT and Claude already have built-in resistance to prompt injection from files.