prompt injection

Prompt injection is an attack where adversarial input is designed to steer a model or model-powered app into ignoring its original instructions and performing unintended actions. These actions can include leaking secrets, executing unsafe steps, or following attacker-supplied goals.

The injected content doesn’t have to be visible or readable to a human, as long as the model parses it. Injection is usually deliberate, but a user can also trigger it inadvertently by supplying input that the model reads as an instruction.

Variants include direct prompt injection, where malicious instructions are entered through the model interface. Indirect prompt injection hides instructions in retrieved or linked content that the system ingests during workflows like RAG or tool use.

Multimodal injection is a growing third surface, where the payload hides in an image, audio, or another non-text modality that a multimodal model processes alongside benign text. Prompt injection exploits the lack of a strict boundary between instructions and data.

Mitigations should adopt a defense-in-depth approach, including:

• Input and output filtering and sanitization
• Isolating and clearly delineating system and user instructions
• Enforcing least-privilege access for tool integrations and sandboxes
• Requiring human-in-the-loop approval for high-risk or privileged actions
• Implementing allow or deny lists for tool use
• Verifying the provenance or trustworthiness of external content
• Hardening retrieval pipelines
• Monitoring and conducting adversarial testing to detect residual risks

Related Resources

Tutorial

How to Add Features to a Python Project With Codex CLI

Learn how to use Codex CLI to add features to Python projects via the terminal. Master AI-powered coding without needing a browser or IDE plugins.

intermediate ai tools

For additional information on related topics, take a look at the following resources:

• Prompt Engineering: A Practical Example (Tutorial)
• Building Type-Safe LLM Agents With Pydantic AI (Course)
• Build an LLM RAG Chatbot With LangChain (Tutorial)
• Use Codex CLI to Enhance Your Python Projects (Course)
• How to Add Features to a Python Project With Codex CLI (Quiz)
• Practical Prompt Engineering (Quiz)
• First Steps With LangChain (Course)
• Build an LLM RAG Chatbot With LangChain (Quiz)