16.19 Untrusted Document Handling and Sandboxing (Week 16 • Advanced RAG & Security)
1. First-Principles Concept & Why It Matters
Untrusted Document Handling and Sandboxing (Advanced RAG & Security) covers File-size/page/decompression-bomb limits (`.docx` is a ZIP archive!), magic-byte verification, isolated parser workers, XML data-tag framing, and zero tool-trigger privileges from untrusted docs. In production Generative AI engineering with Python, it enables teams to Blocking a malicious `40 KB` `.docx` zip-bomb that expands to `4 GB` XML in memory before crashing the ingestion worker! Mastering the underlying mathematics, data structures, model/API mechanics, and evaluation/security invariants directly in Python ensures you can build, debug, and optimize real-world LLM, RAG, and Agent systems independently of any single framework.
2. Production GenAI Engineering Use Case
Blocking a malicious `40 KB` `.docx` zip-bomb that expands to `4 GB` XML in memory before crashing the ingestion worker!
3. Granular Subtopics & Learning Checkpoints
- File-size/page/decompression-bomb limits (`.docx` is a ZIP archive!): Core engineering foundation of Untrusted Document Handling and Sandboxing: Blocking a malicious `40 KB` `.docx` zip-bomb that expands to `4 GB` XML in memory before crashing the ingestion worker!
- Magic-byte verification: Understand how 'Magic-byte verification' works from first principles inside Untrusted Document Handling and Sandboxing before relying on high-level wrappers.
- Isolated parser workers: Understand how 'Isolated parser workers' works from first principles inside Untrusted Document Handling and Sandboxing before relying on high-level wrappers.
- XML data-tag framing: Understand how 'XML data-tag framing' works from first principles inside Untrusted Document Handling and Sandboxing before relying on high-level wrappers.
- Production GenAI Pitfall & Guardrail (Untrusted Document Handling and Sandboxing): Prevent common production failure: Trusting client-supplied `Content-Type: application/pdf` header without checking `%PDF-` magic bytes and uncompressed byte caps.
- Hands-On Python Implementation & Verification (Untrusted Document Handling and Sandboxing): Write a hardened `validate_upload_bytes(filename, data)` guard checking magic bytes and max size.
4. Reference Python 3.12 / GenAI Implementation
def is_valid_pdf_upload(data: bytes, max_mb: int = 20) -> bool:
return data.startswith(b'%PDF-') and len(data) <= max_mb * 1024 * 1024
print(is_valid_pdf_upload(b'%PDF-1.7 sample...'), is_valid_pdf_upload(b'MZ-exe-payload'))
5. Common Production Pitfalls & Guardrails
- Trusting client-supplied `Content-Type: application/pdf` header without checking `%PDF-` magic bytes and uncompressed byte caps.
- Skipping boundary, empty-input, or schema/ACL validation when implementing Untrusted Document Handling and Sandboxing in Advanced RAG & Security.
- Deploying Untrusted Document Handling and Sandboxing changes without automated pytest checks, golden evaluation metrics, and latency/token-cost tracking.
6. Mandatory Hands-On Practical Exercise
Write a hardened `validate_upload_bytes(filename, data)` guard checking magic bytes and max size.
7. Technical Interview & System-Design Checkpoints
- Q: What engineering problem does Untrusted Document Handling and Sandboxing solve in Advanced RAG & Security, and what fails in production GenAI systems when it is misused?
Key Focus: Blocking a malicious `40 KB` `.docx` zip-bomb that expands to `4 GB` XML in memory before crashing the ingestion worker! Avoid common mistake: Trusting client-supplied `Content-Type: application/pdf` header without checking `%PDF-` magic bytes and uncompressed byte caps. - Q: Walk through the underlying mathematics/mechanics of Untrusted Document Handling and Sandboxing (File-size/page/decompression-bomb limits (`.docx` is a ZIP archive!), magic-byte verification, isolated parser workers, XML data-tag framing, and zero tool-trigger privileges from untrusted docs) before relying on any framework.
Key Focus: Explain File-size/page/decompression-bomb limits (`.docx` is a ZIP archive!), magic-byte verification, isolated parser workers, XML data-tag framing, and zero tool-trigger privileges from untrusted docs step-by-step with a concrete Python 3.12 / PyTorch / RAG / Agent code example. - Q: How did you implement, evaluate, and secure Untrusted Document Handling and Sandboxing inside your mandatory GenAI project or capstone?
Key Focus: Reference hands-on implementation: Write a hardened `validate_upload_bytes(filename, data)` guard checking magic bytes and max size.