Context Compression Before the LLM: Cutting Tokens Without Cutting Recall

Book: RAG Pocket Guide: Retrieval, Chunking, and Reranking Patterns for Production Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools Me: xgabriel.com | GitHub You retrieve the top 10 chunks, paste them into the prompt, and send it to the model. Each chunk is 400 tokens. That is 4,000 tokens of context for a question whose answer lives in two sentences buried in chunk 6. You pay for all 4,000 on input. You also pay a quieter tax: the model has to find the answer inside a wall of near-miss text, and longer contexts degrade answer quality even when the right fact is present. Stanford's "Lost in the Middle" work showed it clearly. As input context grows, models reliably use information at the start and end and lose track of facts stuck in the middle (Liu et al., 2023). So the chunk that ranked sixth, sitting