AI builder essentials: tokens, context windows and RAG 101

Tokens are an important foundational concept that underlie how we interact with LLMs. @greggyb and myself recorded a quick video to demystify what a token actually is and unpack several other related concepts. What's in the video What is a token? Tokens are the basic unit of input and output that LLMs operate on. What is retrieval-augmented generation (AKA RAG)? Providing a model with extra data that isn't already in the model's training dataset, and is relevant to a specific query. What are context windows? Context windows are how many tokens a model can use for input and output. Context window size varies from model to model, but it has increased a lot in recent years. What is tokenmaxxing? Does using more tokens really mean you're being more productive? (Spoiler alert: no.) To paint a bigger picture: more context you give a large language model, the better your responses are likely to be. The design patterns for providing that context have evolved at a rap