Word Embeddings Explained: The Math Behind AI, LLMs, and Chatbots

Originally published on malcolmlow.net When a search engine retrieves a document about automobiles in response to a query about cars, it is not matching text character by character. Somewhere beneath the interface, the system understands that these two words are semantically related. The mechanism behind that understanding is the word embedding — and once you see the geometry, you cannot unsee it. This article walks through the key mathematical operations that make embeddings work: distance, similarity, arithmetic, scaling, and the dot product. Each concept is illustrated with concrete numerical vectors so the math is visible, not just described. Real embeddings typically use hundreds of dimensions; the 3- and 4-dimensional examples here preserve all the structure while staying readable on a page. 1 · What is a Word Embedding? A word embedding is a representation of a word as a vector — an ordered list of numbers — in a high-dimensional space. A typical embedd