Do Drafters Work on Small Models? Gemma 4 E2B and E4B on Two Small GPUs

This article provides a step by step guide to serving Gemma 4 E2B and E4B with Google's QAT assistant drafters through llama.cpp's speculative decoding, and measuring what the drafter is worth on two small NVIDIA cards: a desktop RTX 3050 OEM and a laptop GTX 1650 Ti. A suite of Python MCP tools is built to simplify management of the llama.cpp hosted deployment. This is the second part of a short series. Part 1, Gemma 4's MTP Drafter, Explained, covers how speculative decoding and Gemma 4's multi-token prediction (MTP) drafter work and how the exact Q4_0 repack is built. This part has the measurements. https://github.com/xbill9/gemma4-dev A drafter is the guesser in speculative decoding: it proposes the next few tokens, and the main model checks them all in one pass. Do drafters work on small models? Yes. Gemma 4 E2B and E4B are already small, and their drafter is a 4-layer model of 59 MB. Served one request at a time, it makes both models faster on both cards, and GSM8K and BFCL s