I wrote a puzzle generator whose acceptance rate is 0.26%. It throws away 99.7% of everything it produces, and that is the design working as intended, not failing. Generating five valid puzzles takes 1,947 attempts and 84 milliseconds.
The point is not the puzzles. The point is that the generator makes no correctness guarantee at all, and a verifier makes every one of them. Once you split those two responsibilities, "make the generator smarter" stops being the obvious optimisation — and that is exactly the position you are in when the generator is an LLM.
The loop
verigen is a Go CLI that produces cryptarithmetic puzzles — alphametics, the SEND + MORE = MONEY genre, where each letter stands for a distinct digit and the sum has to hold. The known answer to that one is 9567 + 1085 = 10652.
There is one rule, and everything else follows from it:
The generator guarantees nothing. Every guarantee lives in the verifier.
The generator throws plausible-looking letter co
Discussion
Say something first
It all starts with you—share your thoughts now.