Deterministic by default

Every app I build now has a language model somewhere inside it. In each one the model does a sliver of the work, and everything else is plain code that returns the same answer every time it runs. That split turned out to be the most important design decision in any of them, and the app that taught me the most about it was the one I expected to be all model.

A game that should have been all imagination

Realmcaller is a voice driven tabletop game in the Dungeons and Dragons style. A model plays the game master, narrating the world, voicing the characters, and inventing whatever the players walk into. On paper that is the ideal use of a language model. The whole product is imagination.

In practice the game fell apart whenever the rules were left to the model. An early version handed combat to the model's judgment, with the difficulty setting written into the prompt as guidance. I played a whole fight on Hard and spent three turns doing deliberately suicidal things. Zero damage, then a level up. The shop had the same problem. An emphatic instruction about how it should behave, sitting deep in a long prompt, was ignored twice in a row. With the rules loosely held, the quest devolved within a few turns into a story where nothing was at stake, and a game with nothing at stake stops being a game.

The fix was to take the numbers away from the model, starting with combat. Hit rolls, damage, difficulty scaling, who is still standing at the end of the round. All of it is computed in code, and the model is handed the result to narrate. Combat now runs mechanically end to end, and the model does the one job it is good at, which is making the outcome feel like a story. Imagination turned out to be a thin layer on top of a rules engine, and the rules engine is what lets the imagination land.

The same split in a chemistry app

Pool Pilot taught the lesson from the other direction. It reads a test strip with a vision model, and I assumed the scan was the product. The data says otherwise. Four in ten of the people who have ever logged a test have never pointed the camera at a strip. They read the pads themselves, type the numbers in, and use the app for what comes next, which is arithmetic. Given a reading and a pool volume, the dose is a formula. Same pool, same reading, same answer, every time.

The model's job in Pool Pilot is perception. Test strip pads never quite match the chart on the bottle, and a vision model matches that mess better than the hand written color math I tried first. Its output is checked against the ranges a strip can physically show, and then the formula takes over. The model reads. The code doses.

I learned how hard that boundary has to be from the chat assistant. Its prompt opened with a rule that it must never compute a specific dose. Later I added a safety note at the bottom that said, roughly, never recommend chemical quantities unless the user has a recent test on file. That note was a license. "Never, unless" is permission with a filter, and the filter was satisfied exactly when the assistant was opened from a test result. The prompt also carried the dosing formulas as reference material, so the model had the recipe and the ingredients. An output filter was supposed to catch dose statements. Measured against realistic replies, eleven of fourteen slipped through, and on a match it prepended a disclaimer and printed the number anyway.

The fix was structural. The formulas came out of the prompt, so the capability is gone. The rule became one unconditional sentence. The filter now replaces a reply outright, in all five of the app's languages. A rule the model is asked to follow is a suggestion. A rule the code enforces is a rule.

A rule of thumb

When I am deciding where the model goes, I ask one question about each step. Is there a right answer?

Yes means code. Unit conversion, whether a trial has expired, how much chlorine to add, whether this user can see this screen. No means maybe the model. Reading a photo, summarizing a long note, narrating a fight, guessing what someone meant from a messy sentence.

The trap is the middle, the steps that look like judgment and have a right answer once you think for a minute. "Is this reading dangerous?" is a threshold. "Which reminder fires first?" is a sort. "How much damage does a goblin do?" is a table. Most of the middle collapses into code if you push on it, and every step that collapses behaves the same way for the ten thousandth user as it did for the first.

Better models move the boundary a little each year. Things I would have hand coded two years ago, like turning a free text note into a structured reminder, are now reliable enough to hand over. They also make it tempting to hand over things that belong in code, because the model would probably get them right. Probably is the problem. A formula gets the dose right every time and costs nothing. A model gets it right nearly every time, and nearly right at scale is a support inbox.

So the default is deterministic. The model gets the steps where a good answer is the product, code checks everything it returns, and the game master never rolls the dice.