Here's a mistake we made, so you don't have to. We built a Japanese-learning workflow, and one of its jobs was furigana — the little hiragana readings that sit over kanji so a learner knows how to say them. The obvious move was to ask the model for them, right there in the prompt. It worked. It also quietly became one of the most expensive things the pipeline did.
Furigana gets spelled out character by character, and to a language model every one of those hiragana is a token. A single sentence can carry fifteen or more furigana characters — fifteen tokens generated on top of everything the model was already saying. On a short response, that's often ten percent of the entire output. We were paying a frontier model, by the token, to transliterate — a job it can get wrong, and a job a tiny library does for free.
That bill is worth staring at, because it points straight at the real shape of token optimization. The biggest savings almost never come from a cheaper model or a clever cache. They come from noticing you're handing the model work it never needed to touch. Here are the four moves, roughly in order of how much they'll save you.
Move 1 — Don't re-feed data the model doesn't need
When your whole workflow is one giant prompt, this leak stays hidden — everything lands on one bill. The moment you do the right thing and break that prompt into steps, a subtler one opens up. You've got a sentence, its reading, some grammar notes, a couple of example suggestions — and the tempting move is to pour that whole blob into every block so each step 'has the context.' But a block's input is billed as input tokens, and if the block repeats any of it back, you're billed again on the way out. Data that only needs to reach the end of the pipeline gets taxed at every stop along the way.
Noukai has a block type for exactly this: Passthrough. Instead of feeding data into a prompt to carry it forward, a Passthrough block routes it around the model — it arrives at the block that actually needs it, or at the final response, without ever becoming input or output tokens. The grammar notes you generated in step two reach the response in step five without three models in between re-reading and re-typing them.
| How you carry the data | What later blocks do with it | What you pay |
|---|---|---|
| One giant prompt | Everything happens in a single call | Paid once — but you can't isolate, test, or fix any one step |
| Split into blocks, resend everything | Each block re-reads the whole blob and echoes it back out | Paid as input at every block, and again as output |
| Split into blocks, Passthrough the rest | Only the fields a block transforms enter its prompt | Paid once, at the step that actually uses it |
The rule of thumb: a field belongs in a block's prompt only if that block is going to transform it. If you're just moving it, renaming it, or handing it back to the caller, pass it through.
Move 2 — Don't make the model do deterministic work
Back to the furigana. The reason it stung is that furigana isn't reasoning — it's a lookup. Kanji maps to reading by rules a dictionary settled long ago; there's no judgment for the model to add. Which makes it the perfect candidate for the oldest rule in pipeline design: keep deterministic work out of the model.
pykakasi is a small Python package that turns kanji into hiragana. It runs inside a Code block, costs zero tokens, returns instantly, and — unlike the model — never invents a reading that looks plausible and is wrong. We deleted the furigana instruction from the prompt, dropped in a Code block, and that ten percent simply evaporated.
from pykakasi import kakasi
kks = kakasi()
# kanji in, hiragana out — no tokens spent, no reading hallucinated
result = kks.convert("日本語を勉強する")
furigana = "".join(item["hira"] for item in result)Furigana is just the example that bit us. The principle is general: transliteration, parsing, validation, arithmetic, date math, string formatting — anything with one correct answer and no judgment — belongs in a Code block, not a prompt. A model doing arithmetic is slower, pricier, and occasionally wrong. The same three lines of code are none of those.
Move 3 — Right-size the model, one block at a time
Every block in a Noukai pipeline carries its own model, and that's a lever most teams leave untouched. The block that extracts the words out of a sentence doesn't need your best reasoning model — a small, fast one handles it for a fraction of the price. Save the frontier model for the block that actually reasons: the one writing the grammar explanation, or judging whether a learner's answer is right.
Because the model is a per-block dial and not a project-wide setting, this costs you nothing to try. Point the extraction block at a cheaper model, run your test cases, and compare. If quality holds, you've cut that block's bill without touching a single prompt.
Move 4 — Give the output nowhere to ramble
Left unconstrained, a model pads. It opens with 'Certainly! Here's the analysis you asked for,' walks you through its reasoning, and signs off with a summary you never requested — every word of it billed as output. A tight output schema closes that door. When a block declares it returns { word, reading, part_of_speech }, that is all it can return. There's no room left for preamble.
So the schema you write for correctness — so the next block knows exactly what shape it's receiving — pays you back a second time in tokens. The model fills the fields you named and stops. Narrow the schema to what you'll actually use downstream, and the output shrinks to match.
Bonus — talk to your model like a caveman
Everything above is structural: it changes what the model is asked to do. There's a smaller, funnier lever too — how you say it. Prompts collect filler the way any writing does. 'Please could you kindly go ahead and analyze the following sentence for me' carries about three tokens of actual instruction. Trim the padding, ask for terse output while you're at it, and both sides of the exchange get lighter.
| Padded prompt | Trimmed prompt |
|---|---|
| Please could you kindly analyze the following Japanese sentence and provide the reading along with any grammar notes | Analyze this sentence. Return the reading and grammar notes. |
There's a tool that leans all the way into this, called caveman, whose whole philosophy is 'Caveman no make brain smaller. Caveman make mouth smaller' — keep the thinking, cut the chatter. Be honest about the ceiling, though: trimming words is polish, not foundation. It can even add a little overhead of its own, and it won't touch the model's reasoning tokens. The four moves above are where the real money is. Caveman-speak is the wax after the wash.
The cheapest token is the one you never send
Put the four moves together and they rhyme, because each is a way of not sending the model something. Route data around it instead of through it. Hand deterministic work to code. Point cheap blocks at cheap models. Give the output nowhere to wander. The furigana bill taught us that the expensive way: the model will happily charge you to do work it was never the right tool for, and it won't warn you. You have to go looking.
If you want the wider discipline this sits inside — treating prompts as versioned artifacts, measuring every change against a suite of real cases, shipping the exact pipeline you tested — that's what the AI engineering guide covers. Token optimization is one habit within it: build the pipeline so every token you spend is buying something only a model could give you.