Where Jev fits in an agent loop

01Three jobs, one of them is a decision

A language model inside an agent loop is doing three different jobs with one mechanism, and the jobs have nothing in common except that they currently use the same interface. The first is choosing what to do next: which tool, which skill, which parameters. The second is checking whether something is allowed: does this draft break a rule, is this call safe to make. The third is producing content: write the reply, write the code, write the summary.

Only the first two are decisions. Their answer spaces are closed — the tools exist, the rules exist — and the answer is short. The third is generation and stays generation. The whole argument for putting a decision model in an agent loop is that two of the three jobs never needed a text generator, and paying generation prices for them is what makes agents feel slow.

prompt ──▶ [ Jev: which skill / tool / params ] ──▶ agent executes
                                              │
                            uncertain ────────┴──▶ chat model (the tail only)

02The tail is the whole economics

The pipeline that shows up repeatedly in the case list is a two-stage one: classify with a decision model, then send only the low-confidence remainder to something larger. One build classified a hundred emails in 1.42 seconds, escalated the uncertain ones to a frontier chat model, and reported 96 correct out of 100 for roughly seven cents.

The cost of that design is set almost entirely by the size of the tail, not by the price of either model. If ninety percent of calls land above your threshold, the expensive stage barely runs and the decision model is doing the work. If thirty percent land below it, your pipeline spends most of its money on the thing you were trying to avoid, and the decision model is acting as a filter. Those are different systems with the same code, and the share is the number that tells you which one you built.

Which is why threshold choice deserves its own measurement rather than a guess. In our own task set the share of decisions scoring below 0.90 differed by task — from under five percent on one classification task to nearly half on the hardest one. A single global threshold would have been wrong for at least one of them.

The cheap way to find your threshold before committing to the architecture is to take a few hundred real inputs, run them through the decision model once each, and look at the histogram of confidences rather than at the answers. If the distribution is bimodal — most decisions near one and a thin scattering in the middle — a threshold will separate cleanly and the two-stage design is worth building. If it is broad and flat, there is no cutoff that keeps the tail small without discarding good decisions, and you are better off either narrowing the question or accepting that everything needs the larger model.

03A routing step you can actually instrument

The version that holds up keeps the routing decision separate from the execution and logs both. That way a wrong route is a row in a table rather than an unexplained bad answer twenty seconds later.

const route = await jev({
  state: { prompt, available_tools: tools.map(t => ({ name: t.name, summary: t.summary })) },
  questions: {
    tool:   { type: 'choice', criteria: Object.fromEntries(tools.map(t => [t.name, t.summary])) },
    params: { type: 'choice', criteria: paramPresets },   // or a score per parameter
  },
})
log({ prompt, route, correct: await laterJudge(route) })   // routing needs its own eval set
if (route.tool.confidence < THRESHOLD) return escalate(prompt)   // the tail, explicitly

04Failure modes worth designing for up front

A wrong routing decision is invisible until output degrades. The agent does not crash; it calls the second-best tool, or the right tool with the wrong parameters, and the answer that comes back is plausible and wrong. There is no error to catch, which is why routing needs its own labelled set and its own accuracy number rather than being folded into a general “does the agent work” impression.

Adding a routing step adds a round trip. If the decision model is called sequentially before every tool call, you have inserted latency into the hot path — the one thing the design was supposed to remove. The builds that report speedups either make that call concurrent with something else, or route on a coarser boundary than every single step.

Guardrails have the opposite failure. A check that never fires is indistinguishable from a check that works, so a guardrail needs a set of inputs that should be rejected, and it needs that set to be run on every change. Without it you have added a component whose only observable behaviour is costing money.

And the last one is the quietest: once a model is selecting tools, the tool descriptions become the interface. Two tools whose summaries look similar will be confused, and the fix is in how you write the summaries, not in the threshold. Treat the criteria map as API documentation, because that is what it is.

There is a useful way to tell which side of the line a job falls on: ask whether you could write a test for the answer. “Which tool handles this prompt” can be tested against a labelled set. “Write a good reply” cannot, at least not in a way that survives a change of tone or audience. The jobs you can test are the ones a decision model can own, and the ones you cannot are the ones that need generation and a human reading the result.

05Where the rest of the pattern goes

This page is the general case. The browser-agent page is the same loop with a better observation step, the pre-install gate page is the same idea narrowed to a single allow-or-refuse, and the document classification page is the tail-routing design applied to a queue instead of a conversation.