Decision Models

So the AI Community found a new shiny toy. This time, it’s called “Decision Models”. Maybe you’ve heard of Jev or any of its many competitors.

Except this time, the shiny toy might actually be something real.

Because a lot of the work that happens inside a business is not about writing prose (even though “summarize this email” is still one of the most popular use cases).

There’s an internet meme saying an average human makes about 35,000 decisions a day. Guess how many the average business makes.

And even if most of these decisions are cheap and happen within seconds – in total they’re still expensive in terms of costs, delays, or risk.

That’s where decision models come in. This article is for you if you want to wrap your head around them and learn how to turn them into a benefit for your business.

Let’s dive in!

Decision models – at least in the form as they got popularized by Jev – are essentially LLMs that are tuned to pick an answer from a list you provide, instead of writing a text from scratch.

You give it context, a question or task, and the possible answer options and it returns the appropriate answer along with a score for all options.

Example:

You provide a decision model with a customer email.

The customer says:

Then you provide the following question:

Q: Which team should handle this request?

Along with 3 answer options:

A. Billing
B. Technical
C. Sales

Then Jev would return something like this:

(Feel free to try it out using this practical Jev playground here.)

Decision models promise to deliver similar accuracy to frontier models at a fraction of the cost and with much higher speed.

For example, 1 million decisions that cost about $110 with OpenAI’s cheapest non-decision model GPT-6 Luna would cost only $30 with Jev.

Routing a support ticket can take ~2 seconds with Luna, but just 0.25 seconds with Jev.

Magic?

Under the hood, decision models are generally still LLMs.

Andriy Burkov explained it quite well. Essentially, there are two points to understand:

1) Speed and quality trade-off. The time an LLM needs to generate its first token is the compute it spends on the decision. Given two models of similar size, the one that finishes faster did less thinking. That’s why GPT-6 Luna (red) still outperforms Jev (blue) on all decision categories in terms of decision accuracy:

But if you’re looking at speed and cost, decision models like Jev are ahead:

2) Any regular LLM can do the same trick. Just give it one request per option and a shared cached prompt, and you read back only the first token: a number from 0 to 9. A softmax turns those numbers into probabilities (which are more or less reliable, depending on the task at hand).

So a decision model is not a new kind of AI. It’s an LLM wrapped in a practical package.

Which means anyone can build it.

Anyone did.

After Jev got the internet crazy, here’s a list of popular contenders that were released shortly afterwards:

So who’s best? Depends on who measures.

Artificial Analysis got a dedicated Decision Index set up that tests decision models across knowledge, reasoning, tool calling, and classification tasks:

Interestingly, Anthropic’s smallest “regular” model Haiku 5.5 scores 3 points above Jev overall. But it also takes 11 times longer and costs 4.4 times as much. So you decide whether that difference is worth it.

Anyway, the bottom line of these benchmarks is that you want to treat them as directional advice, not absolute.

What matters is how these models perform on your data.

And data is probably the biggest hurdle right now.

Because decision models work on exactly the data you’d rather keep close. Invoices, customer emails, contracts, payment transactions, you name it. And I don’t see many companies suddenly onboard an entirely new AI lab they've never heard of before if they can get (more or less) the same performance from the vendor they’re using right now, or – even better – without any vendor, using a model they run by themselves.

Because open models don’t have that problem as they can process data completely offline – on your own hardware.

So the question isn’t “which model wins”. It’s “which accuracy do I need, and what do I want to pay for it?”

Before you get too excited about decision models like Jev & co, here’s some quick napkin math:

(Illustrative numbers. Plug in your own.)

The point is that for many decisions, the model bill itself is a rounding error. The escalation or error rate is where the costs are.

And accuracy can be a tricky metric.

Imagine that out of every 100 customer support tickets you’re processing, only 2 are really urgent. Model A marks everything as “not urgent”. That gives you 98% accuracy, but both urgent tickets are missed. Model B catches both urgent cases but flags 5 routine ones by mistake: The result is 95% accuracy. Which model would you prefer? The one with the lower accuracy – Model B!

So count false positives and false negatives separately. What matters is how well the model balanced them and what kind of error costs what for your business. Those numbers will tell you more than any generic benchmark.

There are 5 magic decision words that will tell you whether you need “decision making” as an AI mode for your business. Look for tasks that need verbs like:

Here are a few common examples from popular business categories:

You can see that the pattern here is always the same. It's a decision model at work.

Do you need a decision model or not?

Well, that’s up to you to decide.

Your answer depends on how many decisions you need to make, whether you prioritize speed above accuracy, and – above all – whether you’re actually ready to hand off the decision to an AI.

Which is a topic for another day.

See you next Saturday,
Tobias