llama.cpp Now Runs Decision Models

Every agent loop hits the same wall: something has to pick one option out of a handful. Which queue does this ticket belong in, does this comment stay up, what does the agent do next. llama.cpp can now run models built for exactly that job.

The support is already there for five open models, and they span a huge range of sizes — from 144M up to 27B parameters — with more coming.

What Happened

llama.cpp added support for decision models. Instead of writing you a paragraph and leaving you to guess what it meant, these models take a typed question and hand back a probability for every option you offered.

The tasks the source names are the everyday ones: routing tickets, moderating content, and choosing an agent's next step. Five open models are supported so far, running from 144M to 27B parameters. More are on the way.

Why It Matters

A chat model gives you text, and then you write a parser to turn that text into a decision. Decision models skip the parsing step — the output is already a number per option, so your code can compare them and move on.

The size range is the other half of it. A 144M model and a 27B model are not the same machine's problem, so you can pick how much hardware this part of your pipeline is allowed to eat. For anyone already running llama.cpp locally, this is a new category of model to try without adding a new tool to the stack.