Decision Models in Hugging Face

Large Language Models excel at generating text but are not always the best tool for structured decisions such as routing requests or evaluating urgency. Decision models address this by evaluating predefined options and returning a probability distribution over them, avoiding token-by-token generation. This approach, inspired by TypeSafe AI’s Jev and the System One concept, shifts focus from generating answers to assigning probabilities to known choices. The community has adopted the term 'decision model' to describe this computational role.

Decision models differ from traditional classification by allowing output categories to be defined at inference time rather than being fixed during training. Unlike zero-shot classification, which uses natural language inference to match inputs to hypotheses, decision models support multiple question types—choice, scoring, or probability estimation—within a unified interface. This flexibility enables applications to handle dynamic categories without retraining the model.

The interface of decision models centers on a structured state input and named questions about that state, with three fundamental primitives: Choice, Score, and Noul. Choice operates like traditional classification, determining which predefined option best fits the input. Score evaluates an ordered scale, while Noul estimates the probability of a statement being true. This design organizes probabilistic classification into a consistent framework.

The state input for decision models can be a string, structured JSON, or other modalities for supported omni-modal models. Questions describe what the model should evaluate, such as which department should handle a customer request. By returning probabilities over predefined options, decision models provide a reliable way to automate structured decisions without relying on text generation.