How We Cut Inference Spend by Right-Sizing Our Models

Salesforce’s Agentforce previously relied on a single rented model for all tasks, leading to rising inference costs that grew linearly with usage. Instead of passing costs to customers, the company rebuilt its architecture to use targeted open-source models for distinct functions such as intent detection, safety screening, and response evaluation. This approach reduced expenses while improving efficiency and control over the agentic system’s performance and roadmap.

The company deployed the HyperClassifier, a fine-tuned model for intent detection and routing, which operates within the Einstein Trust Layer. It classifies requests across up to 200 labels and routes them to the appropriate subagent in approximately 26 milliseconds—55 times faster than a general frontier model. Accuracy for safety topic detection improved from 95% to 99%, and drift in multi-turn conversations decreased from 20% to 10%, enhancing both speed and reliability for enterprise workflows.

To address security risks, Salesforce introduced a dedicated injection-detection model trained on common attack surfaces such as CRM fields and email bodies. The model classifies six attack vectors—including prompt leakage and privilege escalation—and assigns an injection score logged in an audit trail. A specialized toxicity-screening model was also implemented to evaluate responses for harmful content, mitigating the tendency of LLMs to overlook their own biases or errors in self-assessment.

For response validation, Salesforce developed TextEval, a 20-billion-parameter model that checks whether answers are grounded in real data, properly cited, and aligned with instructions. The model evaluates instruction adherence and task resolution with higher accuracy than general models judging their own output. Additionally, a text ranking model was migrated to Amazon Bedrock to improve retrieval reranking, ensuring the right data reaches the reasoning engine efficiently across global deployments.