Everyone's adopting AI agents. Almost no one can prove the ROI. Here's why
Companies adopt AI agents rapidly but struggle to prove ROI due to misaligned KPIs and lack of business context, according to Databricks’ Amber Roberts.
Amber Roberts, Databricks’ Staff Technical Marketing Manager, argues that traditional productivity metrics like speed fail to capture the value of AI agents, which vary by team and individual needs. She notes that KPIs such as development speed or inference latency differ across roles, making a single productivity measure impractical. Roberts emphasizes that AI’s impact depends on context, expertise, and the specific tasks being automated, rather than a universal benchmark.
Roberts highlights a disconnect between AI buyers and users, where competitive pressure drives adoption without immediate ROI. She describes a trial-and-error phase where companies assess effectiveness over six months to years, often prioritizing staying ahead of competitors over measurable returns. This gap reflects broader challenges in aligning AI investments with actual business outcomes.
Roberts redefines productivity in the AI era, shifting from individual output to leveraging AI assistants to scale expertise. She explains that while AI automates repetitive tasks, it cannot replace judgment or deep subject-matter knowledge. The new baseline measures how effectively professionals apply their expertise with AI support, rather than raw production volume.
Roberts stresses that business context is critical for AI agents to deliver value, comparing them to human assistants who need institutional knowledge. She warns that generic AI tools lack the specificity required for company-specific tasks, such as local regulations or internal processes. Without this context, AI agents risk producing irrelevant or low-quality outputs, undermining their productivity claims.