Abhi Kumar on AI Costs, Reliability and Outcomes and Making AI Earn Its Place in the Enterprise: A Serious Insights Interview

An AI system can get cheaper to run but cost more to trust. A smaller model saves little if it prolongs a customer’s call, sends more work to employees, or completes the wrong task with confidence. In this Serious Insights interview, Voicing AI co-founder Abhi Kumar challenges the assumption that lower model costs translate into better business results. Starting with Uber’s efforts to separate AI usage growth from spending growth, he makes a distinction enterprise leaders cannot afford to miss: controlling costs is an engineering achievement, not a procurement shortcut—and the savings must survive scrutiny beyond the AI budget.

The conversation gets especially revealing when we move from coding assistants to live customer interactions, where an answer cannot be unheard, and a mistake can affect someone’s money, healthcare, or travel. Kumar explains why escalation should come before model routing, what enterprises need to own to preserve their ability to change vendors, and when he would tell a CIO not to scale. He also faces questions about his own company’s accuracy claims, acknowledging that “zero hallucinations” should not have appeared on its website. The discussion is practical, unusually candid, and succinct about how to distinguish AI that costs less from AI that earns its place in the enterprise.

Top takeaways from the interview:

• Measure outcomes, not just usage costs. Kumar argues that cost per correctly completed business outcome is the meaningful business metric. Lower token or session costs can conceal longer conversations, poorer service, and work shifted to human employees.
• Design for uncertainty before routing for savings. Smaller models need clear boundaries, and uncertain or sensitive requests need explicit escalation paths. In Kumar’s approach, uncertainty triggers escalation—not another attempt to make an inadequate model succeed.
• Demand evidence you control—and know when not to scale. Build versioned evaluations from real interactions, scrutinize the methodology behind vendor claims, and retain ownership of evaluation data and portability assets. Kumar’s warning sign for stopping expansion: unpredictable failures that the team cannot explain.

The Abhi Kumar interview

Uber says agent requests increased 9.4 times while spending “relatively stabilized since April.” That sounds more like a cost correction than proof that usage became inexpensive. What does the Uber example genuinely tell us about enterprise adoption, and what would be the wrong lesson to draw from it?

• Your read is right. It’s a cost correction, not a discovery that AI got cheap.
• What it proves: usage and spend can be decoupled. That’s it. That’s still a big deal because most people assume they’re locked together.
• What it doesn’t prove: that the decoupling is free. Someone spent months of engineering time on it. That’s a cost; it just doesn’t show up in the AI line item.
• Wrong lesson to draw: “we can 10x usage at no cost.” Right lesson: “our spend curve is a choice, not a law.”
• The other wrong lesson is thinking it’s replicable by procurement. It isn’t. It’s an engineering outcome.

The pitch makes routing sound like the answer, but Uber also changed context limits, caching, tool loading, batching, grounding, and spend visibility. It describes dynamic routing as work still in progress. How much of the result can reasonably be credited to model selection?

• Honestly? A minority. I’d guess caching and context limits did more of the heavy lifting, and I’d say that even though routing is my argument.
• Caching is the one people underrate. Deduplicated requests are free requests. In a lot of enterprise traffic, the repetition rate is embarrassing.
• Uber themselves say dynamic routing is still in progress. Worth taking that at face value rather than reading backward from the result.
• Where I’d defend routing: it’s the most durable lever, not the biggest one. Caching gets you a step change once. Routing keeps paying as your traffic mix shifts and as new models land.
• The real finding isn’t any single technique. It’s that they were measuring closely enough to know which lever moved what. Most teams can’t attribute their own savings.

Uber’s example comes largely from software development. You work with customer-facing voice systems in banking, healthcare, aviation, and telecommunications. Which parts of Uber’s experience transfer to those environments, and where does the analogy break down?

• What transfers: the measurement discipline. Tiering as a concept. The idea that most requests are simpler than your architecture assumes.
• What doesn’t transfer: retry economics. In software dev, a wrong answer costs you a retry and a few seconds. Tests catch it. A human reviews the diff.
• On a live call, there’s no retry. The customer heard it. You’re mid-conversation with someone’s mortgage.
• Latency budget is completely different too. A coding agent can think for 30 seconds. On a call, 800ms of silence and the person starts talking over you.
• And there’s no equivalent of a regulator in a dev workflow. Nobody audits your pull requests for fair lending compliance.
• So the tiering logic carries over. The tolerance for being wrong doesn’t.

You say every task does not need the biggest model. Fair enough—but who decides that a task is simple or low-risk enough for a smaller model? What signals allow the system to downgrade, and what sends the task back to a more capable model?

• Not a human deciding case by case. A classifier deciding, against rules a human wrote once and reviews.
• Three inputs to the downgrade decision: what’s the intent, is the action reversible, does it touch regulated data. Any one of those trips and it doesn’t go small.
• Signals that send it back up: low confidence on intent, schema validation failure on a function call, the caller rephrasing or repeating themselves, an intent we don’t recognize, sentiment going the wrong way, anything involving an amount above a threshold.
• The important design choice is the default. Uncertainty escalates. It doesn’t guess, and it doesn’t ask the small model to try harder.
• Escalation has to be invisible to the caller. If the handoff is audible, you’ve traded cost for experience, and that’s a bad trade.
• Build this before you build the router. Everyone does it the other way round, and it’s why routing gets a bad name.

A smaller model may be cheaper per token but more expensive overall if it requires additional turns, retries, corrections, or human intervention. Should enterprises measure cost per token, cost per session, or cost per correctly completed business outcome?

• Cost per correctly completed business outcome. It’s the only one tied to whether the thing worked.
• The other two aren’t useless; they’re just for different people. Cost per token is an infra metric. Cost per session is a capacity planning metric. Neither should reach the board.
• You’ve put your finger on the actual trap: a cheaper model that takes four turns instead of two isn’t cheaper. It’s more expensive, and it’s also worse.
• Voice adds a wrinkle software doesn’t have. Extra turns cost the customer time, not just you tokens. That shows up in satisfaction before it shows up in spend.
• I’d track a fourth thing alongside it: escalation rate. If containment is falling while cost per session drops, you didn’t save money; you moved it to headcount.

Model performance changes quickly, so a routing benchmark can become stale almost immediately. How should a company build an evaluation set from its real work, and how often should it retest its model choices?

• Build the set from your own transcripts. Not synthetic, not a public benchmark. A few hundred real interactions, labeled for correct outcome.
• Oversample the failures. The set should be roughly a third things that went wrong, because those are what actually discriminate between models.
• Freeze it. Version it. It becomes a fixed reference you re-run, not something you rebuild every quarter or you lose comparability.
• Refresh cadence: add new cases monthly as your traffic shifts, re-run the whole set on every model version change, full re-baseline quarterly.
• Pin model versions in production. Silent upgrades break routing assumptions, and you find out from a customer.
• And keep the set in your own hands, not the vendor’s. An eval you can’t take with you isn’t leverage.

In a regulated workflow, a cheaper model that is “mostly right” may still be unacceptable. Which decisions should never be dynamically routed, and where must human approval, escalation, or rollback remain mandatory?

• Anything irreversible. Money leaving an account, a policy canceled, a prescription instruction, a flight changed.
• Anything that discloses PII back to a caller who hasn’t been fully authenticated.
• Anything carrying a mandatory disclosure — adverse action notices, recording consent, medical triage language. Those are scripted, or they’re a violation.
• Anything where the caller is flagged as vulnerable, or has said something that suggests distress. That routes to a human, not to a bigger model.
• Complaints. The moment someone says they want to complain, it’s a person. Regulated industries treat complaint handling as its own regime, and AI shouldn’t be inside it.
• Mandatory human approval on all of the above, plus a rollback path and an immutable audit log of what the system did and which model did it. If you can’t reconstruct the decision afterward, you can’t deploy it in a bank.

Voicing AI’s website advertises 97% function-calling accuracy and zero hallucinations. How do you define those measures, what are the denominators, and were the results produced in live customer interactions or controlled evaluations?

• Fair question, and I’ll answer it directly rather than defend the marketing copy.
• The 97% comes from internal evaluation on our 70B variant, run against a curated set of function-calling tasks. Controlled conditions, not live customer traffic. It should be labeled that way on the site, and it wasn’t.
• “Zero hallucinations” shouldn’t be on there at all. It isn’t a measurable claim, and no honest vendor can make it. We’re pulling it.
• What I’ll stand behind, with the methodology attached: average resolution rate around 87% across deployments, and MOS 4.7 for our voice quality.
• Broader point, and I’d say it about the whole category: most vendor accuracy numbers are controlled evals presented without a denominator. If a buyer can’t see the task set and the sample size, the number is decoration.
• We got this wrong on our own site. Fixing it is not a good look, but leaving it up would be worse.

Voicing AI says it can operate entirely within a customer’s infrastructure without third-party model APIs. Does that make model routing easier or harder, and how does the customer preserve model portability instead of exchanging one form of vendor lock-in for another?

• Routing gets easier. You control the whole fleet, so you can move traffic between tiers without renegotiating anything or waiting on someone’s rate limit.
• Portability gets harder unless you design against it deliberately, and you’re right to push on that.
• What we do: OpenAI-compatible interface, so swapping a model doesn’t mean rewriting the integration. Standard schemas for function calls. Nothing proprietary in the prompt format.
• The customer keeps their eval set, their data, and any adapters trained on it. Those are the assets. If they can take the eval set elsewhere and run it, they can leave.
• Honest test of any vendor on this: ask what happens to your fine-tuned weights and your evaluation data if you terminate. If the answer is vague, that’s your answer.
• And I’d say on-prem is the weaker form of lock-in anyway. You have the thing running in your own environment. That’s a much better position than an API you can’t inspect.

Suppose a CIO gives you 90 days to demonstrate that this approach works. What workload would you begin with, what would you measure, what reliability floor would you establish, and what result would make you recommend against scaling it?

• Start somewhere boring and high-volume. Balance enquiries, appointment confirmations, order status. Read-only or trivially reversible.
• Explicitly not: anything that moves money, in the first 90 days. Not because it can’t work, because you haven’t earned the audit trail yet.
• Measure four things. Correct-outcome rate against human review of a sample. Containment. Escalation rate and whether escalations are clean. Cost per correctly completed outcome.
• Reliability floor: correct-outcome rate on the sampled review has to beat the human baseline for the same workload, not an abstract target. If it can’t beat the team you already have, it isn’t ready.
• What would make me say don’t scale: if the failures are unpredictable rather than clustered. Clustered failures are a fix. Scattered ones mean you don’t understand the system, and scaling multiplies something you can’t see.
• Second thing that would stop me: if escalations are going up while containment looks fine. That means it’s completing things badly, and that’s worse than not completing them.
• I’d rather lose a deal at day 90 than have a bank find that out at month nine.

About Abhi Kumar

Abhi Kumar is the Co-Founder of Voicing AI, an enterprise voice AI company focused on helping organizations deploy reliable AI solutions in regulated industries. His background spans enterprise software and venture capital, including roles at Oracle, Microsoft, and M12, Microsoft’s venture arm, where he evaluated and invested in enterprise technology companies globally.

Through his experience assessing which technologies achieved enterprise adoption, Abhi developed a deep understanding of the challenges organizations face when implementing AI at scale. Today, he focuses on bridging the gap between AI innovation and production reality by building voice AI solutions designed for real-world enterprise environments, including banking, healthcare, aviation, and telecommunications.

For more serious insights on AI, click here.

For more serious insights on learning, click here.

Did you enjoy the Abhi Kumar interview? If so, like, share, or comment. Thank you!