Evaluating Language Models for Regulated Work: Accuracy, Auditability, Cost
In finance and professional services you can't ship an AI feature on vibes. Evaluating models for regulated work means measuring accuracy, auditability and cost as one system.
In a consumer app, an occasionally wrong AI answer is an annoyance. In finance, lending, or professional advice, it is a liability with someone's name attached. That difference changes how you choose and evaluate a model entirely. You cannot ship on the demo feeling that "it seems to work." You need evidence, and the evidence has three dimensions that have to be weighed together: accuracy, auditability, and cost.
This is how we think about evaluating language models for work that sits inside a regulated environment.
Accuracy has to be measured, not felt
The first discipline is refusing to trust impressions. A model that answers ten test questions well feels accurate; that feeling tells you almost nothing about how it performs across the thousands of real variations it will meet in production. Replacing the feeling with a measurement means building an evaluation set — a collection of representative inputs with known correct outputs — and scoring the model against it systematically.
Building that set is the actual work, and it is domain work, not model work. It requires someone who understands the subject deeply enough to say what a correct answer is across the messy range of real cases, including the edge cases where being wrong is most expensive. A generic benchmark cannot do this for you, because your risk lives in the specifics of your domain. The eval set is where that risk gets made visible and measurable, and a team that hasn't built one is flying on vibes no matter how sophisticated the rest of the stack looks.
Auditability is not optional in regulated work
In a regulated setting, being right is not enough; you have to be able to show why the system produced a given answer. If a decision is questioned — by a client, a regulator, or your own compliance function — "the AI said so" is not a defensible position. The system has to leave a trail: what inputs it saw, what sources it drew on, what it produced, and when.
This reframes the model-selection question. A model that is marginally more accurate but opaque may be a worse choice than one slightly less accurate but fully traceable, because in regulated work an unexplainable decision is a decision you cannot stand behind. Auditability has to be designed in from the start — retrofitting a trail onto a system that wasn't built to keep one is painful and usually incomplete. The best time to decide you need traceability is before you've built anything, not after a decision gets challenged.
Cost is a real constraint, and bigger is not free
The third dimension is the one that quietly decides whether a system is viable at scale. Frontier models are expensive to run, and cost compounds with every request. A feature that is comfortably affordable in a demo, run a handful of times, can become uneconomic when it runs thousands of times a day across a real user base.
The disciplined question is not "which model is best" but "which model is good enough for this specific task at a cost that scales." Frequently the honest answer is a smaller, cheaper model applied to a tightly scoped problem, rather than a frontier model applied to everything. That is not a compromise; it is engineering. Spending frontier-model money on a task a smaller model handles reliably is waste dressed up as quality.
The three trade off against each other
The reason these three dimensions have to be evaluated together is that they pull against one another. The most accurate model may be the most expensive and the least transparent. The most auditable setup may add latency and cost. Optimising any one in isolation produces a system that fails on the others — accurate but unaffordable, cheap but opaque, traceable but slow.
Evaluation, done properly, is the process of finding the point where all three are acceptable for the specific job, in the specific domain, at the specific scale you operate. It is slower and less exciting than picking the model with the best headline benchmark. But in regulated work it is the only responsible way to choose, because the cost of choosing on impressions is not a bad demo — it is a decision you have to defend later, to someone who is entitled to ask.