Grading calls with your own LLM judge, and what an examiner will ask

Most teams running AI voice agents already run an LLM judge of some kind: a prompt that reads a transcript and returns met or missed for each item on a checklist. It's cheap and fast, and on clear-cut rules it's often right. The trouble starts when someone outside engineering relies on the output, such as a compliance examiner, a client's vendor-audit team or your own counsel.

Where does an in-house LLM judge work well for call QA?

An in-house judge works well for triage, weekly trends and rules with plain textual evidence, and it struggles once someone outside engineering has to rely on its verdicts. An LLM judge is a language model prompted to grade another system's output against written criteria. The best-known evidence for the method is the MT-Bench study, where GPT-4's verdicts on open-ended chat answers agreed with human experts 85% of the time with ties set aside, above the 81% the experts reached with each other (Zheng et al., NeurIPS 2023). That result is about which of two chat answers people prefer, not about compliance rules, but it's a real result.

For call QA, a judge earns its keep in three places. It reads every call and surfaces the few a reviewer should hear. It shows a rule's miss rate moving week to week. And it handles rules with plain textual evidence, like whether the agent said it was a debt collector. Keep yours running for those. Where it struggles, the research explains why.

What does research say about LLM judge reliability and bias?

Research finds five recurring weaknesses: judges are swayed by answer order, answer length and their own outputs, they often disagree with subject matter experts, and they struggle to label compliance violations turn by turn.

• Order: in the MT-Bench study, GPT-4 gave the same verdict when two answers swapped places only 65% of the time. Wang et al. made Vicuna-13B beat ChatGPT on 66 of 80 queries, with ChatGPT as judge, just by reordering the answers (Wang et al., 2023).
• Length: padding an answer with rephrased list items fooled Claude-v1 and GPT-3.5 in 91.3% of cases and GPT-4 in 8.7% (Zheng et al.).
• Self-preference: judges can recognize their own outputs, and the better they recognize them, the more they favor them (Panickssery, Bowman and Feng, 2024). That matters if one model family runs your agent and grades it.
• Expertise: subject matter experts agreed with LLM judges 68% of the time in dietetics and 64% in mental health (Szymanski et al., 2024). Across 20 annotation tasks, JUDGE-BENCH found reliability varied widely and concluded that judges "should be carefully validated against human judgments" (Bavaresco et al., ACL 2025).
• Compliance: CompliBench tested judges on 318 synthetic multi-turn dialogues in airline, healthcare and insurance with injected guideline violations. Gemini-3-pro, the top general model in two of the three domains, labeled every turn correctly in 49% to 57% of conversations, and GPT-5 in 28% to 47% (Yang et al., 2026). A 2026 study of voice-agent judges found reliability "metric- and configuration-dependent rather than uniform" (Purwar, Singh and Srivastava).

None of this says don't use a judge. It says a verdict is a measurement with error, and an examiner will want to know how big the error is and who controls it. The next five questions are the ones to prepare for.

Who wrote each rule, and who approved it?

In most in-house setups an engineer wrote each rule into a prompt and nobody formally approved it, yet the first question about any verdict in an exam is where the rule came from. The prompt keeps changing, too. Shankar et al. named the pattern "criteria drift": "users need criteria to grade outputs, but grading outputs helps users define criteria" (Shankar et al., 2024). That's healthy while you design a rubric and a problem in an exam. What holds up:

• Each rule has a number, its exact text and the standard operating procedure (SOP) line or regulation it came from, such as 12 CFR 1006.18(e) for the debt collection disclosure.
• A named compliance owner approved it on a date, and its text never changes without a new rule version.
• The prompt the judge sees is built from the approved text, not edited by hand.

Which rule version and grader version produced a verdict?

You can answer only if every verdict stores its rule version and its grader version (model ID and prompt hash), because the rule text, the prompt and the model all change. Models don't stay put. Anthropic retires models with at least 60 days' notice for publicly released ones, and OpenAI gives at least six months for generally available models (Anthropic, OpenAI). Pinning sampling settings isn't a full answer either: on Anthropic's API, a non-default temperature, top_p or top_k on Claude Opus 4.7 and later models returns an error (Anthropic).

When the grader changes, regrade a reference set of calls under both versions before trusting any trend line across the change. It's the same reason a fair release comparison needs one grader version on both sides. If verdicts are part of how you show compliance, plan to keep them and their versions as long as the underlying records. Regulation F, the Consumer Financial Protection Bureau's debt collection rule, requires collectors to retain records that evidence compliance or noncompliance until three years after the last collection activity on a debt (12 CFR 1006.100).

Can every verdict point to evidence, and what if there isn't any?

Every verdict should point to a quoted line or a tool record, and when the evidence can't settle a rule, the judge should say so instead of guessing. An explanation is not evidence. Turpin et al. found chain-of-thought explanations "can systematically misrepresent the true reason for a model's prediction," with hidden biasing features cutting accuracy by as much as 36% on BIG-Bench Hard tasks while the explanations didn't acknowledge them (Turpin et al., NeurIPS 2023). Ask the judge for a quoted line and a timestamp instead, then check in code that the quote appears in the transcript. A verdict whose quote isn't there is a failed verdict, not a missed rule. Read tool-call rules from the tool log, not from what the agent said about the tool.

Some rules a transcript can't settle, in two different ways. Speech-to-text drops the one word that matters, so a person has to hear that stretch of audio. Other rules need records a transcript never holds: calling hours need the call time and the consumer's time zones, and call-frequency rules under Regulation F and consent rules under the Telephone Consumer Protection Act (TCPA) need the dial log and consent records (see TCPA consent for AI voice calls). A judge forced to choose met or missed will guess at both. Give it two more verdicts: "needs review" for a stretch a person must hear, and "not checked" for a rule whose records weren't sent, which never counts as passed or failed. Track how often each rule lands in each.

Agent: Hi, this is Maya, an automated assistant calling for Quillfeather Services on a recorded line. Am I speaking with Daniel Reyes?

Customer: Yes, that's me.

1.1 met · Opening: business named and recording announced in the first turn

Agent: Thanks. Please confirm your date of birth.

Customer: March 4, 1986.

1.2 met · Right party: verify_identity returned a match at 00:14

Agent: Thank you, Daniel. Quillfeather Services is a debt collector. This is an attempt to collect a debt, and any information obtained will be used for that purpose. Your balance with Alder Card is $1,284.16. Can you pay that today?

2.3 met · Disclosure: quoted at 00:19, "Quillfeather Services is a debt collector", before the balance at 00:26

3.2 missed · Payment plan: full balance requested at 00:27 with no plan offered first

Customer: I [inaudible] owe all of that. I could pay something on the 3rd.

4.4 needs review · Dispute: "I [inaudible] owe all of that" at 00:31 may be a dispute; a person must hear the audio

Agent: Understood. Would $107 on October 3 work?

Customer: Yeah, that works.

4.1 met · Promise logged: log_promise returned success at 00:48

5.1 not checked · Calling hours: needs the call time, the account's address and every number's area code

How often does the judge agree with your own reviewers?

You only know by measuring it: have QA reviewers grade a sample of calls without seeing the judge's verdicts, then compare rule by rule. Percent agreement is the obvious measure and the most misleading when misses are rare. Cohen's kappa is a statistic that measures agreement between two raters after removing the agreement expected by chance, where 0 is chance and 1 is perfect (McHugh, 2012). Thakur et al. found that judges "with high percent agreement can still assign vastly different scores" (2024). The number that matters most is the false pass: a call the judge passed and a reviewer failed. A false fail costs a reviewer a minute. A false pass is a violation nobody hears again.

Illustrative calibration for one rule:

Agreement is 97.7% and kappa is 0.63, with a 95% interval of 0.54 to 0.73 from the Fleiss, Cohen and Everitt large-sample standard error. The benchmarks of Landis and Koch (1977) call that substantial; McHugh's stricter scale for health research calls it moderate. Yet the judge passed 18 of the 60 calls reviewers failed: 30% of the real misses, with a 95% Wilson interval of 20% to 43%. A judge that passed every call would score 97.0% agreement and a kappa of 0. Report false passes by rule with the calls listed, and grade the same sample with two reviewers, since their agreement with each other shows how much disagreement comes from the rule itself rather than the judge.

Is the grader independent of the team that ships the agent?

In most in-house setups it isn't: the team that ships the agent also writes the judge's prompt, picks its model and sets its thresholds, so the release goes out when the number looks good and the number is theirs. Nobody has to act in bad faith for that to drift. NIST's voluntary AI Risk Management Framework says "processes for independent review can improve the effectiveness of testing and can mitigate internal biases and potential conflicts of interest," and asks for assessors who weren't front-line developers (MEASURE 1.3) (NIST AI 100-1). NIST says the framework is being revised.

Bank supervisors call this "effective challenge." In April 2026 the Federal Reserve, OCC and FDIC replaced SR 11-7 with revised model risk guidance that asks for challengers with "sufficient independence to maintain objectivity" (SR 26-2). The same guidance puts generative and agentic AI models outside its scope, and the agencies said they plan a request for information on AI (OCC Bulletin 2026-13). So how supervisors will treat an LLM judge isn't settled. The principle is.

In practice: compliance approves rule text, someone outside the agent team reviews grader changes, and the grader doesn't run on the agent's model. Grade a sample of human calls on the same rules too. The disclosure in 12 CFR 1006.18(e) is owed whoever speaks it, and "the agent misses 1.9%" means little without the human floor's number beside it. Loops was built as that outside grader: rules the customer's compliance lead approves, a quoted line behind every verdict, and human calls graded on the same rules (how it compares).

This is general information, not legal advice.

Start with calibration. Pull every call your reviewers have already scored, run your judge on them blind, and count the false passes for your five most important rules. That count tells you how much of your judge's output you could hand to an examiner today. For the wider program around it, see how to audit AI voice agent calls against your SOP, or get a second grader to compare against with Loops' free 30-day audit.