LLM-as-judge: AI evaluating AI

Second instalment in the series on testing AI systems. In the first we built the yardstick: golden dataset and test set. Here we deal with who holds it once volume grows, that is, a model evaluating another model. The person who stays in the loop is the subject of the third article.

The starting problem

You have shipped an assistant that answers questions about internal documentation. It works. Then you change the system prompt to handle an edge case a user reported, and you pause for a second before releasing.

How do you know you have not made the other two hundred answers worse?

The classic options do not work:

• Exact comparison: there is no single answer. "The endpoint requires a Bearer token" and "You need to pass a Bearer token in the Authorization header" are both correct and share almost nothing at string level.
• BLEU, ROUGE and friends: they count n-gram overlap. They tell you how much two texts resemble each other, not whether the second one is right. A perfect answer written with different words from the reference scores terribly. A wrong answer recycling the vocabulary of the context scores well.
• Human review: works beautifully, and is exactly what you cannot do on every commit across two hundred cases.

Hence the idea: if judgement requires understanding language, let us use something that understands language.

What it actually is

LLM-as-judge means using a language model as the evaluator of another system's output (or of its own). The judge receives:

• the original input (the user's question)
• the output to be evaluated (the system's answer)
• possibly a reference (the "gold" answer, or the context retrieved by the RAG)
• an explicit judgement criterion

and returns a verdict: a score, a pass/fail, or a preference between two alternatives.

Put like that it sounds trivial. The difference between a judge that works and one that produces noise dressed up as a metric lies entirely in the details of how you build it.

The main variants

Pointwise scoring

"Score the completeness of this answer from 1 to 5."

The most intuitive form and the most fragile. Models are terrible at calibrating numeric scales: ask for a score from 1 to 10 and you get a distribution squashed onto 7 and 8. The granularity you believe you have is an illusion: if your system goes from 7.8 to 7.9 after a change, you have learned nothing.

It works decently only if you anchor every level of the scale to a concrete behavioural description ("5 = answers the question and cites at least one source from the context; 3 = answers partially; 1 = does not answer or contradicts the context"). Without anchors, you are asking for an opinion.

Pairwise comparison

"Between answer A and answer B, which is better against this criterion?"

Far more reliable. The model does not have to place an output on an abstract scale, it only has to express a relative preference — a task it is significantly better at. It is the form of choice for comparing two versions of your system, two prompts, two models.

It has one notable defect (position bias, more on that below) which absolutely must be mitigated.

Rubric / binary criteria

Instead of asking "how good is this answer", you break the notion of quality into atomic yes/no questions:

• Does the answer contain claims unsupported by the provided context?
• Does the answer actually address the question asked?
• Does the answer cite at least one source?
• Does the answer contain advice of a legal nature?

Statistically this is the most robust approach. Each check is simple, verifiable, and when it fails you know exactly what went wrong. You then aggregate however you prefer: percentage of checks passed, or cascading logic (if check 1 fails, the case is rejected and there is no need to evaluate the rest).

The cost is the design work upstream: you have to actually decide what "good" means in your domain. That work is useful regardless.

Reference-based vs reference-free

With a reference: you compare the output against a gold answer. It requires an annotated dataset — the subject of the first article in this series — but the judgement is far more stable.

Without a reference: you evaluate the output in itself, or against the retrieved context. It is the only practicable option in production, where the questions are new and you have no gold answer.

In RAG the most useful case is hybrid: the "reference" is the retrieved context, and the criterion is groundedness — every claim in the answer must be supported by that context. This kind of judge is reliable because the task is almost mechanical: verifying an inclusion, not judging beauty.

G-Eval and variants

An approach that has settled in: have the judge produce an explicit chain of reasoning before the verdict, and — where you have access to logprobs — weight the score by token probabilities, obtaining a continuous value instead of a discrete step. It reduces the coarse-calibration problem.

The transferable point, even without logprobs: reasoning must be asked for before the verdict, never after. If you ask for the score first and the justification second, you get a post-hoc rationalisation.

Jury / panel

Several different judges vote, and you aggregate (majority, average, or escalation to a human on disagreement). It reduces the variance and the bias of any single model. It costs more — keep it for high-impact rubrics.

Agent-as-judge

The recent evolution for agentic systems: evaluating not only the final answer but the whole trajectory — which tools were called, in what order, with which parameters, how errors were recovered from. Rubrics designed for chatbots ("helpfulness", "fluency") are unsuited to an agent that has to chain APIs, and 2026 research is pushing towards rubrics generated for the specific task rather than fixed for the whole system.

When to use it (and when not to)

The practical rule is: do not use a judge where a deterministic check exists.

The mature pattern is a three-level hybrid: deterministic checks for everything mechanically measurable, a judge for what requires understanding, a human for the small percentage that either of the two flagged as doubtful.

The concrete use cases

• Regression testing in CI. The most solid case. You have two hundred test cases, you change something, the judge tells you whether you got worse. You do not need a true absolute number, you need a repeatable comparison.
• Comparing models or prompts. You have to choose between two vendors, or work out whether the cheaper model is enough. Pairwise over a fixed set.
• Filtering synthetic data. You generate a hundred thousand examples and keep the fraction the judge considers valid.
• RLAIF / rubrics-as-rewards. The judge produces the preference signal for alignment in place of human annotators. Here the judge does not measure, it trains — and its flaws become flaws of the final model.
• Production monitoring. You sample a percentage of real traffic and evaluate it asynchronously on faithfulness, tone, policy violations. You see problems go past before a customer reports them.
• Online guardrails. The judge blocks or forces regeneration before the user sees the answer. Here latency and cost matter a great deal, and you need a small, fast judge.

Stories from the field

The judge that rewarded whoever talked most

On an internal assistant for technical documentation we had a pointwise judge with a generic "answer quality" criterion, scale 1-5. As we iterated on the system prompt the average score climbed steadily: 3.4 → 3.7 → 3.9. Everything looked fine.

Then somebody read the answers.

They had become enormous. The model had learned — through us, since we were optimising on the score — to add preambles, recaps, context notes and disclaimers. The judge was rewarding verbosity because a long answer looks more complete. Real users, who wanted one line and a link, were unhappier than before.

The lesson: the score was climbing because we were optimising the metric, not the product. We replaced the single criterion with four binary checks, one of which was explicitly "does the answer contain information the question did not ask for?". The average score collapsed and went back to being informative.

The day we measured the judge

For months we used the judge's numbers in slides without ever asking whether they were true. Then we did the obvious thing: two hundred cases labelled by hand by two people, compared against the judge's verdict.

On the groundedness criterion agreement was excellent — it is an almost mechanical task. On the criterion "the answer is useful to the user" agreement was barely above chance. The judge was measuring something, but not what we thought, and not something it made sense to take decisions on.

The most valuable by-product was discovering that the two of us humans did not agree with each other on a fifth of the cases. Before asking a model to apply a criterion, the criterion has to exist. Half the value of the exercise was writing the operational definition of "useful".

The cost nobody had budgeted

A judge with chain-of-thought, on a top-tier model, on every production answer. On paper, "AI evaluating AI costs nothing". In practice the judge generated more output tokens than the system it was evaluating, because it reasoned before answering, and the line item had become comparable to the product's own.

The fix is obvious once you think about it: sampling (a fraction of live traffic), the large judge only in calibration runs, a small specialised judge in production, and — above all — the judge's cost tracked as a separate line item instead of hidden in the total.

The biases: the part that hurts

This section is the most important one. A judge is not a neutral instrument: it is a language model with all its systematic tendencies.

Position bias. In pairwise, the model tends to prefer the first (or the second) answer regardless of content. It is not random noise, it is systematic, and it varies from model to model. Mandatory mitigation: always evaluate both orderings. If the verdict flips when you swap A and B, the comparison is void and has to be treated as a tie or escalated.

Verbosity bias. Longer answers win, even at equal content. In benchmarks that test this effect explicitly, sensitivity to length often turns out stronger than sensitivity to position. Some models are more resistant than others and penalise filler, but none is immune. Mitigation: criteria that explicitly mention concision, and control of the length distribution across the groups you compare.

Self-preference / self-enhancement bias. A model tends to score its own outputs higher. The most convincing explanation in the literature is not narcissism but perplexity: models reward the texts they find more familiar, that is, more probable under their own distribution. Which means the bias also hits texts they did not generate but which are stylistically similar. Mitigation: a judge from a different family than the generator. Always.

Preference leakage. A more insidious variant: if you generated the synthetic data with model X and use as judge a model related to X, the judge is contaminated. The same holds between a "teacher" model and a distilled one.

Sycophancy. The judge believes what you write to it. If the text under evaluation confidently states that it cited a source, the judge tends to take it at face value without verifying. Invented citations get through.

Overconfidence. The confidence scores a judge declares are badly calibrated upwards. Do not treat them as probabilities.

Prompt sensitivity and drift. Rephrasing a criterion changes the results. Changing the judge model's version changes the results. Operational consequence: the judge prompt has to be versioned like code, and every change of judge model invalidates the metric history until you recalibrate.

Non-determinism. Same input, different verdicts. Temperature at zero helps but does not eliminate the problem.

Goodhart, or: the underlying problem

When a measure becomes a target, it ceases to be a good measure.

It holds for every metric, but with judges it is particularly vicious, for two reasons.

The first: the optimisation loop is extremely fast. Change the prompt, look at the score, change again. In a few hours you have run dozens of iterations of selection against the judge's weaknesses without noticing. You are not improving the system, you are doing directed evolution towards the evaluator's blind spots.

The second: when the judge produces the reward for training (RLAIF, rubrics-as-rewards), the model optimises directly against it, with far more force and patience than you have. Every exploitable shortcut will be exploited.

The only real antidote is keeping an independent channel of truth: a sample of human review that never enters the optimisation loop, and a periodic look at real answers. If the score goes up and users do not notice, the score is lying.

How to build a judge you trust

An operational checklist, in order of importance.

1. Evaluate the evaluator. This is the step almost everyone skips. You need 100-200 hand-labelled cases, ideally from two people, and a measure of agreement between judge and humans (accuracy, Cohen's kappa). The thresholds used in practice: below 0.6 kappa the judge is not usable for decisions; above 0.8 it is solid. Without that number you do not have a metric, you have a generator of reassuring figures.

2. Measure agreement among your humans too. If two internal reviewers do not agree, the criterion is ambiguous and no model will apply it stably. The judge cannot be more consistent than the definition you gave it.

3. Prefer binary and pairwise to numeric scales. A well-defined pass/fail is worth more than a vague 1-10.

4. Break "quality" into atomic, independent criteria, each with its own definition, positive examples and — above all — edge examples. The borderline examples are worth more than the obvious ones.

5. Reasoning before the verdict, structured output in JSON against a fixed schema. It does not only serve the quality of the judgement: it serves you, when you have to debug something that does not add up.

6. A judge from a different family than the generator.

7. Alternating order in pairwise. Non-negotiable.

8. Version the judge prompt, the model, the parameters. A change in any of the three invalidates historical comparisons.

9. Recalibrate periodically. Models get updated underneath you, your data changes, criteria wear out. A calibration round monthly, or at any significant change.

10. Sample, do not evaluate everything. In CI you can afford full coverage of the test set; in production a sample is more than enough.

11. Run the judge asynchronously, unless it is a blocking guardrail. Do not make the user pay the latency of your metric.

12. Track the judge's cost as a separate line item.

A minimal structure, just to fix the shape:

Provided context: {context}
Question: {question}
Answer to evaluate: {answer}

For each criterion, justify in one sentence and then give the verdict.
1. Is every factual claim in the answer supported by the context?
2. Does the answer introduce information absent from the context?
3. Does the answer address the question asked?

Reply only with:
{"reasoning": "...", "supported": bool, "hallucinated": bool, "relevant": bool}

Note what is not there: no request for an overall score, no scale, no vague adjective like "good" or "high-quality".

Where this is heading

Three directions worth watching.

Rubrics as reward. The shift from "the judge measures" to "the judge trains". Rubrics become the bridge between the language of company policy and a signal a machine can optimise. The flip side is that every ambiguity in the rubric becomes a structural flaw in the resulting model.

Hybrid standards with verifiable rewards. Where correctness is mechanically checkable (maths, code, tool use) deterministic verification is used; rubrics remain for the open-ended part. The combination beats either approach on its own.

Adaptive rubrics. Instead of applying the same three dimensions to any task, generating the criteria for the specific task. For an agent chaining APIs the sensible dimensions are call selection, parameter correctness and error recovery — none of which appears in a generic "helpfulness" rubric.

A colleague with well-documented quirks

LLM-as-judge is the right tool for measuring relative differences between versions of your system, at scale, at contained cost, on criteria no syntactic metric can capture.

It is the wrong tool if you treat it as an objective measure of absolute quality, if you have never compared it against real human judgement, or if you use it in place of a deterministic check you could have written in ten lines.

The judge is not the truth. It is a patient, tireless, very fast colleague, with a few well-documented quirks — and it has to be kept in check exactly as you would keep in check a colleague with those characteristics.

A judge still has to be compared against something: the golden dataset from the first article is that something, and without it there is nothing to calibrate against. In the third the person remains, and the judge does not replace them.

Sources

Papers

• Zheng et al. (2023), Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena — arXiv:2306.05685. The founding work: it identifies position, verbosity and self-enhancement bias, and measures agreement between judges and humans.
• Liu et al. (2023), G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment — arXiv:2303.16634. Chain-of-thought plus score weighting by token probabilities.
• Shi et al. (2024), Judging the Judges: A Systematic Investigation of Position Bias — arXiv:2406.07791. Shows that position bias is systematic, not random noise.
• Wataoka & Takahashi (2024), Self-Preference Bias in LLM-as-a-Judge — arXiv:2410.21819. The link between self-preference and perplexity.
• Li et al. (2025), Preference Leakage: A Contamination Problem in LLM-as-a-judge — arXiv:2502.01534. Contamination between the synthetic-data generator and the evaluator.
• Kim et al. (2024), Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models — arXiv:2405.01535 · repo. Open evaluators trained on user-defined rubrics.
• Tan et al. (2024), JudgeBench: A Benchmark for Evaluating LLM-based Judges — arXiv:2410.12784. A benchmark for evaluating the judges themselves.
• Lambert et al. (2024), RewardBench: Evaluating Reward Models for Language Modeling — arXiv:2403.13787.
• Park et al. (2024), OffsetBias: Leveraging Debiased Data for Tuning Evaluators — arXiv:2407.06551. Bias mitigation at the level of the judge's own training data.
• Koo et al. (2023), Benchmarking Cognitive Biases in Large Language Models as Evaluators — arXiv:2309.17012.

Articles and practical guides

• LLM-as-a-Judge in 2026: How It Works, When It Fails, and How to Calibrate — tables on calibration, kappa thresholds, costs and deployment patterns.
• LLM-as-Judge Best Practices: Calibration, Bias, and Cost — operational checklist.
• LLM-as-judge: A complete guide to evaluation best practices — overview of the methods and of validation against human baselines.
• What Is LLM-as-a-Judge? Definition and Best Practices — an introduction focused on structured output and rubrics.
• LLM-as-a-Judge: Top evaluation techniques and best practices — comparison between G-Eval, decision-graph metrics and pairwise judges.
• Rubric-Based Evaluations & LLM-as-a-Judge — rubrics as a bridge between policy and verifiable rewards.