Replacing Claude Opus with a cheaper, smaller model usually means giving up accuracy.
Hightouch took a different approach: keep Opus, but do not send every record through it.
The company built a lightweight JEV—a model in the Judgment, Judge, or Evaluator class—that can request Claude Opus assistance on demand. It first assesses whether an identity-resolution result is reliable. Only ambiguous, conflicting, or high-risk cases are escalated to Claude Opus.
According to Hightouch's published experiment, prompts, and evaluation details, the JEV achieved task accuracy close to Claude Opus while costing roughly one-quarter as much for inference. That works out to about a 75% reduction.
The savings did not come from using 25% less Opus. Most requests were filtered out before escalation.
Two Models, Not Redundant Work
Common multi-model systems query two models at the same time, then use voting or a judge to select an answer. The problem is that simple cases are paid for twice, while their latency adds up.
Hightouch's design assigns the models different roles in sequence rather than running both in parallel.
The first layer is a lightweight JEV. It performs identity resolution and returns both a result and a confidence score. The JEV is not tied to a fixed model; it acts as a routing layer that decides, based on task difficulty and its own assessment, whether Claude Opus is needed.
Claude Opus forms the second layer. It handles records that the JEV considers uncertain and takes responsibility for the final decision.
The easiest detail to copy incorrectly is not the model names, but the request flow:
record -> JEV -> confidence check
|-> high confidence: accept JEV result
|-> low confidence: request Claude OpusIf every sample is escalated, the cascade has no cost advantage. If the escalation criteria are too strict, errors can slip through. The metrics that matter are:
- JEV accuracy when operating on its own;
- The proportion of cases escalated to Opus;
- The distribution of errors before and after escalation;
- The average cost per resolution;
- P95 latency, rather than average latency alone.
Hightouch published its experimental setup, prompts, and evaluation details, along with quantified accuracy and cost comparisons. That matters because, without them, “the same accuracy at 75% lower cost” is just an unreproducible marketing claim.
Original article: LLM Judges for Identity Resolution
How the One-Quarter Cost Figure Works
The roughly 75% reduction is not an arbitrary discount. It results from changing the request structure.
Suppose resolving a batch of identities with Claude Opus alone requires 100 units of inference cost. In a two-layer system, the JEV processes every record first, and only a proportion, p, is escalated to Opus. The combined cost can be approximated as:
total cost ≈ batch JEV cost + p × Opus escalation costThe key variable is p. If simple rules or lightweight models can resolve most identity matches reliably, the escalation rate will be low and Opus spending will fall sharply. Hightouch's final report puts total inference cost at about one-quarter of the cost of using Opus alone.
These results should not be interpreted as “Opus became 75% cheaper.” Model pricing did not change. What changed was call volume: the system handled cases that did not require Opus before invoking it.
There is also an important engineering detail. Identity-resolution tasks typically contain many straightforward records. When identifiers such as email addresses and phone numbers have clear formats, there are often only a few candidate entities. The difficult cases involve spelling variations, missing fields, shared household addresses, and multiple candidates with similar scores. The JEV can handle the first category, leaving the second to Opus.
This is why experimental data matters more than an architecture diagram. The prompts used by the JEV, the definition of confidence, and the escalation conditions all directly affect that 75% figure.
Why Identity Resolution Suits Model Cascades
Identity resolution is not open-ended question answering. It has a relatively clear objective: determine whether two records in a customer database refer to the same entity.
At the same time, it has plenty of ambiguous boundaries.
For example, “John Smith” may refer to several customers. A matching address does not prove two people have the same identity, and different email addresses do not necessarily mean two separate people. Rule-based systems work well for exact matches but struggle with missing cross-system fields, nickname variations, and data-entry errors. Sending all data directly to Opus forces an expensive model to handle large volumes of mechanical decisions.
A model cascade takes advantage of this distribution:
- When formats are clear and the candidate is unique, match directly or use a lightweight model;
- When fields differ slightly and the JEV can assess the case, return the JEV result directly;
- When candidates conflict or data is seriously incomplete, escalate to Claude Opus;
- For high-risk business cases, require human review or a stronger model regardless of confidence.
The last point is crucial. Low-confidence routing cannot replace compliance review. If merging identities affects billing, account recovery, or access to private data, escalation criteria should be driven by business risk rather than a model-reported score alone.
The discussion also reached the Hacker News front page, receiving 165 points and 40 comments. The original thread is available at Hacker News item 50397429. The most useful questions in the comments are not about whether models will replace data engineers, but about how to set escalation thresholds and whether the approach transfers to other tasks.
The JEV Router Is the Part Worth Copying
Switching directly to a cheaper model usually costs capability. A model cascade retains an escalation path, making it more suitable for production systems.
The approach can be adapted to several common tasks.
Customer support quality assurance: The JEV first checks whether a support ticket violates a rule. Clearly compliant cases can pass directly, while complex tone, missing context, or cases involving compensation can go to Claude Opus.
Data cleaning: A lightweight model first handles standard formats, duplicates, and deterministic formatting errors. Semantic deduplication and conflicting-record merges are escalated.
Code review: The JEV runs static checks and screens out low-risk changes, while architectural impact, security vulnerabilities, and cross-module logic go to a stronger model.
When migrating the approach, copy the routing mechanism—not the fixed pairing of Claude Opus with a particular JEV.
I would build a minimal implementation in four steps:
- Establish a single-model baseline and record accuracy, cost, and latency.
- Have the lightweight model output a result, a rationale, and a confidence signal.
- Derive escalation conditions from actual errors rather than setting thresholds by intuition.
- Run ablation experiments on the same sample set to compare the gains at different escalation rates.
A simple routing implementation looks like this:
def resolve(record, jev, opus):
result = jev.judge(record)
needs_opus = (
result.confidence < 0.72
or result.candidate_count > 1
or "identity" in result.risk_flags
)
if needs_opus:
return opus.judge(record, candidate_result=result)
return resultThe 0.72 threshold is illustrative, not an experimental parameter from Hightouch. It must be calibrated with a labeled dataset before deployment. A better approach separates confidence from business risk: however confident the model is, it cannot override an explicit compliance threshold.
Editorial Perspective
This experiment matters more than the simple idea of replacing a large model with a small one. It suggests that the unit of optimization for production AI systems may be the entire decision chain rather than an individual request.
My view is that model cascades will evolve from an internal Agent technique into a cost-control layer. Customer support, quality assurance, data cleaning, and code review share a common structure: request difficulty is unevenly distributed, a small number of difficult cases account for most errors, and those cases are also the best candidates for expensive inference budgets.
This differs from parameter fine-tuning. Fine-tuning primarily changes the capability boundary of a single model, whereas a cascade lets the system choose cost dynamically. Hightouch's published materials also include prompts and evaluation details, giving readers at least a starting point for reproducing the results.
The public data still do not establish that a 75% reduction transfers to every task. Identity resolution has clear answers and relatively stable difficulty tiers; open-ended writing and complex planning may not. If routing thresholds are poorly calibrated, cost savings can turn into rework and manual review. The metrics worth watching extend beyond the inference bill to P95 latency, escalation rates, error types, and system behavior when the budget ceiling is breached.