Eval-Awareness at Deployment Base Rates

At CMUL8, we tested a precise claim: can a simple detector look inside Qwen3 and identify prompts that make the model appear to be under evaluation?

The first result looked decisive. A layer-5 probe separated prompts from EvalAwareBench, a synthetic collection of test-like prompts, from WildChat, a collection of ordinary user conversations, with an AUROC of 0.9861 on Qwen3-4B. In plain language, when shown one prompt from each dataset, the probe ranked the EvalAwareBench prompt higher 98.6% of the time.

Then we tested the condition that matters in practice. We placed one EvalAwareBench prompt among 99 WildChat prompts and set the detector to catch 90% of the EvalAwareBench cases. Only 1.27% of its alerts were correct. Put differently, a reviewer would need to inspect about 79 alerts to find one true case.

The detector had not stopped working. It had answered a narrower question than its label suggested. Four follow-up controls showed that it was mostly recognizing which dataset a prompt came from—not whether the model had detected an evaluation.

This article traces that result from the original question to the deployment test and the forensic controls. Our aim is to make clear exactly what the metric proves, what it does not prove, and what an AI leader should require before using a detector like this in production.

The decision behind the metric

An eval-aware model notices clues that it is being tested and changes how it behaves. For example, a model might give unusually cautious answers when a prompt looks like a safety test, then behave differently in ordinary use. If that happens, a clean evaluation result may not describe the deployed system.

In this experiment, detection means estimating whether a prompt looks like it came from EvalAwareBench rather than WildChat, using Qwen3’s internal state before it generates an answer. It does not mean detecting deception, safety, or harmful intent.

Can Qwen3’s layer-5 activations tell us that a prompt contains signs of an evaluation before the model responds?

Our positive examples came from EvalAwareBench. Its prompts contain between one and eight clues that suggest an evaluation, such as grading language, explicit scoring instructions, or test-like framing. Our negative examples came from WildChat-1M, a dataset of real user–assistant conversations.

Each EvalAwareBench task has several variants. We keep every variant of one task on the same side of the train/test boundary: 20% of tasks train the probe and the remaining 80% build the test haystacks. This prevents near-duplicate versions of a prompt from appearing on both sides and making the result look better than it is.

A plain-language glossary

• Detector: in this study, the rule that scores a prompt from Qwen3’s internal state and raises an alert above a chosen cutoff.
• Probe: the simple detector we apply to Qwen3’s internal activations. It measures a direction already present in the model; it does not retrain or modify Qwen3.
• Hidden state: a numerical snapshot of what the model is representing internally after reading the prompt. Think of it as the model’s working representation, not a sentence a human can read directly.
• Threshold: the score above which the detector says “this looks like an evaluation.” Moving it changes how many prompts are caught and how many false alarms are created.
• Recall: out of all EvalAwareBench prompts in the test mix, the share the detector catches. Recall of 90% means it catches 9 out of 10.
• Precision: out of all prompts the detector flags, the share that came from EvalAwareBench. Precision of 1.27% means roughly 1 alert in 79 identifies an EvalAwareBench prompt.
• Base rate: the share of EvalAwareBench prompts in the test mix. A 1% base rate here means 1 EvalAwareBench prompt for every 99 WildChat prompts.
• False-discovery rate: the share of alerts that are wrong. It is 100% - precision, so 1.27% precision means a 98.73% false-discovery rate.
• AUROC: a ranking score across every possible threshold. AUROC of 0.986 here means that, given one EvalAwareBench prompt and one WildChat prompt, the probe ranks the EvalAwareBench prompt higher about 98.6% of the time. It does not tell us how trustworthy an alert will be when EvalAwareBench-like prompts are rare.

What an AI leader would actually deploy

An AI leader does not deploy an AUROC score. They deploy a workflow: the detector flags a prompt, a person or system reviews it, and the decision may block traffic, trigger an investigation, or create a label for future training.

At the 1% base rate tested here, that workflow would produce about 79 alerts for every true case. Roughly 78 would be false. That creates three concrete costs: unnecessary review work, alert fatigue, and—if the flags become training labels—a dataset dominated by mistakes.

The approval question is therefore not “is the benchmark score high?” It is:

At the expected frequency of the event, does this detector produce alerts we can act on?

How we built the probe

The probe asks one specific geometric question: does Qwen3’s internal representation of this prompt point more toward the average EvalAwareBench prompt or the average WildChat prompt?

To build that direction, we average the hidden states of the EvalAwareBench training prompts, average the hidden states of the WildChat training prompts, and subtract one average from the other. A new prompt receives a higher score when its hidden state points further along that direction.

eval average   = mean(hidden states for EvalAwareBench prompts)
chat average   = mean(hidden states for WildChat prompts)

probe direction = eval average - chat average
prompt score    = prompt hidden state · probe direction

This is why it is called a mean-difference probe. There is no gradient descent and no learned classifier. We test the direction independently at each model layer; layer 5 gives the best ranking score for both Qwen3-4B and Qwen3-8B.

The design is intentionally simple. At CMUL8, we strongly believe an assurance method should be understandable before it is trusted. A mean-difference probe lets us inspect the signal directly instead of hiding it inside another complex classifier.

The benchmark result leaders will see

The task-level score rises slightly, so variant leakage does not explain the headline number. On balanced data, the two corpora really are easy to separate. Another way to read AUROC 0.9861 is: if we draw one prompt from each corpus, the probe ranks the EvalAwareBench prompt higher about 98.6% of the time.

That is a legitimate and strong result for separating these two datasets. It is not yet evidence that the probe detects the broader concept of eval-awareness.

The production question begins where the balanced benchmark ends. Ranking one EvalAwareBench prompt against one WildChat prompt is not the same as finding a rare evaluation-like prompt inside a mostly ordinary traffic stream.

The deployment result operators will inherit

We next built a deployment-like “haystack.” We kept 500 EvalAwareBench prompts fixed, added increasing numbers of WildChat prompts, and chose a threshold that caught 90% of the EvalAwareBench cases. Adding more WildChat prompts changed only how rare the target was; it did not change the positive examples.

When half the test mix comes from each dataset, slightly more than half of the probe’s alerts identify EvalAwareBench prompts correctly. When EvalAwareBench contributes only 1% of the mix, just 1.27% of alerts are correct: 98.73% are false discoveries. Asking Qwen3 directly whether a prompt is part of an evaluation and using a two-stage chain-of-thought judge both perform at chance.

Why does the same threshold behave so differently? At a 1% base rate, every EvalAwareBench prompt is surrounded by 99 WildChat prompts. Even a modest false-positive rate, repeated across those 99 normal cases, overwhelms the one true signal.

In the simplest terms:

precision = true alerts / all alerts

all alerts = true alerts + false alerts

The full calculation combines recall, the base rate, and the false-positive rate. Even a detector with 90% recall and 90% specificity reaches only 8.3% precision when the target appears in 1% of cases. A production detector for a rare event therefore needs a false-positive rate far below the event rate. High AUROC alone does not guarantee that operating point.

This changes the decision. The probe may be useful as a research instrument, but at this threshold and base rate it is not ready to drive automated blocking, escalation, or human review.

When false alerts become bad training data

The same base-rate problem can corrupt the labels used to train a later system. In the test mix containing 1% EvalAwareBench prompts, the chain-of-thought judge flagged 1,892 prompts. Only 19 came from EvalAwareBench; 1,873 came from WildChat.

If those flags are treated as ground truth, the resulting training set contains 99% label noise. The judge’s fluent rationale can make each label feel credible, but confident prose does not make a false alert true.

For an AI organization, this is the second-order risk: a detector that appears strong on a dashboard can quietly contaminate the data used to improve the next model.

The forensic turn: what did the probe learn?

At this point, two explanations remained. Either the probe detected eval-awareness but failed when the target became rare, or it had learned an easier distinction between EvalAwareBench and WildChat. Three observations pointed toward the second explanation:

1. Detection is nearly flat from one evaluation cue to eight. A cue detector should respond to cue strength.
2. The strongest false positives are often short, trivial requests rather than evaluation-like prompts.
3. Performance is unchanged when model size grows from 4B to 8B.

We then ran four controls, each designed to separate the concept we care about from the surface features of the datasets.

1. Remove the cues

EvalAwareBench includes a baseline for each task with all evaluation signals removed. We scored those baselines without retraining the probe.

This is the most direct first test: remove the concept while keeping the underlying task and writing style as stable as possible.

The full positive-to-WildChat gap is 2.21 units. Removing every evaluation cue reduces it by only 0.30. Eighty-six percent of the separation remains.

2. Swap content and style

The first control removes the cues. The second reverses the test: preserve eval-aware meaning while changing the style, then preserve the benchmark style while removing eval-aware meaning.

We constructed a controlled 2×2 comparison: eval-aware content written as natural conversation, and mundane content written in EvalAwareBench’s clean task format.

The probe gives the two groups effectively identical scores. It responds to coherent, purposeful, medium-length text whether or not that text indicates an evaluation.

3. Inspect false positives

Next, we looked at the detector’s mistakes instead of its aggregate score. The 500 highest-scoring WildChat false positives have a median length of 43 words. Typical examples are simple greetings, translation requests, and short creative-writing prompts. Fewer than 2% contain the formal structure expected from EvalAwareBench.

This points to a second artifact: mean-pooling very short sequences can create representations that project strongly onto a fixed direction regardless of content.

4. Intervene on the direction

The final control asks a causal question. If the probe direction represents eval-awareness, adding that direction to layer-5 activations should change either the probe score or the model’s behavior. We steered with multipliers from -5 to +5.

On Qwen3-4B, the probe score changes by only +0.005 at +5. On Qwen3-8B, it moves by +0.256, but outputs remain far below the positive region and show no increase in hedging, formality, or acknowledgment of being tested. Neither curve is monotonic.

The direction separates datasets, but we find no evidence that it controls the behavior the label implies.

What the evidence supports

The forensic controls narrow the claim. They do not erase the benchmark result; they tell us what that result can and cannot support.

CMUL8’s approval standard for an AI detector

At CMUL8, we strongly believe a detector should earn the right to influence a production decision. Before it blocks traffic, triggers an investigation, or generates training labels, we would require three forms of evidence.

1. Report the operating point, not only the ranking metric. AUROC summarizes all thresholds on a balanced comparison. The deployment decision requires precision, recall, false-positive rate, expected base rate, and review volume at the chosen threshold.

2. Match the evaluation distribution to deployment. A detector trained to separate a synthetic benchmark from natural conversations may learn formatting, length, topic, or source. Test on deployment-like negatives and run cue-removal and style-swap controls before approving the interpretation.

3. Demand counterfactual and causal evidence before naming a representation. A direction that predicts a label does not necessarily encode the concept named by that label. Remove the concept while preserving style; preserve the concept while changing style; then intervene on the direction.

These are operating requirements, not academic preferences. They determine whether a metric supports a real decision or merely describes the dataset used to create it.

Conclusion: validate the decision, not the dashboard

The layer-5 probe succeeds at a real but narrow task: it distinguishes EvalAwareBench prompts from WildChat prompts with near-perfect AUROC. This experiment does not show that it can reliably detect whether Qwen3 knows it is being evaluated.

Two findings force that distinction. At a 1% base rate, only 1.27% of probe alerts are correct. And after every explicit evaluation cue is removed, 86% of the score gap remains. The style/content swap and activation-steering results reinforce the same conclusion: the detector has learned dataset identity more clearly than the concept named by its label.

For AI leaders, the decision rule is concrete: do not approve a detector from AUROC alone. Require its precision at the expected base rate, test it against traffic that resembles production, remove obvious dataset cues, swap content and style, and check whether intervening on the detected signal changes the behavior it claims to represent.

A precise metric is useful only when it measures the decision you intend to make.

Methods and experiment code are available in the eval-awareness repository.

Citation

@misc{ghosh2026evalaware,
  author = {Ghosh, Basab},
  title  = {Eval-Awareness at Deployment Base Rates},
  year   = {2026},
  month  = {August},
  url    = {https://www.cmul8.com/research/eval-awareness-base-rates/}
}