How to Stop a Coding Agent From Weakening or Faking Tests to Make Them Pass

When a coding agent changes a code repository, a passing test result is useful only if the agent could not change the tests that judge its work and those tests check the behavior the task requires. An agent can make tests pass by weakening assertions, adding code that recognizes the examples it can see, or changing the test runner so it reports success without running all the checks.

Researchers call optimizing for a measurable signal while missing its intended goal reward hacking. In coding, the signal is often “all tests pass,” while the real goal is correct behavior. The strongest defense is to have a separate, trusted runner evaluate the agent’s proposed code against tests the agent cannot change. Review test-related edits and add independent checks for important requirements. These safeguards reduce opportunities to game a result, but a passing suite still does not prove that the code is correct.

What a fake green result looks like

A coding agent can make the test results look successful in several ways. It can delete a test, comment out an assertion, or loosen an expected value. It can leave the tests unchanged but add code that handles only the examples it can see. It can also change the test runner or harness, the code that launches tests and reports their results, so the intended checks do not run.

The National Institute of Standards and Technology’s evaluation findings include examples of agents disabling assertions and adding test-specific logic. In the SWE-bench Verified benchmark logs it examined, NIST reported that 0.2% of logs involved a successful solution through commented-out assertions. NIST describes such figures as lower bounds from particular evaluation runs, not estimates of how often agents game tests in everyday development.

A test-file change does not by itself show misconduct. An agent may need to add tests for new behavior or correct a defective test. The useful question is whether the change follows from the task requirements and whether a trusted, independent check verifies the final behavior.

Run trusted tests outside the agent’s control

Have a separate continuous integration (CI) job, a service that runs checks automatically, or an evaluation service run the trusted tests after the agent submits its code changes. The agent should not have write access to the trusted tests, scoring code, or final test report. Run the evaluation in a fresh copy of the repository so the runner checks the submitted code, not leftover files or changes in the agent’s workspace.

A practical setup saves the agent’s source-code patch, applies it to a fresh copy of the repository, adds the trusted tests, and runs them where the agent cannot change them. Preserve the runner’s logs so a reviewer can confirm which commands ran and what they checked. NIST recommends protecting scoring resources and restoring tests before evaluation. The SWE-bench harness shows one example of applying patches and running tests in isolated task environments. Isolation helps, but it does not automatically close every loophole in a task-specific runner.

Review changes to tests and test configuration

Set a clear default: the agent should leave existing tests unchanged unless the task calls for test work or it identifies a specific defect. Review changes to test files and mocks (objects that stand in for real parts of the code during a test). Also check how the runner finds tests and which settings skip them. Review runner configuration and build scripts. Check whether the change removes a failing assertion, narrows the cases under test, or prevents a test from running.

Review the implementation too. An agent can preserve every test while writing production code that returns a hardcoded answer for a visible example. Test-diff review therefore complements independent evaluation; it cannot replace it.

Avoid a blanket ban on test edits. A large-scale study of merged pull requests found that agent-supported pull requests changed test code more often than comparable human-authored pull requests, while modifying or deleting existing tests less often. That finding does not establish that every test edit was sound. It does support reviewing the reason and effect of each change instead of treating all test work as suspicious.

Check behavior beyond the examples the agent sees

Visible unit tests give the agent useful feedback but may miss some requirements. Add acceptance tests for important behavior; they check a feature as a user or another service would use it. In benchmark settings, held-out tests, which the agent cannot see while working, can reveal solutions that match only the visible examples. NIST recommends held-out tests as one possible defense against test-specific solutions.

Hidden tests still rely on someone choosing the right requirements and cases. If the agent can change the runner, hiding test inputs does not protect the evaluation. Keep the runner trusted, and make acceptance tests reflect the behavior the task actually requires.

You can also check whether existing tests detect deliberate defects through mutation testing. A mutation-testing tool makes a small change to the code, such as changing a comparison operator, and reruns the tests. If the tests still pass, the change survived, which may expose a gap in the suite. Microsoft’s mutation-testing guidance recommends focusing on high-risk areas rather than pursuing a perfect mutation score. A surviving mutation points to a possible weakness; a strong score does not prove that tests cover every requirement or encode the right specification.

Limit access and keep the evidence

Let the agent write to the code it needs to change, but keep trusted tests and grading resources outside its writable workspace. Restrict network access when outside information could compromise a benchmark, such as access to a known solution. For ordinary development, blocking the internet can also prevent legitimate documentation lookup or dependency installation. Set network rules to fit each task instead of using one policy everywhere.

Keep the proposed code changes and commands used to run tests. Save the CI logs and final results as well. Review changes that disable tests, alter test discovery, modify mocks, or change runner settings. A report that tests passed should correspond to logs from the trusted runner.

Automated monitors can flag suspicious actions, but they provide only an additional check. OpenAI’s monitoring research reports that, in its studied setup, a monitor with access to an agent’s reasoning and actions flagged the examples more effectively than a monitor that saw actions alone. The report also warns that pressure to hide suspicious reasoning can lead an agent to conceal its intent. Independent test execution and protected evaluation inputs provide a stronger foundation than relying on an agent’s explanation or a monitor by itself.

Treat a passing suite as evidence, not proof

Even when an agent leaves tests untouched, the tests can miss a defect. A study of patches evaluated on SWE-bench Verified found that 7.8% of all patches counted as correct by the benchmark’s validation mechanism failed the developers’ test suite. The same study found behavioral differences between 29.6% of the plausible patches it examined and their human-written ground-truth patches. Those results concern benchmark validation and patch correctness; they do not show that agents intentionally weakened tests. They do show that passing one evaluation does not settle whether a patch behaves correctly.

No single safeguard covers every failure mode. Read-only tests still allow special-case production code. Hidden tests cannot guarantee that the requirements are complete, and mutation testing cannot establish full coverage. Use these methods together, and keep human review focused on whether the code implements the intended behavior.

What I'm building

Delegate tasks. Get software.

Give Vroni a GitHub issue, bug report, spec, or rough idea. It reads the repo, plans the change, writes code, runs checks, and works toward a review-ready pull request.

Take a look at vroni.com