Security model and known limits
Who OpenHarnX trusts, what it prevents, what it only detects, and what is still open. The full threat model is in the repository.
OpenHarnX is a gate. Its job is to stop a change that games its tests. The attacker it is designed against is the author of the change: a coding agent, or a contributor's pull request.
Trust boundary#
Trusted: the accepted contract and the protected copies of its tests, held in the OpenHarnX store outside the repository; the OpenHarnX verifier; the protected checker environment; the owner's signing key.
Untrusted: everything in the candidate, including tests and configuration it adds, and anything an agent wrote. Repository text and tool output are data, never instructions.
The verifier runs checkers in a sandbox with no network, writes only to its own run folder, and cannot read ~/.ssh or OpenHarnX's keys. Checkers get an explicit environment allowlist, never the caller's secrets.
Prevented, detected, open#
"Prevented" means an attack was attempted and stopped in a test that also ran without the sandbox, where it succeeded. "Detected" means it can happen but is recorded or reported.
| Examples | |
|---|---|
| Prevented | Edited or deleted locked tests; a candidate file replacing pytest or a module it imports; the verifier changing the candidate; pull request code reading the runner's secrets; a changed lockfile with a protected environment; approvals from someone without write access |
| Detected | Skips, suppressions and loosened configuration (BLOCKED); a test that passed before and fails now; evidence edited after verification (INVALID); changes after verification (STALE); a changed checker environment (INVALID); a sandbox that did not start (never READY) |
| Open | Locally, an agent with a shell running ohx init --lock-tests itself or removing the hook; administrators pushing past branch protection; a pull request running its own edited workflow until review stops the merge |
Known limits#
- Tests can be weak. A passing verdict does not show the tests are good enough to catch a wrong change. The mutation check is advisory.
- Weakening detection covers listed patterns. Weakened assertions, moved suppressions and import-time side effects of new test files are not detected.
- What was accepted still runs. A
conftest.py, pytest plugin oraddoptssetting present at acceptance runs inside pytest. - Without
environment = "uv", changes to the checker environment made before acceptance are not caught. - Code that detects the test runner (behaving differently when pytest is running) is not flagged yet; it is on the roadmap.
- Platforms. Claims hold for the tested platforms only. Self-hosted or shared runners that keep state between jobs, Windows, and kernel or sandbox vulnerabilities are out of scope.
- Deliberate adversarial agents probing the policy over long sessions were not tested. The suite tests known techniques.
Read further#
- Threat model: every threat with its control, status and the tests behind it.
- Release criteria: what the first release had to meet, with evidence.
- Trusted base and required checks: the rules for CI.
Found a way around a claim marked prevented or detected? Report it privately.