Protect an existing test suite
Make the tests you already have the contract, so a change cannot pass by editing, skipping or deleting them.
ohx init --lock-tests is the zero-setup mode. You write no contract file, and nothing is added to the repository.
ohx init --lock-testsWhat gets locked#
- The
tests/folder as it is in the working tree, tracked or not, is copied into the OpenHarnX store. Later runs use these copies, not the files in your repository. - The per-test results of a run become the baseline: which tests passed, failed or were skipped.
- Check configuration such as pytest options,
conftest.pyfiles,ohx.tomland lint or type-check suppressions, so a change that loosens them is found. - The checker's interpreter environment is fingerprinted, unless
environment = "uv"builds a protected one fromuv.lock. - For experimental TypeScript, JavaScript and Go projects, the tracked test files at their paths.
Who locked is recorded with the git user and shown by ohx audit.
The checks that come with it#
ohx init --lock-tests prints the checks it created:
| Check | Mandatory | What it does |
|---|---|---|
weakening | yes | Looks for removed tests or assertions, new skip or xfail markers, new suppressions and loosened check configuration |
no-new-failures-locked-tests | yes | Every locked test that passed at locking still passes when the locked copies run |
no-new-failures-tests | yes | Every test in tests/ that passed at locking still passes, and is not skipped or gone |
locked-tests | no | The run of the locked copies, kept as evidence |
tests | no | The run of the working tree's own tests, kept as evidence |
Projects can add their own checks, such as a linter or type checker, as [[obligations]] in ohx.toml; see the configuration reference.
Then verify any change#
ohx verify --sandbox srtA change that keeps everything passing is NO REGRESSIONS. One that breaks or weakens a locked test is BLOCKED.
Tests that already fail#
A test that fails at locking and still fails is reported as failing before and after, not blamed on the change. If the suite could not run at all when you locked it (for example, it imports code the task will add), OpenHarnX locks anyway and says that a later run counts once every test passes.
Changing the tests on purpose#
Run ohx init --lock-tests again after you have reviewed the test change. The new state becomes the contract and the lock is recorded. In CI, a maintainer approves instead.