Define acceptance criteria

Agree tests for the task before the agent starts. A change that passes them, and breaks nothing, is READY.

Locking the existing suite shows that nothing broke. It does not show that the task is done. For that, write tests for the requested behaviour first and lock them as acceptance tests.

Write the tests first#

Write the acceptance tests in a self-contained file before the agent starts, for example tests/test_percent_coupon.py:

python

from pricing import order_total


def test_percent_coupon():
    assert order_total([(1000, 1)], coupon_percent=10) == 900


def test_percent_then_fixed_coupon():
    assert order_total([(1000, 2)], coupon_cents=300, coupon_percent=10) == 1500

They are expected to fail now, because the behaviour does not exist yet.

Agree and lock them#

shell

ohx contract new --mode task \
  --title "Percentage coupons" \
  --summary "order_total takes a percentage coupon as well as a fixed one" \
  --acceptance tests/test_percent_coupon.py \
  --accept

This writes contracts/0001-percentage-coupons.toml from your ohx.toml defaults, copies the acceptance tests into the store and records the regression baseline. --acceptance takes a file or a folder and can be given more than once. --mode is bugfix (the default) or task. The summary becomes the report's changelog entry.

Commit the contract file so the team can see what was agreed.

Verify#

After the agent's change:

shell

ohx verify --sandbox srt

With the agreed tests passing and nothing else broken, the verdict is READY. The brief lists each agreed test that passed. If an agreed test fails, the verdict is BLOCKED, and editing the repository's copy of the test changes nothing: the locked copy is the one that runs.

READY still leaves questions#

The READY report for the demo's genuine fix, captured with ohx 0.1.1, still ends with a question. The advisory mutation check, which ran without any setup, changed the lines the fix touched and found two changes no acceptance test noticed:

text

## Decisions for you

1. Do these changed lines need a test? Changing them did not make any acceptance test fail: 2 of 8 mutants of changed lines survived: pricing.py:9 `0` -> `1`; pricing.py:14 `0` -> `1`

READY means the agreed tests pass. It does not say whether those tests are good enough. The mutation check is the only measure the report has of how much the tests notice, and it never changes the verdict.

Changing the agreement#

If the expected behaviour itself was wrong, agree a new contract revision rather than editing the test. A report cannot waive a failed mandatory check.