Testing and Quality Gate

SOIT treats schema and service drift as a merge blocker. Run the blocking checks locally before opening a pull request — CI runs the same gates.

Blocking backend gate

Run from soit/server:

uv sync
uv run lint-imports --config importlinter.ini
uv run pytest tests/unit -q
uv run pytest tests/entrypoints -q
uv run pytest tests/integration -q
uv run pytest tests/test_spec_validation.py tests/test_model_scope_audit.py tests/test_trace_emission.py -q

The last line is the platform-invariant suite: it verifies spec validation, model scope audit, and trace emission — the Trace-Everything rule is tested, not just documented.

The support-ticket regression evaluator is also blocking. It uses the deterministic model:test:* path, executes a fixed golden prompt set, and exits non-zero if any case fails:

uv run python scripts/evaluate_support_ticket_regression.py --json-output ../artifacts/support-ticket-regression/report-current.json
uv run pytest tests/integration/test_support_ticket_regression_evaluator.py -q

Every passing report must include a policy answer with a citation and a ticket workflow case with tool-call, child-workflow, audit, citation, and cost evidence.

Blocking frontend gate

Run from soit/web:

npm ci
npm run typecheck
npm run build
npx playwright install chromium
npm run test:e2e

Important frontend coverage includes route rendering, API client behavior, responsive layout, i18n copy, and critical user flows.

Strict debt checks

These run in CI as non-blocking debt reports until the existing backlog is burned down:

uv run ruff check app tests
uv run mypy app tests

New direct import-boundary violations still fail CI — existing debt is explicitly baselined in server/importlinter.ini.

Contract rules

Public API schemas validate transport payloads only.
Application services resolve published versions into internal runtime requests before execution.
Runtime services must not read fields that only exist in HTTP payloads or frontend types.
Agent execution bindings (model, tool, knowledge, workflow, plugin) come from the published agent version, never from execute-payload overrides.

Quality-gate commands themselves are not recorded as Observe runs — Observe validates application runtime data only. After running the demo or the regression evaluator, open /observe?range=24h in the web UI and confirm the run is visible, then inspect its detail page for response events, run steps, tool calls, child workflow runs, costs, citations, and audits.

Docs gate

Run from soit-docs:

npm run types:check
npm run build