Testing processes
Write test cases for a process — inputs, the answer each gate gets, the path and outcome the run must take — run them against any version without touching a live system, compare two versions on the same suite, and make publishing require a green suite.
A change to a process changes what happens to real cases. A test suite says, before you publish, what the process must do: for these inputs, with these answers at the gates, the run goes through these steps and ends like this. You run the suite against any version — a draft too — and compare two versions on it side by side.
A suite run is never live:
| What a run does | What a suite run does instead |
|---|---|
| Calls the connectors bound to each step | Every connector is shadowed: it records what it would have sent and sends nothing. |
| Parks at a gate and waits for a person | The gate is answered from the case, checked the way the sign-off card checks a signature. |
| Scores agent steps with the runtime's step scorer | Answers each agent step from its recorded fixture (unless the case says llm: "live"). |
| Is a row in Runs, in analytics, in someone's queue, on the SLA clock | It is none of those. No run row, no park, no notification, no run_started or gate_signed entry. |
What it leaves is the suite result and one audit entry, process_suite_run: who ran it,
on which version (number and executable hash), and the result counts.
A test case
Open a process and go to the Tests tab. New case opens a JSON editor that checks the
case as you type; the server checks it again on save and names every problem (422).
{
"name": "High risk: the analyst approves, the number is released",
"inputs": { "risk": "high", "confidence": 0.9 },
"llm": "fixtures",
"answers": [
{ "step": "signoff", "visit": 1, "outcome": "approved", "form": { "ticket": "FR-77" } }
],
"fixtures": {
"score": {
"tool_results": [{ "tool": "fraud-scoring-model", "ok": true, "signal": { "risk": "high" } }],
"confidence": 0.95
}
},
"expect": {
"status": "completed",
"path": ["intake", "score", "risk", "stage", "signoff", "execute", "monitor"],
"path_mode": "exact",
"outputs": [
{ "step": "execute", "path": "$.tool_results[2].mode", "equals": "shadow" },
{ "step": "signoff", "path": "$.form.ticket", "equals": "FR-77" }
]
}
}| Key | What it says |
|---|---|
inputs | The run's starting facts, as POST /process-runs takes them. |
auto_resolve | Default true — the autopilot, as the console runs it. |
answers | One per gate pass: step (its id or its gate: label), visit (default 1 — a rework loop passes a gate again), outcome (approved, denied or a branch label), form (the gate form's values), approvers (see N-of-M gates). |
fixtures | Recorded outputs of agent steps, keyed "<step>" or "<step>#<visit>": tool_results (one per tool, matched by tool, else by position) and an optional confidence (0..1). A result's signal feeds the run state, so a fixture can route a gateway. |
llm | "fixtures" (default): steps with a fixture answer from it, the others run on the shadow connectors with the deterministic scorer — the suite is deterministic and free. "live": every step is scored by the runtime's step scorer, as a real run is. |
expect.status | How the run ends: completed, awaiting_human (it waits at a gate the case does not answer), max_steps or dead_end. |
expect.path | Step ids in order. path_mode: "exact" — every step, nothing else; "in_order" — these steps, in this order, others may come between. |
expect.outputs | Per step (and visit): a JSONPath ($.a.b, $.list[0], $['odd key']) that equals a value or contains one (a substring, a list element, or a sub-object). An agent step's output is {tool_results, decision, resolver}; a gate's is {decision, branch, form}. |
A scripted answer is checked like a signature: an outcome that names no branch of the gate, or
form values the form refuses, is a failure of the case (answer_refused, naming the field),
never a signature.
Make a case from a run
Make a case from a run (or POST /process-tests/{slug}/cases/from-run with a run_id)
turns a run that already happened into a case: its inputs, every answer its gates were given
— by step and visit, with the form values and, on an N-of-M gate, who approved — its path
and final status as the expectation, and its agent steps' outputs as fixtures. Run the suite
on the next version to see whether it still does the same.
Record fixtures from a live run
Record (or POST /process-tests/{slug}/cases/{id}/record with a version) runs the case
once live — the runtime's step scorer, connectors still shadowed — and stores what each
step produced as the case's fixtures; the case runs from them (llm: "fixtures") afterwards.
A live run that crashed records nothing (409). The record is audited as a case change
(process_test_case_changed, action recorded).
Run the suite
Pick a version and Run suite (POST /process-tests/{slug}/runs with {"version": 4}).
The result:
status—passed(every case passed),failed, orerror(a case could not run — it crashed; an error is never a pass);- per case:
pass/fail/error, the first divergence — the step, what the case expected and what the run did (kind:path,status,output,answer_refused,approvals) — the path it took, its duration, and any answers or fixtures it never used.
The first divergence is found in this order: the path (every later difference follows from it), then the final status, then each output assertion.
A suite with no cases is refused (409); a version that is not runnable is refused (422,
with its violations). If the audit entry cannot be written, the result is not stored (503).
Compare two versions
Compare (or POST /process-tests/{slug}/compare with {"a": 3, "b": 4}) runs the suite
on both versions — two stored, audited suite runs — and lines them up per case, each side
with its status and first divergence. A row is marked changed when the verdict, the
divergence or the path differs: that is the behaviour change the new version brings.
N-of-M gates
On a gate with an approvals policy (approvals: {required: 2, …}, see
N-of-M approvals) an answer says who approves:
"approvers": 2— two distinct testers, each with the role the policy's slot needs;"approvers": ["[email protected]", {"by": "[email protected]", "role": "admin"}]— named people; a delegate is{"by": …, "on_behalf_of": …}and counts as both.
The approvals are checked like the sign door checks them — different people when
distinct_people is on, the role of each slot when order is sequential — and recorded
for the gate's visit; the engine clears the gate only at quorum, as on a real run. Fewer
approvals than required is a failure of kind approvals ("2 approvals" expected, "1
approval" given). A denial by anyone ends the gate whatever the count.
Publishing requires a green suite
An owner or admin can turn on Publishing needs a green test suite on the Settings page
(PUT /governance/policy with {"publish_requires_green_suite": true}; off by default; the
change is audited as policy_changed). With it on, publishing a version — and rolling back
to one — is refused (409, saying why) unless the latest suite run on that version:
passed;- ran that version's exact executable graph (its hash);
- ran the cases the process has now — a case added, changed or deleted since makes the result stale.
The Versions panel shows each version's suite (suite passed, suite stale, suite failed,
not tested) and why Publish waits. When the results cannot be read the panel says not
read — never "not tested" — and with the rule on, publishing waits. The process_published
entry records suite_required and the suite_run_id it relied on.
API keys
| Scope | Routes |
|---|---|
processes:read | GET /process-tests/{slug}/cases, …/cases/{id}, …/runs, …/runs/{id} |
processes:test | POST /process-tests/{slug}/runs, POST /process-tests/{slug}/compare |
A key never writes a case (create, change, delete, record, from a run): those are for people above viewer. A viewer reads cases and results but does not run a suite.
import minctrl
ci = minctrl.MinctrlClient("https://app.minctrl.com", api_key="mk_live_…") # processes:test
result = ci.tests.run("prior-auth", version=4)
if not ci.tests.passed(result):
for case in result["results"]:
if case["status"] != "pass":
print(case["name"], case["divergence"] or case["error"])import { createMinctrl, suitePassed } from "@minctrl/sdk";
const mc = createMinctrl({ baseUrl: "https://app.minctrl.com", apiKey: process.env.MINCTRL_API_KEY });
const run = await mc.tests.run("prior-auth", 4);
if (!suitePassed(run)) process.exit(1);Promoting between environments
Move a published process from one Minctrl installation to another as a signed release bundle — Ed25519 signatures, trusted installations, import into review, connector mapping, the promotion log and a drift check.
LLM agent steps
Let Claude do an agent step's work — a structured answer checked against your schema, tool calls through your connectors, irreversible actions only proposed for a person to sign, every call recorded with its cost, and spend bounded by budgets.