# Cancel and terminate

> Stop a run gracefully with cancel, or end it now with terminate when no worker can.

Source: https://docs.pyrula.io/workflows/guides/cancel-and-terminate/

| | Cancel | Terminate |
|---|---|---|
| Ends the run | At its next step boundary, by the run's worker | At once, by the store |
| Needs a worker | Yes, for this workflow and version | No |
| Running step | Finishes | May finish its side effects; result discarded |
| Cleanup (`finally`, compensations) | Runs | Skipped |
| Journal ends with | `run:error`, `code="cancelled"` | `run:error`, `code="terminated"` |
| Retryable | No | Yes |

Cancel is a request. With no worker for the workflow and version, or a step that hangs while its
heartbeat keeps renewing, a cancelled run never ends; use terminate. Once latched, cancel wins: a
later step failure or timeout still ends the run `cancelled`.

## Cancel

```python
await client.cancel("fulfill_order", run_id, reason="wrong customer")
```

```bash
pyrula runs cancel <run_id> --reason "wrong customer"
curl -X DELETE "$BASE/fulfill_order/stream/<run_id>?reason=wrong%20customer"
```

The reason lands on the `run:cancel` event and the terminal's `detail`. First reason wins.

## Terminate a run

```python
result = await client.terminate("fulfill_order", run_id, reason="worker fleet lost")
result.outcome    # TerminateOutcome.TERMINATED, or ALREADY_ENDED on a repeat
result.children   # CloseResult(cancelled=0, terminated=2, abandoned=1)
result.run_id     # the chain's live tail, if the run continued as new
result.ended      # (name, run_id) of each child this call terminated
```

```bash
pyrula runs list --workflow fulfill_order --status in_progress --since 1h
pyrula runs show <run_id>
pyrula runs terminate <run_id> --reason "worker fleet lost"    # asks first; --yes skips
curl -X POST $BASE/fulfill_order/runs/<run_id>/terminate -d '{"reason": "worker fleet lost"}'
```

`handle.terminate(reason=...)` does the same for a run you hold; `AgentClient.terminate(agent,
turn_id, reason=...)` posts to `/{agent}/turns/{turn_id}/terminate`.

The store writes the terminal event and deletes the run's lock atomically, so no worker needs to
be connected. `pyrula runs terminate` talks straight to the store (`--store` or
`PYRULA_STORE_URL`), with no runtime or Cloud. It exits 1 for an unknown run and 2 when it needs
confirmation and has no terminal.

The journal gets one `run:error` with `code="terminated"`; `detail` holds `terminated_at`,
`reason` (cut to 500 characters), `requested_by` and, for a child ended with its parent, a
`cause`. Keep secrets out of reasons.

The store needs the `terminate` capability (Memory, SQLite and Valkey have it), else
`TerminateUnsupported` is raised before anything is written and the HTTP route answers 501.

### Effect on the worker

The deleted lock fences the run: a worker still running it, or returning after a crash, has
every write refused, and its next heartbeat stands it down. A step already running can't be
interrupted, so its side effects still happen and its result is discarded.

### Retrying a terminated run

`client.retry` or `pyrula runs retry` serves every step recorded before the terminate and
reruns the step that was in flight, so a non-idempotent write in flight may run twice (see the
retry's `in_doubt` list). A run terminated while waiting on a child retries from before the
first open child, which starts its own child. A run that was cancelled first can't be retried.

## Child runs

On a graceful end the worker applies each open child's `parent_close` policy (see
[Parallel steps and child runs](https://docs.pyrula.io/workflows/guides/parallel-and-children.md#what-happens-to-a-child-when-its-parent-ends)).
`terminate` has no worker, so the store does it:

1. The parent is terminated first, so it starts no more children.
2. `abandon` children are left running; `cancel` and `terminate` children are terminated.
3. Each terminated child's own open children get the same treatment, depth first.

A child whose submit hasn't reached the store yet gets a held cancel and starts `cancelled`
when the submit lands. An unclaimed hold expires after `completed_ttl` on Valkey (24 hours by
default) or `held_ttl` on `MemoryStore` and `SQLiteStore`.

Each ended child carries `requested_by="parent"` and a `cause` naming the parent run. Repeating
`terminate` on a terminated run reapplies the child policies, which finishes a cascade a crash
cut short. On a run that ended any other way it writes nothing and returns `ALREADY_ENDED`.

## From Pyrula Cloud

A runtime connected to Cloud offers the `turn.terminate` control when its store has the
`terminate` capability. The message names the run, an optional `options.reason` and
`options.requested_by`; the journal records that actor, or `cloud` if none is given.

Because the store writes the terminal, the connector sends it to Cloud itself before
acknowledging: the run, the live tail of a continued chain, and each terminated child. The
`pyrula runs terminate` command has no Cloud connection, so a row it leaves open settles on the
next terminal sweep of a runtime that holds the store.

A terminated run reaches Cloud as `run_error` with code `terminated` and a `detail` object. The
reason is redacted like any failure detail and ships at capture `io` and above. Below `io` the
actor ships only when it is the runtime's own (`cloud`, `parent`, `api` or `cli`). A child
cancelled because its parent ended reaches Cloud as `cancel` with a `detail` holding
`cancelled_at`, `reason`, `requested_by` and `cause`.
