# Kafka runtime guarantees

> What @kafka_workflow and @kafka_agent guarantee in each delivery mode.

Source: https://docs.pyrula.io/workflows/guides/kafka-guarantees/

The Kafka runtime provides exactly-once *processing* and at-least-once *output delivery*
by default. `@kafka_agent` uses the same runtime contracts as `@kafka_workflow` with an
additional LLM loop.

`yes` means the guarantee holds; `partial` means it holds within the boundary under
Limits; `no` means it does not hold and Limits says why.

| Binding | processing | output delivery | poison | handler error |
|---|---|---|---|---|
| `kafka_workflow` | yes | partial | yes | yes |
| `kafka_workflow[exactly_once]` | yes | yes | yes | yes |
| `kafka_workflow[durable=False]` | no | partial | yes | yes |
| `kafka_agent` | yes | partial | yes | yes |
| `kafka_agent[exactly_once]` | yes | yes | yes | yes |

## Column definitions

- **processing:** Exactly-once: a redelivered record whose run completed is not re-run.
  - Its result is read back from the run store.
  - A retried record whose run is partial resumes from its checkpointed steps; completed steps do not re-execute.
  - `durable=False` rows are at-least-once: the handler and every inline step re-run.
- **output delivery:** A failed delivery never commits past its record.
  - `at_least_once` rows: a failed delivery commits nothing, so the record is redelivered and re-produced.
  - `exactly_once` rows: output records and input offsets commit in one transaction; an abort leaves neither.
- **poison:** A record that fails decode goes to the DLQ topic, with the error, before the workflow runs.
  - The run store never sees it; the rest of the batch proceeds and commits.
- **handler error:** After `max_retries`, the `on_error` disposition holds.
  - `dlq` routes the record with the error; `fail` stops the worker without committing past it; `skip` commits past it.
  - A run that pauses (`ctx.sleep`, `ctx.interrupt`, `ctx.wait_for_signal`) is an error on this path and never stalls its partition.

## Install and tested versions

All bindings: built in. Supported brokers: Kafka 3.9+ (tested on 3.9 and 4.4), Confluent Platform 7.9+.

## Notes

- **`kafka_workflow[durable=False]`:** durable=False has no run store: the handler re-runs on every retry or redelivery, by design

## Limits

- **`kafka_workflow` - output delivery; `kafka_workflow[durable=False]` - output delivery; `kafka_agent` - output delivery:** at-least-once: a crash between produce and commit re-produces on redelivery; dedupe downstream on key, or choose delivery="exactly_once"

## Runner behavior

- Handler error: `on_error="fail"` stops the runner's worker, not only the one record.
- Delivery: a failed produce, offset commit or aborted transaction rewinds the partition
  to the uncommitted offset and retries with a backoff instead of stopping the runner; a
  fatal error, or eight failures in a row, stops it and `run()` raises
  `KafkaRunnerFailed` so the process exits non-zero. A rewind re-runs the handler for the
  whole drain, including records that went to the dead-letter topic (a run that ended in
  error is not reused), so a failing record repeats its handler attempts on each retry -
  at-least-once processing, not a leak.
- Processing: `kafka_workflow(durable=False)` runs with no run store and reruns on
  redelivery.
- Poison: a poison record sharing a drain with good records does not stall the batch or
  take the consumer down with it.
- Processing: an existing `@workflow` stacked under `@kafka_workflow` still coerces its
  persisted input the same way a bare handler does.
- Output delivery: exactly-once holds across a cooperative rebalance between live
  members: every input id on the output exactly once, read-committed.
- Output delivery: each partition has its own transactional producer, with transactional
  id `group:topic:partition`, and `init_transactions` fences any earlier producer using
  that id. This is per-partition transactional-id fencing (KIP-98) only. The offset commit
  goes through the producer using the consumer's group id and coordinator, but the
  TxnOffsetCommit request is pinned to v0 and carries no generation or member id, so
  KIP-447 group-generation fencing is not implemented. The producer is created on a
  partition's first delivery, not at assignment.