# Kafka

> A Rust-backed Kafka client that tracks confluent-kafka, with explicit timeouts and EOS.

Source: https://docs.pyrula.io/kafka/

`pyrula.kafka` is a Kafka client written in Rust, with native asyncio and an Arrow path
and no librdkafka. The API tracks `confluent-kafka`. Two things differ: every networked
call is bounded by a timeout, and batch APIs (`produce_many_*`, `poll_batch`,
`poll_arrow`) keep per-record work out of Python. Throughput numbers are on
[Benchmarks](https://docs.pyrula.io/kafka/benchmarks.md).

```python
from pyrula import kafka
```

`pip install "pyrula[kafka]"`. Wheels cover Linux (glibc and musl), macOS
and Windows on x86_64 and ARM (aarch64 Linux needs glibc 2.28+). AWS MSK IAM auth is an
extra: `"pyrula[kafka,msk]"`.

## Pages

- [Quickstart](https://docs.pyrula.io/kafka/quickstart.md): produce a batch, poll it back.
- [Producer](https://docs.pyrula.io/kafka/producer.md) and [Consumer](https://docs.pyrula.io/kafka/consumer.md): delivery, batching, groups, offsets.
- [Transactions](https://docs.pyrula.io/kafka/transactions.md): exactly-once produce and consume.
- [Formats](https://docs.pyrula.io/kafka/formats.md): raw bytes, Avro, JSON.
- [Admin](https://docs.pyrula.io/kafka/admin.md): topics, partitions, configs.
- [Configuration](https://docs.pyrula.io/kafka/configuration.md) and [Authentication](https://docs.pyrula.io/kafka/authentication.md) (SASL, mTLS, OAuth).
- [Examples](https://docs.pyrula.io/kafka/examples.md): producer+consumer round trips, EOS, Avro.
- [Client guarantees](https://docs.pyrula.io/kafka/guarantees.md): delivery behavior by producer and consumer setting.
- [Migrating from confluent-kafka](https://docs.pyrula.io/kafka/confluent-compat.md): the drop-in shim.