Kafka

pyrula.kafka is a Kafka client written in Rust, with native asyncio and an Arrow path and no librdkafka. The API tracks confluent-kafka. Two things differ: every networked call is bounded by a timeout, and batch APIs (produce_many_*, poll_batch, poll_arrow) keep per-record work out of Python. Throughput numbers are on Benchmarks.

from pyrula import kafka

pip install "pyrula[kafka]". Wheels cover Linux (glibc and musl), macOS and Windows on x86_64 and ARM (aarch64 Linux needs glibc 2.28+). AWS MSK IAM auth is an extra: "pyrula[kafka,msk]".

Pages

• Quickstart: produce a batch, poll it back.
• Producer and Consumer: delivery, batching, groups, offsets.
• Transactions: exactly-once produce and consume.
• Formats: raw bytes, Avro, JSON.
• Admin: topics, partitions, configs.
• Configuration and Authentication (SASL, mTLS, OAuth).
• Examples: producer+consumer round trips, EOS, Avro.
• Client guarantees: delivery behavior by producer and consumer setting.
• Migrating from confluent-kafka: the drop-in shim.