Kafka

pyrula.kafka is a Kafka client backed by Rust. The API tracks confluent-kafka, so existing knowledge carries over. Two things differ on purpose: every networked call takes an explicit timeout, and there’s a native record path for throughput.

from pyrula import kafka

On a 12-partition, 100k-record benchmark it runs ahead of confluent-kafka on every produce path (roughly 1.1x to 1.4x) and well ahead on Avro, at parity on plain consume, and an order of magnitude past the pure-Python clients. The Rust core does the work a Python produce/consume loop pays for per record. Full numbers and method on the Benchmarks page.

Pages

• Quickstart: produce a batch, poll it back.
• Producer and Consumer: delivery, batching, groups, offsets.
• Transactions: exactly-once produce and consume.
• Formats: raw bytes, Avro, JSON.
• Admin: topics, partitions, configs.
• Configuration and Authentication (SASL, mTLS, OAuth).
• Examples: producer+consumer round trips, EOS, Avro.

Python defaults, the type stubs, and the Rust fallbacks are kept in sync, so what the docs say matches what the runtime does.

To move data between Kafka and a table or warehouse, see Pipelines.