Benchmarks

Records per second, higher is better. Local confluentinc/cp-kafka:7.7.0, Apple M1 Max, 12 partitions, median of three runs; rows move 10-20% between runs. pyrula rows use the bulk produce and batch poll APIs. Bold is the faster of pyrula and confluent. No Linux run is published yet.

The harness is benchmarks/kafka/benchmark.py; run it against your own broker:

python benchmarks/kafka/benchmark.py --broker localhost:9092

--help lists the suites and sizes; the Avro suites need --schema-registry.

100,000 records, 1 KB

Near the broker’s ceiling (~155-175 MB/s), so the faster clients converge.

Suite	pyrula	confluent	aiokafka	kafka-python	vs confluent
Produce bytes	140k	144k	84k	6.1k	0.99x
Produce bytes, acks=1	150k	146k	n/a	n/a	1.02x
Idempotent produce, 6 partitions	157k	149k	n/a	n/a	1.09x
Produce JSON	125k	115k	77k	5.4k	1.08x
Transactional produce	91k	64k	n/a	n/a	1.41x
Produce Avro	128k	125k	n/a	n/a	1.02x
Consume records	138k	126k	103k	6.4k	1.02x
Consume raw bytes	165k	156k	n/a	n/a	1.00x
Consume raw bytes, 12 partitions	146k	156k	n/a	6.4k	1.04x
Consume + JSON decode	137k	131k	82k	5.8k	1.19x
Consume Avro	108k	110k	n/a	n/a	1.00x

1,000,000 records, 128 B

JSON and Avro rows encode a small dict, so records are under 128 B.

Suite	pyrula	confluent	aiokafka	kafka-python	vs confluent
Produce bytes	1.12M	1.08M	194k	31k	1.04x
Produce bytes, acks=1	1.06M	962k	n/a	n/a	1.10x
Idempotent produce, 6 partitions	943k	968k	n/a	n/a	0.98x
Produce JSON	1.07M	403k	189k	34k	2.75x
Transactional produce	210k	138k	n/a	n/a	1.50x
Produce Avro	1.87M	427k	n/a	n/a	4.43x
Consume records	769k	731k	474k	41k	1.06x
Consume raw bytes	939k	907k	n/a	n/a	1.06x
Consume raw bytes, 12 partitions	872k	875k	n/a	41k	1.04x
Consume + JSON decode	1.24M	113k	299k	50k	10.2x
Consume Avro	1.04M	144k	n/a	n/a	7.7x

n/a: not measured (kafka-python has no transaction API).