OSS HTTP Streams Benchmark

A uniform HTTP benchmark for Ursula, Durable Streams, and S2 Lite S3. It covers bulk append/read, small-event append, and SSE live-tail delivery.

Benchmark setup

Workload

Bulk append

8 KiB records, 100 sequential latency samples, then a 15-second write throughput phase.

Catch-up read

512 KiB records, read after data is visible; throughput validates status and response length.

Small events

1 KiB append workload reported as request/s, not MiB/s.

SSE live tail

512 B text events with open SSE subscriptions; reported as append-to-delivery latency. Ursula uses three subscribers across replicas.

Environment

Role	Instance type	vCPU	Memory	Network
Service instance	c5.xlarge	4	8 GiB	Up to 10 Gbit/s
Load generator	c6in.4xlarge	16	32 GiB	Up to 50 Gbit/s

Server configuration

Ursula

3 EC2 nodes
Writes go to the leader
Reads are distributed across replicas
Local log/store, memory hot payload backend, S3 cold-storage config

Durable Streams

Single EC2 process
Single-node reads and writes
File-backed data directory

S2 Lite S3

Single EC2 process
Single-node S2 Lite API
S3 object-store backend

Baseline comparison at c256

Metric	Ursula vs S2 Lite S3
Write throughput	1.5x higher
Read throughput	3.7x higher
Small-event append	1.2x higher req/s
Append p99 latency	1.8x lower
SSE live-tail p99	2.1x lower
Client-observed errors	0 vs 0

Metric	Ursula vs Durable Streams
Write throughput	10.3x higher
Read throughput	19.8x higher
Small-event append	17.2x higher req/s
Append p99 latency	7.6x higher (worse)
SSE live-tail p99	3.8x higher (worse)
Client-observed errors	0 vs 89

Availability posture

Target	Serving availability estimate	Acked-write durability	Failure posture
Ursula	~99.99%	RPO 0 while a Raft majority survives; writes are committed through the replicated quorum path.	Tolerates one service instance or availability-zone loss when two of three voters remain reachable; leader loss causes a short election and retry window.
S2 Lite S3	~99.9%	RPO 0 for acknowledged writes with object storage enabled; writes are durable in object storage before acknowledgment.	The benchmarked Lite process is a single serving point. Instance loss stops service until restart or replacement, but acknowledged data can be recovered from object storage.
Durable Streams	~99.9%	Durability depends on the local file-backed store. The official file store documents a crash-atomicity caveat for producer state recovery.	The benchmarked server is one process with local state. Process restart can recover local data, while instance or AZ loss depends on volume recovery, backup, or external replication.

Trend

UrsulaDurable StreamsS2 Lite S3

c256 results

Target	Write MiB/s	Read MiB/s	Small-event req/s	SSE p99 ms	Errors
Ursula	19.59	3,309.93	4,294	41.18	0
Durable Streams	1.90	167.35	250	10.71	89
S2 Lite S3	12.89	891.72	3,617	87.69	0

Full results

Concurrency	Target	Append p99 ms	Read p99 ms	Write MiB/s	Read MiB/s	Small-event req/s	SSE p99 ms	Errors
4	Ursula	54.24	0.58	1.35	1,871.03	203	40.83	0
4	Durable Streams	6.84	7.83	1.80	171.15	233	12.60	0
4	S2 Lite S3	128.95	0.67	0.52	770.72	72	100.49	0
8	Ursula	58.12	0.56	2.27	3,046.62	382	41.80	0
8	Durable Streams	6.57	8.19	1.73	169.95	216	10.61	0
8	S2 Lite S3	107.89	0.78	1.04	862.31	135	187.74	0
16	Ursula	59.11	0.60	3.66	3,521.28	705	40.62	0
16	Durable Streams	7.80	8.14	1.74	174.76	224	10.67	0
16	S2 Lite S3	114.85	0.78	2.00	909.06	275	114.29	0
32	Ursula	52.78	0.51	6.34	3,510.55	1,199	44.23	0
32	Durable Streams	6.81	8.11	1.75	178.15	223	10.92	0
32	S2 Lite S3	107.30	0.95	3.45	941.92	455	146.38	0
64	Ursula	54.59	0.60	9.08	3,456.31	2,177	40.29	0
64	Durable Streams	6.59	7.98	1.74	169.95	225	10.74	0
64	S2 Lite S3	118.43	0.76	3.50	913.12	302	97.84	0
128	Ursula	58.68	0.70	14.73	3,442.20	3,489	21.72	0
128	Durable Streams	6.73	9.29	1.89	171.44	260	10.42	0
128	S2 Lite S3	114.09	0.91	6.27	949.27	1,826	98.50	0
256	Ursula	55.66	0.56	19.59	3,309.93	4,294	41.18	0
256	Durable Streams	7.35	8.37	1.90	167.35	250	10.71	89
256	S2 Lite S3	102.02	0.70	12.89	891.72	3,617	87.69	0

Reproduce

Use crates/perf-compare as the load generator. Run it from a separate EC2 instance in the same private network as the tested services, after starting each target with the server configuration above.

cargo run --release -p perf-compare --bin perf_compare -- \
  --ursula http://URSULA_LEADER:4437 \
  --ursula-read-bases http://URSULA_NODE_1:4437,http://URSULA_NODE_2:4437,http://URSULA_NODE_3:4437 \
  --durable http://DURABLE_STREAMS:8080 \
  --s2 http://S2_LITE:3000 \
  --payload-bytes 8192 \
  --small-payload-bytes 1024 \
  --read-payload-bytes 524288 \
  --sse-payload-bytes 512 \
  --latency-count 100 \
  --throughput-secs 15 \
  --sse-count 100 \
  --sse-readers 3 \
  --concurrency 256

Repeat the same command with --concurrency set to 4, 8, 16, 32, 64, 128, and 256. The benchmark emits JSON containing append latency, read latency, write throughput, read throughput, small-event request/s, SSE delivery latency, and client-observed errors for each target.