What a dropped stream costs you
Three client reconnect strategies, four injected failure modes, and exact accounting of what each one loses: gap slots, missed events, duplicate delivery and time to catch back up.
Triton shipped built-in Yellowstone auto-reconnect and Helius sells a replay buffer, which makes “how bad is a disconnect” a live question with almost no published numbers behind it. So we measured it.
This is not a provider comparison
We do not hold paid accounts on Helius, QuickNode, Triton, Chainstack or Shyft, so we cannot measure their reconnect behaviour and we do not pretend to. Any cross-provider reliability ranking published without those accounts is guesswork.
What is measured here is the client-side reconnect strategy, against a local stream server whose failures we trigger at an exact slot. That is the part you own, it is the same code whichever provider you point it at, and it decides whether a four-second blip costs you nothing or costs you hundreds of slots.
The headline
Naive reconnect
Reconnect and subscribe from the live head. The default in most tutorials, including ones we have published.
Resume from last slot
Track the last processed slot and resubscribe from it. What a provider replay buffer or Triton-style auto-reconnect buys you.
Dual overlapping streams
Keep two subscriptions open permanently and deduplicate by signature. Costs double and cannot be beaten on coverage.
There is no free option. Naive reconnect loses data. Resuming recovers it and hands you duplicates. Overlapping streams eliminate gaps entirely and cost double plus a flood of duplicates. Pick based on whether your consumer is idempotent and whether a gap is recoverable for you — not on which number looks best.
Per failure mode
Median-coverage run of 3. Time to parity is the honest recovery figure: time to resume can include bytes already in flight when the fault landed, which flatters a stall.
clean-kill
Socket closed cleanly mid-stream. The easy case: the client learns immediately.
| Strategy | Gap | Missed | Duplicates | Recovered | To parity | Coverage |
|---|---|---|---|---|---|---|
| Naive reconnect | 2 slots | 8 | 0 | 0 | 45.6ms | 97.96% |
| Resume from last slot | 0 slots | 0 | 0 | 0 | 26.5ms | 100% |
| Dual overlapping streams | 0 slots | 0 | 380 | 0 | 42.7ms | 100% |
abrupt-reset
Socket destroyed with no FIN. The client only finds out on a failed read, so detection itself costs time.
| Strategy | Gap | Missed | Duplicates | Recovered | To parity | Coverage |
|---|---|---|---|---|---|---|
| Naive reconnect | 2 slots | 8 | 304 | 0 | 45.2ms | 97.96% |
| Resume from last slot | 0 slots | 0 | 304 | 0 | 46.3ms | 100% |
| Dual overlapping streams | 0 slots | 0 | 952 | 0 | 46.5ms | 100% |
silent-stall
Connection stays open and simply stops delivering. The worst case, and the one a client with no read-idle timeout never detects.
| Strategy | Gap | Missed | Duplicates | Recovered | To parity | Coverage |
|---|---|---|---|---|---|---|
| Naive reconnect | 13 slots | 52 | 0 | 0 | 266.8ms | 86.73% |
| Resume from last slot | 0 slots | 0 | 0 | 0 | 267.2ms | 100% |
| Dual overlapping streams | 0 slots | 0 | 336 | 0 | 266.5ms | 100% |
repeated-flap
Four faults in quick succession, to check that reconnect backoff does not itself become the outage.
| Strategy | Gap | Missed | Duplicates | Recovered | To parity | Coverage |
|---|---|---|---|---|---|---|
| Naive reconnect | 17 slots | 68 | 1,116 | 0 | 120.1ms | 84.96% |
| Resume from last slot | 0 slots | 0 | 1,196 | 0 | 119.3ms | 100% |
| Dual overlapping streams | 0 slots | 0 | 3,964 | 0 | 423.4ms | 100% |
What to actually do about it
- 1
Add a read-idle timeout, today
This is the highest-value line of code in this entire page. A stalled connection reports itself as healthy — TCP is fine, your logs are quiet, and you are not receiving data. Nothing detects it except a timer that fires when no bytes have arrived for N milliseconds. We use 250ms in the harness; pick a value comfortably above your slot interval and tear the connection down when it fires.
- 2
Track the last slot you processed
Persist it, not just hold it in memory, because the process that crashes is the one that needed it. Without it you cannot ask to resume, and reconnecting from the head silently discards the outage.
- 3
Deduplicate by signature before you act
Any strategy that recovers a gap will re-deliver events at the boundary. If your consumer places orders rather than updating rows, duplicate delivery means double execution. A bounded LRU of recent signatures is enough.
- 4
Cap your backoff and add jitter
Exponential backoff that grows unbounded converts a series of short blips into an outage you caused yourself — visible in the repeated-flap row above. Cap it low (we cap at 400ms) and jitter it so a fleet of workers does not reconnect in lockstep and stampede the endpoint.
- 5
Decide whether a gap is recoverable for you
If you can backfill from JSON-RPC afterwards, a gap is an annoyance. If it is permanent data loss, you need a provider with replay — Helius LaserStream or Triton Fumarole. We do not offer one, so on this axis we are not your answer.
Method and limitations
Setup: a local stream server emitting 4 events per slot every 20ms with 5,000slots of replay available. Faults are injected at an exact slot, which is the whole reason for using a mock rather than waiting for a real network to misbehave. Every metric is computed against the server's own record of what it produced, so a gap is counted, not inferred.
Loopback, not the internet: timings exclude real DNS, TCP handshake and TLS. Add roughly a round trip plus TLS for a real endpoint. The relative ordering of the three strategies is what transfers, not the absolute milliseconds.
Steady rate, not bursty: the mock emits at a fixed rate. Real Solana traffic is bursty, and a burst landing exactly at a reconnect is harder than anything measured here — so treat these as a floor on what an outage costs.
Resume assumes the server can replay: in this harness it always can. In the real market most providers cannot, including us. That asymmetry is the finding, not a flaw in the measurement.
Machine: AMD EPYC 7763 64-Core Processor, 2 threads, linux 6.17.0-1022-azure x64. Run in CI.
Run it yourself
npx tsx benchmarks/reliability/harness.ts --repeats 3 # or against any Yellowstone-compatible endpoint, including a competitor's YELLOWSTONE_ENDPOINT=... YELLOWSTONE_TOKEN=... \ npx tsx benchmarks/reliability/harness.ts --mode live
If you run it against another provider, the result is yours to publish — and we would like to see it. Corrections to benchmarks@subglow.io.
FAQ
What happens if my Yellowstone gRPC stream disconnects?
You lose everything produced between the disconnect and the moment you resubscribe, unless you explicitly ask to resume from where you stopped. Measured with fault injection across four failure modes, a naive reconnect — reconnect, subscribe from the live head — dropped up to 17 consecutive slots and 68 events in a single incident, ending at 84.96% coverage in the worst scenario. Resuming from the last processed slot recovered 100% of events in every scenario, but only because the server in this test can replay; most providers cannot, which is the real constraint.
Which failure mode is worst?
The silent stall — a connection that stays open and simply stops delivering data. A closed socket tells your client something is wrong immediately. A stalled one tells it nothing: TCP reports a healthy connection, your logs look normal, and you sit there believing you are streaming. The only thing that detects it is a read-idle timeout, which most example clients do not have. In our run the naive strategy lost 13 slots to a stall against 2 to a clean close. If you add one thing to your client after reading this, add the idle timeout.
Does resuming from the last slot cause duplicate events?
Yes, and you have to handle it. Resuming recovered every event in our test, but delivered 1500 duplicates across the run, because a resume boundary overlaps whatever was already in flight. Dual overlapping streams are worse still at 5632 duplicates by design. If your consumer is not idempotent — if it places an order rather than updating a row — duplicate delivery will double-execute. Deduplicate by transaction signature before acting, not after.
Is this a comparison of Helius, QuickNode and Triton reliability?
No, and we want to be blunt about that because it would be the obvious thing to fake. We do not hold paid accounts on those providers, so we cannot measure their reconnect behaviour, and any cross-provider reliability ranking published without those accounts is guesswork. What this measures is the client-side strategy against a stream whose failures we control exactly — which is the part you own, and which behaves the same whichever provider you point it at.
Do I need a provider with a replay buffer?
It depends entirely on whether a gap is recoverable for you. If you can backfill from JSON-RPC afterwards, or if missing a few hundred slots of an analytics feed is survivable, you do not. If a gap is permanent data loss — an indexer, an accounting system, anything that has to be complete — then you do, and your options are Helius LaserStream with roughly 216,000 slots of replay, or Triton Fumarole, which is built for durable resumable streams. Subglow offers no replay buffer, so on this axis we are not the answer, and we would rather say that than let you find out during an outage.
How aggressive should reconnect backoff be?
Less aggressive than you think, and this is measurable. Our repeated-flap scenario fires four faults in quick succession, and capped exponential backoff meant the naive strategy took 120.1ms to reach parity with the live head versus 45.6ms for a single clean kill. Backoff that grows too fast turns a series of 200ms blips into a multi-second outage of your own making. Cap it low — we cap at 400ms — and add jitter so a fleet of workers does not reconnect in lockstep.