Reconnect and replay

Starting state#

You follow a job, an application or an organization. Notifications are at-least-once, unordered and lossy: the same event can arrive twice, late or never. The persisted sequence of each channel is the truth, and work continues whether or not a tab is open.

Operations#

A cursor is the decimal sequence of the last event you hold for the channel. 0 replays from the first retained event.

Procedure#

1. Subscribe first. Join the channel with your grant and buffer whatever arrives. Do not apply it yet.
2. Load the snapshot. It answers the authorized state of the channel in data, the sequence that state reflects, and the matching cursor.
3. Apply only what follows. Discard buffered events whose sequence is not greater than the snapshot's, then apply the rest in sequence order. Subscribing before the snapshot is what makes this race lose nothing.
4. Discard duplicates by id and by sequence.
5. Close gaps. When an event arrives whose sequence is not the next one, hold it and replay from your cursor. Apply the answer in order, then the events you held. A pointer, the broadcast event beyond-cdn-pointer/1, is the same situation announced explicitly: it names a sequence that did not travel, because the event was oversize or its notification was stale. It is not an event. If its sequence is above your cursor, replay from your cursor; otherwise ignore it.
6. Start over when told to. On 410 CURSOR_EXPIRED, on a resync.required event, or when you hold more out-of-order events than you are willing to buffer, load a new snapshot and continue from step 3.

The snapshot data holds management documents of the channel's scope: job for a job channel; application, jobs and environments for an application channel; organization, jobs and credit for an organization channel.

A replay page answers events, the new cursor, and more, which is true when further events follow. A replay returns events only: pointers are never persisted and never appear in it.

Without a subscription#

The transport makes updates arrive sooner. It is never required for correctness. This client follows a job from persisted events alone:

// Follow a job from persisted events only: load a snapshot, then replay after
// its cursor. A broadcast subscription makes this faster; it never replaces it.
const api = process.env.CDN_API_ORIGIN;
const token = process.env.CDN_TOKEN;
const channel = `job:${process.env.CDN_JOB}`;

const headers = { Authorization: `Bearer ${token}` };
const seen = new Set();

async function read(path) {
	const response = await fetch(new URL(`/v1/streams/${channel}/${path}`, api), { headers });
	const document = await response.json();
	if (response.ok) return document;
	if (document.error.code === 'CURSOR_EXPIRED') return undefined;
	throw new Error(`${response.status} ${document.error.code} — ${document.error.message}`);
}

let snapshot = await read('snapshot');
let cursor = snapshot.cursor;
console.log(`snapshot at sequence ${snapshot.sequence}: job is ${snapshot.data.job.state}`);

const final = ['job.succeeded', 'job.failed', 'job.cancelled', 'job.limit_exceeded'];
let ended = ['succeeded', 'failed', 'cancelled', 'limit_exceeded'].includes(snapshot.data.job.state);

while (!ended) {
	const page = await read(`events?after=${cursor}`);

	if (!page) {
		// The cursor is older than the retained events: start again from a snapshot
		snapshot = await read('snapshot');
		cursor = snapshot.cursor;
		continue;
	}

	for (const event of page.events) {
		if (seen.has(event.id)) continue; // the same event can arrive twice
		seen.add(event.id);
		console.log(`#${event.sequence} ${event.type}`);
		if (final.includes(event.type)) ended = true;
	}

	cursor = page.cursor;
	if (!page.more && !ended) await new Promise(resolve => setTimeout(resolve, 2000));
}

The same loop is what you run after a reconnect: keep your cursor, replay after it, and continue.

Expected outcome#

After a reconnect of any length you hold the same state as a client that never disconnected, without having applied any event twice. Final events such as job.succeeded, job.failed and release.ready are in the replay even if no notification for them ever reached you.

Limits#

• Replay needs the same access as the snapshot. After a revocation both answer 403 ACCESS_REVOKED or 404 NOT_FOUND.
• Persisted job logs have their own paged endpoint, jobs.logs, with a longer retention than events.