Retries and replay

Esfera delivers each webhook at least once and keeps trying until your endpoint accepts it or the schedule runs out. This page gives the retry schedule, explains why you may receive the same event more than once and how to deduplicate, how to read the delivery log and replay a delivery, and the limits on endpoints. Read it before you put a receiver in production.

What counts as success

An attempt succeeds when your endpoint answers with a 2xx status within 10 seconds. Everything else is a failed attempt: any other status, a connection error, a TLS failure, or no answer in time. A failed attempt is scheduled again, up to the limit below.

The retry schedule

A delivery is attempted at most 8 times. After a failed attempt Esfera waits, then tries again. There are seven waits, because the eighth attempt is the last:

Attempt	Wait after it fails	About this long after the first attempt
1	60 seconds	0
2	5 minutes	1 minute
3	15 minutes	6 minutes
4	1 hour	21 minutes
5	2 hours	1 hour 21 minutes
6	4 hours	3 hours 21 minutes
7	8 hours	7 hours 21 minutes
8	none: the delivery is marked failed	15 hours 21 minutes

The waits add up to 55 260 seconds, about 15 hours 21 minutes from the first attempt to the last. Times are approximate: they are scheduled delays, not a clock you can set an alarm by. Each attempt is signed afresh, so a late retry still passes the signature tolerance.

An endpoint that is down for an afternoon will catch up when it returns. One that is down for more than about 15 hours will miss the event, and the delivery ends as failed. You can still replay it.

Two other ways a delivery ends as failed:

• You revoked the endpoint while retries were pending. The next attempt marks it failed with lastError "endpoint revoked" and sends nothing.
• A delivery has stayed pending for 26 hours, which means its retry message was lost. The server then marks it failed with lastError "retry schedule exhausted (message lost)", so that it does not look like it is still retrying. Replay it.

At-least-once delivery

Your endpoint can receive the same event more than once. It happens when your 2xx is lost on the way back, when your handler is slow and an attempt times out after it has already done the work, and when you replay a delivery. Esfera never sends two deliveries for the same event on purpose: there is at most one delivery per endpoint, conversion and event, and retries and replays re-send that one delivery.

Make your handler idempotent. The deduplication key is the pair:

(event, data.conversion.id)

conversion.completed for a conversion happens once. If you see it again for the same conversion id, it is a repeat: skip it. A simple way is a table with a unique constraint on that pair, written before you do any work.

// Insert first, act second. A repeat hits the unique constraint and is acknowledged without doing the work again.
const key = `${event.event}:${event.data.conversion.id}`;
const fresh = await db.insertIfAbsent("webhook_events", key, event);   // your store
if (fresh) await queue.push(event);                                    // do the work off the request path
res.sendStatus(200);

Two more rules follow from the design:

• Order is not guaranteed. A retried conversion.analyzed can arrive after conversion.completed. Do not assume the sequence. Each event carries the conversion as it was when the event was created, so compare updatedAt or call GET /v1/conversions/{id} if you need the current state.
• Answer quickly. Record the event and return 2xx inside 10 seconds. Do slow work asynchronously.

The delivery log

GET /v1/webhooks/{id}/deliveries shows what happened to each delivery: its status (pending, delivered or failed), how many attempts, the last HTTP status your endpoint returned (lastStatus) and the last failure (lastError). Use it first when an event seems to be missing. The fields are described in Events and payload.

curl "https://api.esfera.audio/v1/webhooks/$WEBHOOK_ID/deliveries?pageSize=50" \
  -H "Authorization: Bearer $ESFERA_API_KEY"

Replaying a delivery

POST /v1/webhooks/{id}/deliveries/{deliveryId}/replay sends a delivery again to the same endpoint. Use it after an outage on your side, or when your handler lost an event.

curl -X POST "https://api.esfera.audio/v1/webhooks/$WEBHOOK_ID/deliveries/$DELIVERY_ID/replay" \
  -H "Authorization: Bearer $ESFERA_API_KEY"

await esfera.webhooks.deliveries.replay(webhookId, deliveryId);

• It answers 202 with { "ok": true } when the delivery is queued. The delivery goes back to pending until the attempt finishes.
• You can replay a delivery that was already delivered. Your endpoint may have lost the event after it answered 2xx, and replay is the way to recover it.
• It re-sends the same event with the same conversion snapshot, signed afresh. The (event, conversion id) pair is unchanged, so a deduplicating handler treats it as a repeat. If you replay because your handler lost the event, remove or ignore your own record of it first.
• attempts keeps counting. A replay is one more attempt on the same delivery, not a new schedule. A delivery that already used all eight attempts gets exactly one more try. If that fails, it is failed again, with no retries. A delivery that succeeded on its first try, once replayed, still has the rest of the schedule available if the replay fails.
• A delivery that is already pending, whether queued by an earlier replay or mid-retry, answers 409 replay-in-flight. Wait for it to settle.
• If the delivery queue is unavailable, or refuses the message, you get 503 queue-unavailable or queue-send-failed. Nothing changed, so you can repeat the call.
• An unknown webhook or delivery answers 404.

Endpoint limits

Rule	Detail
Active endpoints per account	3. A fourth is refused with webhook-limit. Revoked endpoints do not count.
Scheme	https only.
Hosts	A public hostname. IP addresses of any kind and localhost are refused. So are Esfera's own domains, esfera.audio and everything under it, and esfera-5qr.pages.dev.
Refusal	invalid-webhook-url, with the reason in detail.

The check is on the URL as you register it. It does not look up where a hostname points, so a name that resolves to a private address is not caught at registration. Register only endpoints that are reachable from the public internet.

DELETE /v1/webhooks/{id} revokes an endpoint. It is never deleted outright, so its delivery log stays readable, and a second revoke is a 404. Revoking frees one of the three slots.

Related

• Events and payload and Verifying signatures.
• Webhooks reference for the endpoint operations.
• Retention: missing conversion.expiring is another reason to download early.