2026-09-19 — Engineering · Tutorials

Design a webhook receiver that survives outages

Async task results arrive as webhook deliveries. If your endpoint is down when a delivery lands, retries cover you — but a well-built receiver shouldn’t depend on retries at all.

The durable pattern

1. Verify, ack, enqueue. The HTTP handler does three things: verify the signature, return 200 immediately, and put the raw event on a queue (SQS, a Redis list, a Postgres table). Processing happens in a worker, not the request.
2. Idempotent storage. Store results keyed by task ID — a retried delivery upserts the same row instead of duplicating.
3. Respond fast. Answer in milliseconds. Slow receivers hit delivery timeouts and generate retry storms for no reason.

@app.post("/hook")
def hook(request: Request):
    verify_signature(request)          # raises 401 on failure
    event = request.json()
    queue.push(event)                  # durable: survives process death
    return "", 200                     # ack before doing real work

When retries aren’t enough

For long maintenance windows, don’t rely on retry back-off — list tasks directly. GET /v1/async/tasks returns your queue with statuses, so a recovery job can reconcile anything missed: fetch every task marked completed whose result you never stored.

The failure modes worth testing

• Receiver returns 500 → delivery retries; verify your idempotency survives the duplicate
• Two deliveries of the same event → second must no-op
• Event for an unknown task ID → log and 200; orphan events happen on key reuse
• Signature failure → 401, alert — either the secret rotated or something’s probing you

Signature verification code is in the security post; delivery format and retry schedule in the webhooks guide.

Keep reading