Delivery and security

How Kustomer delivers outbound webhooks. HMAC-SHA256 signatures, the retry schedule, at-least-once delivery, ordering, timeouts, automatic disabling and how to debug failed deliveries.

Kustomer delivers each event as an HTTPS POST and retries failures for about an hour. Your receiver should do three things: verify the signature, return 2xx quickly, and tolerate duplicates and reordering.

The request

HTTP
POST /kustomer HTTP/1.1
Host: hooks.example.com
Content-Type: application/json
X-Kustomer-Webhook-Event: kustomer.conversation.update
X-Kustomer-Webhook-Signature: 3f1c9a…e07b
X-Source: kustomer

{"id":"6717c2b1e5b1f00012ab3511","name":"kustomer.conversation.update",…}
HeaderValue
Content-Typeapplication/json
X-Kustomer-Webhook-EventThe event name, so you can route before parsing
X-Kustomer-Webhook-SignatureLowercase hex HMAC-SHA256 of the raw body, keyed with the webhook's token
Your custom headersWhatever you set in headers when creating the webhook

Verify signatures

The token returned when you create the webhook is the signing secret. You can also read it later with GET /v1/outbound-webhooks/{id}, or in Settings › Platform › Outbound webhooks › Edit › Header information.

Text
signature = hex( HMAC_SHA256( key = token, message = raw request body ) )
import crypto from 'node:crypto'

export function verifyKustomer(rawBody: Buffer, signature: string | undefined, token: string): boolean {
  if (!signature) return false
  const expected = crypto.createHmac('sha256', token).update(rawBody).digest('hex')
  return signature.length === expected.length && crypto.timingSafeEqual(Buffer.from(signature), Buffer.from(expected))
}

Always compare in constant time (timingSafeEqual, compare_digest, hmac.Equal). A plain === leaks how many leading characters matched.

Rotate the token

There's no rotate endpoint. To rotate, create a second webhook with the same URL and events, update your receiver to accept either token, then delete the old webhook.

Respond fast

Kustomer waits up to 30 seconds for a response. Return 200 as soon as the signature checks out, then do the work from a queue. Slow responses tie up deliveries and count as failures when they time out.

WorkerQueueYour receiverKustomerWorkerQueueYour receiverKustomerPOST event (signed)verify signatureenqueue raw body200 OK (milliseconds)deliverdedupe on event id, apply
Diagram source
sequenceDiagram
  participant K as Kustomer
  participant R as Your receiver
  participant Q as Queue
  participant W as Worker
  K->>R: POST event (signed)
  R->>R: verify signature
  R->>Q: enqueue raw body
  R-->>K: 200 OK (milliseconds)
  Q->>W: deliver
  W->>W: dedupe on event id, apply

Retries

A delivery fails on a timeout, a connection error, or a status of 400 or higher. Kustomer retries a failed event up to 8 attempts in total, over about 64 minutes:

AttemptDelay before itTime since the first attempt
1—0
21 min1 min
31 min2 min
42 min4 min
55 min9 min
610 min19 min
715 min34 min
830 min64 min

Delays vary by about ±10% so retries from many events don't arrive in lockstep. If all 8 attempts fail, Kustomer drops that event. It isn't redelivered later, so plan to reconcile after an outage. A delivery whose host name doesn't resolve (a DNS failure) isn't retried at all.

InteractiveRetries for one event
Your endpoint:503 until attempt4
T+0minPress play to send the event.
compressed time · 15 min ≈ 1.2 s
  1. 1
    Attempt 1T+0 min
    ↓ wait 1 min
  2. 2
    Attempt 2T+1 min
    ↓ wait 1 min
  3. 3
    Attempt 3T+2 min
    ↓ wait 2 min
  4. 4
    Attempt 4T+4 min
    ↓ wait 5 min
  5. 5
    Attempt 5T+9 min
    ↓ wait 10 min
  6. 6
    Attempt 6T+19 min
    ↓ wait 15 min
  7. 7
    Attempt 7T+34 min
    ↓ wait 30 min
  8. 8
    Attempt 8T+64 min
To scale: the same attempts on a real clock0–70 min
010203040506070
A delivery fails on a timeout (no response in 30 s), a connection error, or any status of 400 or higher, 4xx included. Failed events are retried up to 8 times in total.
Same event every attemptsent 0×
X-Kustomer-Webhook-Event: kustomer.conversation.update
{"id":"6717c2b1e5b1f00012ab3511", …}
Latest transaction
none yet

Delivery is at least once: even a 200 can be followed by a duplicate if the response is lost. Deduplicate on the envelope's id.

Delays are shown at their nominal values; real delays vary by about ±10%. Disabling is judged on a webhook's failures sustained over time, not on one event, so it isn't placed on this timeline.

Duplicates and ordering

Delivery is at least once. The same event can arrive twice, for example if your 200 was lost on the way back. Deduplicate on the envelope's id:

Node.js
async function handle(event: KustomerEvent) {
  const firstTime = await redis.set(`kustomer:event:${event.id}`, '1', { NX: true, EX: 60 * 60 * 24 })
  if (!firstTime) return
  await apply(event)
}

Order isn't guaranteed either: a retried event can arrive after a newer one for the same record. Compare data.attributes.updatedAt with what you stored and skip older snapshots:

Postgres
INSERT INTO conversations (id, status, updated_at, doc)
VALUES ($1, $2, $3, $4)
ON CONFLICT (id) DO UPDATE
  SET status = EXCLUDED.status, updated_at = EXCLUDED.updated_at, doc = EXCLUDED.doc
  WHERE conversations.updated_at < EXCLUDED.updated_at;

Automatic disabling

If your endpoint keeps failing, Kustomer turns the webhook off so it stops sending to a dead destination. This is based on failures sustained over time, not a short blip: a brief outage or a burst of errors on a busy webhook won't disable it, but an endpoint that stays down for tens of minutes will. Successful deliveries count in your favor.

When a webhook is disabled:

  • enabled becomes false and isError becomes true.
  • Admins see an error notification in Kustomer.
  • Events that happen while it's off are not queued.

Fix the endpoint, then re-enable it with PATCH /v1/outbound-webhooks/{id} and { "enabled": true }, or with the toggle in Settings › Platform › Outbound webhooks. Re-enabling resets the failure history.

Debug deliveries

List the 25 most recent attempts for a webhook:

get/v1/outbound-webhooks/{webhookId}/transactions
200 OK
{
  "data": [
    {
      "type": "outbound_webhook_transaction_log",
      "id": "5f2c62067aba570013e64bb1",
      "attributes": {
        "webhookId": "5f0f572d3919cc00123fd52e",
        "eventName": "kustomer.conversation.update",
        "sentAt": 1729522931273,
        "status": "failed",
        "nextRetry": "1729522991000"
      }
    }
  ]
}

status is succeeded or failed. nextRetry is a millisecond timestamp for the next attempt, or null once the event succeeded or ran out of attempts.

Common causes of failures:

SymptomLikely cause
Every delivery fails with 401Your receiver's signature check is hashing parsed JSON instead of the raw body
TimeoutsDoing work before responding; move it to a queue
403 or no requests arriveA firewall, WAF or bot protection is blocking Kustomer; allow the webhook path
Deliveries show succeeded but never reach your codeThe URL redirects (for example to add a trailing slash). Redirects aren't followed but count as delivered, so point the webhook at the final URL
Some events never arriveThey exhausted their retries during an outage; reconcile

Reconcile after an outage

Webhooks are for low latency, not completeness. After an outage longer than about an hour, or after re-enabling a webhook, fetch what changed with search using an updated_at window that starts before the outage began. Because your handler ignores older snapshots, overlapping with events you already processed is harmless.

Network and security checklist

  • HTTPS endpoint with a valid certificate
  • Signature verified over the raw body, in constant time
  • Token stored in a secret manager, not in code
  • 200 returned within a few seconds; work done asynchronously
  • Deduplication on event id
  • Older snapshots ignored using updatedAt
  • Alerting on non-2xx rates and on the webhook being disabled
  • A reconciliation job for outages