Webhook Reliability Playbook

Duplicate Shopify Webhooks

Use this playbook when the same Shopify-related work appears to happen twice. The important question is not only whether the payload repeated; it is whether the business side effect repeated.

Written by TechYelp Engineering

duplicateidempotencydeduplicationretry
Platform Shopify
Pipeline area Retry and processing layers
Typical impact Duplicate ERP order, inventory deduction, fulfillment, refund, email, or shipping label
Suggested owner Application owner with access to delivery attempts, queue logs, worker logs, and downstream business records
Diagnostic complexity High
Recovery risk Very high if cleanup requires reversing external side effects

Verified Shopify identifier facts

Shopify documents X-Shopify-Webhook-Id as the individual delivery identifier for deduplicating repeated delivery copies and X-Shopify-Event-Id as the event correlation identifier shared by deliveries from the same merchant action Shopify duplicate docs.

Current Webhook Guard product behavior verified from implementation: Shopify duplicate grouping uses the extracted Shopify event correlation ID, X-Shopify-Event-Id, scoped by platform and store. Use that grouping as correlation and product-behavior evidence; verify the raw X-Shopify-Webhook-Id before concluding two records are repeated copies of the same Shopify delivery.

First five minutes

  • Collect Shopify delivery IDs, Shopify event correlation IDs, business object IDs, Webhook Guard event IDs, and Webhook Guard downstream attempt numbers.
  • Check whether the repeated records share the same Shopify delivery ID or only the same business object.
  • Determine whether any operator manually retried or replayed the event.
  • Stop additional retries for the unsafe operation while you classify the duplicate.
  • Check downstream records before deciding to suppress or replay anything.

Evidence to collect

Evidence Where to find it What it proves What it does not prove
Shopify delivery ID and event correlation ID Request headers, Shopify delivery logs, or Webhook Guard event detail Whether the exact delivery repeated or several deliveries came from one merchant action Whether downstream processing was idempotent
Webhook Guard duplicate record Webhook Guard Duplicate Events tab Webhook Guard observed repeated inbound events grouped by its current platform event key That no downstream side effect repeated before or outside Webhook Guard
Destination attempt timeline Webhook Guard event detail Whether Webhook Guard retried downstream delivery and which attempts reached the endpoint That the endpoint safely ignored repeated attempts
Business-operation key Application database, ERP order external ID, fulfillment reference, or idempotency table Whether two side effects used the same intended business operation Which retry layer caused the duplicate without delivery evidence

Focused failure-stage map

Shopify deliveryCompare Shopify delivery ID and event correlation ID.Downstream attemptsCheck whether Webhook Guard retried delivery to the endpoint.Worker executionFind repeated worker attempts for the same business-operation key.Side effectsVerify ERP, inventory, fulfillment, refund, email, and label outcomes.

Fast decision tree

  1. Do the repeated requests have the same X-Shopify-Webhook-Id? If yes, classify them as repeated deliveries of one Shopify delivery; if Webhook Guard only shows the platform event key, verify the raw headers before making this conclusion.
  2. Do they have different delivery IDs but the same X-Shopify-Event-Id or business object? If yes, classify as related deliveries from one merchant action or topic behavior.
  3. Did Webhook Guard show multiple downstream attempts for one event? If yes, classify whether the destination failed to acknowledge or Webhook Guard retried after failure.
  4. Did the application create duplicate side effects despite only one successful delivery? If yes, inspect worker retry, queue redelivery, race conditions, and missing idempotency.
  5. Before cleanup, identify which downstream record should survive and which related inventory, fulfillment, tax, customer, or notification state must be corrected.

Root-cause matrix

Delivery retries

Possible cause Evidence that supports it Evidence that eliminates it Corrective action
Shopify retried after timeout or non-2xx Same delivery identifiers or Shopify retry attempts around a failed response Only one Shopify delivery exists and duplicate work appears only inside the application Respond only after durable acceptance and make downstream operations idempotent.
Webhook Guard retried downstream delivery Multiple Webhook Guard delivery attempts with increasing attempt numbers One successful downstream attempt and duplicate business commands emitted later Fix endpoint acknowledgment and make retry handling idempotent.

Application processing

Possible cause Evidence that supports it Evidence that eliminates it Corrective action
Queue or worker redelivery repeated the side effect One HTTP delivery but multiple worker executions for the same business key A unique constraint or idempotency table shows only one command was accepted Add transactional idempotency around the external side effect.
Deduplication dropped a legitimate later update Different event or trigger timestamps for the same resource but suppression treated them as identical Same delivery ID and same raw payload were repeated after a failure Deduplicate delivery retries by delivery ID, but use business rules and timestamps for resource updates.

Illustrative example — not a real customer incident

Example diagnosis

EvidenceObserved result
Incident evidenceOne Shopify delivery ID.
Incident evidenceOne successful Webhook Guard downstream attempt.
Incident evidenceTwo worker executions for the same business-operation key.
Incident evidenceTwo ERP create requests.

Diagnosis

The duplicate side effect originated in customer application processing, not duplicate Shopify delivery.

Why that diagnosis follows

Only one inbound delivery and one Webhook Guard downstream attempt exist, while the repeated evidence starts inside the customer worker and ERP request layer.

Safest next action

Stop the duplicate-producing worker path, choose the surviving ERP record, add transactional idempotency, and reconcile dependent state.

What not to do

Do not change Shopify duplicate suppression when only one inbound delivery occurred.

Detailed diagnostic procedure

  1. Group all related requests by Shopify delivery ID, Shopify event correlation ID, business object ID, Webhook Guard event ID, destination attempt number, and downstream business key.
  2. Build a timeline of platform delivery, Webhook Guard attempts, endpoint responses, queue messages, worker attempts, and external side effects.
  3. Classify each repeated action as deduplication, idempotency, correlation, or replay-safety evidence.
  4. Check whether a resource legitimately changed more than once, such as an order update after order creation.
  5. Repair downstream state using the business object and surviving operation key, then record the duplicate source.

Immediate containment

  • Disable only the duplicate-producing worker or replay action if it is still active.
  • Block new side effects for the affected business key while allowing unrelated events to continue.
  • Preserve every repeated request and operator action for the incident record.

Permanent fix

  • For application-owned deduplication, use Shopify delivery IDs to deduplicate repeated Shopify delivery copies; treat Shopify event IDs as correlation IDs.
  • Use stable business-operation keys to make ERP order creation, fulfillment, refund, inventory, email, and label creation idempotent.
  • Add uniqueness constraints around irreversible side effects.

Architecture improvement

  • Separate duplicate detection from idempotent business command execution.
  • Store event correlation IDs and downstream external IDs together.
  • Make manual replay require a recovery gate and audit note.

Safe recovery gate

  • I classified the retry layer.
  • I identified the surviving downstream record.
  • I checked every irreversible side effect.
  • I have a durable business-operation key.
  • I can reconcile dependent inventory, fulfillment, refund, email, and label state.

What Webhook Guard can and cannot prove

Can show

  • Inbound events Webhook Guard classifies as duplicates using its current platform event key.
  • Downstream retry attempt numbers and responses.
  • Whether Webhook Guard classified a repeated inbound copy as duplicate and recorded the occurrence.

Cannot prove

  • It cannot prove that the customer application avoided duplicate side effects after accepting a delivery.
  • It should not be treated as proof that every Shopify subscription delivery shared the same X-Shopify-Webhook-Id.
  • It cannot decide whether a later resource update is legitimate without business context.

Webhook Guard evidence for this incident

Webhook Guard Duplicate Events and downstream attempt history help separate repeated inbound copies from Webhook Guard retries to the destination endpoint.

Limitations

  • Duplicate classification cannot prove the customer application avoided duplicate side effects.
  • Current duplicate grouping should be treated as product-behavior evidence and correlated with raw Shopify delivery IDs before changing application logic.

Claim sources

Verify webhook deliveries · Webhooks delivery structure · Webhook Guard User Guide

User Guide

Open Duplicate Events guidance

Production runbook

  1. Group by identifiers.
  2. Classify the retry layer.
  3. Find repeated business side effects.
  4. Choose the surviving downstream record.
  5. Add idempotency before reopening retries.

Sources and last verification

Sources and verification