Webhook
A webhook is a way for one system to notify another in real time by sending an HTTP POST request to a pre-configured URL whenever a specific event happens, rather than the receiving system having to poll for updates.
Overview
A webhook inverts the usual client-server request pattern: instead of a client repeatedly asking a server "has anything happened yet?" (polling), the server proactively sends an HTTP request — usually a POST with a JSON payload — to a URL the client previously registered, the moment a relevant event occurs. This is far more efficient than polling for infrequent or unpredictable events, since no requests are wasted checking for updates that haven't happened, and consumers find out about events with much lower latency.
Typical flow
- A consumer registers a callback URL with a provider (for example, "notify me at
https://myapp.com/webhooks/stripewhen a payment succeeds"). - When the event occurs, the provider sends an HTTP POST to that URL with a payload describing the event.
- The consumer's server validates the request (commonly via a signature header, to confirm it genuinely came from the provider) and processes the event, then returns a 2xx response to acknowledge receipt.
- Providers typically retry with backoff if the receiving endpoint doesn't respond successfully, since delivery isn't otherwise guaranteed to succeed on the first attempt.
Webhooks are widely used for payment notifications (Stripe), CI/CD triggers (GitHub pushes triggering a build), chat integrations (Slack messages), and marketing/CRM automation (a form submission triggering a HubSpot workflow).
Loading webhook data for analysis
Because webhook payloads are typically JSON, a common pattern is to land raw incoming events (e.g., in a table, file, or queue) and then parse and analyze them with a SQL engine. DuckDB reads and unnests JSON naturally, which makes it convenient for exploring or aggregating webhook event logs after ingestion:
Copy code
SELECT
payload->>'event_type' AS event_type,
count(*) AS event_count
FROM read_json('webhook_events/*.json')
GROUP BY ALL
ORDER BY ALL;
Why it matters
Webhooks are the standard integration pattern for event-driven communication between independent systems on the web, avoiding the inefficiency and latency of polling-based integrations.
Related terms
Event-driven architecture is a software design pattern in which services communicate by producing and reacting to events, rather than calling each other directly, enabling loosely coupled, asynchronous systems.
Latency →Latency is the time delay between initiating a request or operation and receiving its result — for example, the time from sending a database query to getting the first byte of its response back.
FAQS
Most providers include a cryptographic signature (often an HMAC computed with a shared secret) in a request header, which the receiving server recomputes and compares to confirm the payload genuinely came from the expected sender and wasn't tampered with or forged.
Most providers implement automatic retries with increasing backoff when the receiving endpoint doesn't respond with a success status, though there are typically limits on how many retries occur before an event is considered undeliverable.
