Bry

Posted on Aug 25 Originally published at Medium

Azure Event Grid: Push-Based Events Without the Kafka Overhead

#cloud #azure #typescript #backend

Key Points

Event Grid is a push-based event router, not a streaming log. Azure's actual Kafka-equivalent is Event Hubs — conflating the two leads teams to pick the wrong service for high-throughput streaming.
Pricing is $0.60 per million operations after the first 100,000 free per month, and delivery is at-least-once, never exactly-once — every subscriber you build must be idempotent, full stop.
A custom topic, a webhook subscription, and dead-letter configuration take four CLI commands — dead-lettering specifically is the one step almost every quickstart skips.
Choose Event Grid when you need to react to discrete events (a blob uploaded, an order placed). Choose Event Hubs when you need to process a continuous stream and replay it.

Prerequisites

CLI/SDK version tested against: az-cli 2.6x.x
An Azure resource group and a storage account for dead-letter destination (az storage account create)
A publicly reachable HTTPS webhook endpoint to receive events — the examples use a placeholder Azure Function URL

Introduction

I've had the same conversation with two different teams: "we need event-driven architecture on Azure, so we need Kafka." Neither actually needed Kafka. Both needed to react when something happened — a file landed in storage, an order got placed — not to process an unbounded stream of records with replay and consumer-group semantics. That's Event Grid's job, and it does it in far less setup than standing up Event Hubs or a managed Kafka cluster would require.

The confusion is understandable. Azure has three services that all get called "event-driven": Event Grid, Event Hubs, and Service Bus. Only Event Hubs is the Kafka-equivalent. Event Grid is closer to a fan-out push notification system — publish an event, Event Grid delivers it via HTTPS webhook (or a handful of other supported destinations) to every matching subscriber, with retries and dead-lettering built in.

This will explain how to build a real topic and subscription from the CLI, including the dead-letter setup nearly every quickstart skips, and draw the line clearly between what Event Grid is for and what it isn't.

Where Event Grid Sits

Diagram: one topic, multiple independent subscriptions, each with its own delivery and dead-letter configuration.

Diagram: Event Grid pushes discrete events to subscribers; Event Hubs is the pull-based, replayable stream for continuous data — they solve different problems.

Building It From the CLI

# 1. A custom topic — the entry point events get published to.
az eventgrid topic create \
  --name orders-topic \
  --resource-group rg-eventgrid-demo \
  --location eastus

TOPIC_ID=$(az eventgrid topic show \
  --name orders-topic \
  --resource-group rg-eventgrid-demo \
  --query id --output tsv)

TOPIC_ENDPOINT=$(az eventgrid topic show \
  --name orders-topic \
  --resource-group rg-eventgrid-demo \
  --query endpoint --output tsv)

TOPIC_KEY=$(az eventgrid topic key list \
  --name orders-topic \
  --resource-group rg-eventgrid-demo \
  --query key1 --output tsv)

# 2. A storage container for dead-lettered events — set this up before
#    the subscription, not after something starts failing silently.
az storage container create \
  --name eventgrid-deadletter \
  --account-name eventgriddemostorage

# 3. A webhook subscription with retry policy and dead-letter destination.
az eventgrid event-subscription create \
  --name order-notifications \
  --source-resource-id "$TOPIC_ID" \
  --endpoint "https://order-processor.azurewebsites.net/api/handle-order-event" \
  --max-delivery-attempts 10 \
  --event-ttl 1440 \
  --deadletter-endpoint "/subscriptions/${SUBSCRIPTION_ID}/resourceGroups/rg-eventgrid-demo/providers/Microsoft.Storage/storageAccounts/eventgriddemostorage/blobServices/default/containers/eventgrid-deadletter"

# 4. Publish a test event to confirm the wiring.
curl -X POST "$TOPIC_ENDPOINT" \
  -H "aeg-sas-key: $TOPIC_KEY" \
  -H "Content-Type: application/json" \
  -d '[{
    "id": "evt-001",
    "eventType": "OrderCreated",
    "subject": "orders/order-abc123",
    "eventTime": "2026-08-01T12:00:00Z",
    "data": { "orderId": "order-abc123", "status": "pending" },
    "dataVersion": "1.0"
  }]'

Four commands, and unlike the EventBridge build in AWS EventBridge, publishing here goes over plain HTTPS with a SAS key header rather than through a dedicated CLI verb — Event Grid's publish path is the topic's own HTTPS endpoint, not an az eventgrid publish command.

At-Least-Once Delivery Is Not Optional Reading

Event Grid retries failed deliveries — up to --max-delivery-attempts times, with exponential backoff, before giving up and routing to the dead-letter destination. That retry behavior means your webhook handler will, eventually, receive the same event more than once. Not as an edge case — as the documented contract.

Diagram: Event Grid's retry-then-dead-letter sequence. A handler that isn't idempotent will process the same logical event twice on any successful retry after a prior partial failure.

Design the handler to key off id (the event ID) and treat re-delivery of a seen ID as a no-op. This is the single most common production bug I've seen with Event Grid consumers — not a delivery failure, but a handler that wasn't written to expect its own retries.

Event Grid vs Event Hubs vs Service Bus

	Event Grid	Event Hubs	Service Bus
Model	Push (webhook/fan-out)	Pull (streaming log, consumer groups)	Pull (queue/topic, message broker)
Kafka-equivalent	No	Yes	No
Replay	No	Yes (retention window)	No (messages consumed once, unless using sessions/dead-letter)
Delivery guarantee	At-least-once	At-least-once (consumer-managed offsets)	At-least-once, FIFO available with sessions
Best for	React to discrete events (blob created, order placed)	High-throughput continuous streams, analytics pipelines	Ordered work queues, transactional messaging
Pricing model	Per-operation ($0.60/M after free tier)	Per-throughput-unit + per-million-events	Per-tier flat + per-operation (Premium)

Recommendation: If your use case is "when X happens, notify Y," that's Event Grid. If it's "process an unbounded stream and be able to replay from three days ago," that's Event Hubs. If it's "guarantee this work item gets processed exactly once, in order, by exactly one worker," that's Service Bus.

Common Mistakes

Mistake 1: Reaching for Event Grid to replace Kafka
Event Grid has no replay and no consumer-group semantics. If a stakeholder says "we need Kafka on Azure," the answer is Event Hubs, not Event Grid — don't let the word "events" in both names cause a wrong pick.

Mistake 2: Building webhook handlers that assume single delivery
At-least-once means exactly what it says. A handler that isn't idempotent against the event id will eventually double-process something, usually during a transient network blip that has nothing to do with your code.

Mistake 3: Skipping dead-letter configuration
Without --deadletter-endpoint, an event that exhausts all delivery attempts is simply gone. No default fallback storage exists. Configure it at subscription creation, not after the first unexplained missing event.

Mistake 4: Setting --event-ttl too short for slow downstream recovery
The default and commonly-used TTL is 1440 minutes (24 hours). If your webhook endpoint can plausibly be down longer than that during an incident, either extend the TTL or accept that events older than it are dead-lettered, and build your recovery runbook around checking that container.

Production Considerations

Performance: Event Grid's push model means latency is dominated by your subscriber's response time, not Event Grid itself — a slow webhook handler triggers Event Grid's own retry logic, which compounds load on an already-struggling endpoint.

Security: Validate the aeg-event-type header for the initial SubscriptionValidation handshake event, and verify the event signature or use Azure AD-based authentication on the webhook endpoint rather than relying solely on the SAS key being unguessable.

Cost: At $0.60/million operations after 100K free per month, Event Grid is inexpensive even at meaningful scale — the cost driver worth watching is retry volume from a flaky subscriber, not the base event rate.

Monitoring: Track PublishSuccessCount, DeliveryAttemptFailCount, and DeadLetteredCount metrics per subscription. A rising DeadLetteredCount with no corresponding alert is how "the event system is just quietly dropping stuff" becomes a support ticket weeks later.

Full Example: Idempotent Webhook Handler

import express, { Request, Response } from 'express';

const app = express();
app.use(express.json());

/** Tracks processed event ids to make delivery idempotent — swap for Redis/DB in production. */
const processedEventIds = new Set<string>();

/**
 * POST /api/handle-order-event — Event Grid webhook target.
 *
 * @remarks
 * Handles the SubscriptionValidation handshake on first subscribe, then
 * processes OrderCreated events. Re-delivered events (same `id`) are
 * acknowledged with 200 but not reprocessed — required given Event Grid's
 * at-least-once delivery guarantee.
 *
 * @returns 200 on successful handling or validation; 400 on malformed payload.
 */
app.post('/api/handle-order-event', (req: Request, res: Response) => {
  const events = req.body as Array<{ id: string; eventType: string; data: unknown }>;

  for (const event of events) {
    if (event.eventType === 'Microsoft.EventGrid.SubscriptionValidationEvent') {
      const validationCode = (event.data as { validationCode: string }).validationCode;
      res.status(200).json({ validationResponse: validationCode });
      return;
    }

    if (processedEventIds.has(event.id)) {
      continue; // Already handled — Event Grid retried a prior delivery.
    }

    processedEventIds.add(event.id);
    console.log(`Processing order event ${event.id}`);
  }

  res.status(200).send();
});

app.listen(process.env.PORT ? parseInt(process.env.PORT, 10) : 3000);

Full source with the teardown script: GitHub → cloud-apis/azure-event-grid-cli/

Conclusion

Event Grid is the right tool when you need to react to something that happened, delivered via push with retries and dead-lettering handled for you — not when you need a replayable, high-throughput stream, which is Event Hubs' job. Build every subscriber assuming at-least-once delivery from day one, configure dead-lettering before you need it rather than after an event goes missing, and the four-command CLI setup above will get you further, faster, than most teams expect from an "enterprise event routing" service.

DEV Community