DEV Community

Pirate Prentice
Pirate Prentice

Posted on

n8n Kafka Node: Produce and Consume Messages in Your Workflows (Free Workflow JSON)

#webdev

If your stack includes Apache Kafka — or you're building event-driven pipelines — the n8n Kafka node lets you produce and consume messages directly inside your workflows without writing a single line of broker code.

This guide covers everything: connecting to Kafka, producing messages, consuming with triggers, and real-world patterns. Free workflow JSON at the end.

What the n8n Kafka node does

n8n ships two Kafka-related items:

  • Kafka node — produces (sends) messages to a Kafka topic
  • Kafka Trigger node — consumes messages from a topic and fires a workflow execution for each one

Together they let you bridge Kafka with any other system n8n supports: Slack, Postgres, HTTP endpoints, Google Sheets, S3, and hundreds more.

Prerequisites

  • n8n instance (cloud or self-hosted)
  • A running Kafka broker (local Docker, Confluent Cloud, AWS MSK, Redpanda, etc.)
  • Broker host/port and topic names
  • Optional: SASL credentials if your broker requires authentication

Step 1 — Create a Kafka credential

  1. In n8n, go to Credentials → New → Kafka
  2. Fill in:
    • Client ID — any identifier (e.g. n8n-client)
    • Brokers — comma-separated host:port list (e.g. localhost:9092)
    • SSL — enable if your broker uses TLS
    • Authentication — choose SASL/PLAIN or SASL/SCRAM if required; enter username/password
  3. Click Test to verify the connection, then Save

Step 2 — Produce a message with the Kafka node

Drag a Kafka node into your workflow. Configuration:

Field Example value
Credential (your saved Kafka cred)
Topic orders
Message ={{ JSON.stringify($json) }}
Key ={{ $json.orderId }} (optional, for partition routing)
Headers key/value pairs (optional metadata)

The Message field is a string. If you're sending JSON objects, wrap them with JSON.stringify() in an n8n expression.

Example: send every new Stripe payment to a Kafka topic 

Stripe Trigger → Kafka node
  Topic: payments
  Message: {{ JSON.stringify($json) }}
  Key: {{ $json.id }}
Enter fullscreen mode Exit fullscreen mode

Every confirmed payment lands in your payments topic instantly, ready for downstream consumers (analytics, fraud detection, fulfillment).

Step 3 — Consume messages with the Kafka Trigger

The Kafka Trigger polls a topic and fires one workflow execution per message.

  1. Add a Kafka Trigger node as the first node in a workflow
  2. Set:
    • Topic — orders
    • Group ID — n8n-consumer-group (Kafka tracks offset per group)
    • Session Timeout — default 30000 ms is fine for most cases
    • Read Messages From Beginning — enable only if you need to replay all history
  3. Activate the workflow

n8n polls Kafka and triggers for each new message. The node outputs:

{
  "message": "{\"orderId\":\"abc123\",\"amount\":49.00}",
  "topic": "orders",
  "partition": 0,
  "offset": "42",
  "key": "abc123",
  "headers": {}
}
Enter fullscreen mode Exit fullscreen mode

Note: message is a string. Parse it downstream with a Code node: return [{json: JSON.parse($input.first().json.message)}]

Step 4 — Parse and route Kafka messages

A typical consume-and-route workflow:

Kafka Trigger
  └─ Code node (parse JSON string → object)
       └─ IF node (route by event type)
            ├─ [order.created] → Postgres INSERT
            └─ [order.cancelled] → Slack alert + Stripe refund
Enter fullscreen mode Exit fullscreen mode

Code node to parse the message 

const raw = $input.first().json.message;
const parsed = JSON.parse(raw);
return [{ json: parsed }];
Enter fullscreen mode Exit fullscreen mode

After this node, all downstream nodes see a proper JSON object.

Common patterns

Dead-letter queue re-processor

Consume from a DLQ topic → fix data → produce back to the main topic.

Kafka → Postgres pipeline 

Kafka Trigger → Code (parse) → Postgres (INSERT)
Enter fullscreen mode Exit fullscreen mode

Real-time event ingestion without custom consumer code.

Multi-topic fan-out

One Kafka Trigger per topic, each with its own workflow. n8n's consumer groups handle offset tracking independently.

Kafka → Slack alerting 

Kafka Trigger (topic: errors)
  → Code (parse)
  → IF (severity == 'critical')
       → Slack (post to #incidents)
Enter fullscreen mode Exit fullscreen mode

Troubleshooting

Connection timeout on Test — check broker host/port, firewall rules, and SSL setting.

Messages not appearing after activation — verify the Kafka Trigger workflow is active (toggle in workflow list). Inactive workflows don't poll.

message field is [object Object] — you forgot JSON.stringify() in the producer. Always stringify objects before sending.

Consumer group lag keeps growing — your workflow is slower than the message rate. Use Split In Batches carefully, or consider increasing broker retention and running parallel workflow instances.

SASL auth failure — double-check mechanism (PLAIN vs SCRAM-SHA-256 vs SCRAM-SHA-512) matches your broker config.

Free workflow JSON

Here's a minimal produce + consume roundtrip to get started:

{
  "name": "Kafka Produce + Consume Demo",
  "nodes": [
    {
      "parameters": {
        "topic": "n8n-demo",
        "message": "={\"ts\": \"{{ $now }}\", \"msg\": \"hello from n8n\"}",
        "options": {}
      },
      "type": "n8n-nodes-base.kafka",
      "typeVersion": 1,
      "name": "Produce to Kafka",
      "id": "kafka-produce-1",
      "position": [400, 300]
    },
    {
      "parameters": {
        "topic": "n8n-demo",
        "groupId": "n8n-demo-group",
        "options": {}
      },
      "type": "n8n-nodes-base.kafkaTrigger",
      "typeVersion": 1,
      "name": "Consume from Kafka",
      "id": "kafka-consume-1",
      "position": [400, 500]
    },
    {
      "parameters": {
        "jsCode": "const raw = $input.first().json.message;\nreturn [{ json: JSON.parse(raw) }];"
      },
      "type": "n8n-nodes-base.code",
      "typeVersion": 2,
      "name": "Parse Message",
      "id": "code-parse-1",
      "position": [620, 500]
    }
  ],
  "connections": {
    "Consume from Kafka": {
      "main": [[{"node": "Parse Message", "type": "main", "index": 0}]]
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

Want 10 more pre-built n8n workflows (Stripe, Gmail, Postgres, webhooks, and more)? They're all in the n8n Workflow Starter Pack — $29, instant download, free updates.

Summary

Task Node
Send message to Kafka Kafka node
Receive messages from Kafka Kafka Trigger node
Parse JSON message string Code node + JSON.parse()
Route by event type IF node

The n8n Kafka integration is solid for teams that already run Kafka and want to connect it to the rest of their stack without writing custom consumer/producer services. Hook it up to your database, alerting, or CRM and you have event-driven automation in minutes.

What are you routing through Kafka? Drop your use case in the comments — always curious what setups people are running.

Top comments (1)

Collapse
 
pirateprentice profile image
Pirate Prentice •

What are you using Kafka for in your n8n stack? Event sourcing, async job queues, real-time analytics pipelines? Drop your use case below — always curious what setups people are running in production.