DEV Community: Steven Jenkins De Haro

Demystifying Confluent's Schema Registry Wire Format

Steven Jenkins De Haro — Sun, 09 Nov 2025 11:52:46 +0000

Table of Contents

Introduction
A brief overview on Avro, JSON, and Protobuf
What is Confluent's wire format?
Debugging the wire format
- Prepare the source data
- Analyze the message contents
- Debugging challenge
- Additional commands
Third-party wire format compatibility
Manually handling the wire format
- Producer code snippet
- Consumer code snippet
Best practices & tips
Conclusion
Resources

Within Kafka-based architectures, message serialization is more than just a format choice, it defines the contract between producers and consumers. When that contract is paired with a schema registry, it is enhanced with features like efficient schema evolution, versioning, and compatibility checks. However, these benefits are more easily realized when both sides of the data exchange use it. When a client does not or cannot use the schema registry, or when the integration with Kafka happens via a REST API, there will be additional considerations at the byte level to take into account.

To address these scenarios, this article provides a detailed breakdown of Confluent's Schema Registry wire format and how to use it without integrating with a schema registry. In addition, it will demonstrate how to inspect and decode Kafka messages using tools like xxd, dd, and low-level byte parsing for clearer insight into how data flows through Kafka topics.

A brief overview on Avro, JSON, and Protobuf

Apache Avro, JSON, and Protobuf formats are widely used data serialization technologies that serve slightly different purposes and offer contrasting trade-offs in efficiency, readability, and interoperability.

Avro is a row-oriented, binary format that can optionally embed schemas with the data (or reference it externally), enabling fast, compact, and strongly typed data exchange. It's well-suited for high-throughput pipelines and big data applications due to its small payload size and efficient encoding. JSON, in contrast, is a human-readable, text-based format that prioritizes simplicity and interoperability, ideal for web APIs and configuration files, though its verbosity results in larger payloads and slower parsing compared to binary formats. Protobuf (Protocol Buffers), developed by Google, is a compact binary format similar to Avro but uses numeric field tags instead of field names. Schemas are defined in .proto files that are later compiled into code, providing efficient encoding and making it a popular choice for microservices, APIs, and inter-service communication.

Below is a simple comparison of the test record { "message": "Hello World!" } to illustrate the size differences between formats:

📊 Size Comparison

Format	Raw	Base64	Base64 Encoded Raw Payload	Meta
JSON	29 B	40 B	eyAibWVzc2FnZSI6ICJIZWxsbyBXb3JsZCEiIH0=	N/A
Protobuf	14 B	20 B	CgxIZWxsbyBXb3JsZCE=	0x0a 0x0c
Avro	13 B	20 B	GEhlbGxvIFdvcmxkIQ==	0x18

📝 Note: For the Avro length prefix, the string value is encoded as a long using ZigZag encoding ( $bytes×2\text{length in bytes} × 2$ ), and then written as a variable-length integer (varint/LEB128). The same encoding is applied to all signed numeric types to make negative numbers efficient by using fewer bytes to represent them. See Data Serialization and Deserialization for more information.

As can be seen above, Avro is the most compact in this example. This is mainly because the schema is referenced externally, including the field names. However, had the schema been embedded, it would have been around 159 raw bytes in size. Protobuf, on the other hand, encodes each field in the payload using a tag encoded with Protobuf wire format, which contains the field number and wire type. For length-delimited types, like strings or Embedded Messages, the tag is immediately followed by a length prefix and then the serialized data. This structure makes Protobuf messages partially self-describing, allowing known fields to be parsed without having the compiled .proto files in a pinch. With all this in mind, it's now time to discuss the wire format to understand where schemas and payloads fit in.

What is Confluent's wire format?

The wire format defines a fixed binary layout for serialized messages that will be sent over the network. It’s not tied to Kafka itself, but rather how Confluent's Schema Registry, and compatible systems, encode metadata alongside the actual payload. In other words, instead of producing only a raw Avro, JSON, or Protobuf payload into a topic, an additional header structure is prepended to the payload. This header tells the consumer which schema to use, and in the case of Protobuf, which top-level message within that schema. The payload can then be deserialized correctly regardless of the consumer's language or platform as long as it has access to the Schema Registry and the corresponding schema ID.

The following is a high-level view of the wire format:

💾 For Arvo and JSON payloads:

+----------------------+-------------------+
| Byte Offset          | Content           |
+----------------------+-------------------+
| 0                    | Magic byte (0x00) |
| 1 – 4                | Schema ID (int32) |
| 5 – end              | Avro/JSON payload |
+----------------------+-------------------+

💾 For Protobuf payloads:

+----------------------+----------------------------------------+
| Byte Offset          | Content                                |
+----------------------+----------------------------------------+
| 0                    | Magic byte (0x00)                      |
| 1 – 4                | Schema ID (int32)                      |
| 5 – (5+N-1)          | Protobuf message index (varint)        |
| (5+N) – end          | Protobuf payload (field tags + values) |
+----------------------+----------------------------------------+

Within the wire format, the first byte is called the Magic Byte (0x00), which is just an indicator to say that the wire format is being used, otherwise, a consumer may throw a serialization exception for an unknown magic byte for raw payloads. This section is followed by 4 bytes to store the schema ID from the schema registry in the form of a big-endian signed 32-bit integer. For Protobuf payloads, the next section is for the message index. It is used to identify which top-level message found in the associated .proto file is encoded. This byte is also varint-encoded and is usually 0x00 if there is only one message. The final section stores the actual raw payload.

Debugging the wire format

It's useful to understand how to debug a Kafka message to validate the structure/format of the data being exchanged to see if it meets expectations. For example, to be able to identify a message that uses the wire format versus one that does not. For this, the xxd CLI tool will be used as it comes preinstalled with most operating systems, including Windows via git-bash.

Prepare the source data

To begin, an Avro sample message using the wire format is needed. Use one of the following options to generate or dump an existing sample:

⚙️ Option 1: Generate

printf '\x00\x00\x00\x00\x01\x18\x48\x65\x6c\x6c\x6f\x20\x57\x6f\x72\x6c\x64\x21' > avro-message.bin

The generated message above is associated with the following dummy schema below with a schema ID of 1:

{
  "type": "record",
  "name": "Greeting",
  "namespace": "com.example.messages",
  "fields": [
    { "name": "message", "type": "string" }
  ]
}

⚙️ Option 2: Kafka CLI

# Depending on package, it might be './bin/kafka-console-consumer.sh ...'.
/bin/kafka-console-consumer \
     --bootstrap-server <BOOTSTRAP_URL>:<PORT> \
     --topic <YOUR_TOPIC> \
     --from-beginning \
     --max-messages 1 \
     | tr -d '\n' > avro-message.bin

📝 Note: The last part of the command removes the 0x0a byte that may appear at the end due to a new line introduced by the command.

⚙️ Option 3: kcat CLI (formerly known as kafkacat < 1.7.0)

# Alternative 'docker run -it --rm edenhill/kafkacat:1.6.0 kafkacat -b ...'.
kcat -b <BOOTSTRAP_URL> \
  -t <YOUR_TOPIC> \
  -C -c 1 -o beginning -e \
  > avro-message.bin

Analyze the message contents

With the avro-message.bin now in hand, it's time to debug its contents. Execute the following command to view the hex representation of the data:

xxd -g 1 -c 16 ./avro-message.bin

🔳 Output

00000000: 00 00 00 00 01 18 48 65 6c 6c 6f 20 57 6f 72 6c  ......Hello Worl
00000010: 64 21                                            d!

📝 Note: Depending on the shell used, the meta sections may be color-coded to help identify the sections more easily 🚀.

🔍 Avro Breakdown

                   Confluent's Wire Format
 ____________________________|_____________________________
|                                        Avro              |
|                       __________________|________________|
|                      |                                   |
[ 00 ] [ 00 00 00 01 ] [ 18 ] [ "Hello World!" UTF-8 bytes ]
  ↑           ↑          ↑                  ↑
magic B  schema id   zigzag len       payload data

In the above breakdown, the last two sections are specific to the Avro format, which can also be considered the raw payload data. The wire format is the combination of this and the first two sections. This is the main difference between producing with raw payload data and data that can be associated with a particular schema in the Confluent Schema Registry.

For brevity, let's quickly compare a Protobuf dump against the Avro dump from earlier by running the following command:

printf '\x00\x00\x00\x00\x01\x00\x0a\x0c\x48\x65\x6c\x6c\x6f\x20\x57\x6f\x72\x6c\x64\x21' | xxd -g 1 -c 16

The generated message above is associated with the following dummy schema below with a schema ID of 1:

syntax = "proto3";

package com.example.messages;

message Greeting {
  string message = 1;
}

🔳 Output

00000000: 00 00 00 00 01 00 0a 0c 48 65 6c 6c 6f 20 57 6f  ........Hello Wo
00000010: 72 6c 64 21                                      rld!

📝 Note: If using a wrapper type like google.protobuf.StringValue from google/protobuf/wrappers.proto instead of string or optional string for presence detection with primitive fields, and for more consistent behavior, the output will be slightly different. This is because the wrapper type adds an inner message structure with a single field called value. The end result would be 00 00 00 00 01 00 [0a 0e] 0a 0c 48 65 6c 6c 6f 20 57 6f 72 6c 64 21, where bytes 0x0a and 0x0e for the outer tag and length are prepended to the expected raw payload in this example.

🔍 Protobuf Breakdown

                         Confluent's Wire Format
 __________________________________|_____________________________________
|                                                Protobuf                |
|                              _____________________|____________________|
|                             |                                          |
[ 00 ] [ 00 00 00 01 ] [ 00 ] [ 0a ] [ 0c ] [ "Hello World!" UTF-8 bytes ]
  ↑           ↑          ↑      ↑      ↑                  ↑
magic B  schema id  msg index  tag   length         payload data

The structure is similar to the previous Avro one, except now the wire format introduces one previously omitted section meant only for Protobuf, the message index. Also, notice how Protobuf's length doesn't use ZigZag encoding for the length prefix.

Debugging challenge

As a small challenge, see if you can spot the differences between the following wire-formatted message carrying a JSON payload and the previous examples:

printf 'AAAAAAF7ICJtZXNzYWdlIjogIkhlbGxvIFdvcmxkISIgfQ==' | base64 -d | xxd -g 1 -c 16

Alternatively, use the longer byte approach:

printf '\x00\x00\x00\x00\x01\x7b\x20\x22\x6d\x65\x73\x73\x61\x67\x65\x22\x3a\x20\x22\x48\x65\x6c\x6c\x6f\x20\x57\x6f\x72\x6c\x64\x21\x22\x20\x7d' | xxd -g 1 -c 16

The generated message above is associated with the following dummy schema below with a schema ID of 1:

{
  "$id": "https://example.com/schemas/com.example.messages.Greeting.json",
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "title": "Greeting",
  "type": "object",
  "properties": {
    "message": {
      "type": "string"
    }
  },
  "required": ["message"],
  "additionalProperties": false
}

Additional commands

Finally, here are a few more commands to play with that could come in handy. The commands are mostly for Avro and JSON-based messages, but they can be adapted for Protobuf as well.

📚 PARSED VIEW

xxd -p ./avro-message.bin | sed -E 's/^(.{2})(.{8})(.*)$/Magic:\1 SchemaID:\2 Payload:\3/'

🔳 Output

Magic:00 SchemaID:00000001 Payload:1848656c6c6f20576f726c6421

📚 SECTIONAL VIEWS

# Magic (first byte)
dd if=avro-message.bin bs=1 count=1 status=none | xxd -g 1
# Schema ID (4 bytes after 1)
dd if=avro-message.bin bs=1 skip=1 count=4 status=none | xxd -g 4
# Payload (all bytes after 5)
dd if=avro-message.bin bs=1 skip=5 status=none | xxd -g 1 -c 16

🔳 Output

00000000: 00
00000000: 00000001
00000000: 18 48 65 6c 6c 6f 20 57 6f 72 6c 64 21           .Hello World!

📚 PARSE CONTINUOUS HEX

# Avoids having to add \x before each byte, just remove spaces.
printf '00000000011848656c6c6f20576f726c6421' | xxd -r -p | xxd -g 1 -c 16

🔳 Output

00000000: 00 00 00 00 01 18 48 65 6c 6c 6f 20 57 6f 72 6c  ......Hello Worl
00000010: 64 21                                            d!

Third-party wire format compatibility

The following tables provide a general overview of schema registries and tools that are compatible with Confluent's wire format. This information is based on first-hand testing and or what could be found in the documentation at the time of this writing.

🗂️ Schema Registries

Name	Wire Format Support	Special Considerations
Azure Schema Registry	❌ No	Uses a different approach
Apicurio Registry	✅ Yes, Automatic
Karapace	✅ Yes, Automatic

☕ Tools/Services

Name	Wire Format Support	Special Considerations
Kafka CLI	⚠️ Yes, via Addon	Add Confluent serializers
kcat CLI / kafkacat CLI	❌ No, Manual only	Supports working with bytes
Strimzi Bridge REST API	❌ No, Manual only
Strimzi Connect	✅ Yes, Automatic
AKHQ	✅ Yes, Automatic
Kafka UI / Kafbat UI	✅ Yes, Automatic
Vector	❌ No, Manual only	Avoid value transformations

📝 Note: The list assumes clients are using the correct serializers/converters to integrate with Kafka and the schema registry.

Manually handling the wire format

This section takes theory into practice by showing how a Native or REST-based Kafka client can produce or consume wire-formatted messages without a direct integration with a compatible schema registry. The goal is to prevent breaking other clients in the data exchange that do use one. The examples below have been created to be framework agnostic and generic enough to apply to any type of client. However, with that said, some adaptation will be needed for the target implementation.

Producer code snippet

The code snippet below shows how a producer can manually tack on the required wire format bytes to an Avro-based payload before sending it to a Kafka topic.

Greeting greeting = new Greeting("Hello World!");

...

// Prepend the wire format.
ByteBuffer buffer = ByteBuffer.allocate(1 + 4 + avroPayload.length);
buffer.put(magicByte);
buffer.putInt(schemaId);
buffer.put(avroPayload);

// Ready for sending as a native-based Kafka client.
byte[] message = buffer.array();
// Ready for sending as a REST-based Kafka client.
String base64Message= Base64.getEncoder().encodeToString(message);

...

System.out.println("Avro Payload Length: " + avroPayload.length);
System.out.println("Message Length: " + message.length);
System.out.println("Message (base64): " + base64Message);
System.out.println("Message (hex): " + hexMessage);

🤖 View full code on GitHub

🔳 Output

Avro Payload Length: 13
Message Length: 18
Message (base64): AAAAAAEYSGVsbG8gV29ybGQh
Message (hex): 00 00 00 00 01 18 48 65 6c 6c 6f 20 57 6f 72 6c 64 21

The actual sending of the message will depend on the type of producer client being used. For example, a native Kafka client will wrap the message bytes using ProducerRecord<String, byte[]> record = new ProducerRecord<>("my-topic", message); before using producer.send(record); to send it. On the other hand, a REST-based client will use the base64 encoded version of the message to POST it via the endpoint /topics/{topic-name} using the Content-Type: application/vnd.kafka.binary.v2+json header.

Consumer code snippet

Similar to the producer, this consumer code snippet reverses the wire format encoding process to extract the payload so that it can be deserialized.

byte[] message = Base64.getDecoder().decode("AAAAAAEYSGVsbG8gV29ybGQh");

ByteBuffer buffer = ByteBuffer.wrap(message);
byte magicByte = buffer.get();   // Position goes from 0 -> 1.
int schemaId = buffer.getInt();  // Reads bytes [1..4], position goes from 1 -> 5.

int remainingBytes = buffer.remaining();
byte[] avroPayload = new byte[remainingBytes];
buffer.get(avroPayload); // Writes remaining bytes to byte array.

...

Greeting greeting = reader.read(null, decoder);

System.out.println("Magic: " + magicByte);
System.out.println("Schema ID: " + schemaId);
System.out.println("Payload [Message Field]: " + greeting.getMessage());

🤖 View full code on GitHub

🔳 Output

Magic: 0
Schema ID: 1
Payload [Message Field]: Hello World!

In practice, a native consumer will use an annotation like @KafkaListener(topics = "my-topic") over a method like public void consume(ConsumerRecord<String, byte[]> record) {} to consume messages. REST-based clients will need to poll with GET the /consumers/{consumer-group}/instances/{instance}/records endpoint using the Accept: application/vnd.kafka.binary.v2+json header to consume the message as base64 encoded.

Best practices & tips

Ideally, Kafka clients should use libraries that natively integrate with Confluent's Schema Registry or compatible alternatives when exchanging schema-backed data. If this is not possible, then the following list can help avoid some pains when developing solutions that do not have this integration as an option:

Prefer schema-aware producers in production where possible, as this will be the safest option for consumers.
Consumers should check for the magic byte (message.length > 5 && message[0] == 0x00) to accept or reject a message. Alternatively, use this check to accept both wire-formatted messages and ones that are not, since most of the code for processing a message is the same.
Consumers should also consider using the schema ID for manually validating a message after requesting the schema with a schema registry client.
For native clients, ensure not to accidentally use Confluent's KafkaAvro*, KafkaJsonSchema*, or KafkaProtobuf* based Serializers/Deserializers in clients not using the schema registry, since these libraries depend on one. Instead, use the ByteArraySerializer and ByteArrayDeserializer for this scenario.
For native clients, be explicit about what serializers/deserializers are being used. Frameworks like Quarkus support automatic selection, however, it may default to a String-based one for these types of messages.
Handle the manual serialization/deserialization logic in a separate class using standard interfaces like Deserializer<T> to make it more reusable and portable.
If there are limitations or complexities in handling the wire format manually, create a bridging application or adapter to handle this task on behalf of the clients that need it. This should be preferred over requiring all clients to disable their schema registry integration.

Conclusion

In conclusion, understanding Confluent's wire format simplifies producing, consuming, and debugging schema-backed messages even when a client does not integrate with a schema registry. Remember to consider the roles each component contributes to in a data exchange. For example, Kafka transports the bytes, Schema Registry defines how those bytes are structured, and serializers/deserializers implement the wire format convention. Use byte-level tools and simple checks (magic byte and schema ID) to validate and diagnose messages across heterogeneous clients. Using only native clients that are integrated into the schema registry is ideal, but when this is not possible, prepend or strip the wire format header consistently to preserve compatibility. Applying these practices will go a long way towards reducing schema-related surprises that may creep into the solution at some point in the future.

Resources

The following resources provide additional details, examples, specifications, and tools related to the topics covered in this article.

Create a Kubernetes NGINX controller with Dapr support

Steven Jenkins De Haro — Sun, 30 May 2021 22:20:55 +0000

In this post, I will show how to create an NGINX controller that exposes the Dapr enabled applications in a Kubernetes cluster. Essentially, the NGINX controller will be configured with the same standard Dapr annotations to get injected with the daprd sidecar. By exposing this sidecar, it will allow external applications to communicate with the Dapr enabled applications in the cluster, see the Dapr API reference.

Note: There have been changes to what repo to use for the NGINX installation, the spec requirements for Kubernetes ingress resources, and the Dapr annotation names, so the information in this post is the updated version that should be followed.

Prerequisites

Kubernetes 1.19+ cluster with Dapr configured.
Helm CLI 3x installed.
Kubectl CLI installed and configured to access the cluster.
Optional: OpenSSL for creating self-signed certificates.

Prepare helm

Add the latest helm chart repo for the NGINX controller by running the following commands:

helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

Create an ingress namespace

Ensure that the current kubectl context is pointed to the correct Kubernetes cluster, and run the following command:

kubectl create namespace nginx

Install NGINX controller with Dapr support

Create a file called dapr-annotations.yaml with the following content to set the annotations on the NGINX ingress controller pods:

controller:
  podAnnotations:
    dapr.io/enabled: "true"
    dapr.io/app-id: "nginx-ingress"
    dapr.io/app-port: "80"

Note: The port above tells the daprd sidecar which port the NGINX controller is listening on. See Dapr Kubernetes pod annotations spec for a complete list of supported annotations.

Run the following command, which references the file above:

helm install nginx-ingress ingress-nginx/ingress-nginx \
    -f dapr-annotations.yaml \
    --set controller.replicaCount=2 \
    --namespace nginx

Add --set controller.service.loadBalancerIP=0.0.0.0 to the above command if a static IP is needed, for example, the last IP from a peered subnet with on-premise.

Create NGINX's daprd sidecar ingress resource

Create a file called ingress-dapr.yaml with the following content:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-dapr
  namespace: nginx
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  tls:
  - hosts:
      - api.example.com
    secretName: tls-secret
  rules:
  - host: api.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-ingress-dapr
            port:
              number: 80

Dapr automatically creates the nginx-ingress-dapr service being referenced above. Make sure to modify the hostname placeholders.

Remove the tls section if TLS termination is not needed, otherwise, run the following commands to create a self-signed certificate for testing:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
    -keyout tls.key -out tls.crt \
    -subj "/CN=api.example.com/O=TLS Example"

kubectl create secret tls tls-secret \
    --key tls.key --cert tls.cert -n nginx

Optionally, inspect the created certificate with the following command:

openssl x509 -in tls.crt -text -noout

It is recommended to use the FREE Let's Encrypt certificates that are globally recognized by following the steps defined in the Securing NGINX-ingress or Encrypting the certificate for Kubernetes (Let’s Encrypt) articles.

Now, run the following command to create the ingress resource:

kubectl create -f ingress-dapr.yaml

Note: This will target the sidecar's service, not the NGINX's service for the load balancer.

Testing externally

If a static IP was not specified earlier, then get the external IP created as part of the NGINX controller setup running the following command:

kubectl get svc nginx-ingress-ingress-nginx-controller -n nginx

or,

kubectl get service --selector=app=nginx-ingress,component=controller \
    -o jsonpath='{.items[*].status.loadBalancer.ingress[0].ip}' \
    -n nginx

With the IP, run the follow test using cURL to see if the Dapr API for the cluster is reachable from outside the cluster:

curl http://<ingress ip>/v1.0/invoke/mathService/method/add \
  -H "Content-Type: application/json"
  -d '{ "arg1": 10, "arg2": 23}'

The above assumes that there is a microservice running using the Dapr app-id mathService that has a POST endpoint called add, which is an example taken from the Service invocation API reference.

To test without building an app, install redis, and use the redis state store building block following the How-To: Configure state store and pub/sub message broker article. After, perform the curl commands in this example to create and retrieve the key-value data in redis.

Delete the NGINX controller

For convenience, here is the command to delete the NGINX controller:

helm delete nginx-ingress -n nginx

Don't forget to delete the ingress resources and the nginx namespace created earlier.

Limiting access

If additional security is required around which Dapr APIs or services should be exposed, then the scope of this can be configured globally, or at a very granular level as needed. For more information, see the following articles:

Show your Tweets on your GitHub Profile README

Steven Jenkins De Haro — Sat, 24 Oct 2020 18:58:20 +0000

GitHub recently released a feature that allows you to create a profile for your GitHub account, which is powered by a simple README.md file. As one might have guessed, people have already found ways to create creative profiles with dynamic content using APIs and GitHub Actions. In this quick tutorial, I will show you one way to dynamically display your tweets on your GitHub profile.

Before We Start

You will first need to enable the profile feature on your account by creating a new repo that matches your GitHub username. For example, my username is StevenJDH so my repo is called StevenJDH. Once done, you can quickly create a starter profile using the GitHub Profile README Generator tool to get the needed markdown as you see here:

Generating an RSS Feed Link

The approach being taken today requires an RSS feed link for your tweets, which Twitter does not provide. Fortunately, the Rss.app service can provide us with one. To get this link, do the following:

From the Rss.app website, sign in or sign up with your Twitter account.
Optional: If this is a new account, you will be automatically enrolled into the 7 day trial for the Premium tier. Downgrading later to the Free tier will undo the following steps unless you opt to pay for that tier. I recommend downgrading to the Free tier now to avoid a message like [[Action required] Your RSS.app Trial has Expired - Sat Oct 31 2020](https://rss.app) replacing your tweets, and having to repeat the steps that follow.
Select the + New Feed button on the left after logging in, and then choose the Twitter RSS Feed tile on the right.
In the first field provided, enter the URL to your Twitter profile and click the Generate button.
You should now see your RSS feed link associated with your Twitter account. In trial mode, the data behind this link will refresh every 30 minutes for the first week, but afterward, it will refresh once a day.

Setting Up Your GitHub Profile README

Now for the fun part, adding the tweets to the GitHub profile. This is made possible thanks to the Blog Post Workflow that we are adapting to make it work for Twitter.

Create a folder called .github at the root of your profile repo. Inside that folder, create another folder called workflows. You should have something like this .github\workflows now.
In the workflows folder, create a file called twitter-workflow.yml with the following contents along with your RSS feed link:

name: Latest Twitter Tweets
on:
  schedule:
    # Runs once a day because using Free tier in Rss.app
    - cron: '0 0 * * *'
  workflow_dispatch: # Run workflow manually (without waiting for the cron to be called), through the Github Actions Workflow page directly.

jobs:
  update-readme-with-twitter:
    name: Update this repo's README with latest tweets
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: gautamkrishnar/blog-post-workflow@master
        with:
          comment_tag_name: "TWITTER"
          feed_list: "<<[[PLACE YOUR RSS FEED LINK HERE]]>>"
          commit_message: "Updated with the latest tweets"
          committer_username: "twitter-bot"
          committer_email: "twitter-bot@example.com"

Back at the root of your repo, add the following to your README.md file where you want the tweets to appear:

### 📱 Latest Tweets

<!-- TWITTER:START -->
<!-- TWITTER:END -->

Once everything is saved, you can manually trigger the GitHub action that will replace the above TWITTER comments with your tweets so you don't have to wait. Just go to Actions at the top of your profile repo, select Latest Twitter Tweets on the left under Workflows, and then select Run workflow on the right. Assuming enough time has passed for the Rss.app service to parse your tweets, you should see the last 5 tweets on your GitHub profile.

Conclusion

In theory, the Blog Post Workflow can support any RSS feed-based service, but it already has direct, out-the-box support for many popular services. It also provides tons of configuration options, which I used to adapt the output more for Twitter. Other than the Free tier service only parsing once a day, this approach works well, and it is compact. If you know of any other approaches that do not use cards, or has a better Free tier for parsing tweets, please share it in a comment.

First Post

Steven Jenkins De Haro — Sat, 24 Oct 2020 05:48:45 +0000

A quick test post for the dynamic feeds in GitHub's README.md Profiles.