Let's say you've got a YouTube channel uploading videos in a regular cadence. When a new video is published, you want to cross-post it to other social media channels that you're running. What could be the best way to do so? There are hundreds of commercial tools on the market for online content marketing. There are hundreds of companies to help you, and those companies have their proprietary solutions for it. It could make sense to utilise those tools or companies. However, with various reasons or circumstances, what if you need to build your own one? What if the existing tools don't fulfil your requirements? Then, it's a good time to build the solution by yourself, isn't it?
This post will discuss an end-to-end workflow story, from YouTube video update to other social media exposure. For the workflow, let's use Azure serverless services like Azure EventGrid, Azure Functions and Azure Logic Apps.
If you like to see the source codes of the solution, find this GitHub repository. It's completely open-source, and you can use it under your discretion with care.
This is a WebSub subscription handler for YouTube video feed update | YouTube 비디오 피드 업데이트를 WebSub 기능을 이용해 받아 처리합니다
YouTube WebSub Subscription Handler
This is a WebSub subscription handler for YouTube video feed update.
- 한국어: TBD
- English: TBD
Deploying Azure Functions App
Deploying Azure Logic App for Scheduled Subscription
Deploying Azure Logic App as Event Handler
Your contributions are always welcome! All your work should be done in your forked repository. Once you finish your work with corresponding tests, please send us a pull request onto our
devbranch for review.
YouTube WebSub Subscription Handler is released under MIT License
The MIT License (MIT)
Copyright (c) 2020 DevRel Korea (한국 DevRel 커뮤니티)
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of…
Google registers all YouTube channels to their WebSub Hub. Therefore, if you want to get the update notification from a specific channel, you can simply send a subscription request to the Hub. To subscribe, enter the message handler URL and YouTUbe channel URL and click the
Do It! button. Too easy!
Please ensure that the subscription process is completed only after passing the message handler verification request.
To verify the WebSub subscription request, the WebSub Hub sends an API call to the message handler. When it arrives, the handler MUST deal with the following.
- The verification request sends a
GETrequest with the following query parameters:
hub.topic: The YouTube channel URL.
hub.challenge: A random string generated by the WebSub Hub, used for the verification.
hub.lease_seconds: The validation period in seconds from the time of the request. The request will be void unless the request is not verified within this period.
- The response of the verification request MUST include the
hub.challengevalue to the response body, with the HTTP status code of 200:
- If the response body includes anything other than the
hub.challengevalue, the WebSub Hub won't accept it as the valid response.
- If the response body includes anything other than the
Here's the verification request handling logic in the Azure Function method:
Once the message handler is verified, the WebSub Hub keeps sending the notification message to the handler whenever a new video update is made, from the subscribed channel.
As the mechanism of WebSub follows the same Publisher/Subscriber (Pub/Sub) pattern, it's not that new. The only difference of WebSub is the event data that makes use of the existing ATOM feed format. Therefore, as long as any subscriber understands the feed format, it should be OK. In other words, the subscriber has a strong dependency on the event data format the publisher sends. In the modern application environments, we recommend decoupling between the publisher and subscriber as much as we can, so that each can organically grow independently. In other words, the subscriber don't have to know the ATOM feed format. How can we make them decoupled, then? The event data format or message format needs to be canonicalised. Then, converting the canonical data into the subscriber-specific format should be done by the subscriber's end.
Therefore, we are going to use CloudEvents as the canonical data format. There are two steps for the conversion–1) canonicalisation and 2) domain-specific conversion. Let's have a look.
The purpose of this step is to decouple between WebSub Hub and your application. The XML data delivered from the WebSub Hub is just wrapped with the CloudEvents format. When a new video is updated onto YouTube, it sends a notification to the WebSub Hub, which looks like the following:
As the message handler takes this request through
POST, it is stringified like this:
The request header also contains the following
As it includes the YouTube channel URL as the message source, you need to extract it.
Then, set the event type and content type like the following.
As I mentioned in my previous post, at the time of this writing, the Azure EventGrid Binding for Azure Function currently has a limitation to support the CloudEvents format. Therefore, you should handle it manually:
So far, the WebSub data is canonicalised with the CloudEvents format and sent to EventGrid. The canonicalised information looks like this:
Now, you have cut the dependency on the WebSub Hub.
Let's move onto the next step.
At this step, the XML data is actually converted into the format we're going to use for social media amplification.
The WebSub XML data only contains bare minimum information like the video ID and channel ID. Therefore, you need to call a YouTube API to get more details for social media amplification. An event handler should be registered to handle the published event data on Azure EventGrid. Like the WebSub subscription process, it also requires delivery authentication. One of the good things using Azure Logic Apps as the event handler is that it automatically does all the verification process internally. Therefore, you just use the Logic App to handle the event data.
The Logic App handler's first action is to verify whether the event data is what you are looking for–it should meet your channel ID and event type of
com.youtube.video.published. If either channel ID or the event type is different, this handler stops processing.
If the event data is what you are looking for, the handler passes it to Azure Functions for further manipulation.
The Azure Functions app calls the YouTube API to get more details of the video, manipulates them, and turns it back to Logic App. The converted data looks like:
YouTube data has been massaged for our purpose.
The event handler now needs to help spread the YouTube video update to the world through designated social media. There are two approaches:
- The event handler directly connects to APIs of individual social media, or
- The event handler publishes another event to Azure EventGrid for other event handlers takes care of social media amplification.
Although both approaches are valid, the first one has strong coupling between the handler and amplifiers. If you need to add a new amplifier or remove an existing one, the Logic App handler must be updated, which is less desirable, from the maintenance perspective. On the other hand, The second approach publishes another event containing the converted data. All social media amplifiers act as event handlers, and they are all decoupled. I chose the second one.
In order to publish the converted YouTube video details to Azure EventGrid, the data needs to be wrapped with the CloudEvents format. The screenshot shows the action on how to wrap the video details data with CloudEvents. This time, the event type will be
The next action is to send an HTTP request to Azure EventGrid, with the CloudEvents payload. You can notice that many metadata headers are starting with
ce-, defined in the cross reference check spec over HTTP.
The message handler now completes its workflow. From now on, each social media handler takes care of the new event data.
YouTube video details are now ready for amplification! Each social media handler takes care of the event data by adapting their circumstances. The event data received from EventGrid looks like this:
As Logic Apps provides the Twitter connector out-of-the-box, you don't need to use the API by yourselves. Therefore, simply use the actions like below:
Logic Apps also provides a built-in LinkedIn connector. So, simply you use it.
Unlike the other two connectors, the Facebook connector has been deprecated. Instead, it's now become an open-source project. So, you should use this open-sourced custom connector or something else. Fortunately IFTTT provides the Facebook Page connector, so you just use it.
From the Logic App point of view, calling IFTTT is just another HTTP call. So it's not that tricky. The only thing to remember is that the request payload can only include no more than
The actual process result in the IFTTT end looks like this:
We've amplified to social media of Twitter, LinkedIn and Facebook.
If you want to add another social media, you can simply add another Logic App as the event handler.
So far, we've implemented a workflow solution that posts to designated social media platform when a new YouTube video update is notified through WebSub, by using CloudEvents, Azure EventGrid, Azure Functions and Azure Logic Apps. As steps are all decoupled, we don't need to worry about the dependencies during the maintenance phase. In addition to that, although a new social media platform is planned to add, it wouldn't impact on the existing solution architecture.
If you or your organisation is planning online content marketing, it's worth building this sort of system by yourself. And it would be an excellent opportunity to make a well-decoupled and event-driven cloud solution architecture.