Azure Cosmos DB has a cool feature called the Change Feed which allows us to perform real-time analytics on our container by listening to any changes that occur within the container.
These changes are then produced as a sorted list of documents that were change in the order that they were modified. These are persisted and can be processed incrementally. The output can then be consumed by multiple consumers for parallel processing.
This feature is pretty useful in situations where a change in an item can trigger another event or an additional action. One example could be an item is inserted within our container, and we use this to trigger an alert to a website saying that there’s a new item in our container.
Change Feed functionality is enabled by default on all Cosmos DB accounts. You don’t have to do anything special to set it up and you can use the throughput that you provision on your Cosmos account to read from the Change Feed like any regular Cosmos DB operation.
Currently, the Change Feed includes inserts and update operations made to items within a container (which is what we’ll focus on in this tutorial). If you want to capture deletes on items in your container, you can use a flag that captures soft deletes or set a Time to Live (TTL) period on your items and use that property for the Change Feed to capture. Bear in mind that the Change Feed will need to be processed in a shorter time than the TTL interval.
Changes appear exactly once in the change feed. Managing any check-pointing logic that you may require needs to be done by your application. The Change Feed Processor library is good for this as it provides automatic check-pointing. Don’t worry about this now for this simple example.
Changes can be synchronized from any point in time and within each logical partition key value, it’s sorted by order of modification. If you have a large container, the Change Feed can be processed by multiple consumers and we can have multiple Change Feeds on the same container running at the same time.
There are three ways in which we can work with the Change Feed:
Using Azure Functions is BY FAR the easiest way to get started with Cosmos DB change feed. We essentially create Functions that are triggered on each new event in our container’s change feed. The CosmosDB Trigger allows us to use to Change Feed functionality without worrying about any infrastructure.
For our purposes, I’m going to create two Functions:
- An HTTP trigger function that allows us to Insert and Update new items.
- A CosmosDB Trigger Function that uses the Change Feed to track changes and inserts on our container, and then persists these to a lease container.
I’m using the 2.x run-time for our Functions.
Now even though the Change Feed supports both Gremlin API and SQL API Cosmos DB accounts, The CosmosDB Trigger for Azure Functions only supports the SQL API, so that’s what we’ll use in this demo.
You can view the entire code repository here. This includes the relevant models and helper classes that I use in this project. I’m just using the Azure Cosmos DB local emulator as my Cosmos account. The local emulator is pretty cool for doing local development without spending money on an actual database in Azure.
Let’s start by creating our HTTP trigger functions to create and update our items:
Here we have two Functions, CreateTaskItem and UpdateTaskItem. In our create function, we’re using the HTTP Trigger input to make a POST request to our Cosmos DB database. We’re just creating a simple TaskItem and persisting it to our TaskCollection Collection. I’ve used the CosmosDB binding here just for this example, but if you’re using Azure Functions and Cosmos DB for production use cases, you should consider using a Singleton instance of Cosmos DB. For our Update function, we’re just making a PUT request to a route of task/{id} where id is our id of the Task.
Now let’s create another function that listens to our HTTP Functions and records these changes to the Change Feed:
Here I’ve created a function called ChangeFeedListener that connects to our TaskItemCollection and listens to any changes on that container. It then persists these changes to a LeaseCollection container. Within our Function, every time we create or update a TaskItem in our TaskItemCollection, it prints how many documents were changed (only 1 in our simple use case) and what the id of the document that was changed was.
In production scenarios , we could send this as a message to event hubs, send it via a text using the Twilio binding or even send it as a notification to a web site. But for now, this simple function will do.
We’ve finished both our Functions, so let’s do something with them!
Since I’m running things locally, I’m just going to Trigger my HTTP function using Postman. Let’s start with our Create function and create a task. For that, we’ll make a POST request to our Function app and pass in a JSON body that includes TaskName, TaskDescription and IsCompleted properties.
Hit the send button to send our POST request and provided our Function works, we should see the following JSON response:
Now that our HTTP Function has been triggered, it shouldn’t be too long until our ChangeFeedListener function gets triggered to tell us that there has been a modification. Take a note of the document ID below.
It’s the same as our Id for the TaskItem document we just created! Copy that id in postman and pass it as a parameter for our UpdateTaskItem Function. Let’s change the TaskDescription and IsCompleted value and update our item:
Take a look at the response body that our Function returns. We can see that our item has now been changed with the updated values.
And when we look in the logs for our Function app, we see that the Change Feed has tracked our update on our TaskItem.
So there you have it! A pretty basic example of Azure Cosmos DB Change Feed in action!
Take the time to have a look at the code and feel free to clone it and have a play around with it! As you can see, implementing Change Feed functionality is pretty simple and straightforward using Azure Functions.
We can also use the Change Feed processor in the new Azure Cosmos SDK v3 library to leverage Change Feed features, so in the future I’ll have a go with that and write something up.
If you have any questions, feel free to comment here or reach out to me.
Top comments (0)