Combine realtime pub/sub messaging with Azure Functions & Durable Entities

In this post I'll explain how data is distributed in realtime using pub/sub over WebSockets in a serverless application running in Azure. The context I'll be using is a multiplayer Advanced Dungeons & Dragons (ADnD) style game that is turn-based with realtime state updates.

You'll learn:

• How to use pub/sub in C# Azure Functions to publish messages.
• How to use pub/sub in a JavaScript front-end to receive messages.
• How to persist state in Azure Functions using Durable Entities.

TLDR: Play the live game, or look at the source code in the GitHub repo.

Distributing state in serverless applications is complex

Working with state in serverless applications and across many client devices is a difficult thing. First, serverless functions are usually stateless, so they can scale out easily without side effects, such as inconsistent state between the serverless instances. Secondly, synchronizing state across devices in a reliable and scalable way is a challenge to build yourself. Clients can temporarily lose connection, messages can get lost or be out of order.

This post focuses on using the right cloud services to manage the game logic and synchronization of data in a reliable and scalable way. Let's assume we have the following requirements:

• Developers should be up & running quickly to produce a prototype.
• Operational maintenance should be minimal.
• Game logic should be running in the cloud and exposed via an HTTP API.
• A small amount of game data should be persisted.
• Game data provided by the backend should be distributed in realtime across all players.
• The client devices should cope with temporary connection issues and should receive messages in order.

The tech stack

A good solution that fits the above requirements is to have a serverless application that is quick to build, low in maintenance, and affordable to get started with.

Player actions trigger HTTP-based Azure Functions that handle the game logic. Ably handles the distribution of data between the Functions and the clients.

These are the high-level components used for the game:

• Azure Functions, a serverless compute offering in Azure. This is used to create the HTTP API that clients can interact with.
  • Entity Functions, an extension of Azure Functions, used to persist small pieces of game & player state.
• Ably, an edge messaging solution, that offers serverless pub/sub over WebSockets to distribute data in realtime.
• VueJS, a well-known front-end framework.
• Azure Static Web Apps, a hosting solution that serves the static files.

Diagram 1: Communication between player devices and the serverless application.

The game API

The API of the game exposes several HTTP endpoints which are implemented in C# (.NET 6) Azure Functions:

• CreateQuest; triggered by the first player to start a new quest.
• GetQuestExists; triggered by players who would like to join a quest to determine if they provided a valid quest ID.
• AddPlayer; triggered by the player once they have selected their character and name.
• ExecuteTurn; triggered by the player when they want to attack.
• CreateTokenRequest; provides an authentication token and is triggered when a connection to Ably is made via the front-end.

All Azure Functions are just a couple of lines of code. The majority of the game logic is implemented in the GameEngine, GameState, and Player classes. All functions related to game interaction only call methods in the GameEngine class. The GameEngine class is responsible for the game flow, and updating the state of the game and player objects.

Diagram 2: Communication flow within the HTTP Azure Functions.

Creating a new quest

To illustrate how Azure Functions and the GameEngine, GameState, and Player classes work together, I'll show the CreateQuest functionality starting at the Azure Function, and ending with publishing messages using Ably.

CreateQuest function

The CreateQuest HTTP function is triggered when a player clicks the Start Quest button. A new quest ID is generated client-side and provided in the request.

public class CreateQuest
{
    // Only showing the class members relevant for this blog section.
    // For the full implementation see https://github.com/ably-labs/serverless-websockets-quest/blob/main/api/Functions/CreateQuest.cs

    [FunctionName(nameof(CreateQuest))]
    public async Task<IActionResult> Run(
        [HttpTrigger(AuthorizationLevel.Anonymous, "post", Route = null)] HttpRequestMessage req,
        [DurableClient] IDurableClient durableClient,
        ILogger log)
    {
        if (req.Content != null)
        {
            var questId = await req.Content.ReadAsStringAsync();
            var channel = _realtime.Channels.Get(questId);
            var gameEngine = new GameEngine(durableClient, questId, channel);
            var gamePhase = await gameEngine.CreateQuestAsync();
            return new OkObjectResult(gamePhase);
        }
        else
        {
            return new BadRequestObjectResult("QuestId is required");
        }
    }
}

If the request is valid, this function will call the CreateQuestAsync() method in the GameEngine class.

GameEngine - CreateQuestAsync

The GameEngine is responsible for the majority of the game flow and orchestrates actions using the GameState and Player classes.

public class GameEngine
{
    // Only showing the class members relevant for this blog section.
    // For the full implementation see https://github.com/ably-labs/serverless-websockets-quest/blob/main/api/Models/GameEngine.cs

    public async Task<string> CreateQuestAsync()
    {
        var nextGamePhase = GamePhases.Character;
        await InitializeGameStateAsync(nextGamePhase);
        return nextGamePhase;
    }

    private async Task InitializeGameStateAsync(string phase)
    {
        var monsterEntityId = new EntityId(nameof(Player), Player.GetEntityId(_questId, CharacterClassDefinitions.Monster.Name));
        await _durableClient.SignalEntityAsync<IPlayer>(monsterEntityId, proxy => proxy.InitPlayer(
            new object[] {
                _questId,
                CharacterClassDefinitions.Monster.Name,
                CharacterClassDefinitions.Monster.CharacterClass,
                CharacterClassDefinitions.Monster.InitialHealth
            }));

        var gameStateEntityId = new EntityId(nameof(GameState), _questId);
        await _durableClient.SignalEntityAsync<IGameState>(gameStateEntityId, proxy => proxy.InitGameState(new[] { _questId, phase }));
        await _durableClient.SignalEntityAsync<IGameState>(gameStateEntityId, proxy => proxy.AddPlayerName(CharacterClassDefinitions.Monster.Name));
    }
}

The InitializeGameStateAsync method is responsible for:

• Creating the monster, a Player (a non-player character, NPC).
• Creating the initial GameState that contains the quest ID and the name of the game phase.
• Adding the monster to the list of players in the GameState.

Stateful Entity Functions

The GameState and Player classes are so-called Entity Functions, functions that are stateful. Their state is persisted in an Azure Storage Account that is abstracted away. You can interact with entities in two ways:

• Signaling (SignalEntityAsync), which involves one-way (fire and forget) communication.
• Reading the state (ReadEntityStateAsync), which involves two-way communication.

For more information about Entity Functions, please see the Azure docs.

I chose Entity Functions since they're quick to get started with and ideal for storing small objects. Entity Functions require no additional Azure services, since a regular function app already comes with a storage account. Ideal for a demo like this.

There are downsides to Entity Functions though. First, entities prioritize durability over latency. This means that it's not the fastest way to store data. This is because signaling tasks are sent to a storage queue which is being polled at a certain (configurable) frequency. The default configuration also enforces batch processing of signaling tasks which, in this game context, is not desirable. I changed the maxQueuePollingInterval and maxEntityOperationBatchSize settings in the host.json file to have an acceptable latency and consistency.

{
    "version": "2.0",
    "extensions": {
        "durableTask": {
          ...
          "storageProvider" : {
            "maxQueuePollingInterval": "00:00:01"
          },
          "maxEntityOperationBatchSize": 1
        }
    },
    ...
}

Read the Azure docs to learn more about performance and scale related to Entity Functions.

Player entity function

The Player entity function is responsible for maintaining the state of a player in the game. The game has four players: the monster (NPC), and 3 real players. Each one has their own Player entity function.

[JsonObject(MemberSerialization.OptIn)]
public class Player : IPlayer
{
    // Only showing the class members relevant for this blog section.
    // For the full implementation see https://github.com/ably-labs/serverless-websockets-quest/blob/main/api/Models/Player.cs

    [JsonProperty("questId")]
    public string QuestId { get; set; }

    [JsonProperty("playerName")]
    public string PlayerName { get; set; }

    [JsonProperty("characterClass")]
    public string CharacterClass { get; set; }

    [JsonProperty("health")]
    public int Health { get; set; }

    public async Task InitPlayer(object[] playerFields)
    {
        QuestId = (string)playerFields[0];
        PlayerName = (string)playerFields[1];
        CharacterClass = (string)playerFields[2];
        Health = Convert.ToInt32(playerFields[3]);
        await _publisher.PublishAddPlayer(QuestId, PlayerName, CharacterClass, Health);
    }
}

Once a Player entity is initialized, a message will be published to an Ably channel. The players who have joined the quest are subscribed to this channel and will receive a message that a new player has joined.

Operations on Entity Functions are recommended to be implemented via interfaces to ensure type checking. There are however some restrictions on entity interfaces, as described in the Azure docs. One limitation is that interface methods must not have more than one parameter. Since I want to initialize a Player entity and set multiple parameters in one go instead of setting each one individually, I decided to provide an object array as a parameter. This then requires casting/converting the array elements to their correct type. Not ideal, but it works.

GameState entity function

The GameState entity function is responsible for maintaining the state of the game, such as the quest ID, the player names, and the game phase (start, character selection, play, end).

[JsonObject(MemberSerialization.OptIn)]
public class GameState : IGameState
{
    // Only showing the class members relevant for this blog section.
    // For the full implementation see https://github.com/ably-labs/serverless-websockets-quest/blob/main/api/Models/GameState.cs

    [JsonProperty("questId")]
    public string QuestId { get; set; }

    [JsonProperty("phase")]
    public string Phase { get; set; }

    public async Task InitGameState(string[] gameStateFields)
    {
        QuestId = gameStateFields[0];
        Phase = gameStateFields[1];
        await _publisher.PublishUpdatePhase(QuestId, Phase);
    }

    [JsonProperty("players")]
    public List<string> PlayerNames { get; set; }

    public async Task AddPlayerName(string playerName)
    {
        if (PlayerNames == null)
        {
            PlayerNames =  new List<string> { playerName };
        }
        else
        {
            PlayerNames.Add(playerName);
        }

        if (IsPartyComplete)
        {
            await UpdatePhase(GamePhases.Play);
            await Task.Delay(2000);
            await AttackByMonster();
        }
    }

    public async Task UpdatePhase(string phase)
    {
        Phase = phase;
        await _publisher.PublishUpdatePhase(QuestId, Phase);
    }

    private async Task AttackByMonster()
    {
        var playerAttacking = CharacterClassDefinitions.Monster.Name;
        var playerUnderAttack = GetRandomPlayerName();
        var damage = CharacterClassDefinitions.GetDamageFor(CharacterClassDefinitions.Monster.CharacterClass);
        await _publisher.PublishPlayerAttacking(QuestId, playerAttacking, playerUnderAttack, damage);
        await Task.Delay(1000);
        var playerEntityId = new EntityId(nameof(Player), Player.GetEntityId(QuestId, playerUnderAttack));
        Entity.Current.SignalEntity<IPlayer>(playerEntityId, proxy => proxy.ApplyDamage(damage));
        await Task.Delay(1000);
        var nextPlayerName = GetNextPlayerName(CharacterClassDefinitions.Monster.Name);
        await _publisher.PublishPlayerTurnAsync(QuestId, $"Next turn: {nextPlayerName}", nextPlayerName);
    }
}

The GameState also performs actions such as signaling Player entities and publishing messages, as can be seen in the AttackByMonster method.

Note that the AttackByMonster method contains two calls to Task.Delay(). This is to add a bit of delay between publishing the messages about the automated monster attack. Without the delay, the action would be much quicker compared with what real players do. Since this is a turn-based game, with realtime state updates, this seemed more natural to me.

Publishing messages

The final step of game logic functionality in the API is publishing messages to the players that have joined the quest. Since several classes need access to this functionality, I've wrapped it in a Publisher class for ease of use.

public class Publisher
{
    // Only showing the class members relevant for this blog section.
    // For the full implementation see https://github.com/ably-labs/serverless-websockets-quest/blob/main/api/Models/Publisher.cs

    public async Task PublishAddPlayer(string questId, string playerName, string characterClass, int health)
    {
        if (_ablyClient != null)
        {
            var channel = _ablyClient.Channels.Get(questId);
            await channel.PublishAsync(
                "add-player",
                    new
                    {
                        name = playerName,
                        characterClass = characterClass,
                        health = health
                    }
                );
        }
    }

    public async Task PublishUpdatePhase(string questId, string phase, bool? teamHasWon = null)
    {
        if (_ablyClient != null)
        {
            var channel = _ablyClient.Channels.Get(questId);
            await channel.PublishAsync(
                "update-phase",
                new
                {
                    phase = phase,
                    teamHasWon = teamHasWon
                }
            );
        }
    }
}

Publishing messages is the easiest part of the API. The _ablyClient in the code sample above is an instance of the Ably REST client and responsible for publishing messages to a channel. The REST client is used since code is running in a short-lived Azure Function and doesn't require a bidirectional WebSocket connection. The client retrieves the channel, the quest ID in this case, and PublishAsync is called that accepts an event name and a payload.

Receiving messages client-side

The client-side is subscribed to the messages published via the API using the realtime Ably client that is based on WebSockets. Based on the type of message received, the game progresses to the next phase, and local player state is updated. So, even though this is a turn-based game, updates in the API result in realtime communication with the players to update their local player state.

The clients require a connection to Ably to receive messages in realtime. The createRealtimeConnection function is called when players start a new quest or join a quest.

async createRealtimeConnection(clientId: string, questId: string) {
    if (!this.isConnected) {
        const realtimeClient = new Realtime.Promise({
            authUrl: `/api/CreateTokenRequest/${clientId}`,
            echoMessages: false,
        });
        this.realtimeClient = realtimeClient;
        realtimeClient.connection.on("connected", async (message: Types.ConnectionStateChange) => {
            this.isConnected = true;
            const messageText = `Ably connection status: ${realtimeClient.connection.state}`;
            this.writeMessage(messageText);
            await this.attachToChannel(questId);
        });

        realtimeClient.connection.on("disconnected", () => {
            this.isConnected = false;
            const messageText = `Ably connection status: ${realtimeClient.connection.state}`;
            this.writeMessage(messageText);
        });
    }
},
async attachToChannel(channelName: string) {
    const channelInstance = this.realtimeClient?.channels.get(
        channelName,
        {
            params: { rewind: "2m" },
        }
    );
    this.channelInstance = channelInstance;
    this.subscribeToMessages();
},

When a new instance of the Realtime object is created, the authUrl is set by calling the CreateTokenRequest endpoint in the API. This returns a JWT token that is used to authenticate with Ably. This approach prevents any API keys from being present in the front-end and potentially ending up in source control.

Once a connection is made, the client attaches to the channel (named after the questId) and subscribes to messages published by Ably.

As an example of how the front-end updates when a player attacks, let's have a look how the player-under-attack message is handled.

subscribeToMessages() {

    // Only showing the functions relevant for this blog section.
    // For the full implementation see https://github.com/ably-labs/serverless-websockets-quest/blob/main/src/stores/index.ts

    this.channelInstance?.subscribe(
        "player-under-attack",
        (message: Types.Message) => {
            this.handlePlayerIsUnderAttack(message);
        }
    );
},

handlePlayerIsUnderAttack(message: Types.Message) {
    if (this.teamHasWon !== undefined) return;
    const playerName: string = message.data.name;
    const characterClass: CharacterClass = message.data.characterClass as CharacterClass;
    const health: number = message.data.health;
    const damage: number = message.data.damage;
    const isDefeated: boolean = message.data.isDefeated;
    const messageText = `${playerName} received ${damage} damage`;
    this.writeMessage(messageText);
    this.updatePlayer(playerName, characterClass, health, damage, isDefeated, false, true);
},

updatePlayer(playerName: string, characterClass: CharacterClass, health: number, damage: number, isDefeated: boolean, isAvailable: boolean, isUnderAttack: boolean) {
    if (characterClass === CharacterClass.Fighter) {
        this.$patch({
            fighter: {
                name: playerName,
                health: health,
                damage: damage,
                isAvailable: isAvailable,
                isUnderAttack: isUnderAttack,
                isDefeated: isDefeated
            },
        });
        setTimeout(() => this.fighter.damage = 0, 3000);
    } else if (characterClass === CharacterClass.Ranger) {
        // Update ranger object
    } else if (characterClass === CharacterClass.Mage) {
        // Update mage object
    } else if (characterClass === CharacterClass.Monster) {
        // Update monster object
    }
},

The local Vue store (Pinia) contains definitions for each player. The store is updated with data coming from the messages pushed by Ably.

Vue components such as the PlayerUnit use the local data store to display the character information:

<!-- See https://github.com/ably-labs/serverless-websockets-quest/blob/main/src/components/PlayerUnit.vue for the full implementation. -->

<template>

    <div v-if="!props.isPlayerSelect">
        <p v-if="props.useHealth">
            <span class="health">{{ targetPlayer.health }} HP</span>
            <span class="damage" v-if="showDamage()">-{{ targetPlayer.damage }}</span>
        </p>
        <p class="stats" v-if="props.showStats">
            <span>Damage caused:</span>
            <span class="info">{{ targetPlayer.totalDamageApplied }}</span>
        </p>
        <img v-bind:class="{ isActive: isActive(), isDefeated: targetPlayer.isDefeated }" :alt="targetPlayer.characterClass" :src="getAsset()" />
        <figcaption>{{ targetPlayer.name }}</figcaption>
    </div>

    <div v-if="props.isPlayerSelect">
        <input type="radio" :id="targetPlayer.characterClass" name="character" :value="targetPlayer.characterClass" v-model="store.characterClass" @click="store.playerName=getName()" :disabled="isDisabled()" />
        <label :for="targetPlayer.characterClass">
            <img :alt="targetPlayer.characterClass" :src="getAsset()" />
            <figcaption>{{ getName() }}</figcaption>
        </label>
    </div>

</template>

Running locally

You require the following dependencies to run the solution locally:

• .NET 6. The .NET runtime required for the C# Azure Functions.
• Node 16. The JavaScript runtime required for the Vue front-end.
• Azure Functions Core Tools. This is part of the Azure Functions extensions for VSCode that should be recommended for installation when this repo is opened in VSCode.
• Azurite. This is a local storage emulator that is required for Entity Functions. When this repo is opened in VSCode, a message will appear to install this extension.
• Azure Static Web Apps CLI. This is the command line interface to develop and deploy Azure Static Web Apps. Install this tool globally by running this command in the terminal: npm install -g @azure/static-web-apps-cli.
• Sign up for a free Ably Account, create a new app, and copy the API key.

Steps

1. Clone or fork the Serverless WebSockets Quest GitHub repo.
2. Run npm install in the root folder.
3. Rename the api\local.settings.json.example file to api\local.settings.json.
4. Copy/paste the Ably API key in the ABLY_APIKEY field in the local.settings.json file.
5. Start Azurite (VSCode: CTRL+SHIFT+P -> Azurite: Start)

6. Run swa start in the root folder.

   You'll see this error message, which is a warning really, you can ignore this when running the solution locally:
   

   Function app contains non-HTTP triggered functions. Azure Static Web Apps managed functions only support HTTP functions. To use this function app with Static Web Apps, see 'Bring your own function app'.
   

   The terminal will eventually output this message that indicates the emulated Static Web App is running:
   

   Azure Static Web Apps emulator started at http://localhost:4280. Press CTRL+C to exit.
   

7. Open the browser and navigate to http://localhost:4280/

Deploying to the cloud

To have the entire solution running in the Azure cloud, you'll need an Azure account, and a GitHub account.

The free Azure Static Web Apps (SWA) tier comes with Managed Azure Functions. These are functions that are included in the Static Web App service. The downside of these managed functions is that only HTTP trigger functions are supported. Our game API uses Entity Functions as well. In order for Static Web Apps to work with our API we need to:

1. Deploy the API separately to a dedicated Function App.
2. Use the Standard (non-free) tier of Azure Static Web Apps.
3. Update the configuration of SWA to indicate that a dedicated Function App is being used.

Deployment steps

1. I've created a GitHub workflow to:
   • Create the Azure resources.
   • Build the C# API.
   • Deploy the API to the Function App.

This approach uses the Azure Login action and requires the creation of an Azure Service Principal, as is explained in more detail in this README.

1. The SWA resource was created in the Azure portal using this quick start. This results in a generated GitHub workflow that will be included in the GitHub repository.
2. Follow these instructions to configure SWA to use a dedicated Function App.

Summary

Using serverless technology is a great way to get up and running in the cloud quickly and not worry about server maintenance. By combining serverless functions (Azure Functions) with serverless WebSockets (Ably) you can build a realtime solution that is cost-efficient and can scale automatically.

Although this demo is built around a game concept, other live and collaborative experiences are also a good fit for this tech stack. These include chat apps, location tracking apps, and realtime monitoring dashboards.

You've learned how to publish messages from the Azure Functions back-end and receive messages in the VueJS front-end to create a realtime experience for the players.

I encourage you to fork the repository available on GitHub and see if you can extend it (add a cleric who can heal other players?). Please don't hesitate to contact me on Twitter or join our Discord server in case you have any questions or suggestions related to this project.