DEV Community: RoboDoig

A practical guide to RNNs for neuroscience research in Keras

RoboDoig — Mon, 01 Mar 2021 00:24:16 +0000

Introduction

Recurrent neural network models have increasingly become powerful models for neuroscience research as we saw in this year’s COSYNE meeting. They can be used to e.g. generate model data for testing analysis methods; or infer connectivity between brain areas based on real neural data.

Despite their obvious practical use, it can sometimes be difficult to make the leap from a written down RNN model in a paper to its practical application in code. Moreover, although frameworks have made it easier than ever to prototype machine-learning models, in neuroscience research we are often interested in network statistics other than standard outputs like accuracy and learning rate - we may instead want to extract layer activations and weights at different stages during model training as a proxy for real neuron activity and connectivity.

This guide is an attempt to develop and explain some simple RNN examples in the Keras framework that are inspired by and applicable to neuroscience applications.

Colab notebooks

Example 1 - MNIST
Example 2 - Data Generation
Example 3 - Connectivity

Example 1 - Simple MNIST

To show the general structure of an RNN in Keras, we’ll start with the classic MNIST example. Here we will obtain a labeled sequence of images of hand drawn digits and train an RNN model to predict the represented digit in the image:

In python, let’s start with the necessary imports and loading the MNIST dataset:

import numpy as np
from tensorflow import keras
from keras.callbacks import ModelCheckpoint
from matplotlib import pyplot as plt
from keras import backend as K

# load the dataset
mnist = keras.datasets.mnist
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
sample, sample_label = x_train[0], y_train[0]

Here we simply load the standard MNIST dataset from the keras library and split it into train and test datasets. x_train and x_test are input data (sequences of images), y_train and y_test are target labels (digits from 0-9).

As a sanity check, let’s plot some of the input images along with their labels. Looping through the first 10 examples in the training dataset and plotting with the labels we obtain:

# show examples
n_examples = 10
plt.figure()
for i in range(n_examples):
   plt.subplot(1, n_examples, i+1)
   plt.imshow(x_train[i])
   plt.title(y_train[i])
plt.show()

So far so good. Now let’s define some model parameters and build the actual model:

# model parameters
input_dim = x_train[0].shape[0]
output_size = 10
epochs = 10
units = 64

# build model
model = keras.models.Sequential()
model.add(keras.layers.SimpleRNN(units, input_shape=(input_dim, input_dim)))
model.add(keras.layers.BatchNormalization())
model.add(keras.layers.Dense(output_size))

model.summary()

First we defined an input dimension parameter input_dim based on the input image shape (28x28 pixels). We then set define output_size = 10 since we want 10 classification classes (digits 0-9). epochs defines the number of training repetitions, and units is the number of neurons we want in our RNN.

Next we use Keras’ sequential model interface to stack network layers on top of each other. First we add a standard RNN layer with SimpleRNN, then a normalization layer for the output and finally a fully connected dense output layer with 10 nodes. Invoking model.summary() should print:

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn (SimpleRNN)       (None, 64)                5952      
_________________________________________________________________
batch_normalization (BatchNo  (None, 64)                256       
_________________________________________________________________
dense (Dense)                (None, 10)                650       
=================================================================
Total params: 6,858
Trainable params: 6,730
Non-trainable params: 128

Let’s step through one layer to understand the network we created. We have a SimpleRNN layer with an output shape of (None, 64) and 5952 total parameters. Where does the output shape information come from? Remember we created our RNN layer with 64 neurons, so an output of 64 in one shape dimension makes sense. The first shape dimension showing None corresponds to the batch or trial number, i.e. the number of examples we will feed to the network over training. Keras allows us to be vague with this number and represent it as None so that we can feed in training datasets of different sizes easily without violating the input dimensions of the layer. We took advantage of this fact before already with the line:

model.add(keras.layers.SimpleRNN(units, input_shape=(input_dim, input_dim)))

An RNN input shape in Keras should have 3 dimensions: batch, timestep, feature but we only provided 2 dims of shape input. This is because the batch dimension is implied by Keras, assuming we will feed in datasets of different lengths.

What about the number of parameters for the RNN layer? We have 64 units in the RNN which are recurrently connected (all connected to each other), which gives us 64*64 = 4096 trainable parameters. Our image input is 28x28 pixels, which the RNN represents as 28 sequences of 28 features, therefore 28 feature channels are passed to the 64 RNN units giving an additional 28*64=1792 parameters. In total now we have 1792+4096=5888 parameters. Finally we have the 64 bias terms for the RNN units, giving the final parameter number of 5952.

Training a model in Keras is similarly simple, we just need to add:

# train
model.compile(
   loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
   optimizer="sgd",
   metrics=["accuracy"]
)

history = model.fit(
   x_train, y_train, validation_data=(x_test, y_test), batch_size=1000, epochs=epochs
)

We use the compile function to define the training loss function, the optimizer and our output metrics. Since we are predicting a categorical variable (digit) we use sparse categorical cross-entropy here for the loss function. Our optimizer is standard stochastic gradient descent and we ask the training to keep track of accuracy (% trials the model predicts the digit from the input image). Finally we call model.fit() with the training and validation test sets. Leaving this to train we’ll see some output appear:

Epoch 1/10
60/60 [==============================] - 3s 29ms/step - loss: 2.2067 - accuracy: 0.2987 - val_loss: 1.7675 - val_accuracy: 0.4677
Epoch 2/10
60/60 [==============================] - 1s 20ms/step - loss: 1.3351 - accuracy: 0.5689 - val_loss: 1.3675 - val_accuracy: 0.6531
Epoch 3/10
60/60 [==============================] - 1s 18ms/step - loss: 0.9991 - accuracy: 0.6946 - val_loss: 1.0370 - val_accuracy: 0.7391
Epoch 4/10
60/60 [==============================] - 1s 19ms/step - loss: 0.8169 - accuracy: 0.7563 - val_loss: 0.8105 - val_accuracy: 0.7897
Epoch 5/10
60/60 [==============================] - 1s 20ms/step - loss: 0.6963 - accuracy: 0.7922 - val_loss: 0.6728 - val_accuracy: 0.8142
Epoch 6/10
60/60 [==============================] - 1s 20ms/step - loss: 0.6205 - accuracy: 0.8149 - val_loss: 0.5768 - val_accuracy: 0.8389
Epoch 7/10
60/60 [==============================] - 1s 20ms/step - loss: 0.5545 - accuracy: 0.8340 - val_loss: 0.5140 - val_accuracy: 0.8552
Epoch 8/10
60/60 [==============================] - 1s 20ms/step - loss: 0.5008 - accuracy: 0.8500 - val_loss: 0.4637 - val_accuracy: 0.8644
Epoch 9/10
60/60 [==============================] - 1s 20ms/step - loss: 0.4618 - accuracy: 0.8630 - val_loss: 0.4319 - val_accuracy: 0.8741
Epoch 10/10
60/60 [==============================] - 1s 19ms/step - loss: 0.4256 - accuracy: 0.8751 - val_loss: 0.4006 - val_accuracy: 0.8813

After 10 epochs, we have a model that can predict digit from an image with >80% accuracy, pretty good!

As a sanity check, let’s add some code using the model to predict a digit from one of our dataset images:

# prediction example
test_image = x_train[0]
test_image = test_image[np.newaxis, :]
y_pred = model.predict(test_image)[0]
plt.figure()
plt.imshow(test_image[0])
plt.title(np.argmax(y_pred))
plt.show()

Which outputs:

We created a variable test_image with the first image from x_train. Since this is a 2D image, and the RNN model expects an extra batch dimension we pad the image with an empty extra dimension in the next line. Then we generate the prediction y_pred using model.predict() with our test_image. Next, we plot the image and title it with the output prediction. Remember that the model output is a 10-element vector where each element represents a prediction probability of each of the digits 0-9. To get the actual predicted digit, we use np.argmax() to find the element with the largest probability. As it turns out, this corresponds nicely to the input image!

If you’re a neuroscientist, and the RNN is the ‘brain’ you’re studying, you may not care particularly about the accuracy of the model network. After all we already know that our brains can recognise and classify these digit images. Instead, you might be more interested in the how - how are the network neurons behaving during presentation of each image. In an artificial neural network, layer activations can be thought of as analogous to neural activity in the brain. Activations are the outputs of neurons in a layer after they receive their input from the previous layer.

If you delve into the properties of the model you just trained, you will notice that the activations for each layer are not stored. This makes sense since all we need to store for the model training is the input, output and updated weights between layers. Storing the activations for each training step and input would cost a lot of storage. To get the layer activations for a particular image input, we need to do some extra Keras magic. Sticking with the test_image we already defined, we’ll add some code to explore the activations:

# activation example
layer_idx = 0
out_func = K.function([model.input], [model.layers[layer_idx].output])
out_vals = out_func([test_image])[0]
print(out_vals)

Giving us the output:

[[ 0.56697565  0.09797323 -0.89703596  0.5188298  -0.12455866  0.047839
  -0.7718987  -0.8179464   0.51488703 -0.3178563  -0.13602903  0.7039628
  -0.22956386  0.23199454  0.49808362 -0.1646781   0.18231589 -0.52438575
  -0.7650064   0.26156113 -0.14623232  0.81333166  0.3180512  -0.4301887
  -0.8027709   0.07813827  0.41824704 -0.8176996   0.02754677  0.2746857
   0.64864177  0.59684217  0.51136965  0.6604145  -0.25604498  0.30178267
   0.31990722 -0.7244299   0.78560436 -0.42247573 -0.16835652 -0.197974
   0.1738112   0.61906195 -0.69502765  0.3859463  -0.09267779  0.27790046
   0.09295665 -0.07516889  0.83438504 -0.15787967 -0.553465   -0.67424375
   0.06541198 -0.1020548  -0.7939734  -0.09875299  0.20282765  0.63470924
  -0.33998007 -0.04162058 -0.33605504 -0.15319426]]

Here we set a layer_idx of 0 (corresponding to the first layer in our network, the RNN). Then we used the keras backend library to define a Keras function called out_func that uses the model input to produce the output of the model at the layer index we provide. We then use this function with our test_image. The output is a 64-element vector corresponding to the activations of our 64 RNN neurons after being presented with the test image.

Example 2 - Generating ground-truth data for testing analysis methods

In the 2018 paper “Unsupervised Discovery of Demixed, Low-Dimensional Neural Dynamics across Multiple Timescales through Tensor Component Analysis”, Williams et al. use the statistics of a RNN to test the utility of their tensor component analysis method. The advantage of using an ANN here is that they have access to the ground truth of layer activation and connection weights to compare against the results of the TCA method.

The data generation strategy of this paper is shown below, reproduced from the paper’s 3rd figure:

The input to the network is 40 timesteps of random noise, with some offset such that it has a slightly negative or slightly positive mean value across time. The network is trained to report the sign of the input with a high magnitude. The two rightmost panels show the activation across time of a (+) and (-) responsive cell in the RNN before and after training. Over time, their responses diverge to produce either positive or negative drive to the output cell.

Let’s recreate this model in Keras.

First we need to create the input data and its target labels to train on. Remember that an RNN in Keras expects 3D shaped data with the dimensions corresponding to batch, timestep, feature. Each input has 40 timesteps of a single feature, and we’ll make 100,000 training samples, so we need 100,000x40x1 input data. Each input needs a label for the model training (the overall sign of the input) which need only be a single positive or negative number. To perform the necessary imports and define this dataset:

import numpy as np
from tensorflow import keras
from keras.callbacks import ModelCheckpoint
from matplotlib import pyplot as plt
from keras import backend as K

# create the dataset
data_n = 100000
timepoints = 40
sign_vec = (np.random.randint(2, size=(data_n, 1)) * 2) - 1

input = np.random.rand(data_n, timepoints) - 0.5  # random values centered around 0
input = input + (sign_vec * 0.2)
input = input[:, :, np.newaxis]  # reshape for RNN
output = sign_vec * 3

# plot the dataset
plt.figure()
for i in range(100):
   c = 'k' if sign_vec[i] == 1 else 'r'
   plt.plot(input[i, :], c)

plt.show()

Let’s step through this code. First we create a variable sign_vec to generate a random sequence of positive or negative numbers. We use numpy’s randint function to create 100,000 random integers of either 0 and 1, then scale these so that we get either -1 or 1. Next we define input as a matrix of random numbers of size 100000x40 with mean zero. Then we add the sign_vec sequence to shift each entry slightly positive or negative according to the elements in sign_vec. We add an extra dimension to input such that it satisfies the input requirements for our eventual RNN. Finally, we generate the output labels for training by simply scaling sign_vec so we get either a -3 or +3 label. The end result is a dataset of 40-element vectors that are slightly positive or negative, and a corresponding label describing the sign. The last code segment plots the first 100 examples of this dataset and colours by sign:

Next we need to build the model:

# build model
units = 50
model = keras.models.Sequential()
# have to return sequences to get the actual unrolled response across time
model.add(keras.layers.SimpleRNN(units, return_sequences=True, input_shape=(timepoints, 1)))
model.add(keras.layers.Dense(1))
model.summary()

This is very similar to our previous model with a few key adjustments. This time we’ll use 50 units and define a SimpleRNN layer with input shape of timepointsx1 (40x1) to match the incoming training data. Notice we added an extra argument to the SimpleRNN layer - return_sequences=True. When we provide input data to an RNN with multiple timepoints, internally the RNN is calculating output for the first timestep and passing that output as input for the next timestep calculation. If return_sequences is False, we only pass the final state of those recurrent timestep calculations. In our case, we want to see RNN activations at each timestep so we set this to True to get access to data about the RNN at each step of the sequence. Finally, we add a single unit dense layer as our network output. We only need one unit here since our training labels are just a single positive or negative number. Invoking model.summary() gives:

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn (SimpleRNN)       (None, 40, 50)            2600      
_________________________________________________________________
dense (Dense)                (None, 40, 1)             51        
=================================================================
Total params: 2,651
Trainable params: 2,651
Non-trainable params: 0

In our previous MNIST example we only analysed the network after it had finished training. In this case, we want to compare the network before and after training. Model training in Keras does not automatically save the model structure at each step of training, so we need to add an extra step here to save a snapshot of our model periodically through training:

# set a history callback to save weights after each epoch
weights_file_prefix = "tmp/model_"
model.save(weights_file_prefix + "untrained.h5")
checkpoint = ModelCheckpoint(weights_file_prefix + "{epoch}.h5", save_freq='epoch')

All we do here is define a save path and use model.save() to save the randomly initialized weights of the model before training starts. Then we define a ModelCheckpoint from the Keras callbacks module. All this does is provide a callback function for the training procedure that saves a model snapshot every training epoch.

Now we can move on to training the model:

# train
epochs = 10
model.compile(
   loss="mse",
   optimizer="sgd",
   metrics=["mse"]
)

history = model.fit(
   input, output, batch_size=1000, epochs=epochs, callbacks=[checkpoint]
)

This is again very similar to our MNIST training example with some small adjustments. First, in this case we don’t really care about the output being exactly -3 or +3, we just want it to skew towards those values - therefore we just use mean squared error (“mse”) as both the loss function and metric in model.compile(). In model.fit() we also add our previously defined ModelCheckpoint to the training callbacks.

After the model has finished training we’d like to see whether we reproduced the neuron behavior from Williams et al. To do this, we need to extract a neuron’s activation across time in response to either a positive or negative stimulus:

# try a prediction on the trained network
n_examples = 100
neuron = 0

plt.figure()
for i in range(n_examples):
   input_test = input[i][np.newaxis, :, :]
   y_pred = model.predict(input_test)

   # get activations for rnn layer
   out_func = K.function([model.input], [model.layers[0].output])
   out_vals = out_func([input_test])
   out_activation = out_vals[0][0, :, :]

   c = 'k' if sign_vec[i] == 1 else 'r'
   plt.plot(out_activation[:, neuron], c)
plt.ylim(-1, 1)
plt.show()

Here we loop through 100 examples of training data and use the activation extraction trick to get activations over time for one of the neurons in the RNN. Then we plot the activation time course and colour by the sign of the input vector:

Good! This neuron diverges in its activity over time depending on the input sign much as in the paper.

Next let’s repeat this process for the untrained network. To do this, we’ll load a model snapshot from earlier in the training and apply the same process:

# repeat for the untrained network
model.load_weights(weights_file_prefix + "untrained.h5")
plt.figure()
for i in range(n_examples):
   input_test = input[i][np.newaxis, :, :]
   y_pred = model.predict(input_test)

   # get activations for rnn layer
   out_func = K.function([model.input], [model.layers[0].output])
   out_vals = out_func([input_test])
   out_activation = out_vals[0][0, :, :]

   c = 'k' if sign_vec[i] == 1 else 'r'
   plt.plot(out_activation[:, neuron], c)
plt.ylim(-1, 1)
plt.show()

This is exactly the same as in the previous step but we added model.load_weights() with the first model snapshot we generated before the onset of training. Running this we get:

This replicates the analogous result in the paper in which the same neuron, before training, has overlapping responses to the two input types.

Example 3 - Inferring connectivity

With certain neuroscience techniques (e.g. 2p imaging) we can generate rich data about the activity of groups of neurons over time and in response to different stimuli. In many cases, we’d also like to infer something about the connectivity of these neurons to understand the circuit architecture they operate in.

Without electron microscope or paired-patch recordings how could we think about doing this with a dataset? One approach might be to model sets of neurons with groups of RNNs. Using real data from neuronal recordings we could train the RNNs to match their responses to the real neurons in response to some stimulus. Once responses are sufficiently similar between the real neurons and the RNN, we can analyze the connection weights in the RNN to infer connectivity between the real neurons.

In this example we will imagine that we are able to image two putatively connected populations of neurons simultaneously. We are also able to stimulate certain subgroups of one of the neuronal populations. Therefore, we can observe the effect of different stimulation combinations on the first population on output of the second population. To model this situation, we will generate a ‘teacher’ network in place of real neuronal recordings. This network will be a 2-layer RNN with known connection weights that we’ll feed with random input in the first layer to produce a series of outputs in the second layer. We don’t need to train this network as it will just serve the purpose of producing an input-output dataset from a network with known weights. We will use this dataset to train a second network with the same architecture - the ‘student’ network. This network will learn to reproduce the input-output relationship of the teacher network over the training procedure. When the student network produces comparable responses to the teacher network we can extract its connection weights and see if it has correctly inferred the connectivity of the teacher network.

The architecture of both teacher and student networks is represented below:

We have an input RNN into which we feed input data. This is fully connected (represented by black arrow) to the 2nd RNN which produces our output.

Based on what we’ve done so far, it seems simple to construct this network; we might write something like this:

neurons = 10

model = keras.models.Sequential()
model.add(layers.SimpleRNN(neurons, return_sequences=True, input_shape=(1, neurons)))
model.add(layers.SimpleRNN(neurons, input_shape=(1, neurons)))
model.summary()

This would indeed produce a fully connected 2-layer RNN structure. We also set return_sequences=True on the first layer to ensure all input data sequences are transferred to the second layer. However, if we invoke model.summary() we’ll encounter an issue:

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
simple_rnn (SimpleRNN)       (None, 1, 10)             210       
_________________________________________________________________
simple_rnn_1 (SimpleRNN)     (None, 10)                210       
=================================================================
Total params: 420
Trainable params: 420
Non-trainable params: 0

For 10 neurons in the first RNN layer, we expect to have 10*10=100 recurrent connections which gives us 100 parameters. The remaining 110 parameters come from the connections between the RNN and the input layer, + the bias terms for each connection. Although we didn’t explicitly define it, the input layer is inferred by Keras in our sequential model. Since we set our input_shape=(1, neurons) we told Keras to expect a 10-element input. This input layer is also fully connected to our first RNN layer by default in Keras so we end up with an extra 10x10 connections (+ 10 bias terms).

Normally this wouldn’t be an issue, but in our case we want to exactly control the initial activity in our first RNN layer, since in our model the input corresponds to direct stimulation of neurons in this layer. If the input and RNN layers are fully connected as is currently the case, there will be cross-talk between input elements and their target neurons, as schematised below:

In the fully connected case, each neuron in the 2nd layer is a product of all 4 input elements. For our model we want the input layer and first RNN layer to use a one-to-one pattern so that we can directly control the activation of the RNN. Unfortunately, there is no out-of-the-box way to define this connection relationship with Keras so we’ll have to think of a workaround. Let’s implement this first with a simpler case - connection to a non-recurrent layer.

We’ll begin with building this simple model:

neurons = 4
model = keras.models.Sequential()
model.add(layers.Dense(neurons, input_shape=(1, neurons)))
model.summary()

Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense (Dense)                (None, 1, 4)              20        
=================================================================
Total params: 20
Trainable params: 20
Non-trainable params: 0

We can see that the input is fully connected with the output since we have 20 parameters in the dense layer (4x4 input connections + 4 bias terms). We can also see that the input is not exactly reproduced in the output dense layer because of this connectivity with a quick test:

print(model.predict(np.ones(1, 4)))
[[-0.79105026  0.09893554  1.4398389  -0.5700609 ]]

The weights between the 4 input elements and the 4 outputs in the dense output layer are represented by a 4x4 matrix describing each connection weight. What if we manually replaced that matrix with a 4x4 identity matrix? (Identity matrix is 1 on the diagonal and 0 everywhere else, a matrix multiplied by an identity matrix of the same size equals the original matrix):

# force weights, kernel weights are identity matrix
weights = model.layers[0].get_weights()
input_weights = np.eye(neurons, neurons)
bias_weights = weights[1]
model.layers[0].set_weights([input_weights, bias_weights])

Above, we get the weights from the first layer of the matrix (the input weights and the bias terms). We then set the first layer input weights as the identity matrix, and keep the bias weights the same. If we repeat the same test:

print(model.predict(np.ones((1, 4))))
[[1. 1. 1. 1.]]

We have the desired result! Our input layer is directly represented in our output.

Let’s now write some code to simulate the training process on this simple network and give us some output metrics

# create a dummy dataset
data_n = 10000
train_x = np.random.rand(data_n, 1, 4)
train_y = train_x + 1

print('pretrain weights: ', model.weights[0], model.weights[1])

# train
model.compile(
   loss="mse",
   optimizer="sgd",
   metrics=["mse"]
)

history = model.fit(
   train_x, train_y, batch_size=64, epochs=10
)

print('trained weights: ', model.weights[0], model.weights[1])

Before training we get get the expected result of the input weights corresponding to the identity matrix:

print('pretrain weights: ', model.weights[0], model.weights[1])

pretrain weights:  <tf.Variable 'dense/kernel:0' shape=(4, 4) dtype=float32, numpy=
array([[1., 0., 0., 0.],
       [0., 1., 0., 0.],
       [0., 0., 1., 0.],
       [0., 0., 0., 1.]], dtype=float32)> <tf.Variable 'dense/bias:0' shape=(4,) dtype=float32, numpy=array([0., 0., 0., 0.], dtype=float32)>

But after training:

print('trained weights: ', model.weights[0], model.weights[1])

trained weights:  <tf.Variable 'dense/kernel:0' shape=(4, 4) dtype=float32, numpy=
array([[1.1834619 , 0.18346326, 0.18346316, 0.18346334],
       [0.18353002, 1.1835296 , 0.18352997, 0.18353006],
       [0.18180324, 0.18180329, 1.181803  , 0.18180329],
       [0.17997868, 0.17997865, 0.1799786 , 1.1799785 ]], dtype=float32)> <tf.Variable 'dense/bias:0' shape=(4,) dtype=float32, numpy=array([0.62174916, 0.62174857, 0.62174875, 0.6217486 ], dtype=float32)>

They’ve changed! Although we initially set these weights to the identity matrix, the training algorithm shifted them on each iteration. Keras does allow us to hide certain weights from the training process with trainable=False in the layer definition, but for the case of the RNN this would hide the recurrent weights as well which we don’t want. Instead we will take advantage of Keras’ custom constraints which we can direct to specific groups of weights. Normally these constraints would be used to e.g. regularise or normalise weights after weight updates, but here we will simply brute force the weights to our desired values:

# custom constraint to hold weights at user defined value
class HoldWeight(tf.keras.constraints.Constraint):
 """Constrains weight tensors to a set value/matrix"""
 def __init__(self, set):
     self.set = set

 def __call__(self, w):
   return self.set

Here we created a new class that inherits from the Keras Constraint class. This class takes an argument set which will be our desired weight values. We override the method __call__ and return our desired weights in place of the updated weights w.

We can apply this to our Dense layer as follows:

model.add(layers.Dense(neurons, input_shape=(1, neurons), kernel_constraint=HoldWeight(np.eye(neurons, neurons))))

Applying the same test on this model after training:

print('trained weights: ', model.weights[0], model.weights[1])

trained weights:  <tf.Variable 'dense/kernel:0' shape=(4, 4) dtype=float32, numpy=
array([[1., 0., 0., 0.],
       [0., 1., 0., 0.],
       [0., 0., 1., 0.],
       [0., 0., 0., 1.]], dtype=float32)> <tf.Variable 'dense/bias:0' shape=(4,) dtype=float32, numpy=array([0.99961793, 0.99961793, 0.99961793, 0.99961793], dtype=float32)>

We can see that our input weights remained as the identity matrix, and only the bias weights were altered.

Now we can put all this together and create the teacher-student approach we discussed earlier. First creating the teacher network:

import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers
import numpy as np
import matplotlib.pyplot as plt
from keras.callbacks import ModelCheckpoint

# parameters
neurons = 10
n_generator = 1000000

# construct the teacher network
teacher_model = keras.models.Sequential()
teacher_model.add(layers.Input(shape=(1, neurons)))
teacher_model.add(layers.SimpleRNN(neurons, use_bias=False,
                                  return_sequences=True,
                                  kernel_constraint=HoldWeight(np.eye(neurons, neurons))))
teacher_model.layers[0].set_weights([np.eye(neurons, neurons),
                                    teacher_model.layers[0].get_weights()[1]])
teacher_model.add(layers.SimpleRNN(neurons, use_bias=False))
teacher_model.summary()

# generate teacher output for the training dataset
generator_input = np.random.rand(n_generator, 1, neurons)
generator_output = teacher_model.predict(generator_input)

# plot the first layer recurrent weights
plt.figure()
plt.imshow(teacher_model.weights[2], cmap='hot')
plt.show()

We define a 2-layer RNN network, and use the identity matrix trick on the first layer so that we can directly apply a desired input onto the first RNN layer. Next we create a dataset of random 10-element (corresponding to the 10 neurons in the RNN) inputs in generator_input. We feed those inputs through the network with model.predict() to give the network output from the 2nd RNN in generator_output. Finally we create a heatmap showing the weights between the recurrent layers with teacher_model.weights[2]. We use the index 2 here since the weights are listed in order inside the model, input weights are weights[0], recurrent weights in 1st RNN are weights[1], weights between RNNs are weights[2].

Next, we create the student network and train it on the input-output dataset we just defined:

# set up the student
student_model = keras.models.Sequential()
student_model.add(layers.Input(shape=(1, neurons)))
student_model.add(layers.SimpleRNN(neurons,
                                  use_bias=False,
                                  return_sequences=True,
                                  kernel_constraint=HoldWeight(np.eye(neurons, neurons))))
student_model.layers[0].set_weights([np.eye(neurons, neurons),
                                    student_model.layers[0].get_weights()[1]])
student_model.add(layers.SimpleRNN(neurons, use_bias=False))
student_model.summary()

# set a history callback to save weights after each epoch
weights_file_prefix = "tmp/model_"
student_model.save(weights_file_prefix + "untrained.h5")
checkpoint = ModelCheckpoint(weights_file_prefix + "{epoch}.h5", save_freq='epoch')

# train
student_model.compile(
   loss="mse",
   optimizer="sgd",
   metrics=["mse", "accuracy"]
)

history = student_model.fit(
   generator_input, generator_output, batch_size=1000, epochs=100, callbacks=[checkpoint]
)

This should all look familiar by now. The network architecture is exactly the same as the teacher network. We define a callback to save model snapshots at each epoch, then compile the model and train on the generator_input, generator_output dataset.

We should get pretty good performance after our 100 epochs of training:

Epoch 98/100
1000/1000 [==============================] - 3s 3ms/step - loss: 1.3204e-05 - mse: 1.3204e-05 - accuracy: 0.9763
Epoch 99/100
1000/1000 [==============================] - 3s 3ms/step - loss: 1.2516e-05 - mse: 1.2516e-05 - accuracy: 0.9770
Epoch 100/100
1000/1000 [==============================] - 3s 3ms/step - loss: 1.1846e-05 - mse: 1.1846e-05 - accuracy: 0.9773

Finally let’s compare the RNN-->RNN weights in the teacher model, the student model before training, and after training:

In each case, we represent the connection strength between the RNNs as a heatmap showing the weight between each pair of neurons. The teacher network generated the training dataset, so those weights are our ground-truth. The untrained student network has a largely dissimilar pattern of weights to the teacher network, while the trained student network has a very similar pattern to the teacher. Therefore, training the student network on the input-output examples from the teacher allowed us to infer the connectivity between the layers in our ground truth. As mentioned above, we can imagine a neuroscience experiment where we replace the artificial teacher network dataset with real recordings from 2 populations of neurons, while stimulating one of the populations. We could then train a student network on this dataset to infer connectivity between the real neurons.

Unity Multiplayer: Bottom to Top

RoboDoig — Thu, 14 Jan 2021 16:58:10 +0000

Link to completed project git repository

Introduction
Setting up the Unity project
Installing DarkRift
Notes on server architecture
Our first player connection
Our first server code
First client code
Player lobby
Player ready and starting the game
Movement synchronisation
Server hosting
Installing PlayFab
Verifying containerisation
PlayFab agent communication
First PlayFab deployment
PlayFab client integration
Connecting clients to servers
Final touches on the server
Putting it all together
Conclusion

Introduction

When building a multiplayer game, a number of important design decisions must be made along the way. What server architecture will you use? How will your servers be deployed and scaled as your player-base grows? Which tools do you need to write yourself and which can be handled by 3rd party software? If a completed multiplayer game is composed of a stack of several layers of these decisions, many tutorials focus on a single layer and it can be difficult to see how they fit together to produce a finished product. For example, it’s easy enough to find a detailed tutorial on writing a dedicated multiplayer server, but not necessarily how to also deploy and run that server in production.

This tutorial is an attempt to demonstrate the ‘full-stack’ of a multiplayer game, without getting bogged down in the numerous design decisions for each layer. Throughout this guide, we will build and deploy a simple online multiplayer Unity game. We’ll write a Unity game client; a standalone dedicated server to connect multiplayer clients; and integrate cloud hosting services to deploy and scale those servers. The end result will be a functioning multiplayer game, that creates and balances multiple server instances according to player load. Since we are covering a lot of ground here, I won’t be developing all of these components exhaustively, but it is my hope that seeing how they all fit together will allow you to go back and improve on them in your own titles.

Tools used

Throughout this guide I will be using:

Unity3D - Client, game visuals

DarkRift - Dedicated server

PlayFab - Server hosting, scaling, matchmaking

You can of course swap these out with other software / services, I am using them here as they are the tools for each job that I have found to be the most intuitive and well documented.

General outline

Build a unity scene for the client. Will contain a UI for logging in, joining a multiplayer session and starting the game when all connected players are ready. The scene will also contain a super-simple ‘game’ where players can move around a game area.
Develop client-side code to connect to an external server and send necessary data
Develop a standalone dedicated server to handle player connections and pass data between players
Integrate the client and server with the PlayFab SDK
Upload the server to PlayFab multiplayer services, setup matchmaking and server scaling rules

Setting up the Unity project

Here we will set up our Unity template. You can follow all the steps in this section, but I recommend instead downloading the main scene from the finished git repo at the start of this guide. Once you have that you can skip to the next section.

Start up a new Unity project. In this example we’ll name it multiplayer-tutorial. Open a new scene, call it something like GameScene.

First we’ll add a UI that will eventually handle basic login, matchmaking and joining sessions. Create a Canvas and add 3 UI panels to it as children. Call the first ‘background’, the second ‘StartPanel’ and the third ‘LobbyPanel’. Stretch the background panel to fill the canvas, colour it black, and set the alpha to 255 so it’s not transparent. Resize the StartPanel and LobbyPanel so that they form two columns in the main canvas, with StartPanel on the left. Set their alpha to something low so they are roughly transparent. You should now have a scene looking something like this:

In StartPanel, add an InputField, and two buttons. Name the buttons something like StartSessionButton and LocalTestButton respectively. In LobbyPanel add a child panel called ConnectedPlayersPanel and a button called ReadyButton. Resize and align them in the panels to your liking and adjust the button text to reflect their function. Finally, add a VerticalLayoutGroup component to the ConnectedPlayersPanel and tick the box “Control Child Size >> Width”. You should now have a fairly ugly but perfectly functional UI:

Add two empty objects to your scene hierarchy, one called NetworkManager and one called GameArea. In GameArea, add a floor cube and some small walls to serve as the main play area. Also set your MainCamera to look directly down at the play area:

Installing DarkRift

To build our multiplayer server we’ll use DarkRift networking. Having tried a few networking solutions for Unity I can safely say it’s my favourite by far. It’s well documented, intuitive and free to name a few reasons. If you’re totally new to DarkRift or even to networking in general, they have a great getting started tutorial which you should check out before proceeding; although the first half of this guide is based on it and we'll cover many of the same concepts.

Before we discuss exactly how our server will be structured, let’s get installing DarkRift out of the way. The install process is simple, download and import the DarkRift package from the Unity Asset Store (the Demo folders on import are optional for this project). You should now have a ‘DarkRift’ folder in your Assets. Inside that folder is a .zip file called something like ‘DarkRift Server’. Extract this file somewhere else on your computer, outside your main Unity project folder. Navigate to where you extracted the files and run ‘DarkRift.Server.Console.exe’. You should see a command line window popup telling you that the server is mounted and listening on a port. We have a server!

Notes on server architecture

There are many different ways to design server-client interactions when making a multiplayer game. You can find an excellent summary of these architectures in another DarkRift tutorial here

In this tutorial we will use the dedicated authoritative server design. The server will be a standalone application that can be hosted completely independently of the Unity clients. All networking between players will pass through the server first. E.g. if player 1 moves their character, this message will be sent to the server, which will then copy that message to all connected players. The server also acts as the primary authority for the game state - clients can request that the game state be changed (e.g. when they move, make an attack) but the server ultimately decides whether those actions are valid.

Our first player connection

Back in the Unity scene, select the NetworkManager object in the scene hierarchy. Click Add Component and select ‘Client’. This component should be available if DarkRift was installed correctly. In the Client settings, set Host as 127.0.0.1, and Port to 4296. Also make sure Auto Connect is checked:

Now run the server with DarkRift.Server.Console.exe again if you already closed it. Hit play on your Unity scene and look at the console output. You should see a message saying you connected. Have a look at the command line output of the server, you should also see a message that a new client has connected. We now have a client that can successfully connect to a server!

Our first server code

In DarkRift, we add functionality to our server by creating plugins that are loaded by the main DarkRift process. To create the first plugin, start up a C# code editor, here I am using Visual Studio 2019. Create a new project with the Class Library (.NET Framework) template. Call your project something like MultiplayerPlugin and create it in the main Unity folder for your project (the parent folder containing your Assets folder). In the project, we also need to add references to DarkRift.dll and DarkRift.Server.dll. You’ll find these in the Lib folder where you extracted the DarkRift server (where DarkRift.Server.Console.exe is located).

Create a new class called NetworkManager.cs and have it inherit from Plugin. Make sure you are using the DarkRift and DarkRift.Server namespaces. In order for our NetworkManager to be able to inherit from Plugin we need to add some code that defines a constructor, ThreadSafe check and Version:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using DarkRift;
using DarkRift.Server;

namespace MultiplayerPlugin
{
    class NetworkManager : Plugin
    {
        public override bool ThreadSafe => false;
        public override Version Version => new Version(1, 0, 0);

        public NetworkManager(PluginLoadData pluginLoadData) : base(pluginLoadData) {

        }
    }
}

Let’s test that our plugin can be loaded by the DarkRift server. In Visual Studio, click Build>>Build Solution to compile the project. Now navigate to the MultiplayerPlugin folder down to MultiplayerPlugin>>bin>>Debug. Inside you should see a file MultiplayerPlugin.dll. Copy this .dll to the Plugins folder of the DarkRift server and run DarkRift.Server.Console.exe. You should see a console message that the PluginManager installed our NetworkManager. This will be our basic server development workflow going forward. When we make a change to the plugin, we build the solution and copy the .dll for the plugin over to the Plugins folder of the DarkRift server. The changes should then be loaded the next time we run DarkRift.Server.Console.exe.

Let’s now add some basic code to handle player connections and disconnection from the server. First we’ll add a new class to our project to represent players, call it Player.cs. For now it will simply have two properties to represent network ID and name, and a constructor to set these:

namespace MultiplayerPlugin
{
    class Player
    {
        public ushort ID { get; set; }
        public string playerName { get; set; }

        public Player(ushort _ID, string _playerName) {
            ID = _ID;
            playerName = _playerName;
        }
    }
}

In NetworkManager we’ll add a Dictionary to keep track of players and add two methods to handle connections and disconnections. In the NetworkManager constructor, we’ll also make sure that the DarkRift ClientManager subscribes to these methods:

class NetworkManager : Plugin
{
    public override bool ThreadSafe => false;
    public override Version Version => new Version(1, 0, 0);
    Dictionary<IClient, Player> players = new Dictionary<IClient, Player>();

    public NetworkManager(PluginLoadData pluginLoadData) : base(pluginLoadData) {
        ClientManager.ClientConnected += ClientConnected;
        ClientManager.ClientDisconnected += ClientDisconnected;
    }

    void ClientConnected(object sender, ClientConnectedEventArgs e) {

    }

    void ClientDisconnected(object sender, ClientDisconnectedEventArgs e) {

    }
}

Now, when a client connects to the server, DarkRift will call our ClientConnected method with information about the sender (client) and any additional context arguments e. The same thing with client disconnections and the ClientDisconnected method.

At a minimum, when these methods are called we want to update our players Dictionary with the players that are currently connected. We also want to broadcast to the connected client the connection status of other clients, and tell the other connected clients about the new connection/disconnection. First, let’s go back to the Player class and define how its data will be sent and received by the server. Have the Player class use the IDarkRiftSerializable interface, this will allow us to define some handy methods for how DarkRift should deal with the class’ data:

using DarkRift;

namespace MultiplayerPlugin
{
    class Player : IDarkRiftSerializable
    {
        public ushort ID { get; set; }
        public string playerName { get; set; }

        public Player() {

        }

        public Player(ushort _ID, string _playerName) {
            ID = _ID;
            playerName = _playerName;
        }

        public void Deserialize(DeserializeEvent e) {
            ID = e.Reader.ReadUInt16();
            playerName = e.Reader.ReadString();
        }

        public void Serialize(SerializeEvent e)
        {
            e.Writer.Write(ID);
            e.Writer.Write(playerName);
        }
    }
}

Now let’s implement our ClientConnected and ClientDisconnected methods in NetworkManager:

void ClientConnected(object sender, ClientConnectedEventArgs e)
{
    // When client connects, generate new player data
    Player newPlayer = new Player(e.Client.ID, "default");
    players.Add(e.Client, newPlayer);

    // Write player data and tell other connected clients about this player
    using (DarkRiftWriter newPlayerWriter = DarkRiftWriter.Create())
    {
        newPlayerWriter.Write(newPlayer);

        using (Message newPlayerMessage = Message.Create(Tags.PlayerConnectTag, newPlayerWriter)) {
            foreach (IClient client in ClientManager.GetAllClients().Where(x => x != e.Client)) {
                client.SendMessage(newPlayerMessage, SendMode.Reliable);
            }
        }
    }

    // Tell the client player about all connected players
    foreach (Player player in players.Values) {
        Message playerMessage = Message.Create(Tags.PlayerConnectTag, player);
        e.Client.SendMessage(playerMessage, SendMode.Reliable);
    }
}

void ClientDisconnected(object sender, ClientDisconnectedEventArgs e)
{
    // Remove player from connected players
    players.Remove(e.Client);

    // Tell all clients about player disconnection
    using (DarkRiftWriter writer = DarkRiftWriter.Create()) {
        writer.Write(e.Client.ID);

        using (Message message = Message.Create(Tags.PlayerDisconnectTag, writer)) {
            foreach (IClient client in ClientManager.GetAllClients()) {
                client.SendMessage(message, SendMode.Reliable);
            }
        }
    }
}

You will see an editor warning at this point that ‘Tags does not exist in the current context’. Let’s create a Tags class in the project to store tag definitions for the different kinds of server messages we will send:

namespace MultiplayerPlugin
{
    class Tags
    {
        public static readonly ushort PlayerConnectTag = 1000;
        public static readonly ushort PlayerDisconnectTag = 1001;
        public static readonly ushort PlayerInformationTag = 1002;
        public static readonly ushort PlayerSetReadyTag = 1003;
        public static readonly ushort StartGameTag = 1004;
        public static readonly ushort PlayerMoveTag = 1005;
    }
}

So far we have only used the PlayerConnect and PlayerDisconnect tags, but the other tags we will use in the future are also included here.

Let’s step through our ClientConnected and ClientDisconnected code to understand what we’ve just written. Because we had our ClientManager subscribe to the ClientConnected method, every time a client joins the server this code will run. First we get the client ID from e.Client.ID and we create a new Player class with this ID. We initialize the player name with the string “default” as the client hasn’t yet told us what the player name should be. We add this new Player class to our Dictionary that stores the currently connected players, using IClient as the Dictionary key so we can always reference connected player data by the client ID. Next we create a DarkRiftWriter and take advantage of the IDarkRiftSerializable interface to write the player data to the writer. We create a new Message and store the player data in it, tagging the message with the PlayerConnectTag. We then loop through all connected clients (other than the one that just connected) and send them the message, informing them that this client has just connected. Finally, we loop through all currently connected players and send their information to the newly connected client. At the end of this method, every connected client knows about the existence of every other client. The process is similar for ClientDisconnected, we remove the disconnected client from the players Dictionary by referencing its IClient, and tell all remaining connected players which client has disconnected. Note that the messages in both methods use SendMode.Reliable in the SendMessage call. In DarkRift, this SendMode guarantees that the message will be sent and received so we want to use this for e.g. cases where we send important player information that will cause bugs if not sent properly. SendMode.Unreliable is used in cases where the message being dropped is not game-breaking, e.g. when we send movement update data continuously.

Rebuild the Plugin solution and copy the .dll over the DarkRift server Plugins folder. Run DarkRift.Server.Console.exe and make sure everything loads correctly and nothing crashes. At this point, nothing particularly interesting will happen. Even if we run the Unity client and connect, the client is not yet sending or receiving any information.

First client code

Back in our Unity project, let’s start setting up some player interface and DarkRift client code. First, create a Tags class and copy the same tags from the server plugin project. This will not be attached to any GameObject so remove the MonoBehaviour inheritance statement:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Tags
{
    public static readonly ushort PlayerConnectTag = 1000;
     public static readonly ushort PlayerDisconnectTag = 1001;
     public static readonly ushort PlayerInformationTag = 1002;
     public static readonly ushort PlayerSetReadyTag = 1003;
     public static readonly ushort StartGameTag = 1004;
     public static readonly ushort PlayerMoveTag = 1005;
}

Now our client and server both have identical tag definitions so that they can pass messages around correctly.

Create a class called NetworkEntity. This will be a corresponding store of player data on the client-side, similar to the Player class in our DarkRift plugin. For now we will just add a networkID and player name property, and some methods for setting these properties:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class NetworkEntity : MonoBehaviour
{
    public ushort networkID {get; private set;}
    public string playerName {get; private set;}

    public void SetNetworkID (ushort _networkID) {
        networkID = _networkID;
    }

    public void SetPlayerName(string _playerName) {
        playerName = _playerName;
    }
}

Eventually, this class will be attached to GameObjects that represent connected players in our game.

Now create a class called UIManager and attach it to the Canvas. This class will take care of how the player interacts with the UI, and map functions to the buttons in the UI. We’ll implement a few key methods as shown below:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;

public class UIManager : MonoBehaviour
{
    public static UIManager singleton;

    // UI Elements
    public InputField nameInputField;
    public Button startSessionButton;
    public Button localTestButton;
    public VerticalLayoutGroup connectedPlayersGroup;
    public GameObject connectedPlayerIndicatorPrefab;
    public Button readyButton;

    void Awake() {
        if (singleton != null) {
            Destroy(gameObject);
            return;
        }

        singleton = this;
    }

    void Start() {
    }

    public void PopulateConnectedPlayers(Dictionary<ushort, NetworkEntity> connectedPlayers) {
        ClearConnectedPlayers();
        foreach (KeyValuePair<ushort, NetworkEntity> connectedPlayer in connectedPlayers) {
            GameObject obj = Instantiate(connectedPlayerIndicatorPrefab);
            obj.transform.SetParent(connectedPlayersGroup.transform);
            obj.GetComponentInChildren<Text>().text = connectedPlayer.Value.playerName;
        }
    }

    public void ClearConnectedPlayers() {
        foreach (Transform child in connectedPlayersGroup.transform) {
            Destroy(child.gameObject);
        }
    }

    public void SetInputInteractable(bool interactable) {
        startSessionButton.interactable = interactable;
        localTestButton.interactable = interactable;
        nameInputField.interactable = interactable;
    }

    public void SetLobbyInteractable(bool interactable) {
        readyButton.interactable = interactable;
    }

    public void DisplayNetworkMessage(string message) {
        startSessionButton.GetComponentInChildren<Text>().text = message;
    }

    public void CloseUI() {
        this.gameObject.SetActive(false);
    }

    public void OpenUI() {
        this.gameObject.SetActive(true);
    }
}

Let’s walk through this code and describe its function. First we define a public static UIManager called singleton. For this project, we only ever need one UIManager instance, and we want other classes to be able to access it easily. Therefore, we use the singleton pattern and define a static reference to the UIManager. In the Awake method, we ensure that only one instance of UIManager can ever exist in our game. We also define public references to the interactable components of our UI, the name input field and all the function buttons. Make sure to go back into Unity scene view and assign all these components. For the Connected Player Indicator Prefab, make a Button prefab called ConnectedPlayerButton and drag it into the Connected Player Indicator Prefab slot. This prefab just needs to be the standard Unity UI button:

PopulateConnectedPlayers will take a Dictionary of NetworkEntity definitions and their associated IDs and populate the ConnectedPlayersPanel with a list of connected players, using ConnectedPlayerButton as the visual indicator. ClearConnectedPlayers clears all these indicators from the panel. SetInputInteractable and SetLobbyInteractable toggles whether the left and right panels of the UI are interactable. DisplayNetworkMessage takes a message string and displays it as text on the Start Session button. We also have two functions for closing and opening the entire UI, by calling the GameObject SetActive method. So far so good, but we haven’t actually interacted with our server yet!

Create two new classes in the Unity project called NetworkInterface and NetworkManager respectively. Attach them both to the NetworkManager GameObject we created earlier in the scene hierarchy. Also uncheck the Auto Connect checkbox in the Client component on the NetworkManager object since we don’t want the client to attempt to connect as soon as the game starts, we instead want to tie connection to some player input. NetworkInterface will also use the singleton design pattern so implement that in the same way as UIManager. We will also need a reference to the DarkRift UnityClient that will be cached in the Start method. Finally, we’ll implement a StartLocalSession method and a callback method OnLocalSessionCallback:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using DarkRift;
using DarkRift.Client;
using DarkRift.Client.Unity;

public class NetworkInterface : MonoBehaviour
{
    public static NetworkInterface singleton;
    private UnityClient drClient;

    void Awake() {
        if (singleton != null) {
            Destroy(gameObject);
            return;
        }

        singleton = this;
    }

    void Start() {
        drClient = GetComponent<UnityClient>();
    }

    // Connect with local test server //
    public void StartLocalSession() {
        // Connect to local network
        drClient.ConnectInBackground(drClient.Host, drClient.Port, drClient.Port, true, delegate {OnLocalSessionCallback();} );

        // Update UI
        UIManager.singleton.SetInputInteractable(false);
    }

    public void OnLocalSessionCallback() {

        if (drClient.ConnectionState == ConnectionState.Connected) {
            // Set lobby controls to interactable
            UIManager.singleton.SetLobbyInteractable(true);
        } else {
            // Else reset the input UI
            UIManager.singleton.SetInputInteractable(true);
        }
    }
}

When StartLocalSession is called, the DarkRift client will try and connect to the IP address and listening port defined in the Client component on the network manager. We still want to attempt a localhost connection so these values should still be 127.0.0.1 and 4296. We also define a callback method to be called when we have a server connection result - OnLocalSessionCallback. At present, after we call StartLocalSession the left UI panel is disabled while we wait for a connection result. If the connection is successful we enable the right UI panel, if it fails we re-enable the left UI panel. Finally, we need to link a button to the StartLocalSession function. Go back to UIManager and add this code to the Start method:

void Start() {       
    localTestButton.onClick.AddListener(NetworkInterface.singleton.StartLocalSession);
    SetLobbyInteractable(false);
}

This code tells the Local Test button to fire off the StartLocalSession method in NetworkInterface when it's clicked. We also make sure the lobby panel on the right of the UI is disabled at startup.

Let’s test all this out. Start up DarkRift.Server.Console.exe again. Hit run on the Unity project and click the Local Server Test button. You should see a console message reporting a successful connection, and a corresponding message in the DarkRift server console.

The NetworkManager class will be responsible for receiving and routing messages from the server, and structuring message from the client to server. To handle messages from the server, we need to add our own function to be called when the DarkRift Client’s MessageReceived Event is fired. The NetworkManager will also follow the singleton pattern, so our initial implementation will be thus:


using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using DarkRift;
using DarkRift.Client;
using DarkRift.Client.Unity;

public class NetworkManager : MonoBehaviour
{
    public static NetworkManager singleton;
    private UnityClient drClient;

    public Dictionary<ushort, NetworkEntity> networkPlayers = new Dictionary<ushort, NetworkEntity>();

    // Player prefabs
    public GameObject localPlayerPrefab;
    public GameObject networkPlayerPrefab;

    void Awake() {
        if (singleton != null) {
            Destroy(gameObject);
            return;
        }

        singleton = this;

        drClient = GetComponent<UnityClient>();
        drClient.MessageReceived += MessageReceived;
    }

    void MessageReceived(object sender, MessageReceivedEventArgs e) {
    }
}

Note we also include a NetworkEntity dictionary to keep track of connected players using their network ID as a key - as well as two public GameObjects that will store the prefabs used to spawn in players. Make those prefabs now, they simply need to be a standard Unity cube with the NetworkEntity component attached. Name them LocalPlayerPrefab and NetworkPlayerPrefab and drag them into their respective slots in the NetworkManager. Give them both a rigidbody component as well so they don’t sail through the walls later.

When we (the client) receive a message from the server, we want to check the message tag and then implement some appropriate game logic. Let’s start with what happens when we get a message with the PlayerConnectTag or PlayerDisconnectTag:

void MessageReceived(object sender, MessageReceivedEventArgs e) {
    using (Message message = e.GetMessage() as Message) {
        if (message.Tag == Tags.PlayerConnectTag) {
            PlayerConnect(sender, e);
        } else if (message.Tag == Tags.PlayerDisconnectTag) {
            PlayerDisconnect(sender, e);
        }
    }
}

void PlayerConnect(object sender, MessageReceivedEventArgs e) {
    using (Message message = e.GetMessage()) {
        using (DarkRiftReader reader = message.GetReader()) {
            // Read the message data
            ushort ID = reader.ReadUInt16();
            string playerName = reader.ReadString();

            // Player / Network Player Spawn
            GameObject obj;
            if (ID == drClient.ID) {
                // If this ID corresponds to this client, spawn the controllable player prefab
                obj = Instantiate(localPlayerPrefab, new Vector3(0f, 1f, 0f), Quaternion.identity) as GameObject;
            } else {
                // Else we spawn a network prefab, non-controllable
                obj = Instantiate(networkPlayerPrefab, new Vector3(0f, 1f, 0f), Quaternion.identity) as GameObject;
            }

            // Get network entity data of prefab and add to network players store
            networkPlayers.Add(ID, obj.GetComponent<NetworkEntity>());

            // Update player name
            networkPlayers[ID].SetPlayerName(playerName);
        }
    }
}

void PlayerDisconnect(object sender, MessageReceivedEventArgs e) {
    using (Message message = e.GetMessage()) {
        using (DarkRiftReader reader = message.GetReader()) {
            ushort ID = reader.ReadUInt16();
            Destroy(networkPlayers[ID].gameObject);
            networkPlayers.Remove(ID);
        }
    }
}

Let’s walk this through. In the Awake method we ensured that when the DarkRift client receives a message from the server it will fire the MessageReceived method in our NetworkManager class. Recall that back in our server plugin code we implemented a ClientConnected method that returned player data to all connected clients when a new client joins the server. When we receive a message from the server in the client NetworkManager, we check what tag the message was sent with, PlayerConnectTag or PlayerDisconnectTag. Depending on the tag we call an appropriate method. If we got a player connect message, we extract the message from the MessageReceivedEventArgs, and initialize a DarkRiftReader to read out the data. Recall that the Serialize method of our Player class in the server plugin writes two variables, a ushort ID and a string playerName. Therefore when our client NetworkManager receives this message we use the reader to read out - in order - the ID and playerName. Our client now needs to spawn an object for each connected player in the server. If the received ID matches our own client ID, we want to spawn the local player which will eventually be controllable by the client. Otherwise we spawn a network player, which will be controlled by the server. We add a reference to this instantiated object (which should have a NetworkEntity component) to our network players dictionary, and set the player name from the message we received from the server. In PlayerDisconnect we destroy the Player object and remove it from the networkPlayers Dictionary.

Let’s try this out. Start your server back up with DarkRift.Server.Console.exe. Create a build of your Unity project and run it, then hit play in the editor so you have two clients running on your PC. Click the Local Server Test button in your in both clients. You should see both a LocalPlayer and a NetworkPlayer appear in your scene hierarchy.

This may not seem like much but we have covered a good portion of the logic for a multiplayer game with a dedicated server. The server plugin waits for client connections, and maintains a record of all connected players. The Unity client connects to the server, and listens for messages coming back from the server. When the client connects, the server sends data back in messages about all the other clients on the server. This structure will form the basis for the rest of our development. The server holds the ground-truth data about the game world, and clients can request that data to update their local Unity data. Clients can also send messages to
the server to update the game-state, which is then broadcast to the other connected clients.

Player lobby

When we connect to the server, we want to see a list of all other players who are in the session in the lobby UI panel on the right. We also want players to be able to set a display name when they join.

We’ll start by updating the UI to show a list of connected players. In the MessageReceived method of the client NetworkManager, add code to populate the UI with all connected players, using the method we wrote in the UIManager previously:

void MessageReceived(object sender, MessageReceivedEventArgs e) {
    using (Message message = e.GetMessage() as Message) {
        if (message.Tag == Tags.PlayerConnectTag) {
            PlayerConnect(sender, e);
        } else if (message.Tag == Tags.PlayerDisconnectTag) {
            PlayerDisconnect(sender, e);
        }
    }

    // Update the UI with connected players
    UIManager.singleton.PopulateConnectedPlayers(networkPlayers);
}

If we test our clients again with this addition, you’ll see that the ConnectedPlayersPanel starts to fill up with player indicators as more clients join. But even if we input a name in the InputField before joining, the player indicator always shows ‘default’. Remember that in our server code, when we initialize new Player data, we do so with a default string. Let’s add a way for our client to tell the server what we want our display player name to be.

At the bottom of the server NetworkManager class, add a PlayerInformationMessage class that implements IDarkRiftSerializable and a SendPlayerInformationMessage method that takes a string playerName as an argument:

private class PlayerInformationMessage : IDarkRiftSerializable {
    public ushort id {get; set;}
    public string playerName {get; set;}

    public PlayerInformationMessage() {

    }

    public PlayerInformationMessage(string _playerName) {
        playerName = _playerName;
    }

    public void Deserialize(DeserializeEvent e) {
        id  = e.Reader.ReadUInt16();
        playerName = e.Reader.ReadString();
    }

    public void Serialize(SerializeEvent e) {
        e.Writer.Write(playerName);
    }
}

public void SendPlayerInformationMessage(string playerName) {
    using (DarkRiftWriter writer = DarkRiftWriter.Create()) {
        writer.Write(new PlayerInformationMessage(playerName));
        using (Message message = Message.Create(Tags.PlayerInformationTag, writer)) {
            drClient.SendMessage(message, SendMode.Reliable);
        }
    }
}

This class/method pair is a useful code pattern for passing messages between the client and server. PlayerInformationMessage defines some message data (an ID and player name) as well as how a message of this type should be read (Deserialize) and written (Serialize). The method SendPlayerInformationMessage creates a PlayerInformationMessage and sends it to the server. A few important notes here. You’ll notice that the data and read/write methods for PlayerInformationMessage are very similar to the Player class in our server plugin. Why not just create a single class and send method that is shared between the server and client code? The fact is, we probably could do that, and it might be more efficient in a large, complex codebase. You could imagine building a separate class library called NetworkMessages that defines all the messages and send methods needed, that can be implemented by both the server and the client. However, since we are building a fairly minimal implementation, we’ll try and keep a strict separation between our client and server code for modularity and ease of understanding. Also notice that our PlayerInformationMessage Serialize method does not write the ID field of the class. This is because we are sending this message from our DarkRift client to the server, and the server will receive the client ID with the message. Therefore, sending the ID in the message as well would be redundant.

To send this PlayerInformationMessage, we need to add some code to the NetworkInterface class, in the OnLocalSessionCallback method:

public void OnLocalSessionCallback() {
    if (drClient.ConnectionState == ConnectionState.Connected) {
        // If connection successful, send any additional player info
        NetworkManager.singleton.SendPlayerInformationMessage(
            UIManager.singleton.nameInputField.text
        )

        // Set lobby controls to interactable
        UIManager.singleton.SetLobbyInteractable(true);
    } else {
        // Else reset the input UI
        UIManager.singleton.SetInputInteractable(true);
    }
}

We simply add a line of code that calls the NetworkManager SendPlayerInformationMessage with the current text of the input field if the local network connection is successful. Now we need to define how the server will deal with this message once it is received. In the server plugin NetworkManager, add a line of code in the ClientConnected method:

// Set client message callbacks
e.Client.MessageReceived += OnPlayerInformationMessage;

This line ensures that a connected client will call a method - OnPlayerInformationMessage, whenever a server message is received. Let’s define that method by adding this to the NetworkManager as well:

void OnPlayerInformationMessage(object sender, MessageReceivedEventArgs e)
{
    using (Message message = e.GetMessage() as Message) {
        if (message.Tag == Tags.PlayerInformationTag) {
            using (DarkRiftReader reader = message.GetReader()) {
                string playerName = reader.ReadString();

                // Update player information
                players[e.Client].playerName = playerName;

                // Update all players
                using (DarkRiftWriter writer = DarkRiftWriter.Create()) {
                    writer.Write(e.Client.ID);
                    writer.Write(playerName);

                    message.Serialize(writer);
                }

                foreach (IClient client in ClientManager.GetAllClients()) {
                    client.SendMessage(message, e.SendMode);
                }
            }
        }
    }
}

When this method is called, it extracts the message from the MessageReceivedEventArgs and initializes a reader to read out the message data. Recall that our PlayerInformationMessage only wrote one variable in its Serialize method - the playerName - which is the only thing we read from the reader. We use the MessageReceivedEventArgs to get the ID of the client from where this message originated. We update the received player name in the players Dictionary accordingly. Finally, we create a DarkRiftWriter and write the client ID and updated name into a message, then broadcast this message to all connected clients.

Now let’s return to our client NetworkManager and define how we will deal with this information. First we’ll add a new condition to our MessageReceived method to account for receiving a message with the PlayerInformationTag. Our method should now look like this:

void MessageReceived(object sender, MessageReceivedEventArgs e) {
    using (Message message = e.GetMessage() as Message) {
        if (message.Tag == Tags.PlayerConnectTag) {
            PlayerConnect(sender, e);
        } else if (message.Tag == Tags.PlayerDisconnectTag) {
            PlayerDisconnect(sender, e);
        } else if (message.Tag == Tags.PlayerInformationTag) {
            PlayerInformation(sender, e);
        }
    }

    // Update the UI with connected players
    UIManager.singleton.PopulateConnectedPlayers(networkPlayers);
}

Then let’s create the method for dealing with new player information from the server in the client NetworkManager class:

void PlayerInformation(object sender, MessageReceivedEventArgs e) {
    using (Message message = e.GetMessage()) {
        using (DarkRiftReader reader = message.GetReader()) {
            PlayerInformationMessage playerInformationMessage = reader.ReadSerializable<PlayerInformationMessage>();

            networkPlayers[playerInformationMessage.id].SetPlayerName(
                playerInformationMessage.playerName
            );
        }
    }
}

Hopefully by now this is fairly self explanatory. We retrieve the message and read its data with DarkRiftReader. We extract the data using our previously created PlayerInformationClass based on its Deserialize method. We then update the appropriate network player in our networkPlayers Dictionary based on the received ID and player name.

You can test this out now by starting up some clients, typing a name in the input field and clicking the Local Server Test button. As clients join, their display names should also show up in the Connected Players Panel. Remember that since we have also changed some server code, you will need to build your solution again and copy the MultiplayerPlugin.dll into the DarkRift Plugins folder.

You might have noticed some inconsistencies at this point in the way we read out message data in the client and server code. On the client side we used:

PlayerInformationMessage playerInformationMessage = reader.ReadSerializable<PlayerInformationMessage>();

Whereas on the server we used:

string playerName = reader.ReadString();

Remember though that in our client code, we defined how player information messages should be read in our PlayerInformationMessage class which implements IDarkRiftSerializable. The fact that we implemented this interface is why we can shorten our message reading code slightly to reader.ReadSerializable<PlayerInformationMessage>, but under the hood we are doing the exact same thing, recall the PlayerInformationMessage Deserialize method:

public void Deserialize(DeserializeEvent e) {
    id  = e.Reader.ReadUInt16();
    playerName = e.Reader.ReadString();
}

So ultimately these two approaches are interchangeable for our small project. If you are reading a lot of data from a Message, necessitating many lines of code doing something like reader.ReadString(), reader.ReadBoolean(), reader.ReadSingle() over and over, you may improve readability by defining all these reader operations in a Message class implementing IDarkRiftSerializable - as in our PlayerInformationMessage example.

Player ready and starting game

Our next step is to allow clients to indicate to the server when they are ready to start playing the game. We also want the server to coordinate starting the game for all clients when all are ready.

First we’ll add a new message/method pair to our client NetworkManager:

// Message for telling the server a player is ready
private class PlayerReadyMessage : IDarkRiftSerializable {
    public ushort id {get; set;}
    public bool isReady {get; set;}

    public PlayerReadyMessage() {
    }

    public PlayerReadyMessage(bool _isReady) {
        isReady = _isReady;
    }

    public void Deserialize(DeserializeEvent e) {
        id = e.Reader.ReadUInt16();
        isReady = e.Reader.ReadBoolean();
    }

    public void Serialize(SerializeEvent e) {
        e.Writer.Write(isReady);
    }
}

public void SendPlayerReadyMessage(bool isReady) {
    using (DarkRiftWriter writer = DarkRiftWriter.Create()) {
        writer.Write(new PlayerReadyMessage(isReady));
        using (Message message = Message.Create(Tags.PlayerSetReadyTag, writer)) {
            drClient.SendMessage(message, SendMode.Reliable);
        }
    }
}

This is very similar to our PlayerInformationMessage, but this time we send an indicator boolean rather than a string.

Next, we’ll add a method in our NetworkInterface class that we can link to a UI button to send a ready message to the server:

public void SetPlayerReady() {
    // Tell the server this player is ready to start game
    NetworkManager.singleton.SendPlayerReadyMessage(true);

    // Update UI
    UIManager.singleton.SetLobbyInteractable(false);
}

Let’s set that link up in the UIManager as well, the Start method should now look like this:

void Start() {
    localTestButton.onClick.AddListener(NetworkInterface.singleton.StartLocalSession);
    readyButton.onClick.AddListener(NetworkInterface.singleton.SetPlayerReady);
    SetLobbyInteractable(false);
}

That’s it for the client side code. Now let’s go to the server plugin and add another property to the Player class to track which players have clicked ready. Our Player class should now look like this:

class Player : IDarkRiftSerializable
{
    public ushort ID { get; set; }
    public string playerName { get; set; }
    public bool isReady { get; set; }

    public Player() {
    }

    public Player(ushort _ID, string _playerName) {
        ID = _ID;
        playerName = _playerName;
        isReady = false;
    }

    public void Deserialize(DeserializeEvent e) {
        ID = e.Reader.ReadUInt16();
        playerName = e.Reader.ReadString();
    }

    public void Serialize(SerializeEvent e) {
        e.Writer.Write(ID);
        e.Writer.Write(playerName);
    }
}

Only the server needs to know the ready status of all players, so we don’t need to add logic right now to Serialize and Deserialize this property, since it won’t be sent as a message from the server.

Let’s also add some code to receive player ready messages on the server plugin NetworkManager. We need to add two methods, OnPlayerReadyMessage and CheckAllReady:

void OnPlayerReadyMessage(object sender, MessageReceivedEventArgs e)
{
    using (Message message = e.GetMessage() as Message) {
        if (message.Tag == Tags.PlayerSetReadyTag) {
            using (DarkRiftReader reader = message.GetReader()) {
                bool isReady = reader.ReadBoolean();

                // Update player ready status and check if all players are ready
                players[e.Client].isReady = isReady;
                CheckAllReady();
            }
        }
    }
}

void CheckAllReady()
{
    // Check all clients, if any not ready, then return
    foreach (IClient client in ClientManager.GetAllClients()) {
        if (!players[client].isReady) {
            return;
        }
    }

    // If all are ready, broadcast start game to all clients
    using (DarkRiftWriter writer = DarkRiftWriter.Create()) {
        using (Message message = Message.Create(Tags.StartGameTag, writer)) {
            foreach (IClient client in ClientManager.GetAllClients()) {
                client.SendMessage(message, SendMode.Reliable);
            }
        }
    }
}

Also make sure that clients MessageReceived will call OnPlayerReadyMessage. Your MessageReceived subscriptions should now look like this in the ClientConnected method:

// Set client message callbacks
e.Client.MessageReceived += OnPlayerInformationMessage;
e.Client.MessageReceived += OnPlayerReadyMessage;

OnPlayerReadyMessage uses the hopefully now familiar pattern to read an incoming message and read the data, in this case a boolean indicating whether the client is ready. We set the players ready status according to this boolean and then call CheckAllReady.

In CheckAllReady, we loop through all connected players to see if they all have isReady set to true. If so, we broadcast an empty message to all clients to indicate that the game should start. We don’t need to add any data to this message as we will simply use the StartGameTag without any additional information to signal that the game should start.

Our final step is to have the Unity client start the game when it receives the start game message. Back in the client NetworkManager we’ll add another case to the MessageReceived method to deal with a start game message:

void MessageReceived(object sender, MessageReceivedEventArgs e) {
    using (Message message = e.GetMessage() as Message) {
        if (message.Tag == Tags.PlayerConnectTag) {
            PlayerConnect(sender, e);
        } else if (message.Tag == Tags.PlayerDisconnectTag) {
            PlayerDisconnect(sender, e);
        } else if (message.Tag == Tags.PlayerInformationTag) {
            PlayerInformation(sender, e);
        } else if (message.Tag == Tags.StartGameTag) {
            StartGame(sender, e);
        }
    }

    // Update the UI with connected players
    UIManager.singleton.PopulateConnectedPlayers(networkPlayers);
}

We’ll also add a StartGame method that for now will just close the UI to reveal the game area:

void StartGame(object sender, MessageReceivedEventArgs e) {
    UIManager.singleton.CloseUI();
}

We also want our player GameObject to be controllable when the game starts. Create a new class in Unity called Player, and attach it to the LocalPlayerPrefab we made earlier:

using System.Collections;
using System.Collections.Generic;
using UnityEngine;

public class Player : MonoBehaviour
{
    public bool controllable = false;
    public float moveSpeed = 8f;
    float hMove;
    float vMove;
    Vector3 motion;

    // Update is called once per frame
    void Update()
    {
        if (controllable) {
            hMove = Input.GetAxis("Horizontal");
            vMove = Input.GetAxis("Vertical");

            motion = new Vector3(hMove, 0f, vMove).normalized * Time.deltaTime * moveSpeed;

            transform.position += motion;
        }
    }
}

The Player class here simply allows us to control some basic movement of a GameObject, only when the controllable property is set to true. Back in the client NetworkManager, let’s make sure that StartGame sets the player prefab to be controllable:

void StartGame(object sender, MessageReceivedEventArgs e) {
    UIManager.singleton.CloseUI();

    // Set the local player to be controllable
    foreach (KeyValuePair<ushort, NetworkEntity> networkPlayer in networkPlayers) {
        Player player = networkPlayer.Value.GetComponent<Player>();
        if (player != null) {
            player.controllable = true;
        }
    }
}

All that happens here after closing the UI is that we loop through all NetworkEntity instances and try to get a Player component from each. Since only the local player (the one corresponding to the player client) will have the Player component, once we find it we set it to be controllable so that we as the player are controlling one entity.

Save and build everything and restart your server (again copying over the updated plugin .dll file). Open up a few game instances and try typing in a name, then clicking Local Server Test. Players should start to populate the lobby. Once everyone has hit ready the game should start! Every client should see a game screen and be able to control a little cube character. Note that since we haven’t set a minimum player limit, if there is only one connected player they can start the game by hitting ready without others joining first.

You’ll notice there is something missing though. Each client can control a character but movement is not synchronised between clients.

Movement synchronisation

Our final step for the basic game implementation is to make sure that when a client moves their character, the updated movement is sent to all connected clients so that they can track the position of all players. We’re also going to give player characters different colours so they’re distinguishable.

First we need to change the Player class in our server code. We need to store data about position and colour, as well as assign a starting position and colour on creation. Here is the finished Player class in our server code:

using System;
using DarkRift;

namespace MultiplayerPlugin
{
    class Player : IDarkRiftSerializable
    {
        public ushort ID { get; set; }
        public string playerName { get; set; }
        public bool isReady { get; set; }

        public float X { get; set; }
        public float Y { get; set; }
        public float Z { get; set; }

        public byte ColorR { get; set; }
        public byte ColorG { get; set; }
        public byte ColorB { get; set; }

        public Player() {
        }

        public Player(ushort _ID, string _playerName) {
            ID = _ID;
            playerName = _playerName;
            isReady = false;

            Random r = new Random();

            X = (float)r.NextDouble() * 5f;
            Y = (float)r.NextDouble() * 5f;
            Z = (float)r.NextDouble() * 5f;

            ColorR = (byte)r.Next(0, 200);
            ColorG = (byte)r.Next(0, 200);
            ColorB = (byte)r.Next(0, 200);
        }

        public void Deserialize(DeserializeEvent e) {
            ID = e.Reader.ReadUInt16();
            playerName = e.Reader.ReadString();

            X = e.Reader.ReadSingle();
            Y = e.Reader.ReadSingle();
            Z = e.Reader.ReadSingle();

            ColorR = e.Reader.ReadByte();
            ColorG = e.Reader.ReadByte();
            ColorB = e.Reader.ReadByte();
        }

        public void Serialize(SerializeEvent e) {
            e.Writer.Write(ID);
            e.Writer.Write(playerName);

            e.Writer.Write(X);
            e.Writer.Write(Y);
            e.Writer.Write(Z);

            e.Writer.Write(ColorR);
            e.Writer.Write(ColorG);
            e.Writer.Write(ColorB);
        }
    }
}

Back in our client NetworkManager, we’ll add a new class/method pair to take care of movement messages:

// Message for updating movement
private class PlayerMoveMessage : IDarkRiftSerializable {
    public ushort ID {get; set;}
    public Vector3 position {get; set;}

    public PlayerMoveMessage() {
    }

    public PlayerMoveMessage(Vector3 _postion) {
        position = _postion;
    }

    public void Deserialize(DeserializeEvent e) {
        ID = e.Reader.ReadUInt16();
        position = new Vector3(e.Reader.ReadSingle(), e.Reader.ReadSingle(), e.Reader.ReadSingle());
    }

    public void Serialize(SerializeEvent e) {
        e.Writer.Write(position.x);
        e.Writer.Write(position.y);
        e.Writer.Write(position.z);
    }
}

public void SendPlayerMoveMessage(Vector3 position) {
    using (DarkRiftWriter writer = DarkRiftWriter.Create()) {
        writer.Write(new PlayerMoveMessage(position));
        using (Message message = Message.Create(Tags.PlayerMoveTag, writer)) {
            drClient.SendMessage(message, SendMode.Unreliable);
        }
    }
}

This should seem fairly straightforward now. All we need to know for a movement message is which client moved (ID) and what its new position is. One difference to notice here is that this is the first message that we send with SendMode.Unreliable. All our other messages have contained vital player data that may only be sent once per session. If one of these messages failed we’d end up with a buggy player experience. A movement message, however, is constantly updating so if we miss a few messages it won’t affect our experience much. We also avoid the situation of having to wait for a whole string of reliable network messages to be read which could slow down our movement updates.

Now that we have this message structure implemented, we just need to add a single line to the client Player class to send movement updates to the server. The finished class should look like this:

public class Player : MonoBehaviour
{
    public bool controllable = false;
    public float moveSpeed = 8f;
    float hMove;
    float vMove;
    Vector3 motion;

    // Update is called once per frame
    void Update()
    {
        if (controllable) {
            hMove = Input.GetAxis("Horizontal");
            vMove = Input.GetAxis("Vertical");

            motion = new Vector3(hMove, 0f, vMove).normalized * Time.deltaTime * moveSpeed;

            transform.position += motion;

            NetworkManager.singleton.SendPlayerMoveMessage(transform.position);
        }
    }
}

We still need the server to broadcast movement updates to the other connected clients in a game session. Back in the server plugin NetworkManager we’ll create another message handling method to do exactly this:

void OnPlayerMoveMessage(object sender, MessageReceivedEventArgs e)
{
    using (Message message = e.GetMessage() as Message) {
        if (message.Tag == Tags.PlayerMoveTag) {
            using (DarkRiftReader reader = message.GetReader()) {
                float newX = reader.ReadSingle();
                float newY = reader.ReadSingle();
                float newZ = reader.ReadSingle();

                Player player = players[e.Client];

                player.X = newX;
                player.Y = newY;
                player.Z = newZ;

                // send this player's updated position back to all clients except the client that sent the message
                using (DarkRiftWriter writer = DarkRiftWriter.Create()) {
                    writer.Write(player.ID);
                    writer.Write(player.X);
                    writer.Write(player.Y);
                    writer.Write(player.Z);

                    message.Serialize(writer);
                }

                foreach (IClient client in ClientManager.GetAllClients().Where(x => x != e.Client))
                    client.SendMessage(message, e.SendMode);
            }
        }
    }
}

Here we read the incoming movement message from the client and update its position data in the server, then send the updated position out to all other clients. Make sure that you add OnPlayerMoveMessage to the MessageReceived event in the ClientConnected method. You should have this:

// Set client message callbacks
e.Client.MessageReceived += OnPlayerInformationMessage;
e.Client.MessageReceived += OnPlayerReadyMessage;
e.Client.MessageReceived += OnPlayerMoveMessage;

Finally we need our clients to update character object position based on this updated information. Back in our client we’ll add one last case to the MessageReceived method. This finished method should look like this:

void MessageReceived(object sender, MessageReceivedEventArgs e) {
        using (Message message = e.GetMessage() as Message) {
            if (message.Tag == Tags.PlayerConnectTag) {
                PlayerConnect(sender, e);
            } else if (message.Tag == Tags.PlayerDisconnectTag) {
                PlayerDisconnect(sender, e);
            } else if (message.Tag == Tags.PlayerInformationTag) {
                PlayerInformation(sender, e);
            } else if (message.Tag == Tags.StartGameTag) {
                StartGame(sender, e);
            } else if (message.Tag == Tags.PlayerMoveTag) {
                PlayerMove(sender, e);
            }
        }

        // Update the UI with connected players
        UIManager.singleton.PopulateConnectedPlayers(networkPlayers);
}

We’ll also implement the PlayerMove method to deal with movement updates from the server:

void PlayerMove(object sender, MessageReceivedEventArgs e) {
    using (Message message = e.GetMessage()) {
        using (DarkRiftReader reader = message.GetReader()) {
            PlayerMoveMessage playerMoveMessage = reader.ReadSerializable<PlayerMoveMessage>();

            networkPlayers[playerMoveMessage.ID].transform.position = playerMoveMessage.position;
        }
    }
}

Here, we read the PlayerMoveMessage data, find the corresponding NetworkEntity based on the received network ID, and update its position with transform.position.

In the client NetworkManager PlayerConnect method, let’s also add some code to set the players’ initial position and colour based on the server message. The final PlayerConnect method should look like this:

void PlayerConnect(object sender, MessageReceivedEventArgs e) {
    using (Message message = e.GetMessage()) {
        using (DarkRiftReader reader = message.GetReader()) {
            // Read the message data
            ushort ID = reader.ReadUInt16();
            string playerName = reader.ReadString();

            Vector3 position = new Vector3(reader.ReadSingle(), reader.ReadSingle(), reader.ReadSingle());

            Color32 color = new Color32(reader.ReadByte(), reader.ReadByte(), reader.ReadByte(), 255);

            // Player / Network Player Spawn
            GameObject obj;
            if (ID == drClient.ID) {
                // If this ID corresponds to this client, spawn the controllable player prefab
                obj = Instantiate(localPlayerPrefab, position, Quaternion.identity) as GameObject;
            } else {
                // Else we spawn a network prefab, non-controllable
                obj = Instantiate(networkPlayerPrefab, position, Quaternion.identity) as GameObject;
            }

            // Set the color
            Renderer renderer = obj.GetComponent<MeshRenderer>();
            renderer.material.color = color;

            // Get network entity data of prefab and add to network players store
            networkPlayers.Add(ID, obj.GetComponent<NetworkEntity>());

            // Update player name
            networkPlayers[ID].SetPlayerName(playerName);
        }
    }
}

Build and test everything once more. Open a few Unity game instances and connect to the local server with a few different names. Click ready on all windows and all the clients should be launched into the game. You should be able to control a distinct character on each window, and the movement should be synchronised across all clients. Each character should also have a distinct colour.

So now we have a basic multiplayer game with a dedicated authoritative server. Is it a pretty game? No. Is it fun? Also no. But it does have the basic components of a multiplayer game that you might want to build on.

Let’s pretend it is a good game that we actually want to release. What do we do with it now? First of all, this game only runs as a multiplayer game on a single PC connecting to a local server. We want players anywhere in the world to be able to connect and play together. So we could host our server somewhere and distribute the game out to people. But how do we assign connection information to players now that our server is hosted somewhere? Furthermore, what if our game becomes incredibly popular? How many servers do we need to host? How do we match players up and assign different connection addresses for different hosted servers to all of them.

In the next few sections we will explore some of those questions. By the end, our simple game server will be hosted online and we’ll have a system for matching players together and in game sessions. We’ll also implement a way for our pool of available servers to change dynamically depending on player load.

Server hosting

While one option is to write your own network code providing relay servers or dynamic server allocation, there are a number of established services that take care of some of this hard work for you. In this tutorial we will use PlayFab from Microsoft Azure, but there are several other options, including but not limited to Amazon GameLift, GameSparks, Epic Online Services and SteamWorks.

In the author’s opinion, PlayFab has the clearest documentation and introductory guides, is essentially free to get started with, and includes the features we need for the purposes of deploying a simple multiplayer game. In particular, the ability to set up matchmaking queues without much friction, and the ability to scale active servers up and down automatically. More info on PlayFab and a good introductory overview here

Over the next few steps we will integrate our DarkRift game server with the PlayFab SDK, allowing it to send and receive data from PlayFab hosting services. We’ll then create a server asset package that will allow our game server to be containerised and hosted on PlayFab. We’ll set up server allocation and matchmaking rules to handle connecting players, and finally update our client code to allow us to join properly online game sessions. Don’t worry if none of that makes sense yet, we’ll step through each section ahead.

Installing PlayFab

In your server code editor, download and install the NuGet package com.playfab.csharpgsdk. In Visual Studio 2019: Tools >> NuGet Package Manager >> Manage NuGet Packages for Solution…, then hit Browse and search for the package. Allow all the dependencies to install also. This package will allow our server to communicate and interact with the PlayFab multiplayer platform.

To integrate PlayFab with our game server, the minimum we need to implement is the gamer server SDK Start and readyForPlayers methods. Import the GSDK namespace in the server plugin network manager with the line:

using Microsoft.Playfab.Gaming.GSDK.CSharp;

And then modify the constructor:

public NetworkManager(PluginLoadData pluginLoadData) : base(pluginLoadData)
{
    ClientManager.ClientConnected += ClientConnected;
    ClientManager.ClientDisconnected += ClientDisconnected;

    // Connect to PlayFab agent
    GameserverSDK.Start();
    if (GameserverSDK.ReadyForPlayers()) {
        // returns true on allocation call, player about to connect
    } else {
        // returns false when server is being terminated
    }
}

If you try building and running the server now you will notice that it crashes and closes. Did we just break everything? Now that we’re integrating the PlayFab GSDK into our server code, the server needs a PlayFab agent to run alongside. Once we deploy our server online, that will be taken care of, but for now we need a way to test our server code locally without reuploading it every time we want to test some new code.

To do this we’ll download PlayFab’s local debugging toolset here. Download the MockVmAgent.zip file and extract it somewhere. Inside that extracted folder there should be a MockVmAgent.exe application. This process will ‘pretend’ to be a PlayFab service for our local testing. For it to work we need to set a few more things up.

First, in your DarkRift server folder containing DarkRift.Server.Console.exe, select everything inside and zip it into a single .zip file called DarkRift.Server.Console.zip. The first level of that .zip file should contain DarkRift.Server.Console.exe. In the MockVmAgent folder, find the MultiplayerSettings.json file and open it. First, we need to specify the local file path to our zipped server package:

"AssetDetails": [
        {
            "MountPath": "C:\\Assets",
            "LocalFilePath": "C:\\Path\\to\\zip\\package\\DarkRift.Server.Console.zip"
        }
],

As well as the start command to run our server:

"ProcessStartParameters": {
        "StartGameCommand": "DarkRift.Server.Console.exe"
},

We also need to map our game ports as shown below, (more on this later)

"PortMappingsList": [
        [
            {
                "NodePort": 56100,
                "GamePort": {
                    "Name": "game_port_tcp",
                    "Number": 4296,
                    "Protocol": "TCP"
                }
            },
            {
                "NodePort": 56100,
                "GamePort": {
                    "Name": "game_port_udp",
                    "Number": 4296,
                    "Protocol": "UDP"
                }
            }
        ]
],

Also make sure that the RunContainer parameter is set to false.

When we build our server this time, as well as copying the MultiplayerPlugin.dll over to the Plugins folder of our server, we also need to copy our imported GSDK package and its dependencies to the Lib folder of the server: Microsoft.Playfab.Gaming.GSDK.CSharp.dll and Newtonsoft.Json.dll. After copying those over zip everything up again to create the DarkRift.Server.Console.zip package. Open a powershell window (as administrator) and navigate to where MockVmAgent.exe is located and run it with command .\MockVmAgent.exe. The command line should start to spit out messages with the game state changing from StandingBy to Active to Terminating. When the state switches to active your server should be running and a connection from your Unity client should again be possible.

In this section, we added a minimal interface layer to our server that allows it to communicate with PlayFab multiplayer services. The local debugging toolset we downloaded allows us to start the server process from a PlayFab agent, imitating how servers will be generated when we deploy everything online.

Verifying containerisation

In the previous section, we used PlayFab to run our server to check that we had integrated the PlayFab GSDK correctly. However, remember that we set RunContainer to false in the MultiplayerSettings.json file. When our server is actually deployed it will be containerised in a virtual machine, so next we need to check that our server package will containerise without errors. Once we can containerise the server package, deploying it online should be frictionless.

NOTE: Creating Windows containers with Docker requires Windows 10 Pro Edition. If you don’t have access to this, you’ll need to do the next debugging steps after deploying to PlayFab servers.

To create containers for our server we’ll use Docker. Download Docker for Windows here. Once Docker is downloaded and running, make sure it’s set to run with Windows containers.

NOTE: Docker must be set up to use Windows containers to use PlayFab's local debugging tools. To do this, right-click on the Docker icon in the notifications area / system tray (desktop bottom right) and select Switch to Windows containers.

Back in our powershell window we should still be in the folder where MockVmAgent.exe is located. To set up the Docker networks and get the Docker image for containerising our server run .\Setup.ps1. In MultiplayerSettings.json we need to change a few more settings. Set RunContainer to true this time, and update the ContainerStartParameters with the DarkRift server process command:

"ContainerStartParameters": {
        "StartGameCommand": "C:\\Assets\\DarkRift.Server.Console.exe",
        "ResourceLimits": {
            "Cpus": 0,
            "MemoryGib": 0
        },
        "ImageDetails": {
            "Registry": "mcr.microsoft.com",
            "ImageName": "playfab/multiplayer",
            "ImageTag": "wsc-10.0.17763.973.1",
            "Username": "",
            "Password": ""
        }
},

We’re almost done, but if you try and run MockVmAgent now you’ll notice that the container attempts to start up and then promptly deletes itself. If you navigate to the Agent output folder in the local debugging toolset and find the GameLogs for the last session, there will probably be an error message related to ANSI coloring. Luckily there is a quick fix for this. Go to the DarkRift server folder, find the Server.config file and open it. We simply need to disable fast ANSI coloring in one of our server LogWriters:

<logWriters>
    <logWriter name="FileWriter1" type="FileWriter" levels="trace, info, warning, error, fatal">
        <settings file="Logs/{0:d-M-yyyy}/{0:HH-mm-ss tt}.txt" />
    </logWriter>

     <logWriter name="ConsoleWriter1" type="ConsoleWriter" levels="info, warning, error, fatal">
         <settings useFastAnsiColoring = "false" />
     </logWriter>

    <logWriter name="DebugWriter1" type="DebugWriter" levels="warning, error, fatal" />
    </logWriters>
  </logging>

Now repackage the server files into a .zip and run the MockVmAgent again. After a few ticks, your server should be back in an active state.

NOTE: If you get an error when running MockVmAgent, check that you have packaged your server files into a .zip in the right location specified in MultiplayerSettings.json, and that all existing Docker containers in Docker desktop are closed.

Although the server is active and successfully containerised, if you now try to connect from the Unity client the connection will fail. This is because we have introduced an extra connection layer by containerising the server. The server process itself is still listening on port 4296, but we just packaged that server into a virtual machine that has different network access points. Recall earlier though that we mapped these access points onto the server:

"PortMappingsList": [
        [
            {
                "NodePort": 56100,
                "GamePort": {
                    "Name": "game_port_tcp",
                    "Number": 4296,
                    "Protocol": "TCP"
                }
            },
            {
                "NodePort": 56100,
                "GamePort": {
                    "Name": "game_port_udp",
                    "Number": 4296,
                    "Protocol": "UDP"
                }
            }
        ]
],

The DarkRift server has a default network listener (which you find the definition for in the Server.config file) that is a bichannel listener which receives both TCP and UDP messages on the same port. PlayFab expects us to map TCP and UDP ports separately, but since we are using a bichannel listener in DarkRift we set the same port number for the TCP and UDP channels. In this PortMappingList, we are saying that clients are allowed to connect to our local server container (address is still 127.0.0.1 for local testing) on port 56100. 56100 is mapped twice to DarkRift’s port 4296 for both TCP and UDP messages. If we go back to our Unity client and set the Client port in our NetworkManager object to 56100, we should now be able to connect successfully to our containerised server.

This is basically how we will connect to servers once they are deployed online, the only difference being that we will have multiple instances of the server deployed, each with different IP addresses and access ports. Since we won’t manually set these addresses ahead of time, the client won’t initially know what access point to connect to so we’ll have to implement a way for clients to look for available access points. This will be discussed and implemented in full in a later section of this guide.

Right now though, we have to build out our PlayFab server integration a bit. You might have noticed that if you leave the process running for some time after running MockVmAgenet.exe we get stuck on the Terminating state. In MultiplayerSettings.json, there are some settings that define how our PlayFab agent behaves once its initialised:

"NumHeartBeatsForActivateResponse": 10,
"NumHeartBeatsForTerminateResponse": 60,

The PlayFab agent uses sequential ‘heartbeats’ to check in with our DarkRift server to see if it’s healthy and should continue running. Those two lines in MultiplayerSettings.json are hard-coded values that tell the PlayFab agent to send 10 heartbeats before activating the server. After 60 heartbeats try and shut down the server. We haven’t yet implemented any code to respond to PlayFab heartbeats or to gracefully shut down the server, so the PlayFab agent will sit forever in the Terminating state.

PlayFab agent communication

In our DarkRift server plugin NetworkManager we’ll first create two methods (they can be empty for now), void OnShutdown and bool OnHealthCheck. In the NetworkManager constructor, we’ll register these functions with the PlayFab GSDK. The NetworkManager constructor and new methods should now look like this:

public NetworkManager(PluginLoadData pluginLoadData) : base(pluginLoadData) {
    ClientManager.ClientConnected += ClientConnected;
    ClientManager.ClientDisconnected += ClientDisconnected;

    GameserverSDK.RegisterShutdownCallback(OnShutdown);
    GameserverSDK.RegisterHealthCallback(OnHealthCheck);

    // Connect to PlayFab agent
    GameserverSDK.Start();
    if (GameserverSDK.ReadyForPlayers()) {
        // returns true on allocation call, player about to connect
    } else {
        // returns false when server is being terminated
    }
}

void OnShutdown() {
}

bool OnHealthCheck() {
    return true;
}

For now, OnHealthCheck always returns true, so the server will always report it is healthy. We’re also going to keep OnShutdown simple, all this method will do is attempt to exit the current environment (i.e. shut off the VM):

void OnShutdown() {
    Environment.Exit(1);
}

This should be enough to automatically shut off the container when the PlayFab agent enters the termination state.

As it stands, we have implemented a bare minimum template for a server integrated with PlayFab. We still want to add some functionality later - namely player information updates and a more sophisticated server health check - but before that let’s have a first try at deploying our server.

First PlayFab deployment

Create a PlayFab account following the instructions here to create your first title. Once your title is created you should be presented with the title dashboard. Click the Multiplayer tab on the left panel and enable multiplayer servers:

NOTE: You may have to enter some payment information somewhere during this stage. Development mode is free on PlayFab up to a certain number of unique users / server hours, but this is not a guarantee (from me) you won’t be charged anything for server use. Check the PlayFab billing information to be sure.

Once multiplayer servers have been enabled, navigate to the builds section and click New Build. You should now see a form for entering information about the build. First of all, what is a build? A build is composed of a version of our server package that PlayFab will run, and some associated information like port mappings, virtual machine selection and regions where our server will run. We can have different builds for the same game running simultaneously on PlayFab. This is really useful if we have, for example, a deployed version of our game but we want to test some new server features without taking down the active servers - we could just create a new build with a modified version of our server.

Let’s fill our our build details. Call the build something like server-test. Choose a virtual machine and set the servers per machine. Right now we’re only testing if our server will actually run on PlayFab correctly so we don’t need many cores or servers, we can just choose a standard F2s v2 (2 cores) and have 1 server per machine. Platform will be Winodws and our Container Image will be Windows Server Core. Next we need to upload our server package .zip file - the same one we used when testing containerisation locally with MockVmAgent.exe (DarkRift.Server.Console.exe). Make sure you build the server solution and rezip everything before uploading the file. We also give a server start command. Our .zip package will be mounted in C:\Assets in the container so the start command will be C:\Assets\DarkRift.Server.Console.exe.

We also need to map ports (as we did before with our local containerised server). Recall that our DarkRift server has a bichannel listener on port 4296, therefore we need to define two port mappings for port 4296 - one TCP and one UDP. The port name is not that important but call it something that makes the TCP and UDP channels easily identifiable. Finally we select a server region. Your choice will be limited in development mode, so just pick whatever is closest to you on your spot of the planet. We should also set a maximum server number, and the number of standby servers.

A quick note on standby servers: deploying a server is not instantaneous, so if all servers are busy and a player requests a new server it might be some minutes before it’s usable. PlayFab therefore affords us the option to keep some servers on standby. Changing a server from standby to active is much quicker than deploying a whole new server, so it allows our server pool to be a bit more agile in response to player load.

With our settings complete hit Save at the bottom of the page. You should be taken back to the Build tab and see that we have an initialized build, currently with 0 servers running:

We’ll have to wait a few minutes for PlayFab to set everything up. While its working the build should have the Deploying status:

Eventually our build will deploy and you should see the standby and total servers numbers change:

If we navigate over to the Servers tab and select our deployment region, you should also see the individual servers and their states:

Right now we only have standby servers since no players have attempted to join and transition them to the active state. Which begs the question - how do we connect to these servers as a client? In this tab you can find the connection information from the Connect button on the right, but obviously we are not going to have our players go to our PlayFab account and copy server information into their game client. Instead we need an automated way to assign servers to players and give them connection information. This will be the topic of our next section!

Before moving on, make sure to delete the build we just created, just to make sure we don’t use up server hours while it’s idle.

PlayFab client integration

We need to set up our Unity project with the PlayFab SDK to have it communicate with our PlayFab hosted service. Follow the instructions here to install the SDK (return here when you reach the first API call step). The title settings will correspond to the new PlayFab title we created in the previous section.

Connecting clients to servers

Let’s talk a bit about our strategy for connecting clients to servers. One option is for clients to be able to directly request a new server instance from PlayFab. This is a valid option and is discussed in the PlayFab documentation. Keep in mind though that server hosting does cost money - if we give clients the ability to request servers directly someone might hack our game client and continuously request servers, leaving us racking up a significant server bill. Another option is to write an intermediary service between the client and PlayFab that deals with requesting new servers from PlayFab. We can give this intermediary service our developer secret key and only allow this service to create servers. This service could authenticate users, ensure one user doesn’t create too many servers etc. This is a much better approach but requires us to develop a whole extra service on top of our client and server code. For this tutorial, we will instead take advantage of PlayFab’s existing matchmaking service to deal with client connections.

With this matchmaking service, we can setup a matchmaking queue that listens for attempted client connections and matches clients together with a set of rules (e.g. latency, region). The matchmaking service will then take care of allocating servers and will send the information back to clients that are trying to join. This way we never allow clients to make server creation requests, and we don’t have to spend too much time writing intermediary services between client and server.

Back in our Unity project open up the NetworkInterface class. We first need to add some import statements and a few properties corresponding to PlayFab settings:

using System.Net;
using PlayFab;
using PlayFab.ClientModels;
using PlayFab.MultiplayerModels;

// PlayFab settings
public string region; // The region where we will try to connect
public string matchmakingQueue; // The name of the matchmaking queue we'll use
public int matchmakingTimeout; // How long to attempt matchmaking before resetting
public string playfabTCPPortName; // Playfab's name for the TCP port mapping
public string playfabUDPPortName; // Playfab's name for the UDP port mapping

Now we need to add a series of methods to deal with each step of the PlayFab connection process. First a public method to initiate the entire process of connecting to a PlayFab hosted server:

// PlayFab Connection //
public void StartSession(string clientName) {
    // Attempt to login to PlayFab
    var request = new LoginWithCustomIDRequest { CustomId = clientName, CreateAccount = true};
    PlayFabClientAPI.LoginWithCustomID(request, OnLoginSuccess, OnPlayFabError);

    // Disable input panel
    UIManager.singleton.SetInputInteractable(false);
}

StartSession takes a clientName parameter (our chosen display name) and sends a login request to PlayFab. Note that this is a very basic login request that doesn’t do any sophisticated authentication or account checking. You might want to extend this for a finished game. We also update the UI to be disabled while the connection process runs. The LoginWithCustomID call specifies two callback functions, OnLoginSuccess and OnPlayFabError. OnPlayFabError will be a very general function that we will use throughout the connection process so let’s implement that now:

// PlayFab error handling //
private void OnPlayFabError(PlayFabError error) {
    // Debug log an error report
    Debug.Log("Error!");

    Debug.Log(error.GenerateErrorReport());
}

OnLoginSuccess continues the process by starting a matchmaking request with the login info:

private void OnLoginSuccess(LoginResult result) {
    // If login is a success, attempt to start matchmaking with the client's entity key values
    StartMatchmakingRequest(result.EntityToken.Entity.Id, result.EntityToken.Entity.Type);
}

StartMatchmakingRequest builds the matchmaking request structure:

private void StartMatchmakingRequest(string entityID, string entityType) {
    // Create a matchmaking request
    PlayFabMultiplayerAPI.CreateMatchmakingTicket(
        new CreateMatchmakingTicketRequest {
            Creator = new MatchmakingPlayer {
                Entity = new PlayFab.MultiplayerModels.EntityKey {
                    Id = entityID,
                    Type = entityType
                },
                Attributes = new MatchmakingPlayerAttributes {
                    DataObject = new {
                        Latencies = new object[] {
                            new {
                                region = region,
                                latency = 100
                            }
                        },
                    },
                },
            },

            // Cancel matchmaking after this time in seconds with no match found
            GiveUpAfterSeconds = matchmakingTimeout,

            // name of the queue to poll
            QueueName = matchmakingQueue,
        },

        this.OnMatchmakingTicketCreated,
        this.OnPlayFabError
    );
}

StartMatchmakingRequest creates a matchmaking ticket. A ticket is generally comprised of a list of players who want to play together and the necessary data required to match them with other players. In our case this data is limited to the player ID and type (Entity) and some data about connection region and latency. Note that we hard code the latency to 100 in this example, in a more sophisticated request we would ping the region first to check latency with a quality of service request.

Our matchmaking request also specifies a timeout period (GiveUpAfterSeconds) and the name of the queue to join (QueueName), both of which are properties of our NetworkInterface class that we’ll set later in the Unity editor. Finally we have two callback methods, OnPlayFabError which we already defined and OnMatchmakingTicketCreated:

private void OnMatchmakingTicketCreated(CreateMatchmakingTicketResult createMatchmakingTicketResult) {
    // Now we need to start polling the ticket periodically, using a coroutine
    StartCoroutine(PollMatchmakingTicket(createMatchmakingTicketResult.TicketId));

    // Display progress in UI
    UIManager.singleton.DisplayNetworkMessage("Matchmaking request sent");
}

Once we’ve made and sent a matchmaking ticket, we need to start periodically checking it to determine its status. In OnMatchmakingTicketCreated we begin a Unity coroutine PollMatchmakingTicket to periodically check our ticket status in the background. We also flash a UI message telling the client that the matchmaking process has started.

private IEnumerator PollMatchmakingTicket(string ticketId) {
    // Delay ticket request
    yield return new WaitForSeconds(10);

    // Poll the ticket
    PlayFabMultiplayerAPI.GetMatchmakingTicket(
        new GetMatchmakingTicketRequest {
            TicketId = ticketId,
            QueueName = matchmakingQueue
        },

        // callbacks
        this.OnGetMatchmakingTicket,
        this.OnPlayFabError
    );
}

PlayFab only allows us to check our ticket status something like 6 times per minute, therefore PollMatchmakingTicket first delays the upcoming request for 10 seconds. Then we ask PlayFab for the ticket status with GetMatchmakingTicket, using our ticket ID and target matchmaking queue. Error callback is again the same, otherwise we call OnGetMatchmakingTicket:

private void OnGetMatchmakingTicket(GetMatchmakingTicketResult getMatchmakingTicketResult) {
    // When PlayFab returns our matchmaking ticket
    if (getMatchmakingTicketResult.Status == "Matched") {
        // If we found a match, we then need to access its server
        MatchFound(getMatchmakingTicketResult);
    } else if (getMatchmakingTicketResult.Status == "Canceled") {
        // If the matchmaking ticket was canceled we need to reset the input UI
        UIManager.singleton.SetInputInteractable(true);
        UIManager.singleton.DisplayNetworkMessage("Start Session");
    } else {
        // If we don't have a conclusive matchmaking status, we keep polling the ticket
        StartCoroutine(PollMatchmakingTicket(getMatchmakingTicketResult.TicketId));
    }

    // Display matchmaking status in the UI
    UIManager.singleton.DisplayNetworkMessage(getMatchmakingTicketResult.Status);
}

Here we check for a few potential statuses of the matchmaking ticket. If the ticket status tells us we’ve matched with other players, we run the MatchFound method. If the ticket is canceled we just have to update the UI to allow clients to attempt another session. Otherwise, we don’t yet have a definitive matchmaking status (the ticket is probably still waiting to match) so we restart the PollMatchmakingTicket to poll the ticket in another 10 seconds. We still have to implement the MatchFound method that we’ll call on a successful match:

private void MatchFound(GetMatchmakingTicketResult getMatchmakingTicketResult) {
    // When we find a match, we need to request the connection variables to join clients
    PlayFabMultiplayerAPI.GetMatch(
        new GetMatchRequest {
            MatchId = getMatchmakingTicketResult.MatchId,
            QueueName = matchmakingQueue
        },

        this.OnGetMatch,
        this.OnPlayFabError
    );
}

When we get a successful matchmaking ticket in OnGetMatchmakingTicket we pass that information to MatchFound and ask PlayFab to request the details of the successful match, calling OnGetMatch as a callback:

private void OnGetMatch(GetMatchResult getMatchResult) {
    // Get the server to join
    string ipString = getMatchResult.ServerDetails.IPV4Address;
    int tcpPort = 0;
    int udpPort = 0;

    // Get the ports and names to join
    foreach (Port port in getMatchResult.ServerDetails.Ports) {
        if (port.Name == playfabTCPPortName)
            tcpPort = port.Num;

        if (port.Name == playfabUDPPortName)
            udpPort = port.Num;
    }

    // Connect and initialize the DarkRiftClient, hand over control to the NetworkManager
    if (tcpPort != 0 && udpPort != 0)
        drClient.ConnectInBackground(IPAddress.Parse(ipString), tcpPort, udpPort, true, delegate {OnPlayFabSessionCallback();});
}

private void OnPlayFabSessionCallback() {
    if (drClient.ConnectionState == ConnectionState.Connected) {
        // If connection successful, send any additional player info
        NetworkManager.singleton.SendPlayerInformationMessage(
            UIManager.singleton.nameInputField.text
        );

        // Set lobby controls to interactable
        UIManager.singleton.SetInputInteractable(false);
        UIManager.singleton.SetLobbyInteractable(true);
    } else {
        // Else reset the input UI
        UIManager.singleton.SetInputInteractable(true);
        UIManager.singleton.SetLobbyInteractable(false);
    }
}

At long last we get to some information we can use to connect to a server! By the time OnGetMatch has been called, a number of players will have been matched together with the matchmaking ticket. PlayFab assigns a server to that group of players and we can then request those details with getMatchResult.ServerDetails. We get the IP address, then we loop through all the returned ports in the server details and check which ones correspond to the TCP and UDP port names we specify on the Unity client and on the PlayFab settings. Armed with this data, we can then ask the DarkRift client to connect to the remote PlayFab server.

We also need to link a UI button to start this whole matchmaking process. In Unity, open up the UIManager class we created previously. Link the Start Session button with the StartSession method of the NetworkInterface, passing in the input field text as a parameter. The Start method of the UIManager should now look like this:

void Start() {
    startSessionButton.onClick.AddListener(
        () => {NetworkInterface.singleton.StartSession(nameInputField.text);}
    );
    localTestButton.onClick.AddListener(
        NetworkInterface.singleton.StartLocalSession
    );
    readyButton.onClick.AddListener(NetworkInterface.singleton.SetPlayerReady);
    SetLobbyInteractable(false);
}

Final touches on the server

Previously we implemented about the bare minimum needed for communication between PlayFab and our server to allow it to run. We’re going to add a little spice to our server code now to provide a little more information to the PlayFab service while our servers are deployed. Back in the server plugin code NetworkManager class we need to add a few more properties first, the function of which will become clear soon:

DateTime startDateTime;
bool sessionIdAssigned;

Make sure sessionIdAssigned is set to false in the NetworkManager constructor as well:

public NetworkManager(PluginLoadData pluginLoadData) : base(pluginLoadData)
{
    ClientManager.ClientConnected += ClientConnected;
    ClientManager.ClientDisconnected += ClientDisconnected;

    GameserverSDK.RegisterShutdownCallback(OnShutdown);
    GameserverSDK.RegisterHealthCallback(OnHealthCheck);
    sessionIdAssigned = false;

    // Connect to PlayFab agent
    GameserverSDK.Start();
    if (GameserverSDK.ReadyForPlayers()) {
        // returns true on allocation call, player about to connect
    } else {
        // returns false when server is being terminated
    }
}

Now we’ll add some code to our OnHealthCheck method. Remember that we registered this method with the PlayFab SDK already, and it will be called every time the PlayFab service does a health check on a running server:

bool OnHealthCheck() {
    // How long has server been active in seconds?
    float awakeTime;

    if (!sessionIdAssigned) {
        awakeTime = 0f;
    } else {
        awakeTime = (float)(DateTime.Now - startDateTime).TotalSeconds;
    }

    // Get server info
    // If session ID has been assigned, server is active
    IDictionary<string, string> config = GameserverSDK.getConfigSettings();
    if (config.TryGetValue(GameserverSDK.SessionIdKey, out string sessionId)) {
        // If this is the first session assignment, start the activated timer
        if (!sessionIdAssigned) {
            startDateTime = DateTime.Now;
            sessionIdAssigned = true;
        }
    }

    // If server has been awake for over 10 mins, and no players connected, and the PlayFab server is not in standby (no session id assigned): begin shutdown
    if (awakeTime > 600f && players.Count <= 0 && sessionIdAssigned) {
        OnShutdown();
        return false;
    }

    return true;
}

We added a lot here so let’s step through it. We define a float awakeTime to keep track of how long our server has been active for. If a session ID hasn’t been assigned to the server yet (PlayFab assigns a session ID when a server becomes active) we can set awakeTime to 0 as the server is not currently active. Else, we determine how long the server has been active by subtracting the start time from the current time and converting to seconds.

Now we get the server config from PlayFab using the SDK method getConfigSettings. We attempt to get the session ID from the config, if we successfully get this value, and a session ID was not previously assigned, we know that our server as just become active, and we can set sessionIdAssigned to true and set the startDateTime to the current time to indicate the server became active at this time.

Finally we do a check to see if we want to keep the server running. The final if statement check whether the server was switched to active (i.e. its not a server waiting in StandBy mode), if the current player count is 0, and if the server has been active for more than 10 minutes. If all those checks are true then this server is probably an active game server where all the players left and we don’t need it in our server pool anymore. In that case we ask the server to shut down so we can recycle it into our pool of standby servers, and return false so that PlayFab knows our server process is no longer healthy.
We’re also going to write a method to tell PlayFab how many players have joined the server:

void UpdatePlayFabPlayers() {
    List<ConnectedPlayer> listPfPlayers = new List<ConnectedPlayer>();
    foreach (KeyValuePair<IClient, Player> player in players) {
        listPfPlayers.Add(new ConnectedPlayer(player.Value.playerName));
    }

    GameserverSDK.UpdateConnectedPlayers(listPfPlayers);
}

Here we create a list of ConnectedPlayer (PlayFab’s representation of a player) and loop through our Dictionary of connected players. For each player we generate a new ConnectedPlayer with the player’s name and add it to the ConnectedPlayer list. Then we just call the SDK UpdateConnectedPlayers method with that list. Make sure to add a call to UpdatePlayFabPlayers at the end of both the ClientConnected and ClientDisconnected method so PlayFab has an up to date copy of the players connected to the server.

Putting it all together

Build your server and zip all the files up as before. Deploy a build with the updated server with the same steps as the First PlayFab deployment section. Once your build is running, open the Matchmaking tab in PlayFab. Create a new queue and call it DefaultQueue. For our testing, let’s say that we want 3 players to join before a server is allocated, so set the Min and Max of the match size to 3. Check Enable Server Allocation and make sure the build in the drop down matches the one we just created. To be able to allocate servers, at minimum we need a region selection rule for the matchmaking. Click Add Rule and name this new rule ‘region’. It needs to be of type Region selection rule. The attribute path should be ‘Latencies’ and let’s set a maximum latency of 500:

Think back to our StartMatchmakingRequest method in the client NetworkInterface class:

private void StartMatchmakingRequest(string entityID, string entityType) {
    // Create a matchmaking request
    PlayFabMultiplayerAPI.CreateMatchmakingTicket(
        new CreateMatchmakingTicketRequest {
            Creator = new MatchmakingPlayer {
                Entity = new PlayFab.MultiplayerModels.EntityKey {
                    Id = entityID,
                    Type = entityType
                },
                Attributes = new MatchmakingPlayerAttributes {
                    DataObject = new {
                        Latencies = new object[] {
                            new {
                                region = region,
                                latency = 100
                            }
                        },
                    },
                },
            },

            // Cancel matchmaking after this time in seconds with no match found
            GiveUpAfterSeconds = matchmakingTimeout,

            // name of the queue to poll
            QueueName = matchmakingQueue,
        },

        this.OnMatchmakingTicketCreated,
        this.OnPlayFabError
    );
}

Notice that part of our ticket was a data structure called Attributes where we defined a DataObject called Latencies with a hard-coded latency of 100 for our chosen region. This is why our region selection rule in PlayFab uses the attribute path Latencies. Since we hard coded our latency instead of querying it from PlayFab services, our client will always be able to join the matchmaking queue as the maximum latency we set in PlayFab is 500. Click Create Queue to finish off.

Back in our Unity client we need to set some Editor variables on the NetworkInterface class so our client can find this matchmaking queue:

Make sure your region corresponds to where your build is deployed, and that the matchmaking queue field matches what you just named the queue in PlayFab. Build and run 3 Unity clients, type in a name for each and click the start session on the UI for each of them. After a short time, you should see everyone matched and joined to the lobby. Click ready on all 3 clients and the game should start. In the PlayFab server panel, you should see an active server with 3 players joined. As we add more players PlayFab should continue to add standby and active servers until we reach the server maximum we set. As players leave and servers become empty, our PlayFab integration should start shutting servers back down.

Conclusion

In this guide we built a simple multiplayer game in Unity with a dedicated server. We hosted our server on PlayFab services and set it up to scale with player load. Admittedly our game is not very interesting, but I hope this has helped with understanding how the pieces of a multiplayer game fit together and has given you some insight into how each part could be further developed.

DEV Community: RoboDoig

A practical guide to RNNs for neuroscience research in Keras

Introduction

Colab notebooks

Example 1 - Simple MNIST

Example 2 - Generating ground-truth data for testing analysis methods

Example 3 - Inferring connectivity

Unity Multiplayer: Bottom to Top

Table of contents

Introduction

Tools used

General outline

Setting up the Unity project

Installing DarkRift

Notes on server architecture

Our first player connection

Our first server code

First client code

Player lobby

Player ready and starting game

Movement synchronisation

Server hosting

Installing PlayFab

Verifying containerisation

PlayFab agent communication

First PlayFab deployment

PlayFab client integration

Connecting clients to servers

Final touches on the server

Putting it all together

Conclusion