DEV Community: netcell

An look into using Firebase for Multiplayer Game

netcell — Tue, 12 Mar 2019 17:00:00 +0000

Firebase is awesome, unless you choose the wrong product for the wrong job. A theoretical analysis of Firebase pricing for powering multiplayer games.

Cloud Firestore vs Realtime Database

In October 2017, Firebase launched Cloud Firestore as a new fancy hot cloud database. This new service is operated side by side with Realtime Database and can confuse new user of which to use for their purpose. Firebase’s effort of driving user to the new service makes it even more confusing, delivering an impression that Firestore is there to replace Realtime Database. When you go to any page of Firebase documentation site on Realtime Database, you will see a highlight section on the top of the page encouraging you to explore Firestore.

On Features, Firestore is in fact far better than Realtime Database: better structure, better querying, better rules writing and more concurrent connection. The only downside: its pricing. While Realtime Database is mainly priced by the amount of data you are using and transferring, Firestore is priced by the amount of read/write operations you are making on the database. This makes using Firestore awkward for realtime applications since each client will react and issue read operations on database changes and that costs money.

Who would have thought that Realtime Database is better for realtime applications right?.

Of course no one is using database synchronization for realtime multiplayer games, but turn-based or asynchronous multiplayer games can benefit greatly with Firebase approach. However, one game session can easily invoke 1000 operations, meaning you can support 20 game sessions daily with free plan and 100 game sessions daily with $25/month plan. Realtime Database on the other hand can support 100 CCUs for free plan and 100,000 CCUs for $25/month.

It is the best to use both of these database, Realtime Database for game sessions and Firestore for anything else. With this, you can open connections to realtime database only when the game session starts and close it with firebaseRef.off() when it ends to optimize the simultaneous connections count.

Furthermore, if you are only using realtime database for game sessions, scaling with multiple databases becomes pretty easy since you don’t have to share any data between these database. While Firestore is supposed to be able to automatically scale indefinitely, I don’t think its benefit outweights the cost in this case.

Cloud Functions HTTP Triggers

Since you are making changes to database, you probably want to make some logic not dependent on the client. Cloud Functions provides you with the ability to create HTTP endpoints. However, the functions are billed by invocations, which is really really bad since there is no protection available for the endpoints if you are using it with CORS.

People can make mass requests to these endpoints and you will be billed accordingly. Rejecting from inside cloud functions code doesn’t work since the invocation is already counted. This is not the end of the world though since you can setup CloudFlare and use rate limiting feature to protect your endpoints. This service is only billed by good requests - requests that are allowed to reach to your endpoints.

However, there is another aspect you need to consider: pricing. Free plan gives you 125,000 invocations monthly and $25/month gives you 2 million. This is far less than the amount of write operations you are allowed on Firestore. If you don’t want to use CloudFlare or if you think this limitation might cause trouble, you may want to explore the option to migrate to your own server.

A very interesting point about Cloud Functions endpoints is that you can use frameworks like express.js to wrap these endpoints into a single Cloud Functions HTTP trigger. Since the Firebase admin library can be initialized on any server, you can bring your entire express.js section of your code to another server and it should work just fine. This is important because you can start developing your application with Cloud Function which makes your life much easier, knowing that you can migrate anytime you want.

Programatically Run Motive.io Script in Unity 3D

netcell — Tue, 20 Mar 2018 17:00:00 +0000

Motive.io is a professional platform that allows you to integrate cutting-edge AR technology into compelling location-AR games and experiences.

Motive.io is like Wordpress for geolocation game for Unity 3D featuring a set of tools that allow authoring scripts on their web-based interface and activate them on client.

The thing is, Motive.io is only responsible for hosting the scripts, all the logics are actually ran offline on client based on conditions setup on authouring tool. That is fine for simple single-player games, but for more complex features like multiplayer, you need to be able to control your scripts programmatically , to be aware of their events and feeding them parameters from your own server.

Unfortunately, the service is currently in beta, “launching public access to the platform on March 21st.”, the manual is lacking a lot of contents, the examples only scratch the surface, and there are no SDK documenation. Luckyly, their support is really fast and helpful.

In this series, I will explain how to launch a script programmatically with dyanmic variables.

Anatomy of a Motive Script

Motive script is created on Motive.io Authoring tool.

Normally, you will be using a Main script to launch other scripts, and in example projects, there is a Scripts panel where you can manually activate them. However, if you are programmatically launching a script, you will need it’s unique identifier. Each script has one that can be found on the address bar at scriptId parameter.

I think the term “launching a script” is a little bit misleading. In fact, you will be creating instances of that script, each with an unique runId specified by you. Each instance has its own memory. For example: If you create instances of a Location Task, each instance will have its own location, status of completion, etc.

Operations on Motive Script

There are 3 operation on a script instance you can do: Launch, Stop and Reset.

Launch: creates a new script instance or resume the instance if it is already created
Reset: halt the script and reset the original state of the script.
Stop: halt the script, hide any location marker, but does not reset anything.

To understand the difference between Reset and Stop , consider this example:

An instance of a script that contains two location tasks is launched.
The player has already completed one task in that instance.
If you Stop and then Launch the instance again, you will only see the task that is not completed.
If you Reset and then Launch the instance again, you will see both tasks.

ScriptManager Singleton

Script managing and operation is handle by a ScriptManager singleton with two main methods LaunchScript and StopRunningScript. Their most basic signatures are as follow:

public void LaunchScript(
  string scriptId, string runId, Action<bool> onComplete
);

public void StopRunningScript(
  string scriptId, string runId, bool resetState, Action onComplete
);

Both of them recieves scriptId for identifying the script, runId for identifying the script instance and onComplete Action for callback on completion.

For launching a new instance of a script with runId as unique identifier, or resuming an instance with that id, we use:

ScriptManager.Instance.LaunchScript(scriptId, runId, onComplete);

To stop an instance with id runId, we call StopRunningScript with resetState = false:

ScriptManager.Instance.StopRunningScript(scriptId, runId, false, onComplete);

To reset an instance with id runId, we call StopRunningScript with resetState = true:

ScriptManager.Instance.StopRunningScript(scriptId, runId, true, onComplete);

Try in Example

Start with the GhostHunter Unity template provided by Motive.io, we will be editting the script launcher to launch new instance everytime.

By default, the script launcher of the template only launch and manage one instance with runId="test" for each script.

// TestScriptsPanel.cs
ScriptManager.Instance.LaunchScript(e.Script.Id, "test", null);

If you run the project, each time you launch a script, the Launch button is turned in to a red Stop button. You can remove this behaviour by changing the logic of TestScriptItem.cs like this:

// TestScriptItem.cs
public void UpdateState()
{
  var isRunning = false; // Previously = ScriptManager.Instance.IsScriptRunning(m_script, "test");

  LaunchButton.gameObject.SetActive(!isRunning);
  StopButton.gameObject.SetActive(isRunning);
}

However, this only keep the Launch button, but this button wouldn’t create new script instance and no new location marker will be added. Instead, you would want to replace "test" with a random identifier like this:

// TestScriptsPanel.cs
var runId = System.Guid.NewGuid().ToString();

ScriptManager.Instance.LaunchScript(e.Script.Id, runId, null);

Now, you can run the project and launch any script and new instance of it would be initiated. Managing all of these instances would requires more coding since TestScriptItem.cs only track runId = test instances. We wouldn’t discuss that here, instead, let’s explore how to add dynamic varibable to our scripts.

What’s next?

In the next post in this series, I will explain how to create dynamic variables in Motive.io Authoring Tool and passing values to those variables when launching script instance in Unity 3D.

Wordpress development with Docker

netcell — Sat, 17 Mar 2018 17:00:00 +0000

One of my obsessions lately is keeping my operating system as clean as possible. In order to do that, I perfer to install as few as possible anything on it.

Node.js makes that quest so easy as I only need node installed globally and everything else can be installed locally inside project directory. However, for many other languages, such as Python and PHP, it’s often not the case. Furthermore, when each project requires different environemnt setup, it quickly becomes a painful job managing everything and sometime you break the whole system.

Docker to the rescue.

While mostly used for controlling production and teamwork environments, Docker is also very useful for encapsulating development environment and providing a “clean slate” to make sure nothing breaks unintentionally.

Goals

When building a docker development system, I often consider the following requirements:

Minimum volume mounting: File IO operations on mounted host volumes are slow (on Windows and Mac) so you would only want to mount the very sourcecode files you are going to edit. Others should be copied when builing the image.
No runtime installation: Dependencies and 3rd-party plugins should not be installed at container run time, especially if they require internet connection, to ensure fastest booting time when you resume the development. They should be installed when building the image.
Minimum 3rd-party on host: If possible, dependencies and 3rd-party plugins should be fetched from the internet (via a configuration file like package.json or Gemfile or even listed inside Dockerfile). Otherwise they should be kept as zip files and copied/extracted when building the image. There are two reasons for this:
- Copying a big amount of files from host to image is very slow.
- Keeping a big amount of 3rd-party files in your git directory is messy.
Trackable snapshot: You should be able to take snapshots of your current development progress (especially in Wordpress case as explained below), and the they should be able to be tracked in git repository.

Base Docker Image

For Wordpress development, I use chriszarate/wordpress since it provides a bunch of useful features via envinronment varaibles including:

Automatically activate plugins when container starts. Because no one wants to activate theme manually every single time. However, this becomes irrelevant once you start using database snapshots.
Automatically activate a theme when container starts. Similarly, this becomes irrelevant once you start using database snapshots.
Append additional PHP to wp-config.php.
Preconfigure admin user, admin password, admin email and site url to skip the boring welcome setup screen.

The magic behind this docker image is a set of wp-cli scripts appended into the default entrypoint docker-entrypoint.sh of the official wordpress docker image.

Folder Structure

Here is the folder structure of my Wordpress development directory, there are 3 main sections (which are poorly named in the screenshot but explained nicely in the picture below):

Sourcecodes: The plugins and themes you are currently developing. They are text files (.php, .css, .xml, etc.) that you are going to create and change during most of your project timeline.
3rd-parties: They are the 3rd-party plugins that your site needs, the dependencies of your project or the parent theme for which you are creating child theme. They should be kept as zip files in these folders or specified as urls in Dockerfile.
Snapshots: Including the sql dump from wordpress database and the compressed file of wp-content/uploads folder (in case you are building contents or starting with demo content of a theme).

With this structure, 3rd-parties and Snapshots will be built into the image, while sourcecodes are mounted to allow changes in development.

docker-entrypoint.sh is the default entrypoint docker-entrypoint.sh of the original docker image and will be modified (see below) to fit our needs.

Dockerfile

For a sample Dockerfile, checkout this gist.

Packages

For Dockerfile configuration, I install the following packages:

unzip: As noted above, snapshots and 3rd-parties are saved as zip files (and in the case of remotely fetched, also in zip format). We would need this utility to decompress them.
mysql-client: This package provides neccessary commands required by wp db to export and import wordpress database as .sql snapshot.
[Optional] wget: In the case where dependencires are remotely fetched, we use wget to download them from specified urls.

FROM chriszarate/wordpress:4.9.1

RUN \
    apt-get update && \
    apt-get install unzip wget mysql-client -y && \
    rm -rf /var/lib/apt/lists/*

3rd-parties and Snapshots

We copy each 3rd-party directory of zip files to a temporary folder on the image (or download via wget), extract and delete all zip files. The content of these folders will be copied to real target directories at runtime by docker-entrypoint.sh.

The reason we are not copying them directly to the real target folders is that they would be overwritten by commands in official wordpress image’s docker-entrypoint.sh.

# If copied from folder
COPY ./plugins/ /temp/plugins
# If downloaded via url
wget -P /temp/plugins/ https://downloads.wordpress.org/plugin/jetpack.5.9.zip
# Extract and delete zip files
RUN unzip '/temp/plugins/*.zip' -d /temp/plugins && rm /temp/plugins/*.zip || true;

|| true is added to make sure that when there are no zip file the script would not fail and terminate image build process.

Similarly we copy snapshot files (if available) to the image. The uploads.zip file should be decompressed like above, while wordpress.sql would later be processed by docker-entrypoint.sh.

Entrypoint and Configurations

We need to replace the default docker-entrypoint.sh with our modified script. We don’t have to specify ENTRYPOINT since it is already declared in the official docker image.

COPY docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh

It is also useful to have your own php uploads configuration as uploads.ini in your repository and insert into the image. Obviously you can config other internal system files using the same method: copy a custom file to the absolute path inside the image.

COPY ./uploads.ini /usr/local/etc/php/conf.d/uploads.ini

Entrypoint Script

For a sample docker-entrypoint.sh, checkout this gist.

It’s time to modify docker-entrypoint.sh. Starting with a clone of the script from chriszarate/wordpress, we would need to copy all files in /temp/ directory to where they need to be. These lines need to be placed before plugins and themes activation.

CONTENT_DIR=$ROOT_DIR/wp-content
THEME_DIR=$CONTENT_DIR/themes
PLUGIN_DIR=$CONTENT_DIR/plugins

cp -r /temp/themes/* $THEME_DIR || true
cp -r /temp/plugins/* $PLUGIN_DIR || true
cp -r /temp/base/* $CONTENT_DIR || true

# Activate plugins. Install if it cannot be found locally.
# ...

One thing to notice here is that, all these files and folders are owned by root user, hence will not be writable by Wordpress. If you need to give Wordpress permission over these directories, you need to insert something like this:

chown -R $WEB_USER:$WEB_USER $ROOT_DIR/wp-content/uploads

Finally, when everything is done, at the end of docker-entrypoint.sh, we would want to import wordpress.sql file. We place this line at the end of the file, just before exec "$@", to ensure our database would not be overwritten by anything.

runuser $WEB_USER -s /bin/sh -c "wp db import $CONTENT_DIR/wordpress.sql"

exec "$@"

Docker Compose

For docker-compose configuration, you can find an example at chriszarate’s repository. It’s a combination of 2 services: wordpress for PHP wordpress installation and mysql for database. There are only a few things to change here:

Build your Dockerfile: instead of using chriszarate/wordpress image directly

services:
  wordpress:
    build: .

Mount your sourcecodes according to your project, one by one. Do not mount the entire themes or plugins directory since their versions inside the container are already populated with other files.

volumes:
  - "./src/themes/my-custom-theme:/var/www/html/wp-content/themes/my-custom-theme"
  - "./src/plugins/my-custom-plugin:/var/www/html/wp-content/plugins/my-custom-plugin"

Local only: Only list plugins and themes that are installed inside docker image on WORDPRESS_ACTIVATE_PLUGINS and WORDPRESS_ACTIVATE_THEME environment variables. Otherwise, the entrypoint script will attempt to download them from the internet and prolong the startup time.

Now we are set. Let’s go and clean off any XAMPP or MAMP stack, or just delete everything and reinstall your operating system, or even better throw your machine away for a brand new computer. Have fun.

Developing GameClosure with Docker and Gitlab CI

netcell — Sun, 24 Dec 2017 17:00:00 +0000

Develop games with HTML5 game framework GameClosure using Docker and Gitlab CI.

Docker for Development

Following the guide of GameClosure, I made a Dockerfile that has gameclosure/devkit installed where I copied manifest.json, package.json and run the commands to install dependencies :

COPY --chown=node manifest.json package.json ./
    RUN devkit install
    COPY --chown=node ./package.json ./
    RUN npm install
    CMD ["devkit", "serve"]

My project only needs 2 folders: src for source codes and resources for assets. These are the folders required by GameClosure, everything else are dependencies and installed in the docker image. Using docker-compose, I mount these folders to the container and expose port 9200.

volumes:
        - ./src:/home/node/game/src
        - ./resources:/home/node/game/resources
    ports:
        - 9200:9200

From the host machine, I can access devkit page via browser at localhost:9200 as normal.

Continous Deployment

In order to build the project, I need another docker-compose configuration to mount a build folder and run the build command instead of serve.

volumes:
        - ./build:/home/node/game/build
    command: devkit debug browser-mobile

I didn’t want to build the game manually, so I used Gitlab CI/CD service to have it runs the container, builds the game and deploys to Gitlab Pages for me. I create this .gitlab-ci.yml file with pages job that build the game into *public* artifacts:

Unlike Github Pages which can only build Jekyll, Gitlab Pages can automatically build almost anything with its CI/CD service.

pages:
        stage: deploy
        script:
            - docker-compose -f docker-compose.yml -f docker-compose.browser-mobile.yml up
            - cp -a build/debug/browser-mobile/. public/
        artifacts:
            paths:
                - public

Now, everytime I push to my Gitlab repository, the game will be built and served at Gitlab Pages.

Geolocation in Mobile Game

netcell — Thu, 19 Oct 2017 17:00:00 +0000

🖼️ Overview

The general problem is as follow:

Provides the game a context of player being inside a certain area in a set of predefined records, allowing the game to affect player’s process accordingly.

🗒️ Here, a location is understand as a set of two value (latitude, longitude) which refers to a position on the surface of the Earth.

🛒 What’s on the market?

🎮 Geolocation Games

🔴 🌐 Geocaching

Launched in 2000 and is still kicking well, the game allows players to go find real boxes 📦 hidden around the world. While being more of an outdoor game ⚽ than a video game 🎮 , this is probably one of the longest running geolocation mobile game ever.

🔴 🏙️ Ingress

Before the widely successful Pokemon Go 👎 , Niantic created Ingress with a futuristic theme 🚀 and so many more features: location capturing, structure building, PvP actions, etc.

🔴 📈 Landlord

Landlord allows player to buy venues 🏯 that they visits and earn rent 💵 as people check in 📍 at those location in real time.

🔴🕵️‍♀️ Revenge of the Gang

In this game, the players create gangs 👨‍👦‍👦 to fight each other for territories 🗺️ . The aim of the game is to try to conquer as many cells as possible on a grid map (see more at Grid section).

⚙️ Location as a Services

🔴 🏖️ Google Places API

This service is discussed below 👇 in Marker section.

🔴 🔪 Motive.io 🌟

A location backend with Vuforia AR integration for Unity.

The service provides quite a complete backend solution for location-based game 👍 development with places matching, user location tracking, narrative designs and quest management.

They also promote AR integration with Vuforia but I believe with ARKit and ARCore insight, the modern AR approach implementation is in due.

Sadly, Motive.io is currently still in closed beta.

🔴 ⛓️ Backendless.com 🌟

The geolocation feature of this service seems to cover most 👍 of the backend features you need. However, there are no 🚫 Backendless SDK for game engines right now (though about one year ago, it is said to be under development).

🔴 🗺️ Mapbox

🌟 Mapbox is a toolbox of geo-related tools. It even has mobile SDKs and Unity SDK for displaying maps. Mapbox analysis tool Turf.js allows complicated geo calculation and estimation.

🤖 Technical Patterns

🏁 Grid - Area-based method

Grid map is really popular in games. A grid split the map into multiple consecutive regions called cells with identical shapes (square or hexagon in most cases).

🌟 By using grid method in geolocation, the game can find out which cell player is currently in and affect the player’s process accordingly.

Considering the world map 🗺️ as a 2D plane , latitude and longitude are the coordinates, this problem is similar to checking if a point is within a polygon. ☝️ However, there are cases where the area overlaps where the map wrap around, but these cases can be solved with relatively simple algorithms.

📌 Markers - Distance-based method

🌟 In this method, given a database of locations, the game need to figure out which of those locations are within a certain radius around and nearest to player’s current location.

🔴 🏖️ Google Places API - The third-party way

🌟 With this method, the database doesn’t save the location but ID of the places available on Google Places API.

🌟 Google Places has a very good 🛢️ location database of real-life places. The nearby search-request API 🔎 allows searching places near player’s current location within a radius 👉 which is the problem we are looking into.

- Notice 1: It was possible to add places that doesn’t exist on Google Places. However, that API has been deprecated ⛔ since June 2017. Therefore, if we use Google Places API, our markers will be limited to what exists on Google Places only. On the bright side 🔅, its database is really huge so this may not be that big of a problem.

Notice 2 The Google Places API for iOS enforces a default ⚠️ limit of 1,000 requests per 24 hour period. However, it is possible to request an increase by following a simple review process.

With this method, the database structures and operations would be much simple , delicate the most notable features to the Google’s service. The game backend only needs to map places to desired behaviours.

🔴 ⚗️ Formulas for Spherical distance - The first-party way

There are three formulas for this including: Haversine formula, Spherical Law of Cosines and Equirectangular approximation. These are listed in the document that is referred by Google Maps 🗺️ team.

👉 According to the document:

Spherical Law of Cosines - provided the 👍 best accuracy for small distances and the 👎 worst performance overall.
Haversine formula - is the most popular in the past ⏳ since it avoids large round-off errors in computations in the era where computers 🖥️ did not exist or were very primitive.
Equirectangular approximation - has the 👍 **best performance while provides the 👎 worst accuracy.

🌟 With the current technology, Spherical Law of Cosines became widely used for spherical distance calculation. The Google Maps 🗺️ documentation, regarding this matter, also gave the example SQL query based on Spherical Law of Cosines . 👍

🔴 🔬 Optimization

For whichever formula selected, the problem with these techniques is that the formula requires a lot of trigonometric operations, which are very CPU-intensive 🤖. It is important to ✂️ trim down the records to a small subset prior to use the formula.

🌟 A method for trimming process would be using a square with player’s location at center and only select locations in the database that lies within this square. The calculation, as mentioned in Grid section 👆, is relatively simple.

🚚 Distribution

There are 2 ways to distribute bonuses 💰 to real-life locations 📍 where players can visit:

✋ Manually: By creating location manually, we can establish partnerships 🤝 by putting special bonuses 💰 at partner’s location 📍 .
🤖 Automatically** : Even with partnerships, we still need to populate 🌧️ the map with many regular bonus places to keep players interests inside the geo-game-loop**.

📁 General Distribution

🌟 For seeding 🌱 , in the beginning, we would be distributing bonuses around famous places 🏯 in selective (or all) cities 🏙️ around the world (or in selected countries).

🌟 Using Text Search Requests in Google Places API, we can search for point of interest places (tourist attraction) by querying:

${cityName}+point+of+interest

    # Example requesting famous places in New York City
    https://maps.googleapis.com/maps/api/place/textsearch/json?query=new+york+city+point+of+interest

The response returns 20 places object along with a next_page_token string which can be used to fetch more places.

📌 Markers Distribution

Markers distribution pretty simple: For each famous, we only need to directly assign bonuses 💰 to its location 📍.

🏁 Grid Distribution

Note: I will only cover square grid only. Hexagon grid is more complicated and other than game visual requirement, there is actually no perk over square grid.

Note: Here I use the term subgrid which means a set of cells in the grid that forms a square.

First we decide the size S of a square grid cell S x S and put this grid over the world map 🗺️. There are two options:

The grid cover the whole map 🗺️
- For each city, we find the smallest subgrid that fully cover that city.
A custom grid for each city 🏙️
- For each city, find a square with the size is a multiple of S
- Divide the square into a grid of S x S cells.

After establishing a grid for each city, assign bonuses 💰 to cells that contains famous places 🏯.

Webhook notification from Netlify to Discord

netcell — Fri, 11 Aug 2017 17:00:00 +0000

Moving from Slack to Discord is a huge pleasure. Sure it lack some functionalities that Slack has to offer, but Discord clients are super fast and smooth and comes with voice chat. At the end of the day, performance is what matters the most.

One of the problem I had to deal with is moving the Netlify notifications from Slack to Discord. Here in this post, I will talk about the process of me figuring it out. However, if you are just looking for a solution just like I was, jump forward to TLDR;.

What I thought

At first, I thought that I only need to create a incoming webhook on Discord to get a webhook url.

Afterward, create a outgoing webhook on Netlify.

Copy the webhook url from Discord and paste into Netlify.

You know what? It doesn’t work!

What’s wrong?

The problem is that the JSON format sent from Netlify isn’t compatible with Discord. After searching around, I found this part inside Discord Developer Documentation.

TLDR;

In Netlify, create a Slack intergration webhook (yeah, Slack), paste your Discord webhook url and append /slack

Have fun :)