DEV Community: Project-DC

Studying logs using VitaBoard

Dhairya Jain — Sun, 06 Sep 2020 22:05:14 +0000

This post is the continuation post in a tri-post series about PyGeneses. This post focuses mainly on one of the packages in PyGeneses which is VitaBoard. If you are unfamiliar with PyGeneses and want to know about it, these articles would help you get to speed.

So, now let's start with VitaBoard.

What is VitaBoard?

Vitaboard provides and advanced, and interactive dashboard to study the agents after their training phase is over. After each agent dies. his/her lifecycle, which contains all the actions he/she has performed, is written in a log file. These log files are used as the input in the VitaBoard which then allows the user to visualize the agent’s life in an easier manner. Vitaboard provides the user with a life visualizer, group statistics, and a genetic history visualizer. It allows the users to identify and understand behaviors shown by the particular agent while the other agents and the environment being the factors affecting them.

Getting Started with VitaBoard

To start using VitaBoard, PyGeneses must be installed in your system. If you have forgotten how that is done or have no knowledge about it, you can find it here.

To start the VitaBoard in your system, all you need is PyGeneses and python installed in your system.

Steps to run

Run the following command in your terminal:

user@programmer~:$ vitaboard

Open localhost:5000 or 127.0.0.1:5000 in any browser.

Various features of VitaBoard

VitaViz This is the first screen that the user sees when the VitaBoard opens in the browser.

This is the VitaViz tab. This is used to visualize the lifecycle of the particular agent. An agent’s lifecycle can be visualized by using its log file which is generated when that particular agent dies.
So, to visualize the lifecycle, enter the location of the log file in the first field and set the speed with which you would like to see the lifecycle simulation in the second field (if you enter 1, the simulation speed would be 1 frame/sec).

After entering the details and hitting ‘Run’, a pygame window opens which will display the simulation. The window looks like this.

VitaGroups This is the second tab, VitaGroups. This is used to form the clusters of agents based on their Neural Network embeddings. This tab looks like this. The input here is again the location of log files. The difference is in VitaViz the location given is of a single log file but in VitaGroups, the location of the entire folder is given which has the log files. Make sure that this folder has an embeddings folder inside it along with other log files. After entering the location, press the ‘Get Groups’ button, this will generate and show the cluster graph. These data points are generated by reducing the dimensions of the agent’s trained embeddings into two dimensions using t-SNE (t-distributed Stochastic Neighbour Embeddings). In this graph, the user can click on any node and the name of that agent will be displayed below, clicking on this again will enable the user to visualize the life of that agent which is similar to the feature in VitaViz tab.
VitaStats The next tab is VitaStats. This helps the user to visualize the various statistics about the agents. The location of log files is entered here. The location given as input here is similar to that of the location entered in the VitaGroups tab. After pressing the ‘Get Stats’ button, various graphs will be generated and displayed. Here the user can click on any node of any graph to get a list of the agents born at that particular time-stamp. This list comes below the graphs. Clicking on any list item will enable the user to visualize the life of that agent. The First Graph shows the relation between the average of the ages of the death of players born at a particular time and the time. This can be used to study the trends in the lifespans of the agents. The second graph shows the relation between the variance in death ages of players born at a particular time and the time. This graph can be used to realize the similarities/dissimilarities in the death age of players born under at the same time and living in similar circumstances. The third graph maps the relation between the Quality of Life and the time. Hereby quality of life we mean the count of players born at a particular time who survived for more than 50 timesteps (in ticks).
VitaLineage The last tab is VitaLineage. This is used to visualize the family tree of that particular agent. Give the location of a single log file of the agent and click on ‘Get Tree’. This gives the family tree of that agent. Click on any node to visualize the life of the agent. This was all about VitaBoard and the features it has to offer.his is an open-source project and would also be enlisted in the Hacktoberfest. I hope you all would consider this project and make contributions to improve and enhance this project. The GitHub link of this project can be found here. Hope to see you there.

Getting started with PyGeneses

Siddhartha Dhar Choudhury — Sun, 06 Sep 2020 15:12:48 +0000

What is PyGeneses? What is this blog post about? Well if you have no idea what PyGeneses is then I would suggest you to first go through the introductory post. If you know the answer to these two questions, then you are in the right place. Let’s get started.

In this post we will go through installation process and PyGeneses’ training and hyperparameter tuning packages.

Installation

PyGeneses can be installed using pip in either your local system or a cloud based platform. The steps for installation will be the same for both cloud and own system. Let’s install PyGeneses:-

user@programmer:~$ pip install git+https://github.com/Project-DC/pygeneses

It will soon be available in pypi, till then installation will be done from the official GitHub repo for PyGeneses.

Let's start training

pygeneses.envs

Now that we have installed PyGeneses, let’s write some code to train the PrimaVita agents. First, we will look at the most basic code example to train agents with default hyperparameter setting and REINFORCE algorithm (we will be adding more RL algorithms in near future).

Yeah it is just these 3 lines. The three steps to train agents are:-

1) Import PrimaVita (First life) class from envs.prima_vita.

2) Instanstiate the PrimaVita class.

3) Call the run() method on the PrimaVita object.

That was super simple wasn’t it. This default setting will allow you to train the agents in ‘bot’ mode which means you will not be able to see the environment while training, in case you want to see the environment throughout training you just need to change the mode to ‘human’:-

Let us now look at an example where the default hyperparameters are changed:-

You just have to pass a dictionary with keys as the name of hyperparameter (here the initial energy of agents and the time in ticks after which model will be updated are changed) and the values as the value you want to specify. For a more detailed list of available hyperparameters refer to the official docs. Just one more line and you can customize your training.

pygeneses.hypertune

Now let’s move on to hyperparameter tuning package (hypertune) of PyGeneses. PyGeneses has two different algorithms for trying out different hyperparameter values:-

1) Grid Search: This takes in a pool of hyperparameter values as input and trains the model with all the possible combinations of these values. This is useful when you want to try out different values of a hyperparameter or group of hyperparameters.

Following is a snippet of code for applying a grid search over pool of hyperparameters using hypertune:-

This too in 3 lines of code

Again the three steps are as follows:-

1) Import HyperTune class from pygeneses.hypertune.

2) Instantiate HyperTune class with the name of environment class, hyperparameter list, hyperparameter values (2D list) and number of logs to be generated for each run.

3) Finally call hypertuner( ) method of HyperTune class.

The values are to be listed in order in which the hyperparameters are listed.

Moving on to the next algorithm:-

2) Randomized Search: This is similar to grid search in that it too takes a pool of hyperparameter values but instead of training model with all possible combinations it selects a subset out of those combinations randomly based on a probability.

Here is the code example:-

The only change in this code and the grid search code is the extra parameter randomize_percent which tells the randomized search algorithm the percentage of combinations to pick up randomly from the pool of available hyperparameters.

This was the end to this part of PyGeneses blog post series. In the next part we will talk about VitaBoard — an interactive dashboard for visualization of training results in PyGeneses which helps makes sense of the actions taken by agents during training and studying their behaviour. See you there :)

Introduction to PyGeneses

Siddhartha Dhar Choudhury — Sun, 06 Sep 2020 14:58:43 +0000

If I try to summarize PyGeneses in a single line then it would be something like — “PyGeneses is a PyTorch based Deep Reinforcement Learning (Deep RL) framework that helps users to simulate artificial agents in bio-inspired environments”.

Let’s break down the above line:-

PyTorch: PyTorch is a machine learning framework developed by Facebook AI Research (FAIR). It has an intuitive python like syntax which allows faster prototyping and serving of ML/DL models. We use PyTorch for the deep reinforcement learning algorithms that we provide as part of our framework.
Deep Reinforcement Learning: Deep Reinforcement Learning is a technique in Artificial Intelligence that brings together the best of Deep Learning and Reinforcement Learning. Deep learning deals with the concept of neural networks which are capable of learning complex functions from data, and Reinforcement Learning is based on the idea of learning from experience.
Artificial Agents: Artificial agents are simulated and simplified versions of species found in nature. The rules these agents follow closely mimic that of nature which allows them to display behaviour close to what animals around us (or even human beings) display. Note: These agents are nowhere close to the complexity of living beings in our planet and are just a simplified version of them.
Bio-inspired environments: Bio-inspired environments mimic conditions of survival and rules that we see in our natural environment in simulation.

About the framework

PyGeneses is a collection of four different packages:-

pygeneses.envs

This module allows you to create, configure and tweak the in-built bio-inspired environments. As of now, this only provides a single environment called Prima Vita (First Life), but there’s more coming soon! This lets you set up the entire environment and the species in just a few lines of code and provides both high level API and low level control over the environment. Training using the API includes logging of every action of an agent so that it can be studied using VitaBoard.

pygeneses.models
The ‘models’ module is what allows us to import the neural networks which the species uses to learn what to do. As of now, only the default model’s (REINFORCE) implementation is provided, but we will be adding support for custom pluggable networks from v0.2 onwards.

pygeneses.hypertune
The ‘HyperTune’ package allows us to configure and test out various hyperparameters we can provide for an environment and species (a list of hyperparameters is provided in the Classes section of this documentation). This contains single hyperparameter testing, grid search and randomized search. This allows us to find the best set of hyperparameters to display a type of behavior. This also produces logs which we can study using Vitaboard.

pygeneses.vitaboard
Vitaboard provides an advanced, interactive dashboard to study agents after the training phase. After each agent dies, his/her actions are written into a log file. And vitaboard allows us to visualize the agent’s life. It provides us with a life visualizer, group statistics and a genetic history visualizer. It allows us to identify and understand behaviours exhibited by an agent while interacting with the environment or with other agents in the environment.

Use cases

Pygeneses can be used for a variety of purposes, the limit only being your creativity and imagination. That being said, some of the use-cases of pygeneses can be:

1) To understand basic psychology — Since every agent has a limited actions to choose from in each time step, the simulation can be studied to infer the basic psychological patterns displayed, this can help to understand the similarities/dissimilarities between the agents and real life organisms.

2) To create applications/games based on the framework — You can create basic applications/games based on our framework. An example can be a productivity app that adds food to the simulation only when the user completes a task, hence staking the lives of the agents of the user’s own simulation in exchange for productivity.

3) To learn more about Deep Reinforcement Learning — Users can just tinker around with different hyperparameters and models and observe the results visually, this makes for a interesting way to get an insight into the world of Deep Reinforcement Learning.

Target Audience

Due to the simple high level API of PyGeneses I believe that this tool can be helpful for both programmer and non-programmers who want to understand the nature of behaviours in animals or understand concepts of Deep Reinforcement Learning.

Most of our API requires only 3–4 lines of code to modify the environment according to your needs, thus allowing users with little to no knowledge of ML or Deep RL to use our tool easily.

We (the developers of PyGeneses) believe that understanding the reason of display of certain behaviours is not only an active area of study but also a fun experiment and through our framework we want to provide a platform for carrying out both of these tasks.

Final words

If you are someone who is keen to learn about ML and AI, or a scientist who studies behaviour or even someone who just wants to have some fun experimenting with these artificial agents then get your local installation of PyGeneses today.

For a detailed introduction (installation and code examples) follow the subsequent articles on this:-

1) Getting started with PyGeneses

2) Studying logs with VitaBoard

If you want to contribute to this project or simply want to know more, then follow the official docs.