DEV Community: Tim van Mourik

Visual machine learning (Keras) with GiraffeTools

Tim van Mourik — Wed, 11 Sep 2019 14:14:36 +0000

***NOTE: the main article about GiraffeTools can be found here***

When I started to go down the rabbit hole of ‘machine learning’, it was a puzzling and bewildering experience. After building my first digit recognition network, it did not feel like the concatenation of obscure modules named Conv2D, MaxPooling, or DropOut was very intuitive. Although the outcome brilliantly predicted every new hand-written digit that I fed the model, it did not feel like I knew what I was doing. And more importantly, that I would know how to apply it to my own data.

This is quite a barrier for people that are new to the machine learning space. Add to that the proficiency in coding that is required for running the network on your data. Wouldn’t it be cool if we could lower this barrier?

Introducing: Keras with GiraffeTools

What if you could graphically build your machine learning network? And easily integrate the network with the rest of your code and share the results? If only there were tools for a G raphical I nterface for R eproducible A nalysis F or work F low E xperiments!

That’s what GiraffeTools is: a completely open source web application that allows you to build Keras networks. It automatically creates code that you can save directly to a GitHub repository as a new commit. Your model is now accessible at: https://giraffe.tools/workflow/ [YOUR_GITHUB_HANDLE] / [YOUR_REPOSITORY] / [YOUR_BRANCH_OR_COMMIT].

As an example, take a simple digit recognition network using Keras. The master branch from my keras-giraffetools-digit-recognition repository within my TimVanMourik Github account can be inspected at:

https://giraffe.tools/workflow/TimVanMourik/keras-giraffetools-digit-recognition/master

https://giraffe.tools/workflow /TimVanMourik/keras-giraffetools-digit-recognition/master

How did I construct this?

I dragged-and-dropped Keras modules into the editor
I connected them to generate a sequential network (designed after this tutorial)
GiraffeTools automatically generates the analogous Python-Keras code
The ‘Save to GitHub’ button saves the Python code to my repository (this Python file), as well as the visual pipeline representation (this JSON file)
I manually wrote Python code to load MNIST data, load the GiraffeTools NeuralNet (plain python or Jupyter Notebook)

# Load MNIST data
from keras.datasets import mnist
# Load NeuralNet as defined and saved by the editor
from GIRAFFE.code.neural\_net import NeuralNet

As you would with any Keras network, from here you build, compile, and fit your NeuralNet:

# Preprocess MNIST data
...
# build the model
model = NeuralNet(num_classes)
# Compile the model
model.compile(
   loss='categorical_crossentropy',
   optimizer='adam', 
   metrics=['accuracy']
)
# Fit the model
model.fit(
   X_train, 
   y_train,
   validation_data=(X_test, y_test),         
   epochs=10,         
   batch_size=200
)

All code for this example is available at:

TimVanMourik/keras-giraffetools-digit-recognition

Getting started

The easiest way is to fork the repo above and get started from there! If you want to start from scratch, go ahead: https://giraffe.tools/workflow.

But there is one caveat: at this moment you have to point to the keras nodes in a GIRAFFE.yml file in the root of your repository. How this works under the hood you can read here, in the GiraffeTools main article. The TLDR is: you can use GiraffeTools for any other data analysis pipelines as well if you point to your a library file of function. I pre-made and linked this for the Keras application.

Your input!

I am relatively new to machine learning, so any help to improve this is much appreciated! Via the comment section, GitHub, or the GiraffeTools Slack. Also, I would love to make the same connection to PyTorch, but I have never used it. If you have PyTorch experience, please get in touch and we’ll build it together!

GiraffeTools: a generic online workflow editor

Tim van Mourik — Thu, 29 Aug 2019 10:39:12 +0000

https://giraffe.tools

The analysis of Big Data has been a hot topic for at least a decade. These analyses often consists of a sequences of steps that mold and transform the data into features that have some meaningful interpretation. And maybe you want to extract features for subsequent machine learning analyses.

Would it not be cool to _graphically build analysis pipelines _? This is not a new idea of course. You can find node editors in graphics, genetic analysis, financial tech, and in many more places. Yet, if you have written your own data analysis toolbox, wouldn’t it be great if you could give your users access to your tools and example pipelines in a visual manner?

Introducing GiraffeTools

GiraffeTools is a generic G raphical I nterface for R eproducible A nalysis F or work F low E xperiments! I originally developed GiraffeTools for ‘neuroimaging’. On a daily basis, I work with people that analyse all sorts of brain data (MRI, EEG, MEG, etc.), but quickly noticed that all important toolboxes have completely different APIs and are often written in different programming languages.

So I developed an GitHub-integrated web editor that automatically creates analysis code from a visual workflow…

To which you can add your own toolbox
To which you can add your custom modules to any supported toolbox
To which you can add a custom ‘grammar’ that transforms the graph representation into code for your toolbox.
From which you can save any pipeline to a GitHub repository and easily build examples for your users
In which you can inspect your visual workflow at any previous commit, because your workflow is fully GitHub version controlled!

Your project is accessible at: https://giraffe.tools/workflow /$username/$repository/$branch_or_commit, for example: https://giraffe.tools/workflow/TimVanMourik/SomeGiraffeExample looks like:

https://giraffe.tools/workflow/TimVanMourik/SomeGiraffeExample

Add your own content

Ok, so how does that work? Well, any GitHub project can be a GiraffeTools project by simply putting a GIRAFFE.yml configuration file in the root of your project. There, you can specificy

A files list that points to a file to which you want to write your visual pipeline,
A nodes list that points to toolbox modules that you might want to use in your pipeline,
A grammars list that specifies the interpreter that transforms the graph into code.

**# Content of a GIRAFFE.yml file in the root of your repository.**  
tools:
  workflow:
    # A file in your repository to which the UI state is saved 
    # Currently only a single file is supported 
    files:
      - GIRAFFE/keras.json
    # A list of nodes to load into the editor
    # The path is either relative to the root or a full URL
    nodes:
      - https://raw.githubusercontent.com/TimVanMourik/keras/giraffe-tools/keras_nodes.json
    # You can load your own JavaScript code generator. Documentation will follow soon
    # The path is either relative to the root or a full URL
    grammars:
      - language: 'Keras'
        script: GIRAFFE/test.js
        format: python

Now the good part is that once you built a nodes library and a grammars code interpreter, anyone can build pipelines with your software! Anyone can create a new repository and build a custom workflow with your software that is insightful and immediately shareable (by a simple Github ‘fork’).

Getting started

How what do these files look like? Examples of how to programmatically create a nodes file (Python and MATLAB code) are included in https://github.com/GiraffeTools/Libraries. It requires a JSON specification of your toolbox, the categories in which you may want to subdivide all your functions, a list of nodes (functions) that you want users to be able to drag and drop into the editor, and a list of their input and output ports:

{
  "toolboxes": [
   {
    //specify the name of the toolbox, e.g. Keras
    "name": "Keras",
    //specify a list of categories within the toolbox
    "categories": [
     {
      //specify the name of the category
      "name": "core",
      //specify the nodes within that category
      "nodes": [
       {
        //specify the name and attributes of a single module
        "name": "Dense",
        "category": "core",
        "toolbox": "Keras",
        "web_url": "https://keras.io/layers/core/Dense",
        //add any set of parameters that are required to generate
        //your specific code.
        "code": [
         {
          "language": "Keras",
          "argument": {
           "name": "Dense",
           "import": "from keras.layers import Dense"
          }
         }
        ],
        //add input and output ports.
        "ports": [
         {
          "name": "units",
          "input": false,
          "output": false,
          "visible": true,
          "editable": true,
          "code": [
           {
            "language": "Keras",
            "argument": {
             "kwarg": false,
             "arg": 0
            }
           }
          ]
         },
         ...

These nodes are loaded into the editor to drag’n drop, regardless of whether a grammars code interpreter is specified. Separately, you could work on this interpreter, which is a JavaScript file. In its simplest form it contains a writeCode and writeFiles function. The former translates code the pipeline graph into code, the latter specifies to what file in your repository it is written, in case a user uses the ‘Save to GitHub’ functionality.

module.exports = () => {
  async function writeCode(nodes, links) {
    return "I am creating code here!";
  }
  async function writeFiles(nodes, links) {
    const myFilename = 'GIRAFFE/my_code.py';
    return {
      [myFilename]: await writeCode(nodes, links)
    }
  }
  return {
    writeCode,
    writeFiles,
  }
}

This is plain JavaScript that receives the (nodes, links) props, based on which you can write more elaborate code for your specific purpose. Examples are included in https://github.com/GiraffeTools/CodeGenerators.

Project page

As a bonus, there is the project page in which all your commits to a specific project are listed. This way, you can open the workflow at any previous point in time. You can also take a look at code from different branches of our repository.

Open source

The platform is built to create your own open source analysis pipelines. The GiraffeTools code itself is also entirely open source: https://github.com/GiraffeTools/GiraffeTools! Stars, forks, bug reports, feature requests, or contributions are much appreciated!

Documentation

This article is just an introduction of GiraffeTools. More documentation and Medium posts will follow! This is all work in progress and in active development! Any feedback is welcome!