DEV Community: goyder

Running (ML) in the 90's

goyder — Sat, 13 Feb 2021 05:27:56 +0000

In this series, I'm documenting my experiences with attempting to write and execute a machine learning program (any machine learning program) in Python 1.6 on a NeXT hardware emulator running NeXTSTEP.

Goal

We're almost there. NeXTSTEP is running. Python's compiled.

Our end goal is in reach: we now want to write a basic ML script for Python 1.6.

It's time.

Challenges and decisions

What "ML" do we want to write?

So here's the first question: what "machine learning" script or program should I develop?

First, we should consider computational power is limited. NeXT machines ran at double-digit megahertz speeds. Whatever algorithm we choose needs to be relatively simple.

This is not to mention that even relative to our limited computational power, Python is going to be slow. Nowadays, with supercomputers in our pockets, I think we often forget that Python is a "slow", high level language. When you are working on early 90s hardware, this point will be clear. (So GPU-enabled deep neural networks are off the table for now.)

Ultimately, I decided to implement the logistic regression algorithm. It's a classic binary classifier - often one of, if not the, first taught - and I love its elegance of transforming the classic linear model into the odds ratio into a probability.

For a visual refresher of the whole process, consider this video.

Getting the data

For data, I ran with the classics - I decided I would work with the Iris dataset. This is a data set most often used for multiclass classification, so we'll have to convert it into a binary classification problem.

A copy of the dataset is readily availble from the UCI Machine Learning Repository, and is easy to transfer to Previous with our NFS.

A very basic, but very functional development environment

Key question: if we're going to write code in Previous, how are we actually going to write it?

Interestingly, NeXTSTEP doesn't seem to have an IDE in our modern sense. There's Project Builder, which is certainly intended to be the "hub of application development", but this doesn't have a dedicated text editor environment - it offloads that duty to the "Edit" text editor utility.

If we're just working in text editors, we might as well default to the classic development environment: multiple terminal windows running a shell and Vi. (Vim isn't available by default on NeXTSTEP.)

I wish this looked cooler. I'll have to figure out how to make it neon green text on a black background.

Writing in earnest

With the "development environment" up and going, I got to work on writing the scripts:

1-d-split-dataset.py, to transform our dataset into a form ready for binary classification.
logistic_regression.py, containing the main logistic regression model class.
2-m-logistic-regression.py, to train the logistic regression model on the dataset and use it on some test values.

Adapting to Python 1.6

If you look at the code in the above files, it definitely looks like modern Python. Most of the hallmarks are there - indentation, clean syntax, all the things we love.

There are definitely a few major and minor differences in Python 1.6, though. In particular I noted:

numpy and pandas are sorely missed. (numpy apparently existed for Python 1, but I could not find a copy anywhere. pandas was years off in the era of 1.x.) There's not a strong system for carrying out linear algebra or numerical calculations, hence the preponderance of map and lambda calculations and the minimally-implemented matrix multiplication (really just the dot product) in the logistic_regression.py script.
There's a surprising number of "utility" features not yet implemented - things like list comprehensions, the csv module, and even the zip function are absent and missed!
There's all kinds of minor differences - there's no import <module> as <name> functionality, for example. If you want to import a module, you better like its name, mister!
And of course who could forget when print wasn't a function?

Overall, while it's missing a lot of modern conveniences and there are some syntax differences, it's surprisingly quick to get moving in, provided you have the documentation open.

(I still desperately miss numpy and pandas, though.)

The proof in the pudding

Okay. Final proof. If we call the entry point to our machine learning script, will it run?

Click through for the stunning conclusion!

Yes! Albeit pretty slowly. Regardless, it was wonderful to see these scripts running to completion after all of the hard work and tweaking required to get to this point.

What next?

There's nothing more!

In this first article of this series, I set out to a challenge with three components:

Write and execute a machine learning program;
In Python 1.6,
Compiled from source in NeXTSTEP.

The challenge has been achieved, and this silly trophy has been claimed!

Overall, this has been a really fun experience and a good way to spend a couple of weekends. Besides satisfying my curiosity around NeXT and NeXTSTEP, I've certainly picked up bit of technical know-how as well - in particular, I've broadened my horizons around software compilation (modern and otherwise) and *nix disk management and mounting.

That's all there is - I hope you've enjoyed this series, and I hope you can use my materials provided in these articles to pursue your own NeXT adventures.

Series review

Where have we got to?

At this point we have:

Compiled Previous
Found pre-built images of NeXTSTEP.
Successfully mounted and launched an image of NeXTSTEP 4.3.
Set up a fileshare so we can easily get files onto the system.
Found an image of the NeXTSTEP developer tools.
Installed the developer tools we so have the capability to build Python from source.
Downloaded the source code for Python 1.6 and transferred it onto the NeXT machine.
Compiled a (hacky, slightly incomplete) version of Python 1.6.
Wrote a Python 1.6 script to carry out a logistic regression script.

How did we get here?

In this sesh we:

Downloaded the classic Iris dataset and transferred it into the Previous environment via the NFS.
Found the Python 1.6 reference documentation.
Wrote a script to transform the raw dataset into a form suitable for our logistic regression script.
Wrote a logistic regression model that allows for training and prediction.
Successfully ran the logistic regression model script in Python 1.6 in NeXTSTEP!

What could we explore further?

NumPy, Python's most critical array based tool, apparently existed in one form or another in Python 1.6. It'd be interesting to find and install a compatible version.
Speaking of installing packages, Python 1.6 was a time before pip (and before easy_install, for that matter) - it would be a good exercise to try managing this Python package environment in its earlier form, if only to gain an appreciation for how good we have it now.
It would be fun to explore building a ML tool in NeXT's in-built developer tools, to really get the authentic NeXT ML solution!
If we deployed Previous instance into a Docker container and somehow set this up via an API, we could truly take the claim of the very worst Dockerised ML solution ever (not to mention this would be a fun and interesting technical challenge).

Compiling Python from source, for fun and frustration

goyder — Wed, 10 Feb 2021 14:03:59 +0000

As of the last post, we have:

Previous, the NeXT emulator, up and running.
NeXTSTEP 3.3 up and running.
An NFS server up and running, and the NeXT machine connected to it.
The NeXTSTEP DEVELOPER environment installed and functional.

We're now at the pointy end - it's time to attempt to compile Python 1.6 from source.

Goal

Our goal for this session is simple and clear: compile Python 1.6 from source, and confirm the interpreter works.

Challenges and decisions

Getting the source

The source code for Python 1.6 is actually still available on the main Python website, although it's not well advertised nowadays.

Interestingly, the download page for Python 1.6 has some broken HTML, and it's not possible to click through to agree to the license agreement. However, the download page for Python 1.6.1 is still functioning, so I went with that.

Using the NFS previously set up, it was straightforward to move the source tarball into Previous, unpack it, and start to look through the README.

It's always valuable to look at the README.

Getting going...

Flicking through the README, I note that there's special installation instructions for each platform, including NeXT... however, there's nothing other than a note about "fat binaries" (a term which prompted a Google from me, but not much else).

The build instructions are straightforward: first, I just have to run the script ./configure, which I promptly do. This runs through a huge number of checks, all of which are satisfactory:

And then I just have to run ./make, which begins strongly:

Look at those compilations fly!

...and stopping...

Pretty quickly after compilation, it halts with the following error:

Seems like there's an issue with the NeXTSTEP DEVELOPER provided C standard library implementation - apparently they've mixed in some Objective-C. This is maybe not surprising (Objective-C was the primary language of choice for NeXT, and for Apple products from OS X onwards until Swift came along), but I am a little surprised it would be implemented in such a way to crash regular C compilation.

Fortunately, the fix has been provided in the error message (that's friendly!) and that can be implemented under compiler options in the very well organised Makefile, changing

# Compiler options passed to subordinate makes
OPT=       -g -02 -OPT:0limit=0

# Compiler options passed to subordinate makes
OPT=       -g -02 -OPT:0limit=0 -ObjC

With this change made, calling make has the compilation process buzzing along!

...and stopping again...

However, this doesn't last. Once we get to the Modules section of the compilation process, it bombs out on a compilation error - not enough arguments for a C function, it seems.

I'm not terribly familiar with C compilation, so I'm really coming up the learning cliff on this one, and I take a bit deep dive. After a fair bit of investigation I find:

Something's crashing in the _locales module. The code is making a call to strcoll and crapping out, saying there's not enough arguments. That feels like a very blatant oversight for the reasonably-mature-at-this-point Python codebase, so I take a closer look.
In the Python codebase, I find the line where it's dying, and confirm there's a call to strcoll with two arguments, s1 and s2. The call in _localmodule.C is perfectly normal - totally textbook! This is a bog-standard implementation of a bog-standard C function.
I then dug into the string.h file installed with NeXTSTEP DEVELOPER and it's... odd? The function has a totally different signature: int strcoll(char *to, size_t maxsize, const char *from) This maxsize argument seems to come out of nowhere, and is borking the install.
Looking up this specific implementation, the only reference I find to this signature is in the book "Software Engineering in C", which notes that it "documents the proposed ANSI standard C, which is expected to be ratified in 1987."

I looked up the ANSI standard C implementation of strcoll, and maxsize was nowhere to be seen.

Or, summarising these findings up in a chain of "whys":

What a journey.

Overall, something's a little awry here - I would love to find out the history of this strange implementation, because I could find naught in my searching.

A limiting, but effective, workaround

Thinking there might be a solution in the README, I went back and re-read it. After reading from top to bottom, I found an interesting message for installing under SunOS 4.x:

When using the standard "cc" compiler, certain modules not be compilable because they use non-K&R syntax. You should be able to get a basic Python interpreter by commenting out such modules in the Modules/Setup file, but I really recommend using gcc.

This sounds good. We're certainly getting errors due to non-standard syntax (although perhaps not like they meant).

I'd previously looked into gcc, but couldn't find a non-NeXT provided implementation for NeXTSTEP. However, a "basic python interpreter" could certainly be sufficient.

Sure enough, I was able to jump into the Modules/Setup file and begin commenting out the troublesome modules as they cropped up during compilation. (Interestingly, it seems there's a lot of intended variation in what modules to install, based on the notes within this file.)

This feels janky, but effective. I can only hope the finished product is sufficient for my needs.

Hello, snek

Slowly but surely I made progress until it is complete: the make process ran to completion (approximately 1 hour of uninterrupted compilation time) and I am greeted with a single blessed python executable file.

I run it, and I am greeted with the beautiful sight of a Python interpreter (and one minor error):

[GCC neXT DevKit-based CPP 3.1] on next3_3 - I'd like to think that's a compilation message that isn't often seen.

I ran some basic tests, instantiated and worked with a variety of data types - the interpreter seems good and functional. Success!

What next?

With the Python interpreter up and running, the only thing left to do is write the basic machine learning script in mid-90s era Python and claim victory!

Series review

Where have we got to?

At this point we have:

Compiled Previous
Found pre-built images of NeXTSTEP.
Successfully mounted and launched an image of NeXTSTEP 3.3.
Set up a fileshare so we can easily get files onto the system.
Found an image of the NeXTSTEP developer tools.
Installed the developer tools we so have the capability to build Python from source.
Downloaded the source code for Python 1.6 and transferred it onto the NeXT machine.
Compiled a (hacky, slightly incomplete) version of Python 1.6.

How did we get here?

In this sesh we:

Pulled the Python 1.6.1 source code from the Python website
Transferred the files onto the NeXT machine via our NFS
Found a "workaround", albeit a limiting one, for our issues with compiling Python 1.6 from source
- By commenting out troublesome modules from the Modules/Setup file, we are able to progress and complete compilation.

What could we explore further?

The guidance for compilation noted that the removal of modules might not be necessary if using a bog-standard gcc implementation, rather than NeXT's custom version. It could be worthwhile to find an implementation of gcc for NeXTSTEP 3.3 - I've seen enough rumblings on the NeXT forums to think this is possible.
There are updates available for NeXTSTEP 3.3, and potentially the NEXTSTEP DEVELOPER codebase. Applying these updates may help solve the non-standard C libraries.

Installing the hottest development environment of 1995

goyder — Sun, 07 Feb 2021 12:50:45 +0000

At this point, I've managed to get the NeXT machine emulator going, found and launched NeXTSTEP 3.3, and even connected it a file server on my local network. We're almost ready to start compiling Python 1.6 from source - we just need to make sure we have the tools to do so.

Goal

Our goal in this session is to get some standard developer tools in NeXTSTEP installed so we can compile Python 1.6 from source.

Challenges and decisions

What "developer tools" do we need?

The standard implementation for Python is "CPython" - this is the Python programming language written in C and Python code. (For interest, there are various other implementations such as PyPy or Jython that are worth exploring.)

CPython 1.6 is what we aim to install, so we'll need make sure we have some version of the standard C development tools installed.

...Do we already have them?

If you bought the $4,999 developer edition of NeXTSTEP/OPENSTEP, it shipped with "NeXTSTEP DEVELOPER", a second disk that had all of the NeXT development tools (see section 3.5 here for more details). This disk includes a precompiled instance of GCC, include files, linker libraries, and all of the custom NeXT software that made developing software 10x faster (according to NeXT's marketing team). These tools should be what we need.

However, none of the these programs and utilities are currently installed. This makes sense - the pre-built disk image I used to NeXTSTEP 3.3 was totally fresh and appeared to be at the first config step, so it's up to me to install those tools!

Finding NeXTSTEP DEVELOPER

How do we get our hands on NeXTSTEP DEVELOPER? Well, pretty much the way we got our hands on NeXTSTEP - search through Abandonware sites.

Very quickly I was able to find an ISO (disk image) of NeXTSTEP DEVELOPER on WinWorld, the same site that hosted the pre-built disk image of NeXTSTEP 3.3 I'm running. Thanks, WinWorld!

Installing NeXTSTEP DEVELOPER

We've got an .ISO file sitting on our harddrive - now we have to get that into our virtual environment.

Mounting...

First, we need to mount the image into our environment. To do so, we'll use the SCSI connection options within Previous to create a virtual disk drive, and then mount the .ISO file. Our SCSI connections should look like this:

Effectively giving us a pretty standard machine, with a harddrive and CD drive attached:

NeXT recognises the CD and it shows up as a new drive, just like on a modern Mac:

...and installing

With the disk ready to go, we can now install the development tools. It turns out this is super straightforward - under the path NextCD/Packages/ there are four .pkg files:

DeveloperDoc.pkg
DeveloperLibs.pkg
DeveloperTools.pkg
GNUSource.pkg

And when we open these files, they load in NeXT's in-built Installer app, making installation a breeze. Just make sure to launch them as a user account with sufficient permissions via the OpenSesame utility.

Not knowing what order these should be loaded in (and noting that others were unable to find much information on install order, either) I went ahead and started installing in the following order DeveloperTools, DeveloperLibs, GNUSource, and finally DeveloperDoc.

Each installation was successful on the first run (provided I remembered to run with sufficient permissions!):

All in all, a surprisingly smooth process.

A final test to make sure this works

To make sure that we actually have a working set of C development tools working, let's write the classic stub of "Hello World" and make sure it compiles.

We'll write a minimal helloworld.C:

#include <stdio.h>

int main (void) {
    printf("Hello, world!\n")
    return 0;
}

And fire off the compilation command and verification to check that it works:

localhost> cc helloworld.C -o hello
localhost> hello
Hello, world!

And as we can see from the following screenshot - all good!

We've passed a basic smoke test and can venture into deeper waters.

What next?

With a working C compiler, we can now attempt to compile Python 1.6 from source.

Considering the scope of our "Hello, world" test run, this is a bit like going from building a blinky LED circuit to constructing a Volkswagen Mini, but fortune favours the bold!

Series review

Where have we got to?

At this point we have:

Compiled Previous.
Found pre-built images of NeXTSTEP.
Successfully mounted and launched in image of NeXTSTEP 3.3.
Set up a fileshare so we can easily get files onto the system.
Found an image of the NeXTSTEP developer tools.
Installed the developer tools so we have the capability to build Python from source.

How did we get here?

In this sesh we:

Located an ISO of NeXTSTEP DEVELOPER (here)
Found a variety of useful information sources on it, including this very thorough FAQ.
Mounted the NEXTSTEP DEVELOPER disk and installed the relevant software using NeXT's standard install processes, running them with elevated permissions using OpenSesame.

What could we explore further?

We've barely scratched the surface of the NeXTSTEP DEVELOPER tools. These tools are the technical grandparents of all modern Apple development tools, and it would be fascinating to step through.
- This video gives a good walk through of some of the tools (and happens to cover a lot of the same content of this series!)
I'd like to suss out the ideal text editor - I've been happily working in Vi via the terminal but I'd be surprised if there wasn't a preferred, reasonably well featured text editor built into the NeXTSTEP OS.

Networking through the ages

goyder — Sun, 07 Feb 2021 01:45:31 +0000

In the last article, I managed to finally launch a copy of NeXTSTEP 3.3 on the Previous emulator - which is rad, because:

That's one of the three goals I had! (I'll take my dopamine reward now, please.)
It's very enjoyable to use the NeXTSTEP operating system (OS). I'm not to do a tour of NeXTSTEP because there is lots and lots and lots of wonderful content in this space, but needless to say, it's very neat. It feels robust and full-featured in a way older OSes often... don't.

Pictured: many neat things happening.

However, there is a key bit of functionality we're missing: file transfers.

Goal

Our goal for this session is simple: figure out a simple way to be able to move files from the outside world into my NeXTSTEP instance.

This is going to be required to get files from the internet into the NeXTSTEP environment (like the Python source code, for example).

Challenges and decisions

Why not directly connect to the internet?

Interestingly, it might have actually been possible to find a web browser, connect Previous to the internet, and go download the files directly. However, I was wary of this approach for two reasons:

Safety - this "machine" is decades old, and while I doubt there's too much malware for NeXTSTEP floating around, I don't want to take this ancient, unsecured device outside.
Pragmatism - The world wide web has moved on a lot since 1994. Even if I just accessed 90s era FTP servers, I suspect this would be painful (...but interesting - perhaps this is an experiment for another day)!

Come back to me, o internet of the 90s!

What are the options for transferring files?

Okay, so what else could I do? A few paths come to mind:

"Removable media"

If I was working with a physical machine, the simplest way I would get files to and fro would be via some kind of removable media - CDs, floppy disks, external harddrives, for example.

Now, because I'm working with a virtual machine, I could probably do the same with virtual removable media. Previous offers all the media connection options of an actual NeXT device, mainly via emulating SCSI connections.

Using these SCSI connections, I see two main options:

Removable drives - I could create virtual hard-disks or (shudder) floppy disk images that I could write to on my Ubuntu host machine, load into Previous, and access within NeXTSTEP.
I could also create ISO files to pass files back and forth, very much as if I was burning them onto CDs.

Neither are super appealing options, to be honest - both will require a lot of shuttling back and forth of imaginary "removable media"; I'd prefer to leave that in the 90s. But we could give the hard-disk option a go.

Trying anyway

The most straightforward path I saw was to build a "fake" disk drive, mount it in Ubuntu, and write the relevant files to it; I could then mount it to the NeXTSTep instance.

The main challenge is file systems and partitioning - things have changed between 1994 and 2020!

NeXT is fully featured, and is happy to format a blank disk presented to it; but mounting the consequently formatted disk in Ubuntu is painful because its auto-file-system-detection doesn't readily recognise NeXT's own ancient file-system.

I think this approach has legs, but I decided to explore other options in the meantime.

UPDATE - I eventually cracked this! The key thing is to manually set the file system for mounting to a specific NeXTSTEP option. Unfortunately, this only allows read-only access, which means that this approach is not suitable for our needs. Regardless, example code is as follows:

losetup -f  # find next loop device, e.g. /dev/loopXX
sudo losetup /dev/loopXX NS33_2GB_dup.dd  # assign to loop
sudo mount -t ufs -o ufstype=nextstep /dev/loopXX /mnt/next

Network file systems

Previous had two files in its core directory of promise: networking.howto.txt and filesharing.howto.txt. The former walked through how to connect NeXTSTEP to your local network; the latter how to set up a Network File System server on a Mac and connect the machine to it.

"Surely not," I thought. "No way I could spin up a file server on my local network on a 2018 Mac and have this 26 year OS connect and talk to it happily."

The proposed network solution.

It turns out that, in fact, you can.

A "brief" unsuccessful diversion

The instructions, filesharing.howto.txt, were a guide on how to set-up a NFS on a separate Mac OS X. I do have a Mac, but it made sense to me (and would be far more convenient) to set up the NFS on the Linux machine which Previous was running on.

A desired alternative network solution - a bit more elegant.

However, after a few hours of configuring NFS server options on my Linux machine and combing through the NeXTSTEP forums for guides, I was stuck, with no file transfers to show for it. NeXT simply could not see the NFS server on the same machine.

I still think this should be possible, but the problem could be in Previous, or network emulation, or in connecting a NFS client to a server running on the same machine - honestly, there were too many paths for me to go down to even identify an obvious next option, so I sighed, admitted defeat, and went back to the guide.

Getting this going

After following the instructions that came with Previous, I was able to connect to my Mac from Previous in under 10 minutes. Ha! This was helped along by the fact that NeXTSTEP actually has inbuilt utilities to connect to NFS servers:

Pictured: functional and friendly utilities.

And I was able to happily create a textfile on my Mac and access it via NeXT - a process that felt very similar to inter-generational trading in Pokémon, if we're to be honest.

It's slow as hell, but functional.

What next?

Well, there we are: I can now get files into NeXTSTEP quickly and easily through a fileshare. In particular, this opens the door to accessing the source code we need to install Python.

Series review

Where have we got to?

At this point we have:

Compiled Previous.
Found pre-built images of NeXTSTEP.
Successfully mounted and launched an image of NeXTSTEP 3.3.
Set up a fileshare so we can easily get files onto the system.

How did we get here?

In this sesh we:

Connected Previous to the local network, using the provided instructions.
Set up a NFS server on a Mac and then connected to it from Previous, again using the provided instructions.

What could we explore further?

How much can we connect a NeXT machine to the internet? Can we get a mid-90s era web browser installed and see what's accessible?
- Seems there's some interest in this...
I never did give the virtual floppy/CD/HD approach much of a go - the latter would be a great activity with lots to learn about images, mounting, and filesystems.

Abandonware and You: Installing Yesterday's Technology of Tomorrow, Today

goyder — Thu, 04 Feb 2021 13:03:53 +0000

As outlined in the previous article, I've managed to locate, compile and launch the emulator for the NeXT hardware, charmingly named Previous.

At this point, it's a bit like the point where you've bought and assembled your PC but not yet installed the operating system (OS). What and how are we going to install on this fresh machine?

Goal

The high level goal here is to get some instance of NeXTSTEP up and running on Previous.

That sounds relatively straightforward - Previous is, after all, a emulator designed to run NeXTSTEP, so it shouldn't be that hard, should it?

We'll find out.

Challenges and decisions

Defining hardware vs software and the little inbetween

The first thing I found I had to get clear in my mind was what Previous was, and what it wasn't. This was a little fuzzy because of what you get when you first run Previous.

When you boot Previous with the absolute default settings, you are greeted with a very limited command-line interpreter - is this the boot-up of NeXTSTEP? Is this our OS?

Nah, it turns out - this is the "ROM Monitor".

This is essentially the BIOS of the device. This is best described as firmware - low level software that talks close to the hardware. We can use this to boot into our OS.

Thus, we have:

Previous emulating the hardware,
Previous running the ROM monitor firmware,

But we are missing an OS to run on top of these.

How do we get an Operating System?

The first question one might have is "where do you get 30 year old operating systems?", and the answer is, unsurprisingly, "the internet".

NeXTSTEP comes under the broad banner of "abandonware" - software that is ignored by its owner and manufacturer (the fact that NeXT went defunct 24 years ago may explain this). Crucially, this means that this software will often be archived and freely shared online, which is exactly the case here.

I found a few good options to choose from, including a source that provides boot floppies and another source that provides pre-built hard-disk images. In both instances, we have all kinds of versions of NeXTSTEP to choose.

What version of NeXTSTEP to install?

I've been describing "NeXTSTEP" as if it was a single OS, but the fact is that it can refer to many different versions, from the first public release of 1988 (NeXTSTEP 0.8 - not even 1.0!) to the final version from 1997 (OPENSTEP 4.2).

A huge amount of development went on during this time, and hardware and software compatibility varied, so picking the right version is important.

After doing a bit of digging, I decided on NeXTSTEP 3.3 to start. This was the last version under the "NeXTSTEP" name, and was apparently quite mature and trusted - a lot of the references I see in software and READMEs are to this version.

From scratch or pre-built?

Okay, if we're going to run with NeXTSTEP 3.3, how do we want to go about doing so? We actually have a couple of options:

First, we can get copies of the original boot floppies and do a completely fresh install on a totally empty "hard-drive" using Previous - this is akin to putting in a fresh, blank hard-drive into your new computer and doing a fresh install from the system CD/USB.

Secondly, we can download an image of a hard-drive that already has NeXTSTEP installed and boot that - a bit like putting in a hard-drive from your old machine with Windows already installed.

Ultimately, I went for the latter option - while I found some guides for installing from scratch, this path seemed fraught with challenges that may be NeXT and may be the emulation - a recipe for disaster. Let's take the "safe way" and see how that goes.

Download and mount

I picked the "NeXTSTEP 3.3 HD" download from WinWorld and away I went. (The fact that there have only been 72 downloads of these images in total via WinWorld makes me think I'm down a pretty esoteric path.)

Inside the compressed file, you'll find a file called NS33_2GB.dd - this is a pre-built image with NeXTSTEP 3.3 installed to the point where it's ready for you, the user, to start configuring and using it. It also includes a robust README that explains in very welcome detail exactly how to configure Previous to get the image up and going (neatly moving past the confusing ROM monitor from above).

When I followed those instructions, the boot went smoothly and I was greeting with a config screen:

And, following the initial configuration and booting up the Mailbox app, I was greeting with the "famous" welcome email from the man himself, Steve Jobs:

Looking good - NeXTSTEP 3.3 is up and running! Now to start using it...

What next?

With a fresh install of NeXTSTEP, we now have to figure out how to use it.

Series review

Where have we got to?

At this point we have:

Compiled Previous.
Found pre-built images of NeXTSTEP.
Successfully mounted and launched an image of NeXTSTEP 3.3.

What did we find?

In this sesh, we:

Found a good source of NeXTSTEP boot floppies
- Importantly, we also found a guide to install those boot floppies under Previous.
Found a good source of NeXTSTEP OS and resource images

What could we explore further?

Other versions of NeXTSTEP - how different is 0.8? What about 4.2?
Building from boot floppies - how hard would it actually be to a fresh install from scratch if we wanted to?

Moving forward with Previous

goyder — Thu, 28 Jan 2021 12:55:29 +0000

In this series, I'm documenting my experiences with attempting to write and execute a machine learning program in Python 1.6 on a NeXT hardware emulator running NeXTSTEP.

As I outlined in the first article, my motivations carrying out this project are basically:

Curiosity
Interest in picking up technical skills

Today's article is going to be more focused on the latter motivation, as I figure out the best way to run the 30-something-year-old NeXTSTEP operating system on my current machine (a Linux desktop with 9-ish year old hardware, for reference).

Goal

Our purpose here is relatively straightforward: let's explore our options for emulation platforms to let us run NeXTSTEP, pick one, and get it going.

(I'd love to actually carry all of this out via an actual NeXT machine, but their age, importance, and the fact that not very many were actually built or sold means that it's very difficult to find one.)

Challenges and decisions

What platform to use?

One of the key challenges I encountered straight off the bat is trying to figure out what hardware I want to emulate. Two options jump out at me:

First, there is a emulator available that replicates the NeXT machines, charmingly called "Previous". (Points for that.)
- I'd actually very briefly played around with this years ago but gave up quickly as I didn't really know what I was doing, and the learning curve was significant.
The other option is to run one of the later versions of NeXTSTEP that were made available on non-NeXT hardware. (These versions were known as OPENSTEP, of course not to be confused with the API spec OpenStep) Notably, "non-NeXT hardware" means x86 hardware, meaning it can be run under VirtualBox.

The documentation and guides in both cases seem... dicey. I daresay there'll be interesting debugging regardless.

Depicted: debugging.

Reviewing both of the options, I decided to go with Previous because:

It's the most true to the spirit of this project (it's the closest to authentic hardware, and the non-Intel chipset it's emulating makes this even more pointlessly quixotic!)
The VirtualBox option is only available for later versions of NeXTSTEP which were ported to x86 chipsets.
It does seem to have the most support, features, and, most importantly, active forum community.

Seriously, an active and devoted forum cannot be understated for its value in troubleshooting.

Getting Previous installed

Having decided on Previous, I jumped into figuring out how to get it running on my machine. Off the bat, I was a touch nervous, primarily because the latest news article on the homepage was from 2016, and that started with "very long time since last updated this homepage...". This didn't scream "strongly supported and frequently updated" to me, but at least they're at a 1.X version.

With no prebuilt builds available to download, I set about following the instructions on how to build from source. Again, these instructions are old, but they do look technically possible, so adapted them into script and give it a go. (The script is available below.)

I type ./install.sh into the shell and begin a few minutes of tense waiting while the compilation process proceeds...

Watching intensely, no significant errors crop up...

Stay on target... stay on target...

And bang. The build executes flawlessly first time, to my amazement. I launch the still-warm, freshly built executable and am greeted with a blank launch menu:

I've succeeded. Now I just have to explore what it is I've succeeded in doing.

What next?

Okay, it seems like we have a hardware emulator going - now we have to get some software to run on it.

Stay tuned.

Series review

Where have we got to?

At this point, we have:

Compiled Previous.

(As this project proceeds, this trail of breadcrumbs will be more useful.)

What did we find?

In this sesh, we:

Found the Previous website, and
Adapted the install script.

What could we explore further?

Other emulation options - how well do the Virtualbox emulators work? Are there significant differences?
What's the feasibility of the online implementations? If I write a ML algorithm in one of the Docker-run systems have I created the very worst containerised ML solution possible?

Writing machine learning code on 20 year old software on 30 year old hardware

goyder — Mon, 25 Jan 2021 14:23:38 +0000

In this series, I'm going to document my experiences with attempting to write and execute a machine learning program in Python 1.6 on a NeXT hardware emulator running NeXTSTEP.

Two points up front:

This is a pointless, silly thing to do.
I hope you end up enjoying the ride as much as I do.

There's a good chance that the challenge outlined may be mostly nonsensical to you, and it's not clear why it's a silly thing to do.

Let me paint the broad strokes.

The goal

The challenge I've outlined has three components:

Write and execute a machine learning program;
In Python 1.6;
Compiled from source in NeXTSTEP.

Machine learning

To be clear, the easiest part of this challenge be the machine learning component. I'm very much of the school of thought that the term 'machine learning' covers a broad church which includes the foundational but insanely useful algorithms like linear and logistic regression. These are relatively simple algorithms to implement in any programming environment.

Call this part of the challenge an excuse to frame this whole exercise as "an exploration in the worst possible way in writing ML".

Python 1.6

Python 1.6 - What's interesting about that? Well, Python is a wonderful, fully-featured and robust language - and it's certainly my go-to language, especially in the realm of data analysis and machine learning. Personally, I began using Python in its venerable Python 2.7 version, before making the jump to various 3.x versions full-time over the last five years.

But Python is a surprisingly old language, with its first release being in 1991, or fully 30 years ago!

Warning: may not be an official Python document.

The language has made incredible leaps and bounds in power and functionality even the limited time I've been using it. I'm curious to rewind the clock 21 years to its 1.6 release and explore the idiosyncracies of these early releases.

NeXT and NeXTSTEP

Finally, perhaps the most challenging to explain: NeXT and NeXTSTEP. NeXT was a hardware and software company that existed in the mid 1980s-90s; their core software product was an operating system (OS) called NeXTSTEP. There is a lot to unpack around NeXT, but there are three main reasons why I'm intrigued:

NeXT was the baby of Steve Jobs, and is most often associated with his "second act" between being cast out of Apple and eventually rejoining; it's a period of his life often brushed over, perhaps because of just how poorly he managed NeXT. This is charitably referred to as the time when he made mistakes and learnt to be a leader. NeXT has a long and dramatic history which I cannot pretend to do justice; it's definitely worth exploring if you're interested in computing lore.
Drama aside, the NeXTSTEP OS was genuinely innovative for its time; it was an impressive technical achievement and regarded as a powerful development environment. NeXT machines and NeXTSTEP were associated with many technological touchstones of the 90s: Doom and the World Wide Web (pick your preferred order of importance) were both developed on NeXT products.
Finally, there's a good chance you still use a descendent of NeXTSTEP. NeXTSTEP eventually formed the basis of Apple's Mac OS X, and consequently MacOS, iOS, iPadOS, etc. NeXTSTEP is history, but its lineage goes on.

Also, NeXT had a really neat logo (which of course has an associated story).

But... why?

Why would I burn my limited free time on a silly project like this? That's a very legitimate question. There are two primary reasons.

Pragmatically, technological broadening and skill development is a common and welcome outcome. I always find that carrying out a silly challenge like this introduces a bunch of unexpected technical challenges, and solving them broadens my horizons greatly. With this challenge, I expected I'm going to learn a good amount around compiling programs from scratch and manipulating disk images -- and who knows what else.

To quote Elting E. Morison, we should consider "the story of the three princes of Serendip who set out to find some interesting object on a journey through their realm. They did not find the particular object of their search, but along the way they discovered many new things simply because they were looking for something." In other words - set an arbitrary goal, accomplish other things along the way.

But my main driver is not personal development - it's curiosity. Here is an opportunity to dive backwards in time to explore and use tools and portions of "history". I want to see what was saved and what was lost.

Seeing systems and objects in their "prototypical" forms has long been a passion of mine; as a child, I spent a lot more time exploring the Sonic the Hedgehog 2 prototype community than I ever did playing the completed game!

P.S., did you know they just recently found a Sonic 1 prototype after two decades of searching?

Future articles and next steps

(Pardon the pun.)

Future articles are going to cover the numerous steps on the road to achieve the goals set out above.

Because this is going to knit together a lot of publically available resources, future articles will include some main sections:

Challenges and decisions, covering some of the surprising bumps in the road I assume I will encounter,
A series review, covering what's been done so far,
A summary of findings including all of the key links and resources I've hit upon, and;
Things to explore further, noting things that I was curious about but didn't dig into more.

Hopefully these will help form useful resources for any folks who want to dig around in the same sandpit as me.

See you next time!

Automatic Reporting in Python - Part 3: Packaging It Up

goyder — Sat, 29 Dec 2018 15:08:22 +0000

As outlined in my previous posts (Part I and Part II available on this fine website), the goal of this project is to make an automatic reporting tool.

In this series of guides, the outcome I'm shooting for is a single HTML page that allows me to interrogate and compare the output of machine learning models.

At the conclusion of the previous tutorial, the reporting tool was actually showing some genuine use! It could accept a number of .csv summaries of machine learning summary files and output a single .html page that presented the information is a form that was... functional.

There three main features that I'd like to add to the tool for now:

The report looks incredibly dull. Readability counts! We need to improve the a e s t h e t i c s of the report.
It's hard to dig into the tables - currently it's just 100 rows presented with no tools to search. Some basic search functionality would be grouse.
The datasets that we want to run the report with are currently hard-coded into the script. This needs to be split out to make the tool slightly more flexible.

A brief comment

Compared to the previous posts, this post is much more of an exploratory, learning experience - this post represents a neophyte's attempt to build a working tool, rather than a beautiful depiction of all that is possible. If you can see a much better approach for anything listed in this post, please feel free to share it with myself and all the other readers!

But without further ado, let's take a crack at improving the aesthetics.

Step Seven - Improving the Aesthetics

Those of you with an understanding of HTML pages might know what comes next: Cascading Style Sheets, known more commonly as CSS.

Taking your first steps down a path

As indicated above, the challenge in the context of this tutorial is that this is an enormous field that I personally am not actually particularly well-versed in, having only dabbled and hacked in this space. However, I am familiar with learning new things.

So, if this is your first real introduction to CSS, let me take you down the same path I would recommend in learning any new tech:

Do some background reading on the fundamentals of CSS. Hit up Wikipedia. If you're keen, hit up the CSS standard! Never be afraid to Google "simple introduction to [topic]."
As you read and explore, make note of potential good resources of future information. I would encourage everyone to take a look for an "Awesome List" relevant to your topic - and in case, the Awesome CSS List is here.
With a basic understanding of what the hell is going on under your belt, play around to your heart's content with local files and implement as much as you can in local scratch files. In this case, create small (or large?) HTML pages and figure out how to structure the CSS neatly. I find this really helps to get to know some of the practical realities and challenges of working with the language before I start diving into frameworks.

Being a bit more mercenary

For self improvement, I find nothing beats spending the time working up solutions from scratch. Of course, if you're trying to hack together a solution for a business need, it may not behoove you to spend hours and days coming up with an elegant CSS framework from the beginning.

Instead, we may like to quickly jump off of someone else's CSS framework. Fortunately, there are a number of these available, with a great number of them focusing on being 'minimal'. A quick Google search should get you started down this path.

Integration

How is the CSS file to be integrated our report? There's a few options that are typically at play:

Download the CSS file and keep it as external file. The advantage is that we have our .html and .css files neatly separated, and we have full control over both; the far more significant disadvantage is that now if we want to move our report around, we have to drag a bunch of .css files around with it.
Use a Content Delivery Network (CDN) copy of the CSS file. Most frameworks will offer a CDN link for their file: this is essentially a link to an efficient, readily available copy of the data. The advantage is that you can get a CSS up and going in your page just by dropping a single link in the <head> section of your .html, no mussin' and fussin' with local files. The disadvantage is that you don't have control of the file, and an internet connection is required.
A slightly more complex option is to have local copies of the CSS file, and then write them into the .html file. This could probably be done relatively easily and sustainibly if we got clever with our templating. The advantage is that we'd have our report in a single file, and it wouldn't require an internet connection to use; the disadvantage is that it is going to require a bit more effort to get set up. (This is commonly used approach when creating standalone versions of interactive pages. Write a Jupyter Notebook to .html, inspect the file, and you'll find all the CSS and JavaScript magic packaged up in the <head> section.)

At this early stage of prototyping, I prefer to use CDNs if possible. The advantage of being able to swap CSS frameworks just by changing a single line of code and not having to bother with local files is worth the cost of not being able to play with and edit the framework. Optimisation (in the form of being able to automatically integrate the CSS into the .html file) can come a little later.

To start with, I'm going to use Milligram, a lightweight little framework. To get started using the CDN method, I simply follow the provided instructions to integrate into the CDN. Under our templates/report.html file, I'll add the requisite links into the <head> section:

report.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>{{ title }}</title>
    <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Roboto:300,300italic,700,700italic">
    <link rel="stylesheet" href="//cdn.rawgit.com/necolas/normalize.css/master/normalize.css">
    <link rel="stylesheet" href="//cdn.rawgit.com/milligram/milligram/master/dist/milligram.min.css">
</head>
<!-- body section continues below... -->

And all of a sudden, our plain, early 90's looking webpage has been transformed into something a bit more pleasing to the eye:

But we note there's something still not quite right here - primarily, why does the page (and the table in particular) always take up the whole width of the window? Why doesn't this look right on mobile?

It turns out just adding a bunch of .css files isn't quite enough. We need to make sure the layout of our .html pages match what's expected by the .css layout.

HTML layouts

Like most topics in this space, the layout of your HTML page is a reasonably intuitive concept, while simultaneously being a problem that you spend years diving into. What makes it a bit more challenging is that despite there being a number of somewhat fragmentary explanations, I've struggled to find a simple and/or holistic to the field (although this explanation is currently my favourite gentle introduction, and the Mozilla guide appears to be quite thorough).

For brevity, I'm going to leave most of the further reading to you, the reader (sorry!), and instead focus on what Milligram expects.

If we inspect the code of the Milligram page, we'll see that within the <body>, we can see the HTML of the site is structured roughly as:

<body>
    <main class="wrapper">
        <header class="header">
            <section class="container">
        <section class="container">

Now, based on some of the readings in the above links, and assuming that this is the structure that Milligram is expecting, we can apply the same structure to our own report, giving us something like the following:

report.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <!-- Note the addition of the viewport! --> 
    <meta name="viewport" content="width=device-width, initial-scale=1.0, minimal-ui">
    <title>{{ title }}</title>
    <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Roboto:300,300italic,700,700italic">
    <link rel="stylesheet" href="//cdn.rawgit.com/necolas/normalize.css/master/normalize.css">
    <link rel="stylesheet" href="//cdn.rawgit.com/milligram/milligram/master/dist/milligram.min.css">
</head>
<body>
    <main class="wrapper">
        <header class="header">
            <section class="container">
                <h1>{{ title }}</h1>
                <p>This report was automatically generated.</p>
            </section>
        </header>
        {% for section in sections %}
        {{ section }}
        {% endfor %}
    </main>
</body>
</html>

summary_section.html

<section class="container" id="summary">
    <h2>Quick summary</h2>
    <h3>Accuracy</h3>
    {% for model_results in model_results_list %}
    <p><em>{{ model_results.model_name }}</em> analysed <em>{{ model_results.number_of_images }} image(s)</em>, achieving an
        accuracy of <em>{{ "{:.2%}".format(model_results.accuracy) }}.</em></p>
    {% endfor %}
    <h3>Trouble spots</h3>
    {% for model_results in model_results_list %}
    <p><em>{{ model_results.model_name }}</em> misidentified <em>{{ model_results.number_misidentified }} image(s)</em>.</p>
    {% endfor %}
    <p><em>{{ number_misidentified }}</em> misidentified image(s) were common to all models.</p>
</section>

table_section.html

<section class="container" id="{{ model }}">
    <h2>{{ model }} - Model Results</h2>
    <p>Results for each image as predicted by model <i>'{{ model }}'</i>, as captured in file <i>'{{ dataset }}'</i>.</p>
    {{ table }}
</section>

We had already structured this report to be a collection of largely independent collection of sections - we were even using this terminology! - so it's not a huge drama to add the <section> tags to the system.

Proof that this works

Run autoreporting.py and inspect the report - try it both at full-screen and simulating a mobile screen.

Progress! That's the benefit of using a well-made responsive layout.

GitHub status

Oooft. That was a lot of background reading for not a huge amount of code. All the same, the project should look like this.

Step Eight - Making Our Tables Interactive

Okay. So now we have our tables, and they look pretty good - the challenge is that they're not interactive. For instance, it would be wonderful to have the functionality to filter by a category, or drill down to a specific image.

Now, as with exploring CSS, we have a couple of options. Certainly, we can explore the option of creating all of this ourselves - there are plenty of examples around, and they're not terribly difficult. But, if we're being pragmatic (or pressed by business needs!) we can probably find a pre-built package of what we need.

After a bit of googling, I came across DataTables - this is a plugin for JQuery, a very common JavaScript framework. DataTables looks like it covers most of the functionality we need, and provides a wealth of extensions and plugins for any of the functionality we don't have. All in all, a promising candidate.

Implementing DataTables

Fortunately, it turns out that implementing DataTables is relatively straight forward. From the front page of the DataTables site, we can see that the general principles are that we need to:

Although not spelled out explicitly - DataTables is a JQuery plugin - so first we need load the Jquery .js file.
We'll then load the DataTables .js and .css files.
Finally, we call the DataTables function, pointing it at the HTML id of the table we want to add the functionality to.

That's all relatively straightforward, with just some very solvable wrinkles:

Our tables don't have id tags to refer to.
We need a way to call the DataTables function and point it at the id of the table, in a way that fits with our templating system.

Let's address these one by one.

Importing the relevant files

Before we get to our wrinkles, let's hit our basics. As we imported our files from CDNs previously, we'll do the same for JQuery.

Let's update the <head> section of templates/report.html and add the links:

report.html

<!-- More above! -->
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>{{ title }}</title>
    <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Roboto:300,300italic,700,700italic">
    <link rel="stylesheet" href="//cdn.rawgit.com/necolas/normalize.css/master/normalize.css">
    <link rel="stylesheet" href="//cdn.rawgit.com/milligram/milligram/master/dist/milligram.min.css">
    <link rel="stylesheet" href="//cdn.datatables.net/1.10.19/css/jquery.dataTables.min.css">
    <script src="https://code.jquery.com/jquery-3.3.1.slim.min.js"></script>
    <script src="//cdn.datatables.net/1.10.19/js/jquery.dataTables.min.js"></script>
</head>
<!-- More below! -->

Adding `id` tags to the tables

We need to add an id tag to the <table> objects in our report. How can we do this?

Well, let's work backwards from the templates/table_section.html file. Within that file, we note that we insert our fully-formed HTML tables via the {{ table }} insert.

The {{ table }} insert is generated in autoreporting.py when we call the get_results_df_as_html method of ModelResults class. This method takes the pandas DataFrame and converts it into a string of HTML using the DataFrame.to_html function.

If we inspect the docs for that function, we see that there's an optional argument table_id. Ah yeah, cool man! If we pass the model name as that argument, the HTML table will be generated with the id that we want. The ModelResults class already has model_name as an attribute, so we can include that:

class ModelResults:

    # ...

    def get_results_df_as_html(self):
        """
        Return the results DataFrame as an HTML object.
        :return: String of HTML.
        """
        html = self.df_results.to_html(table_id=self.model_name)
        return html

You can run autoreporting.py and inspect the tables that are generated to confirm that they do indeed have the model name as an id.

Easy! Just a matter of tracing it back from the end result of HTML to the actual source within the code.

Calling the DataTables function

We need to call the DataTables function listed above, pointing at the appropriate id as we've just generated. The actual code to call the function is pretty straightforward. The question is: where do we put it?

This challenge has a couple of bounding constraints on it.

We need to call the function to generate the DataTable using the model name - something like this:

$(document).ready( function () {
    $('#VGG19').DataTable();
} );

Traditionally, JavaScript is placed either in the <head> section, although this is a somewhat controversial discussion. Note that this is tradition - it will actually run anywhere.

This is a challenge because of the structuring of our template. We want to place the JavaScript call for DataTables in <head>, which is in templates/report.html. Right now, when report.html is rendered under the main() function in autoreporting.py, it doesn't know anything about the model names: the render() call only has arguments for title, the overall title of the report, and sections, a list of pre-rendered strings of HTML ready to be inserted into the document. We just need to modify autoreporting.py to pass in the model names and tweak report.html accordingly. We tweak our files thusly:

autoreporting.py

def main():

    # ...

    # Production and write the report to file
    f.write(base_template.render(
        title=title,
        sections=sections,
        model_results_list=[vgg19_results, mobilenet_results]
    ))

report.html

<head>
    <!-- Lots of calls above... -->
    <script>
        {% for model_results in model_results_list %}
            $(document).ready(function() {
                $('#{{ model_results.model_name }}').DataTable();
            } );
        {% endfor %}
    </script>
</head>

Bingo-bango: when we render report.html, the calls to render the DataTable functionality for each existing table is included. Nice!

And now, if we run autoreporting.py and inspect the output, we get something like this:

We can now order our tables, search for categories, filter by image names - we have some rich functionality available, with more available via extensions.

This is a huge advantage for a report! Imagine if you only wanted to check out the common themes of incorrect image categorisations, or rapidly narrow down on a specific image.

GitHub status

Your repo should look a little something like this.

Step Nine - Packaging it up

AKA Step the Last.

The good news: we have the functionality we want and need. We can take a .csv file or two and punch out an interactive report.

The bad news: we've hardcoded it to two files, VGG19_results.csv and MobileNet_results.csv, which limits the functionality.

The final step for this exploration is therefore to turn this hard-coded script into a tool that can be called from the command-line. We want to be able to call the report and an arbitrary number of .csv files and have the report spat-out. So if we called our script and specified the relevant .csv files, we'd get a report successfully written to the /outputs folder - looking a little like this on the command line:

$ python autoreporting.py VGG19_results.csv MobileNet_results.csv
Successfully wrote "report.html" to folder "outputs".

This can be accomplished by utilising command line arguments - or, to put it rather simply, the commands that follow the call to Python. (In the example above, the first argument is autoreporting.py, our script. The second and third command line arguments are VGG19_results.csv and MobileNet_results.csv, respectively.) We have a couple of main ways we can approach this:

We can crunch the arguments manually, using sys.argv. There's absolutely nothing wrong with this approach, sys.argv is really quite simple to use.
We can use a Parser like argparse, primarily to assist in generating help and error messages.

Because I've not used argparse before, I'm interested in giving it a go and testing it for these purposes.

Implementing `argparse`

Most everything we want to work with in argparse is handled within the main() call within autoreporting.py. To make this work, we're going to:

Define and parse the arguments we're interested in (specifically, filepaths to the results .csv files), using argparse;
Convert these filepaths into ModelResults objects that we can use to generate our reports;
Adapt our existing code to output reports using these ModelResults objects.

So, first of all, we need to make sure that argparse is imported. It's been part of the standard library since Python 3.2.

autoreporting.py

import argparse
# ...

Next, within main(), we'll define the parser - we're saying how we want the command line arguments to be interpreted. This code is adapted pretty quick smart from the demo in the argparse docs. We actually only have one argument, by how argparse defines it - just the filepaths to our results .csv files. The key thing to note is that we set the nargs argument to "+", indicating that we can have a undefined number of arguments of this kind, but we do need at least one.

When we call parser.parse_args(), all the arguments are neatly returned as a Namespace object that makes the inputs very easy to access, as we'll see in the following steps.


# ...

def main():
    """
    Entry point for the script.
    Render a template and write it to file.
    :return:
    """
    # Define and parse our arguments
    parser = argparse.ArgumentParser(description="Convert results .csv files into an interactive report.")
    parser.add_argument(
        "results_filepaths",
        nargs="+",
        help="Path(s) to results file(s) with filename(s) '<model_name>_results.csv'."
    )
    args = parser.parse_args()

From args, the Namespace object, we can pull out the filepaths and use them to generate ModelResults objects.

args.result_filepaths holds a list of our filepaths, which we have indicated should point at filenames in the format <model_name>_results.csv. We use the os.path module to manipulate this filepath, extract the model name, and generate the ModelResults object, adding it into a list called model_results as we go.

This filename manipulation can look a little tricky, but inspect the doccies of os.path and you'll see it's mostly clever string manipulation. os.path is full of very, very useful functions that can save you a lot of time with common path manipulations, and help your code to work cross-platform!

    # Create the model_results list, which holds the relevant information
    model_results = []
    for results_filepath in args.results_filepaths:
        results_root_name = os.path.splitext(os.path.basename(results_filepath))[0]
        model_name = results_root_name.split("_results")[0]
        model_results.append(
            ModelResults(model_name, results_filepath))

The logic for the set intersection - how we figure out which images are common across all results files - has be changed to account for the fact that we now have an arbitrary number of ModelResults objects in a list.

To make this work, we quickly extract the misidentified_images property of each object using a list comprehension, and then calculate the intersection of sets based on this resulting list. (Note that we have to use a leading asterix (*) when we call set.intersection() so that each member of the list gets passed in as an individual argument).

    # Create some more content to be published as part of this analysis
    title = "Model Report"
    misidentified_images = [set(results.misidentified_images) for results in model_results]
    number_misidentified = len(set.intersection(*misidentified_images))

Everything below this point is relatively consistent with our previous version, but now we're taking advantage of the fact that we have our ModelResults objects already packed up into the model_results list.

    # Produce our section blocks
    sections = list()
    sections.append(summary_section_template.render(
        model_results_list=model_results,
        number_misidentified=number_misidentified
    ))
    for model_result in model_results:
        sections.append(table_section_template.render(
            model=model_result.model_name,
            dataset=model_result.dataset,
            table=model_result.get_results_df_as_html())
        )

    # Produce and write the report to file
    with open("outputs/report.html", "w") as f:
        f.write(base_template.render(
            title=title,
            sections=sections,
            model_results_list=model_results
        ))
    print('Successfully wrote "report.html" to folder "outputs".')

Oooft! With all of the explanations, this looks a little complex. However, when you compare this code to the previous commit, you'll see there's not a great deal that's actually significantly different here - we've really kept the core principles the same and just played with the packaging a bit.

GitHub status

Your project should look a little like this.

The End?

At the very start of the first post, I indicated that the goal of this project was to create an automatic HTML reporting tool, where the outcome was a single stand-alone HTML file, with info and interactivity.

Well, it's done! We've got a tool that can accept an arbitrary number of standard results files, and spit out a report that crunches them into an interactive format.

Take a breather, push your chair away from your desk, and pat yourself on the back. We've done what we set out to do!

Does that mean we're done? That depends, really.

What comes next?

At this point, we have a tool that works for a very narrow use-case, and assumes perfect inputs and perfect operation from the user. Now, if you're using a tool like this just for yourself, and the inputs to the tool are quite consistent, then that could in fact be perfectly satisfactory - so no more work to be done, you've got something that's fit for purpose.

But of course, there are any number of ways we can work to extend and harden this tool. As I was writing this tutorial, I made notes on some of these. Running losely from simpler to more complex, here are a few notes and ideas:

Could we add a default command-line argument with argparse that allows us to specify the title of the report?
Our .csv inputs need to be named in a perfectly consistent format. How can we restructure our inputs so that we can define the model name and not have it read from the filename?
How can we make this a command-line script that can be run anywhere on our machine - not just in the folder the script is in? If our data is generated and stored elsewhere, it would certainly be more useful to be able to call autoreport on the terminal, rather than trace back to where the script is stored, for instance.
In its current form, we're analysing images - can we add functionality to show the images we're analysing? This would be great for generating hypotheses for why a model failed.
Our reports need an internet connection each time they're open. As part of the templating process, could we pull down the JavaScript and CSS files and embed them into our files?

This is a tiny sliver of the possible extensions - and this is not even to mention that there's plenty of refactoring and tidying to be done across the project. The job of improving your work is never done!

A thankyou and a call to action

This has been the first tutorial of this scope I've ever written. I've had a lot of fun doing so, and to paraphrase Sigur Rós, this has been a good beginning.

But I'm really keen to hear what parts of this you, the tutorial-reader, enjoyed and what parts were challenging or obscure. Feel free to drop a comment or send me a message on what worked and what didn't.

See you next time!

Automatic Reporting in Python - Part 2: From Hello World to Real Insights

goyder — Wed, 13 Jun 2018 14:03:46 +0000

As outlined in my previous post, the goal of this project is to make an automatic reporting tool.

In this guide, the outcome I'm shooting for is a single HTML page that allows me to interrogate and compare the outputs of machine learning models.

At the conclusion of the previous tutorial, we'd gotten up to the "Hello, World!" stage. Some very basic content could be fed into our very basic structure - but now it's time to add some real features.

A note on tutorials and design choices

Rather than just providing a dump of code that works, I'd like to incrementally work through the development, step-by-step.

Personally, I've always found this kind of tutorial more effective as:

It gives an insight of how someone else approaches a problem
It actually reflects how I code - feature by feature, commit by commit.

When I was totally fresh to programming, I worked from a large number of tutorials. The structure of most tutorials gives you the impression that there is one and only one way to accomplish the goal that the tutorial sets out to do - and, admittedly, sometimes there is, depending on how niche and technical the tutorial is.

But more often I've found that any programming job, even the very small, requires choices and trade-offs to be made in the implementation of the solution from the get-go. Should this be a function or a class? Do I import a library or roll my own? Should I be using a dictionary or a list? What kind of flow-on effects are these choices going to have?

One approach to simplify things: mark your assumptions

Now, there's obviously no simple solution to this challenge. However, one approach to making the process of understanding easier is being clear on the assumptions underpinning your project.

For instance, in this project:

I am the only consumer of these reports, and therefore;
My users (me!) will not be hostile
A minimum of error-checking is required, as I can interrogate and troubleshoot myself
An internet connection is available
Portability is not required

Most of these are simplifying assumptions that serve to make clear exactly how thorough or involved my solution needs to be. OF course, as soon as you start looking at providing your code to others, or in an enterprise situation, or if a higher degree of automation is required, the game can change significantly!

It makes the ecosystem in which this tool will exist a bit clearer. When I get stuck and I'm not sure where to go, I'll refer back to these assumptions and see if I've not answered my own question somewhere.

Step Three - Generate some data to work with

As mentioned in Part 1 of this Post, there are a host of use-cases for this tool that come from the machine learning space.

In this case, we'll consider the following scenario:

We have an existing, baseline model that determines whether an image contains a cat.
We regularly develop new models that carry out the same task. These models may or may not be an improvement on your baseline model.
Every time we develop a new model, we run it over an existing set of data and capture the outputs. In this case, the outputs are what the model believes to be in each image.

It would therefore be convenient to be able to:

Automatically review these outputs and get a status report on how the new model did.
Automatically compare the performance of these models against each other. Where did one outperform the other? Are there images that both struggle with?

For the purposes of this study, we'll assess the performance of two models in the well respected field of detecting cats in images. VGG19, the first model, is a well respected and important image classifier. MobileNet is a significantly more lightweight architecture (running approximately six times faster than VGG19, for example) but with reduced accuracy.

The use of these models to generate the data can be found under a separate repo, available here. While the process of generating this data is a bit beyond the scope of this discussion, the end-result is we have two datasets to play with - VGG19_results.csv and MobileNet_results.csv - each stored as CSV files.

For each image in the dataset, we have a column for the predicted ImageNet category ID (imagenet_index), the text description of the category (imagenet_category), and whether this corresponds to a housecat (correct).

Step Four - Set up a structure to put things in

Well - after our digressions - back to the reporter. When we left off, we had the reporter in a functional but featureless state:

Let's make the next move by adding a basic stucture to the report that we can populate with more sections of content. First, we'll update report.html. We'll make some minor tweaks, like specifying an actual title point to be entered and adding a brief preamble, but the real interesting part is below that:

report.html

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>{{ title }}</title>
</head>
<body>
<h1>{{ title }}</h1>
<p>This report was automatically generated.</p>
{% for section in sections %}
    {{ section }}
{% endfor %}
</body>
</html>

We're limited to inserting blocks of text into our template - Jinja allows control flow structures! In this case, we'll feed in some iterable variable sections - note the plural - and then we'll publish each section object within.

But what are we to include in these sections? As a first run, we can generate a section for each model that displays a brief bit of information about the model output, and then displays the results. Let's create a new template, table_section.html,
that we'll use to render this information.

table_section.html

<h2>{{ model }} - Model Results</h2>
<p>Results for each image as predicted by model <i>'{{ model }}'</i>, as captured in file <i>'{{ dataset }}'</i>.</p>
{{ table }}

Nice and simple - this gives us all the structure we need.

Before we get to inserting the tables into the document, let's see what this gives us if we populate it with some dummy data. We'll update autoreporting.py to a) refer to the new updated terms in the updated report.html template, and b) render and input some sections so we can check out the new template.

autoreporting.py

from jinja2 import FileSystemLoader, Environment

# Configure Jinja and ready the loader
env = Environment(
    loader=FileSystemLoader(searchpath="templates")
)

# Assemble the templates we'll use
base_template = env.get_template("report.html")
table_section_template = env.get_template("table_section.html")

# Content to be published
title = "Model Report"
sections = list()
sections.append(table_section_template.render(
    model="VGG19",
    dataset="VGG19_results.csv",
    table="Table goes here."
))
sections.append(table_section_template.render(
    model="MobileNet",
    dataset="MobileNet_results.csv",
    table="Table goes here."
))


def main():
    """
    Entry point for the script.
    Render a template and write it to file.
    :return:
    """
    with open("outputs/report.html", "w") as f:
        f.write(base_template.render(
            title=title,
            sections=sections
        ))


if __name__ == "__main__":
    main()

When we run the script, the two sections we rendered with the table_section.html template will be included in our report, one after the other.

Here, we only have two sections, and they're of the same kind - but we can imagine it'd be no concern to string together more models, or generate blocks that compare performance between models, to suggest just some examples.

GitHub status

At this point, your project will look something like this.

Step Five - Insert the tables into the document

It's time to extract and insert our tables into the report.

The basic approach I've suggested is to use pandas to read in the .csv file as a DataFrame object (the workhorse class of pandas). From this DataFrame object, we can readily export it as HTML using the to_html method. Our modified version of autoreporting.py therefore looks as follows:

autoreporting.py

import pandas as pd
from jinja2 import FileSystemLoader, Environment

# Allow for very wide columns - otherwise columns are spaced and ellipse'd
pd.set_option("display.max_colwidth", 200)


def csv_to_html(filepath):
    """
    Open a .csv file and return it in HTML format.
    :param filepath: Filepath to a .csv file to be read.
    :return: String of HTML to be published.
    """
    df = pd.read_csv(filepath, index_col=0)
    html = df.to_html()
    return html


# Configure Jinja and ready the loader
env = Environment(
    loader=FileSystemLoader(searchpath="templates")
)

# Assemble the templates we'll use
base_template = env.get_template("report.html")
table_section_template = env.get_template("table_section.html")


def main():
    """
    Entry point for the script.
    Render a template and write it to file.
    :return:
    """

    # Content to be published
    title = "Model Report"
    sections = list()
    sections.append(table_section_template.render(
        model="VGG19",
        dataset="VGG19_results.csv",
        table=csv_to_html("datasets/VGG19_results.csv")
    ))
    sections.append(table_section_template.render(
        model="MobileNet",
        dataset="MobileNet_results.csv",
        table=csv_to_html("datasets/MobileNet_results.csv")
    ))

    with open("outputs/report.html", "w") as f:
            f.write(base_template.render(
                title=title,
                sections=sections
            ))


if __name__ == "__main__":
    main()

The main thing that jumps out at me here is that this is starting to get fairly involved for a script. To keep things on the straight and narrow, I'm applying the following rough flow for ordering:

Imports to start
Run any configuration code required
Define any functions required
Run any standalone code
main() function and call to run it at the end of the script.

This raises the question: what should be standalone and what should go in main()? As a rule of thumb, code that we only want executed when we explicitly run the script goes in main(); code (like classes, definitions, objects) that we might like to use elsewhere go in the body, available for import elsewhere. You can find a good many explanations of this functionality around, but I find this to be a thorough discussion of the concept.

Some other minor points on the above:

Note the call to widen the max_colwidth setting via set_option for pandas at the start of the script. Without this, pandas will silently apply some funny formatting if any column exceeds fifty characters. Increasing the max_colwidth is a trick that has only been learnt through painful experience.
There's repetition as we call table_section_template.render() - note that the filename of the same file is entered manually twice, violating the principle of Don't Repeat Yourself. Would it be worthwhile to tweak this so the filename is automatically split out from the filepath?

With this implemented, we can re-run the script, and voila - tables!

GitHub status

If you're playing at home, your project will resemble this.

Step Six - Include a summary and comparison

Well, we have the raw data displayed, but we haven't really answered our key question: which model performs better? Using our 'section' based approach, let's develop a new 'summary' section.

Two questions that often come to mind when comparing models is:

How accurate is each model?
Are there patterns in how the models are inaccurate - in this instance, are there images that all the models get wrong?

This moves beyond simply displaying the raw data: instead, we're starting to draw inferences and make summaries, marking a significant increasing in the complexity of the code. I find this situation tends to induce a kind of analysis paralysis if I'm not careful: I can spend too long thinking about how best to approach the problem to ensure the solution is neat and elegant. Do I dive straight into the code? Do I write a new template? How do I know if the code as it exists now is under or overcooked?

One way to focus this process is to very clearly articulate our end-goal, allowing us to focus on figuring out the most effective way there. So, before diving straight back into the Python script, I spent five minutes scribbling down how I'd like this section to read, and settled on some explicit phrases that I was happy with. After some scribbling, I ended up with this target:

Summary block prototype

Quick Summary

Accuracy

Model 1 analysed 100 images, achieving an accuracy of 97.2%.

Model 2 analysed 100 images, achieving an accuracy of 94.2%.

Trouble spots

Model 1 misidentified 10 images.

Model 2 misidentified 12 images.

8 images misidentified images were common to all.

This seemed straightforward, and implies a neat translation into Jinja template structuring - it makes the next step feel obvious. So with this target clearly in mind, I could easily produce the template:

summary_section.html

<h2>Quick summary</h2>
<h3>Accuracy</h3>
{% for model_results in model_results_list %}
<p><em>{{ model_results.model_name }}</em> analysed <em>{{ model_results.number_of_images }} image(s)</em>, achieving an accuracy of <em>{{ "{:.2%}".format(model_results.accuracy) }}.</em></p>
{% endfor %}
<h3>Trouble spots</h3>
{% for model_results in model_results_list %}
<p><em>{{ model_results.model_name }}</em> misidentified <em>{{ model_results.number_misidentified }} image(s)</em>.</p>
{% endfor %}
<em>{{ number_misidentified }}</em> misidentified image(s) were common to all models.

This template assumes some that I'll produce a list of some objects model_results, which in turn contains properties like model_name, number_of_images, and accuracy. I'll also need some value number_misidentified, referring to the number of images misidentifed across all models: this doesn't quite mesh into a theoretical, singular model_results object, so it stands alone. (For now.)

This seems plausible enough, and implies some useful information I can take back into the code, giving me a good structure to start with. If, as I start to work back into the code, I find that these early and rough requirements don't quite translate well or make things too difficult, I won't hesitate to feed design decisions forward and tweak them. However, in this instance, this looks to be fairly straightforward and I can't imagine any significant issues.

With this in mind, I worked back into autoreporting.py and started making some changes. By this point I feel autoreporting.py is getting a bit long to be posting every time, so I'll work by exception from here on out: you can find this relevant commit here.

Of perhaps greatest importance is the model_results object. I spent a little bit of time debating whether to implement this as a class, dictionary, or named tuple: in the end, I landed on a class. The primary reason for this was that I was ended up bundling the data and the methods in one: and when you have data and methods combined, a class is a good approach.

autoreporting.py - ModelResults class

# ...

class ModelResults:
    """
    Class to store the results of a model run and associated data.
    """

    def __init__(self, model_name, filepath):
        """
        :param model_name: Name of model.
        :param filepath: Filepath to results .csv.
        """
        self.model_name = model_name
        self.filepath = filepath

        self.dataset = os.path.split(filepath)[-1]
        self.df_results = csv_to_df(filepath)  # Filesystem access

        self.number_of_images = len(self.df_results)

        self.accuracy = self._calculate_accuracy()

        self.misidentified_images = self._get_misidentified_images()
        self.number_misidentified = len(self.misidentified_images)

    # Class methods not shown... 

# ...

Now, with this ModelResults class, we can call in our results files only once and have our relevant information readily available, solving the DRY problem mentioned back in Step Five.

autoreporting.py - main() function

# ...

def main():
    """
    Entry point for the script.
    Render a template and write it to file.
    :return:
    """
    # Content to be published
    title = "Model Report"
    vgg19_results = ModelResults("VGG19", "datasets/VGG19_results.csv")
    mobilenet_results = ModelResults("MobileNet", "datasets/MobileNet_results.csv")
    number_misidentified = len(set(vgg19_results.misidentified_images) & set(mobilenet_results.misidentified_images))

    # Produce our section blocks
    sections = list()
    sections.append(summary_section_template.render(
        model_results_list=[vgg19_results, mobilenet_results],
        number_misidentified=number_misidentified
    ))
    sections.append(table_section_template.render(
        model=vgg19_results.model_name,
        dataset=vgg19_results.dataset,
        table=vgg19_results.get_results_df_as_html())
    )
    sections.append(table_section_template.render(
        model=mobilenet_results.model_name,
        dataset=mobilenet_results.dataset,
        table=mobilenet_results.get_results_df_as_html())
    )

    # Produce and write the report to file
    with open("outputs/report.html", "w") as f:
        f.write(base_template.render(
            title=title,
            sections=sections
        ))

Now, we'll run our revised autoreporting.py, and:

A neat summary is what we get!

From this output, we can see:

Both models achieve approximately the same level of accuracy. This is surprising in and off itself - I would have expected VGG19 to be far more accurate.
We have a modest amount of overlap in the misidentified images - about 2/3s are common between the models. This suggests that those images are particularly challenging in some way. It may be worthwhile digging into these and examining whether these images are flawed or unsuitable in some way.

Or, to summarise the above dot points: we're actually getting useful insights from the report. Nice!

GitHub status

At this point, the project will probably look like this.

Next steps

We have a functioning report that produces useful information! That's a good start. However, there are many things to look at next, such as:

The datasets that we want to run the report are currently hard-coded into the script. How can we turn this into a more useful, flexible tool?
It's hard to dig into the tables in any meaningful fashion - the output is very static. Can we improve the model such that we can filter and tweak the data?
The report looks exceedingly dull. How can we spruce it up?

There's always more work to be done.

Automatic Reporting in Python - Part 1: From Planning to Hello World

goyder — Sat, 26 May 2018 09:49:52 +0000

I'd like to document and step through the execution of a simple concept in Python: creating an automatic HTML reporting tool.

The outcome would be a single stand-alone HTML file. An HTML file is a great reporting tool: even stripped of a back-end server, you can package up a good volume of info in a page and have handy-dandy interactivity.

Now, to be fair: this is not a terribly exciting or sexy project. All the same, automatically producing reports is a fantastic trick to have up your sleeve, especially in business environments. Automate the boring stuff, as they say.

Personally, I mainly intend to use this to automate reporting of model performance in the machine learning space, but you could of course use this for any context!

Naturally, tools like this already exist. But I want to make my own, for the experience and flexibility it offers me.

How is this guide presented?

Rather than just providing a dump of code that works, I'd like to incrementally work through the development, step-by-step. You'll see what I see - code what I code!

Personally, I've always found this kind of tutorial more effective as:

It gives an insight of how someone else approaches a problem
It actually reflects how I code - feature by feature, commit by commit.

Who is this for?

A user "knee-deep" in Python is assumed. I'll not run over standard syntax and the like, but I'll be sure to stop and explain anything particularly tricky. Wherever possible, I'll refer back to cleverer and clearer explanations.

Step Zero - Stop for a minute

I'm actually a chemical engineer by background. While I certainly gained and applied a lot of useful and/or esoteric knowledge from this background, by far the most useful bit of advice I ever learnt was probably also the most simple:

When approaching a new problem, draw a picture.

By "draw a picture", I mean to externalise the problem. Sketch it out and get thoughts and ideas out on paper.

Before tackling this (relatively small-scale) project, I sat down and spent 10-15 minutes sketching out how I thought I might approach it, what parts would move, what parts would stay static, and a (very) rough structure of the project.

My own nigh-incomprehensible sketch is as follows.

Some key learnings from early planning

Based on my early "planning", some things came clear:

An HTML templating system makes a lot of sense. These systems are designed to input dynamic content into static template: this sounds exactly like what we're after. In Python, Jinja is a common choice, and may be familiar to anyone who has played with Flask.
In the machine-learning space, I'm constantly finding new things I might want to look out for, or new ways to compare models, or new ways to analyse a dataset - so both the content of a report and the features I can pack into any report should be flexible.
A lot of the data I work with is in .csv format, so having tools to work with that and generate interrogatable HTML blocks from it would be neat.

Step One - Lay a foundation

For pretty much any size project bigger than tinkering in the command line for a couple of minutes, I'm fond of setting up a Github repo and a consistent environment.

This, I think, will be a pretty small-scale project, so I'm not terribly concerned with laying down a full project structure, but it's worth being comfortable with this for further reference.

This:

Encourages good habits in keeping the project clean
Helps me jump from my Mac laptop to my Linux desktop (or any other environment, for that matter) with a minimum of fuss

A few quick goals to start us off:

Set up a virtual environment

We want to create an isolated instance of Python for this project - or "virtual environment" - rather than using whatever general installation is present on this machine. This isolates our dependencies and makes it easier to move the project around. I'm working in Pycharm, which gives you the option to create new virtual environment from the start.

To make this transferable, I'll keep a requirements.txt file in the root of my project directory which details what packages are required in the environment.

There are many, many, many guides on this topic, and they're good to refer to for more specific details.

Set up a Git/Github repo

To implement version tracking and allow ready transport to and fro from Github, we'll initialise a Git repo locally (in the project folder) and create a new repo in Github. The local Git repo can then pushed to Github.

A succint guide can be found on Github.com.

As a matter of course of working in a Git/Github repo, we'll add the following to the root project directory:

A readme.md file that briefly describes the project, and
A .gitignore file that indicates which files and folders should not be tracked by Git and pushed to Github. This might include automatically generated files (from the IDE, or from the virtual environment, for instance), sensitive files and config data, or anything else you've created locally that doesn't need to be shared with the world. For me, this looks like:

# .gitignore

# Don't add the virtual environment, IDE, and Jupyter notebook info
venv
.idea
.ipynb_checkpoints
Write-up.ipynb

Github status

At this point, the skeleton of your project should look a bit like this.

Step Two - "Hello, World" templating

At this point, we have a very simple project structure that we can start populating with code and templates. To carry out a bit of a smoke-test and give me somewhere to build off of, I'd like to create the simplest possible report.

This first implementation should input no information, and output a simple "Hello, world!" HTML page.

A brief primer on templating

This project is built on the concept of templating and template processors. This is an enormous but very crucial field in web publishing.

In a crude explanation: we separate the structure of an HTML page and the content that is to be published. When it comes time to generate a new webpage (or in this context, a report!) we can input the content into a standardised structure and get a new webpage out.

Templating is well and truly ubiquitous now, but if you'd like an example of life without consistent manual or automatic templating, you can find many unironically charming examples in a Geocities archive.

Building 'hello, world' functionality

Building on this distinction between structure and content, we can create very simple examples for both to test the system.

Structure

Our structure, in this case, is a near-empty HTML page with locations for data to be input. We'll call this a "template", and we'll store this and others under a template folder in our root project directory. We'll create a new HTML file under this folder - {project_folder}/template/report.html. The contents of this file are as follows:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <title>{{ content }}</title>
</head>
<body>
{{ content }}
</body>
</html>

Note the elements here within curly braces (in this case double curly braces - {{ and }}). This is where dynamic content will be entered.

Template processors such as Jinja are very powerful and allow many features to be present in templates beyond simple insertion. Take a look at the first example on the Jinja homepage and see if you can make sense of it.

Content

Our content, for now, can be as simple as hello, world!.

We'll create a new file in the root of the project directory called {project_folder}/autoreporting.py and hardcode our 'content' to begin with.

content = "Hello, world!"

Functionality

We have structure and content - now we just need to get them combined using Jinja.

Firstly, create an outputs directory and add it to your .gitignore file. We'll dump what we create in this folder, but there's no reason to share whatever ends up in here in your repo.

Jinja requires an Environment object to store and define things like the config and loader - i.e. where we will pull our templates from. In this case, we're running off the local filesystem, so a FileSystemLoader specified to search in the templates directory will do us just fine.

With our env instance, we can render our basic template and write this to file.

Our autoreporting.py script now looks like this:

from jinja2 import FileSystemLoader, Environment

# Content to be published
content = "Hello, world!"

# Configure Jinja and ready the template
env = Environment(
    loader=FileSystemLoader(searchpath="templates")
)
template = env.get_template("report.html")


def main():
    """
    Entry point for the script.
    Render a template and write it to file.
    :return:
    """
    with open("outputs/report.html", "w") as f:
        f.write(template.render(content=content))


if __name__ == "__main__":
    main()

Note the argument to template.render, content=content. This links the template and our code. The first content refers to the {{ content }} in the template, while the second is the string defined in the script. We define this relationship and the string is fed into the template to make a substitution.

Results so far

If you run this script and all goes well, a new report.html file will be written under outputs/. Open this file and behold what you have created!

This certainly meets the criteria of "very simple, but functional."

If you inspect the HTML of the page, you'll note that the {{ content }} tags in the template were neatly replaced by the Hello, world! string, which is the crux of this affair.

Github status

You've developed a new feature, so it's a good point to commit and push to Github.

Your repo should probably look a bit like this.

Next Steps

The reporter walks and works, but we certainly don't have any sort of meaningful input or flexibility at this point. Our next steps will be to explore how we can read in data and report on it, and start working with JavaScript and CSS.