DEV Community: Daniel McMahon

2+ Years in Professional Development

Daniel McMahon — Wed, 22 Jan 2020 11:02:50 +0000

A follow up on my A Year in Professional Software Development article.
... also yay for 2+ years ...

A Year Later

So here I am a year and a bit on and find myself thinking back a lot over the last couple of years. It's been an interesting rollercoaster ride full of ups and downs.

The key points I highlighted in my older article still hold up and are things that are still relevant today as they were a year ago.

Seek the opinion & advice of your colleagues
You code as a team, not as an individual
Write tests & maintain documentation regularly
Make the working process work for you
Take the bad with the good
The difference between junior & senior is not all that much

That last point though I have more opinions on.

When you're no longer a 'Junior' developer (but also not a senior developer either).

I think it's fair to say at this stage that I'm no longer considered a 'junior' developer (something echo'd by my promotion midway last year in work). I think the distinguishing factor is that you develop a level of certainty to your uncertainty.

I can quickly review the functionality of an API or code library and be able to adopt it onto my codebase fairly quickly, I can write somewhat decent tests for my code where required, and I have an idea of how to prioritise bug fixes/tech debt over new features in my codebases.

More than anything though I have been through the process involved with creating, adopting, maintaining and decomissioning microservices. This has probably been the most valuable source of my learnings over the past year. Seeing how stakeholders can set expectations, how to go about building something that meets the outlined expectations, maintaining it and then opting to see if its worthwhile to maintain the services you build and how unexpected factors can influence decisions along the way.

Continuous Upskilling

I've made it a priority to maintain a certain level of upskilling throughout the past year (although notably less so than my first year). One of the more formal pillars of this would be my completion of the AWS Developer Associate certification. Having used AWS services on and off the last few years this was a nice qualification to obtain outside of work to help fight off the imposter syndrome that comes with the job. Shoutout to the folks at A Cloud Guru I definitely couldn't have passed it without their online udemy course.

I currently have my sights set on completing the Certified Kubernetes Application Developer program later this year.

Personal Projects

When it comes to personal projects I've definitely upped the ante. I went out and purchased a few domains and setup an ecosystem where all my past, present and future work can live - daniel-mcmahon.com. This was inspired by the work over at linktree.

I've updated my GitHub to a more memorable name /dan-mcm and I have multiple live projects including:

In addition to those frontend applications I also the the Inspiration API and my Music Programs which are overdue an update.

Going through the pains of setting up a static site with SSL certification through AWS CloudFront was a worthwhile exercise I definitely recommend to anybody else looking to expand their skillset.

I've taken back up the piano lately and find myself touring the Dublin train stations on a frequent basis - as a result I have a handy business card that I use to direct people towards my revamped Music Centre website.

In addition to providing a centralised hub for my projects, I also took the time to go through them, updating any security dependencies (nobody ever warns you of the maintenance required on old projects). In addition to this I gave the readmes a good polish and even added in some Emoji's and more impressive looking shields.io badges to make them look less 'clinical' - check them out over on my GitHub.

Maintaining Positive Mental Health

I think this is something that isn't talked about enough, and it's particularly important in the tech industry. Imposter syndrome is something that is rampant throughout the industry and it's sort of expected when you enter an industry where the tech is changing rapidly and it's a constant case of the rabbit (developers) chasing the carrot on the stick (new shiny tech).

I think it's important for developers to be realistic with their expectations of themselves. While there are scrum practices surrounding story points and delivery, there are other aspects that you need to maintain and be careful of.

Being kind to yourself is something that is hugely underrated, particularly when you see the portrayal of rockstar developers chugging energy drinks and coding till 2/3am in the morning. Improving is very much a marathon not a sprint. You need to ensure that your output is running at a sustainable rate, there's no point in burning yourself out during the first half of a quarter and being useless for the remainder. Pay attention to your mind and your body and learn when to take the foot off the accelerator.

Personal Perspective

One of my own key focuses with tech was to be a full stack developer, being able to understand how to orchestrate the backend while being able to put up a pretty facade. There are articles out there that talk about being a T developer, that is one that is good at most things but specialises in one very specific technology. I'm still searching for what that key technology might be, but I've found myself enjoying quite a lot of front end work (React/JS), building some tooling to help others write code (AppsScript, Python, Scala), and maintain good DevOps (K8, Docker, ZMON, Grafana) practices.

Tech is an industry where people tend to move on quickly from place to place. I see that around me a lot lately where after a 2 year period quite a few of the people I started with have either changed teams or left. I think thats just the nature of the industry - but it makes you consider your own options and if maybe its time to start a new chapter.

Resources

I often want to give back to the tech community but rarely know how, so here is an attempt at compiling a list of tools and useful resources I've used over the past year.

Shell/Mac

iTerm 2

IDE(s) & Themes

IntelliJ - Java/Scala
- Monokai theme
- Markdown Navigator
Atom - JS/React/YAML/JSON/Python
- City Lights theme
- file-icons
- prettier-atom
- pretty-json
- linter
- go-plus

Build Tools

Scala/Java - sbt
JavaScript/React - npm / yarn
Golang - Go Modules

JS (React) Libraries/Frameworks

For Multiple Routes in React

React Router

Styling Components

Code Styling & Linting

Authentication

Passport

Auth Providers

Browser Automation

Server Side

Express

Test Frameworks

Jest

Code Styling

Scala Libraries/Frameworks

Akka - AKKA HTTP Framework
Play - API Framework
Circe - JSON Parsing

Test Frameworks

Code Styling

Pre Commit Hooks

Pre Commit

API Tools

Databases

SQL
- AWS RDS
NoSQL
- Postgres
- DynamoDB

Documentation Generators

Chrome Plugins

NoCoffee - Useful for visual impairment design work
Axe - useful accessibility tool
Google Tag Assistant - useful when setting up Google Analytics tags
Github File Icons
* *

Scrum Stuff

Stock Photo / Icon Libraries

Color Pallets

Containers & Deployment Systems

Continuous Integration/Development

CMS Solutions

Squarespace
Shopify - this is amazing for e-commerce work - seemless integration with the likes of Printify and AliExpress. You need to check out this latest Dropshipping business trend.

Domain Registers

Streamers

YouTube

Twitch

Suz Hinton - noopkat - IOT

Final Note

I've had a particularly difficult year with personal circumstances but throughout that entire period work has been a solid foundation I could rely on both for support and drive, and I'm grateful towards all my coworkers for that.

I hope 2020 will be a better one, and that I'll get the opportunity to write more and possibly speak at a conference or two!

As always if you have any questions or queries don't hesitate to get in contact in the comments below.

In the meantime I look forward to doing the 3 year review with you!
Dan.

Building a Chrome Plugin - Image Swap it up!

Daniel McMahon — Tue, 05 Feb 2019 17:12:38 +0000

I'd like to start by apologising in advance for the ridiculousness of this Chrome extension, but hopefully it'll serve as a useful reference if you want to go build your own!

From Simple Intentions...

This project was initially intended as a useful reference in the event I ever needed to go and build a new Chrome extension. The basic goal and functionality of the project was to build a Chrome extension that when clicked would swap all the images on the page.

When starting out I had selected a series of GiantBomb images/gifs from the web as a starting point for what would prove to be a dumb inside joke. I wanted to be able to use web based images as well as local images in order to demonstrated the full functionality of building such an application.

Having recently seen the APIs you didn't know you needed article I couldn't help myself but to update the project to also be able to dynamically swap in Bill Murray & Nicolas Cage related images as well. Something I think a wider audience might appreciate.

The Implementation

When it comes to setting up a Chrome extension your starting point lies with the manifest.json. You can think of this as a similar version of a package.json for your standard NPM project.

{
  "name": "GiantBomb-ify",
  "version": "0.1",
  "manifest_version" : 2,
  "description": "Replaces all images with GB Staff",
  "icons": {
    "16": "logo-small.png",
    "48": "logo-small.png",
    "128": "logo-small.png"
  },
  "background" : {
    "scripts" : ["scripts/background.js"]
  },
  "browser_action": {
    "default_icon": "logo-small.png"
  },
  "web_accessible_resources": ["images/*.jpg"],
  "permissions": ["activeTab"]
}

The name, icon, version, description fields should feel somewhat familiar to NPM or React devs out there, they can be populated with pretty generic details. The other fields are worth explaining in a bit more details.

manifest_version: this is specific to the Chrome API, version 2 is the current default version that should be used.
background: this dictates what scripts should be running in the background once the chrome extension is enabled.
browser_action: this dictates the default action that should occur when the chrome extension is added to your browser, in this instance it simply sets a default_icon.
web_accessible_resources: this is required to allow public access to the images stored within the extensions codebase. Without this you may have difficulties accessing localised images.
permissions: this field limits the permissions of the chrome extension to only the active Chrome tab you have open.

The Scripts

As per the manifest.json the background.js script is the first one executed during the extensions addition to a new tab. The script is only being used here to trigger another file 'imageswap.js'. In a more generic setting the script coudl be used to add more functionality to the wider Chrome api and handle items such as firing multiple scripts or injecting global variables etc.

chrome.browserAction.onClicked.addListener(function(tab) {
   chrome.tabs.executeScript(null, {file: "scripts/imageswap.js"});
});

The logic of the imageswap.js contains some very basic JS.

var chromeExtensionId = "iaimgpoecjggdpdlmlhhidciphppgfcf"

var images = document.getElementsByTagName('img');

var gb_images = [
  "https://static.giantbomb.com/uploads/scale_medium/18/187968/2743737-yplezwp.jpg", //drew headset
  "https://pbs.twimg.com/media/BpxlFaHCYAIQvvY.jpg", //brad mohawk
  "https://static.giantbomb.com/uploads/original/7/72053/2671808-rorie.png", //rorie glasses
  "http://static.giantbomb.com/uploads/original/13/139155/2536312-2379004-jeff_headshake.gif", //jeff disapproval
  "http://static.giantbomb.com/uploads/original/17/179932/2783337-4310291077-prUNU.png", // dan creep
  "https://static.giantbomb.com/uploads/original/2/23893/2455096-1935536898-10875.jpg", // vinny alpha protocol
  "http://www.deepfreeze.it/img/j_headshots/alex_navarro.jpg", // alex smile
  "https://magnettheater.com/oldImages/headshots/large_abbyrussell_2017.jpg", // abby face
  "https://static.giantbomb.com/uploads/original/23/233047/2986879-2776158-0471975364-o4gps.gif", // jason approval
  "https://78.media.tumblr.com/e0b2ba4e1fd8b1323b4e9c88477db345/tumblr_oshzt0nUPs1u7bbkho1_400.gif", // ben disapproval
  `chrome-extension://${chromeExtensionId}/images/ryan.jpg` // local file example - ryan
]

for (var i = 0, l = images.length; i < l; i++) {
  var index = Math.floor(Math.random() * Math.floor(i%gb_images.length+1))

  // GB implementation (default)
  images[i].src = gb_images[index];

  // Nicolas Cage implementation
  // images[i].src = "https://www.placecage.com/" + images[i].width + "/" + images[i].height

  // Bill Murray Implementation
  // images[i].src = "https://fillmurray.com/" + images[i].width + "/" + images[i].height

  var prev = i
}

The chromeExtensionId variable is used for accessing local files, if you want to use local images you will have to update this value with your own chromeExtensionId once you have followed the instructions in the projects readme. This is used for populating the local image file ryan.jpg.
The images variable is used to get all the image tags on a page.
The gb_images array is used as our set of 10 GiantBomb staff images for switching
A for loop is used to iterate through all the images on the page, and for each one it uses a randomly selected index from our gb_images array and displays that instead.
The alternative code for implementing the Bill Murray or Nicolas Cage images follows a similar pattern except in their scenarios the height and width values of the images are used and passed into the corresponding urls to generate the associated images.

Sample Results

Of course I couldn't sign off without showing some quick screenshots from the results of this horrific extension...

A few things you can note from these screenshots:

Yes I simply searched red in google then fired the plugin - it provides a more interesting/psychotic effect when used on Twitter/Facebook but I don't have the time to censor my own personal profiles for the sake of screenshots.
In both cases for Bill Murray and Nicholas Cage due to the way the corresponding API's Placecage and Fillmurray work you will see a lot of repetition on images of the same size, because these APIs take in width and height parameters this is inevitable. Some custom logic could be added to modify this.
In the case of GiantBomb as there is a limited number of images the output is a bit bland and not super mixed, this can easily be updated with the inclusion of more images instead of the 10 hardcoded.
In the case of GiantBomb you may notice one broken image. This is corresponding to the local file that is referenced - I have left this error in to emphasise as mentioned above that you need to ensure to update the chromeExtensionId to match your local version to have this work correctly.

Possible Implementations

So the use case here is pretty basic and there are limited reasons to want to swap out all the images on a page, however if you wanted to dig dipper and experiment more there are a few other modification you could make to try improve the functionality from this basic foundation - here are some that immediately spring to mind:

Add in a blacklisted image url list - you could create your own censorship extension
Use the extension to write the discovered URLs on a page to a DB, might be a useful way to obtain training data for some image learning models

Closing Thoughts

I don't get to spend a lot of time lately developing with JavaScript but I love being able to get started quickly with something fun, relatively easy and having a practical output by the end. This specific extension isn't released on the Google Chrome store however their documentation is relatively helpful and should help you publish your own extension should you wish to go down that route!

You can checkout the project & the readme with setup instructions here.

As always be sure to leave your feedback and thoughts below.

'Till next time!

Do you unit test private methods?

Daniel McMahon — Tue, 08 Jan 2019 14:43:16 +0000

Encountered an interesting issue at work recently when it came to unit testing a new repo that was setup. The repo in question was relatively small and functioned as a basic cronjob, however in order to follow some best practices I decided to opt in for including some basic unit tests.

In doing so I mistakenly wrote a unit test for a private Python class signified with the standard underscore prefix! As I haven't coded in Python in a while I had forgotten about this syntax, but I was a little stunned to hear from a colleague that best practice seems to be to purposefully not test private methods.

I'm curious to hear the thoughts from the Dev community on this one? I get by their very nature you should probably opt to test the Class that the method belongs to end to end rather than opt for a unit test, but what about when you're looking for simple test functionality in a timely manner?

Curious to hear from the community on this one! (across different languages/frameworks too - not just Python)

Lucas - A Webscraper in Go

Daniel McMahon — Thu, 06 Dec 2018 18:35:25 +0000

Lucas

Lucas is a webscraper built using Go and the Colly library. He's also an adorable spider on YouTube!

Why?

I wanted to experiment a little with Go as it's a programming language used by certain teams at my office and not something I often get hands on with. I figured it would be fun to try deconstruct a random online website and see if I could come up with a programmatic way to abstract the HTML data in a way that could be perceived as useful.

The project started out as a fun experiment but after taking it as far as I have I decided to stop working on it for a few reasons which I'll outline below.

The Morality of Web Scraping

As will be apparent to many out there, web scraping is a bit of a 'grey' area morally speaking. From all the research and reading I've done on the subject I usually boil it down to this:

'if you are not violating fair terms of use, abiding by a companies robots.txt file & notifying them that you are doing so, then you should be in the clear'

This is a highly generalised statement for this complex issue but I find its a nice guideline to try follow.

When it comes to e-commerce online websites can hold some highly profitable data, data which they may not even necessarily surface on the frontend of their stores, but they will still have the JSON floating around in their static page content.

Some types of information e-commerce stores may leave floating around in their HTML include:

Stock levels (things like low in stock flags or even exact stock numbers!)
Size availability
Pricing/Sales Rates

For general user purposes this data can go unnoticed but it can be a simple case of right clicking on a storefront and selecting 'view page source' to access this data.

The 'Drawbacks' of Structured data

When it comes to designing a web scraper you want to try analyse the structure of your 'target' sites pages. Ask yourself these kinds of questions:

Is there an easy pattern to follow to access product specific pages
Is there a replicated structure across multiple pages to allow easy html parsing

On the more technical side of things consider the following:

Does the website update its HTML often?
Can you account for missing data during your web scraping?
How will you store the data? Will a DB be fast enough?
Do you plan to handle multi-threading/distributed scraping?

How the scraper works

So onto the fun part - the code! I will talk in generalised terms here so the practices used can be applied to any form of e-commerce store.

Data Storage

In order to setup a way to store the scraped websites data I decided to roll with a postgres DB, for no other reason than I was familiar with it and it was easy to setup via a docker-compose file.

lucas:
  container_name: lucas
  image: postgres:9.6-alpine
  ports:
    - '5432:5432'
  environment:
    POSTGRES_DB: 'lucas_db'
    POSTGRES_USER: 'user'

With this basic PSQL db I was able to setup a basic table by running the following command with the following input file

psql -h localhost -U user lucas_db -f dbsetup.sql

dbsetup.sql:


\c lucas_db

CREATE TABLE floryday(
  index serial,
  product text,
  code text,
  description text,
  price decimal(53, 4)
)

As you can see from the table there were a few basic details I decided to scrape from the web pages in question:

index: this was just used as a unique id
product: name of the product item
code: code of the product in question
description: the description text of the product
price: the price of the product

As listed above there are other additional fields you might be interested in abstracting in your own experiences like size & availability.

Go Dependencies

I was a little sloppy in my service setup in that I did not rely on a Go service dependency management tool like dep, instead I just took care of manually installing them (as there were only 3 dependencies it wasn't so bad). These were the three main external libraries I used they can be installed with the command go get <repo-name>:

github.com/gocolly/colly - webscraping package
github.com/fatih/color - command line colors package
github.com/lib/pq - postgres driver package

To make this setup a little easier I setup a Dockerfile to keep track of the installation

FROM golang:1.11

MAINTAINER Daniel McMahon <daniel40392@gmail.com>

WORKDIR /opt/lucas

ADD . /opt/lucas

ENV PORT blergh

# installing our golang dependencies
RUN go get -u github.com/gocolly/colly && \
  go get -u github.com/fatih/color && \
  go get -u github.com/lib/pq

EXPOSE 8000

CMD go run lucas.go

The main logic

In short the main code does the following:

Starts at a seed url
Scans all the links on the page
Looks for a page that matches a certain regex i.e. -Dresses we know from some basic checks that these pages all have a similar page structure and are usually product pages that we are interested in
Define a Struct to represent our clothing values of interest
Write the Struct to the postgres DB
Continue up to a size of 200 writes

Here is the main bulk of the code with comments explaining the logic - it is not optimized and still quite rough around the edges but its key functionality is in place:

lucas.go

// as our scraper will only use one file this will be our main package
package main

// importing dependencies
import (
    "encoding/json"
    "log"
    "os"
    "fmt"
    "strings"
    "github.com/gocolly/colly"
    "github.com/fatih/color"
    "database/sql"
    _ "github.com/lib/pq"
    "strconv"
)

// setting up a datastruture to represent a form of Clothing
type Clothing struct {
    Name                    string
    Code                    string
    Description     string
    Price                   float64
}

// setting up a function to write to our db 
func dbWrite(product Clothing) {
    const (
      host     = "localhost"
      port     = 5432
      user     = "user"
      // password = ""
      dbname   = "lucas_db"
    )

    psqlInfo := fmt.Sprintf("host=%s port=%d user=%s "+
    "dbname=%s sslmode=disable",
    host, port, user, dbname)

    db, err := sql.Open("postgres", psqlInfo)
  if err != nil {
    panic(err)
  }
  defer db.Close()

  err = db.Ping()
  if err != nil {
    panic(err)
  }

  // some debug print logs
  log.Print("Successfully connected!")
    fmt.Printf("%s, %s, %s, %f", product.Name, product.Code, product.Description, product.Price)
    sqlStatement := `
    INSERT INTO floryday (product, code, description, price)
    VALUES ($1, $2, $3, $4)`
    _, err = db.Exec(sqlStatement, product.Name, product.Code, product.Description, product.Price)
    if err != nil {
      panic(err)
    }
}

// our main function - using a colly collector
func main() {
        // creating our new colly collector with a localised cache
    c := colly.NewCollector(
        // colly.AllowedDomains("https://www.clotheswebsite.com/"),
        colly.CacheDir(".floryday_cache"),
    // colly.MaxDepth(5), // keeping crawling limited for our initial experiments
  )

    // clothing detail scraping collector
    detailCollector := c.Clone()

        // setting our array of clothing to size 200
    clothes := make([]Clothing, 0, 200)

    // Find and visit all links
    c.OnHTML("a[href]", func(e *colly.HTMLElement) {

        link := e.Attr("href")

        // hardcoded urls to skip -> these arent relevant for products
        if !strings.HasPrefix(link, "/?country_code") || strings.Index(link, "/cart.php") > -1 ||
        strings.Index(link, "/login.php") > -1 || strings.Index(link, "/cart.php") > -1 ||
        strings.Index(link, "/account") > -1 || strings.Index(link, "/privacy-policy.html") > -1 {
            return
        }

        // scrape the page
        e.Request.Visit(link)
    })

    // printing visiting message for debug purposes
    c.OnRequest(func(r *colly.Request) {
        log.Println("Visiting", r.URL.String(), "\n")
    })

    // visting any href links -> this can be optimised later
    c.OnHTML(`a[href]`, func(e *colly.HTMLElement) {

        clothingURL := e.Request.AbsoluteURL(e.Attr("href"))

        // this was a way to determine the page was definitely a product
                // if it contained -Dress- we were good to scrape
        if strings.Contains(clothingURL, "-Dress-"){
            // Activate detailCollector
            color.Green("Crawling Link Validated -> Commencing Crawl for %s", clothingURL)
            detailCollector.Visit(clothingURL)
        } else {
            color.Red("Validation Failed -> Cancelling Crawl for %s", clothingURL)
            return
        }

    })

    // Extract details of the clothing
    detailCollector.OnHTML(`div[class=prod-right-in]`, func(e *colly.HTMLElement) {
        // some html parsing to get the exact values we want
        title := e.ChildText(".prod-name")
        code := strings.Split(e.ChildText(".prod-item-code"), "#")[1]
        stringPrice := strings.TrimPrefix(e.ChildText(".prod-price"),"€ ") 
        price, err := strconv.ParseFloat(stringPrice, 64) // conversion to float64
        color.Red("err in parsing price -> %s", err)
        description := e.ChildText(".grid-uniform")

        clothing := Clothing{
            Name:                   title,
            Code:                   code,
            Description:        description,
            Price:                  price,
        }

        // writing as we go to DB
        // TODO optiize to handle bulk array uploads instead of one at a time
        dbWrite(clothing)

        // appending to our output array...
        clothes = append(clothes, clothing)
    })

    // start scraping at our seed address
    c.Visit("https://www.ourclothingwebstore.com/Dresses-r9872/")

    enc := json.NewEncoder(os.Stdout)
    enc.SetIndent("", "  ")

    // Dump json to the standard output
    enc.Encode(clothes)

}

Dangerzone

After getting the basic functionality setup and working it was at this point I decided to try experiment with multiple product seed pages. I discovered that this particular store laid out its main products on a page called: https://www.ourclothingwebstore.com/Dresses-r9872/ however this could be paginated with a simple /p2 at the end of the url, or /p3, /p4 all the way up to p80 something! On average there was around 40+ products on each of these pages. I had implemented a simple for loop that iterated over this seed url updating it each time. I could in essence with a little more logic setup the crawler to hit all the Dress products this store had on sale (and similarly I'm sure with a small regex tweak the logic could have been applied to other fashion categories the retailer had on offer).

It was at this point that I decided to stop work on the project as I had achieved the basic goals I set out to do and learned perhaps a little too easily how malicious this innocent project could turn. It was turning into a one way stop into essentailly having a DB that contained the entire storefront of this ecommerce site.

There are definitely optimisations that would be required to achieve this goal but I imagine with the use of goroutines you could get some parallelisation of this scraper happening to speed up the process to potentially scrape the entire website in a short timespan.

Closing Thoughts

I had some fun trying to reverse engineer the websites HTML structure and figure out how they were displaying product pages and how to go about abstracting the right product hrefs to crawl and the correct data to be able to obtain and write to a DB. It was enjoyable but it all felt a little too close to the sun for my legal liking.

I have deliberately not referenced the 'real' websites name in this example for the sake of their anonymity but the underlying principles should be applicable to most major online ecommerce retailers.

I was amazed with how easy it was to get up and running with the Colly library - I definitely suggest testing it out but be careful with what data you decide to scrape/store and that you investigate your targets robots.txt file to ensure you have permission to hit their website.

Any thoughts/opinions/comments feel free to leave below.

See you next time!

Integrating Docker with your Personal Projects

Daniel McMahon — Tue, 04 Dec 2018 20:22:41 +0000

Why?

Docker themselves outline the use cases for adopting their container based infrastructure over on their main website. I can provide a brief snapshot of my own opinions on why it's a useful tool and how it can be used to optimize your project workflow as well as how it can make your projects stand out after just leaving college.

The Benefits

I'm going to avoid the ship shipping ships container metaphors here. If you've ever heard or come across Docker or Kubernetes the landscape is littered with these metaphors. I'll try keep things a little more simplistic in outlining some of the benefits of this tooling:

Dependency Validation: Docker allows you to create images based on a build of your project. This means all your dependencies are self contained, allowing you to build the project locally as a docker image, run it as a container and verify that it works and has all the correct dependencies. This prevents weird oversights you might have with globally installed packages on your local machine that aren't defined in the projects package.json, sbt or dep file. If the Docker image runs on your Windows PC, it should run fine on your Linux EC2 instance or other devices you plan to run the Docker image on.
Version Control: don't just rely on GitHub! As mentioned above you can build Docker Images which are essentially snapshots of your projects builds at various points in time. Imagine a scenario where you need to demo your service locally, you pull the latest version from master, build it and it crashes... you don't have time to revert to an older commit, build the service and then launch it... instead you can just pull the older commit image and run it - as its all self contained it will be much faster in building/running. It's also a handy way to quickly compare old features and new features -> it can also be an interesting way of viewing how a project originally looked 5+ years ago compared to how it is now.
CI/CD: If you're using continuous integration/development solutions and you accidentally deploy a broken build in a pipeline, it can be quite time efficient to simply refer to an earlier docker image rather than rebuild the project from scratch again. A popular method of development is to wrap up your projects as docker images, upload them to a cloud solution (such as the public Docker Hub or a more private solution hosted on a service like AWS Elastic Container Registry) and then use those images in the prod environment. This is a particularly powerful solution when paired with a service like Kubernetes that may be running replicas of services across multiple pods or EC2 instances. It's simply more efficient using a docker image then pulling the repo 3 times, installing and building all the dependencies.
Imitating Prod Dependencies Locally & Quickly: quite often when working in a professional environment you may be working on a front end codebase that requires access to a a postgres db for storing some site documentation while also requiring access to a storage solution like Redis for managing user session storage. It can be quite time consuming setting up your development environment as a result of these dependencies, however Docker have a useful tool docker-compose that will allow you to run a postgres and redis setup with a single command 'docker-compose up -d' with a yaml file containing less than 10 lines of code.

I'll quickly acknowledge at this point that Docker is not the be all and end all solution. Inductor has an interesting article Do you really need docker or kubernetes in your system link that I highly recommend checking out.

As a college graduate or new developer being able to demonstrate interest and knowledge in how you would plan to make your project scalable and deployable in a production environment can give you the edge over other candidates in interviews. Having a basic Dockerfile setup in your personal projects gives you another unique talking point and skill to reference in your interviews.

Integrating Docker with your beginner projects

I took on the challenge to go about integrating Docker into some previous projects I've worked on to help provide some end to end insight on the practical use cases of the tooling and how easy it can be to add this extra functionality. I've previously written articles on the development of these basic projects so beginners can gain a better insight into how a project can develop new features over time by reviewing these past articles.

There is a general pattern adopted across all three of these projects which you will notice (and which emphasises how easy it can be to generate these files)

Use a base image i.e. node/openjdk/golang
Set the Maintainer
Set the working directory and add the relevant files
Set any ENV vars i.e. ports/versions -> this allows easy updates later
Install your dependencies using 'RUN'
Expose your required port/ports
Run your main command i.e. npm start

Now lets have a look at the generated files.

JavaScript: React Portfolio - Code - Article

This project evolved from a simple create-react-app.
The generated Dockerfile looks as follows:

Dockerfile

FROM node:8

MAINTAINER Daniel McMahon <daniel40392@gmail.com>

WORKDIR /opt/react-portfolio

ADD . /opt/react-portfolio

ENV PORT 3000

RUN npm install

EXPOSE 3000

CMD npm start

There is an NPM package that might help you auto generate Dockerfiles in future. Checkout the dockerfile-generator package available over at NPM.

Scala: The Inspiration API - Code - Article

This project evolved as a basic Scala/Play application built from SBT. This service has a dependency on a postgres DB that can be setup locally using docker-compose.

Dockerfile

FROM openjdk:8

MAINTAINER Daniel McMahon <daniel40392@gmail.com>

WORKDIR /opt/inspiration-api

ADD . /opt/inspiration-api

ENV SBT_VERSION 0.13.15

# Install sbt
RUN \
  curl -L -o sbt-$SBT_VERSION.deb http://dl.bintray.com/sbt/debian/sbt-$SBT_VERSION.deb && \
  dpkg -i sbt-$SBT_VERSION.deb && \
  rm sbt-$SBT_VERSION.deb && \
  apt-get update && \
  apt-get install sbt && \
  sbt sbtVersion

EXPOSE 9000

CMD sbt run

docker-compose.yaml

inspiration:
  container_name: inspiration
  image: postgres:9.6-alpine
  ports:
    - '5432:5432'
  environment:
    POSTGRES_DB: 'inspiration_db'
    POSTGRES_USER: 'user'

In order to correctly have the Docker container running the service communicate with a local DB setup you will need to run it with the following port setup:

# setup db dependencies
docker-compose up -d
psql -h localhost -U user inspiration_db -f dbsetup.sql

# build and tag image locally
docker build -t inspiration_api:v1 .

# port forwarding Docker to localhost:9000
# note: this seems to be broken atm - I'll update when fixed!
docker run -ti -p 9000:9000 -p 5432:5432 <docker-image-id>

# publish docker image to docker hub
docker push <docker-repo>

Interested in automating this process? There are a few Scala libraries that will auto generate and push docker images for you. Checkout marcuslonnberg's sbt-docker plugin.

Golang: Lucas - Code - [article incoming...]

This project is a basic web crawler built using Go and the Colly library. This service has a dependency on a postgres DB that can be setup locally using docker-compose.

Dockerfile

FROM golang:1.11

MAINTAINER Daniel McMahon <daniel40392@gmail.com>

WORKDIR /opt/lucas

ADD . /opt/lucas

ENV PORT blergh

# installing our golang dependencies
RUN go get -u github.com/gocolly/colly && \
  go get -u github.com/fatih/color && \
  go get -u github.com/lib/pq

EXPOSE 8000

CMD go run lucas.go

docker-compose.yaml

lucas:
  container_name: lucas
  image: postgres:9.6-alpine
  ports:
    - '5432:5432'
  environment:
    POSTGRES_DB: 'lucas_db'
    POSTGRES_USER: 'user'

In order to correctly have the Docker container running the service communicate with a local DB setup you will need to run it with a network flag in host mode:

# setup db dependencies
docker-compose up -d
psql -h localhost -U user lucas_db -f dbsetup.sql

# build docker image
docker build .

# run docker container and portforward port 3000
# the host network flag will allow your container to speak to your local DB
docker run -ti -p 8000:8000 --network="host" <docker-image-id>

# publish docker image to docker hub
docker push <docker-repo>

Closing Thoughts

As you can see from the above its relatively straight forward to get your applications up and running on Docker. It can be a little tricky when you first get used to the concepts behind Docker but once you get the hang of it the advantages it offers are fantastic.

There is a super cool tool I stumbled upon this week that will let you analyse the quality of your docker images and give you a % rating for how well optimized it is i.e. if you have too much bloat from installed packages etc. Go check it out - the libraries called 'dive'.

The natural next step from 'Dockerising' your applications is to look at deploying it through some sort of CI/CD solution. I hope to try take a look at setting up your own Kubernetes cluster using a library called kops and deploying the above services through it.

Any feedback/thoughts/suggestions feel free to comment below.

See you next time!

A Year in Professional Software Development

Daniel McMahon — Sat, 01 Sep 2018 13:35:06 +0000

A Year in Professional Software Development

I survived a year working in Professional Software Development - Huzzah!

I feel there may be some value in recalling my own personal experiences a year after being thrown into the software engineering workforce after coming straight out of a 1 year computer science conversion program. I was lucky enough to be hired by the company I conducted work experience at straight out of my course and since then its been a long and mainly prosperous road.

There are a few things that I feel are worth sharing so those new to the industry can feel some security in their mindsets and how they might evolve and adapt over their coming work year. Here are the key lessons I’ve learned in the past year:

Seek the opinions & advice of your colleagues

This may seem obvious but having worked with a wide variety of developers from the typical rockstar dev who pops up services and repos overnight with zero test coverage, to the devs who provide agonising feedback over a single line PR change to some production code its important that you weigh the opinions of your colleagues against your own.

When you are a new developer everyone seems more experienced but the more exposure you get to codebases and different people's approaches to their workflow you start to see patterns between the developers who provide genuine value to the company/your team and those who are churning out code for their own means whether thats to just get a service out the door or to try and emphasise their own unique contributions during a certain quarter or sprint. The experiences both sides of the spectrum have to offer can be enlightening and helpful towards your own personal development and workflow style.

You code as a team. Not as an individual

I’ve worked across multiple teams this past year and found that teamwork really is essential to achieving your goals. (Duh). I’ve found that teams who spend time planning out quarters or sprints and factor in the skills and talents of their members accurately are in a much better position to scope work out accurately and overall provide a positive work atmosphere.

As a new developer you will need to lean heavily on the support of your colleagues. There is an inevitable build up time before you are able to make meaningful contributions of your own however it is important to realise that your colleagues have a responsibility to assist you and its not something you should feel shy or embarrassed about requesting. In the same way they provide support to you, you should be able to provide further support to new devs who join in the future.

Its a dog eat dog world out there and us developers have to stick together. And on the note of team consideration…

Write tests & maintain documentation regularly

For your own sanity and the sanity of your co-workers do not test code in production if you can avoid it.

You need to ensure the code you commit is to a certain production quality standard with a high level of test coverage and decent documentation. This is beneficial for so many reasons:

Any future changes to the codebase that may impact your changes will be instantly picked up when changes are made and your tests fail. When services pop up overnight with limited test coverage and get thrown into production down the road, you or even worse your team mates will have to pick up to the technical debt down the road to implement the tests when something breaks later on.
Quality documentation make the codebase easier to understand how to deploy services locally, in production or just provide insight into what your application is and how you can test its functionality.
Team understanding. If you are a new developer you will benefit greatly to exposure of codebases with quality testing and good documentation. You should consider it a responsibility to make your work to a standard that new developers can easily adopt and work with. Thats not to say your code should always be simple but take care with the basic things including use of descriptive variable names and the use of details local/production deploy instructions.

To understand your code you need to be able to write tests for it. This is something I will openly admit to struggling with from time to time as often its easier to just write a block of code then it may be to mock up a server call to test it. Always try relate back to the test triangle when it comes to your work.

Make the working process work for you

Whether your team has adopted an agile kanban, scrum or waterfall style of development its key that you focus on making the existing system work to your advantage. There are many methods of working in software development but the most common one I’ve seen is the use of a Kanban board (Jira, Trello, LeanKit) with specific rules around epics, stories, tasks, bugs and how they are all individually handled.

Learn the systems in place, learn how to write the various tickets, learn how to use the systems in place to optimise your workflow. If theres a streamlined process for logging and tackling bugs be sure to follow that system. I would never suggest blindly taking the systems in place to be optimal, and by all means you should question at every corner if the particular ‘way of working’ implemented by your team or larger org currently makes sense.

When it comes to handling things like rituals i.e. standups, refinement, planning, retrospectives, post-mortems adapt and use the processes that make the most sense for those on your team. There is little point in engaging in daily standups with your team if nobody is getting value from it. Similarly if there are issues with planning taking too long or being of minimal value speak up and provide alternative options to try and handle the problems encountered.

Never just go with the flow, especially if the flow is detrimental to team productivity.

Take the bad with the good

By its very nature software development is going to be a bumpy road full of the highest highs and lowest lows. I’ve had plenty of experiences where some block of code was not behaving as expected and banged my head against the wall for hours only to discover a typo somewhere in a variable name. I’ve also had the struggle with getting weird results from querying Postgres DBs with what should be a simply order by query only to discover the JS library I was using had its own quirks. There were even times when I tried applying stuff from college like the use of xmlhttp requests only to discover that in React it was so much faster to just whip out an axios library to do the same job. Don’t even get me started on the amount of CORS related issues I’ve had to debug the past year…

These experiences are only a handful of personal experiences I’ve encountered along the way, all of which drove me up the wall at the time, but from each one I’ve learned a lot and adjusted my own personal debugging approaches. If you haven’t heard of it you need to go and checkout Rubber Duck Debugging.

But at its very core software engineering is problem solving. As a junior developer you will not learn if you don’t make mistakes. The important factor is to try and ask for assistance only when you need it. Don’t be afraid to tackle a problem head on before seeking the advice of the puzzle-masters that are the more senior developers. You don’t get better at puzzles by being given the answers, you get better by being exposed to them and adapting your thinking towards how to solve them.

The difference between Junior & Senior is not all that much…

My point here is that graduate developers are all to quick to label themselves as ‘juniors’ despite spending perhaps up to 4 years working on independent projects or college related material. Try not to think of yourself as a junior developer but instead focus on the steps you can take to become a better developer

The biggest difference I found between ‘junior’ and ‘senior’ developers aside from experience is the perspective used to approach problems. Senior engineers possess an attribute of persistence when they encounter a bug and use their wealth of knowledge (i.e. advanced google/stack-overflow skills) to continuously poke at solutions to various problems.

Consider the scenario you hit a bug, just can’t get that API to work, you’ve been trying for a couple days and just cant wrap your head around it. You eventually cave in, ask a senior dev for help. They leave for a few minutes to check something, come back and like magic solve your issue with a less than 10 minute turnaround. You can bet your ass the reason they were able to fix that solution was they had encountered it before, remembered googling it and using result #3 to resolve it.

Now I’m not using a blanket statement to say this is how it works in all cases and developers shouldn’t be overly reliant on these resources, but it is always worth considering that senior developers dont necessarily know all the answers but rather they know how to debug or search for solutions in different ways to you. It is a very valuable experience sitting with your peers during pair programming and watching the approaches they adopt to approach debugging some broken code. These debug skills will be some of the most valuable you can obtain.

The most surprising things…

When you’re a fresh eyed college graduate there are certain things that you quickly become exposed to that you may not expect in the professional development world. Heres a few points that surprised me along the way that you may want to watch out for:

You will be dealing with large codebases for even the most simplest projects. Give it time to understand new codebases - try not to be intimidated by the sizing of some projects compared to what you may be used to dealing with in college
The simplest tasks will often take longer than you initially expect. I still struggle on a regular basis with trying to assign accurate story points to tech tickets. Estimating the time it takes to complete a task is something that comes with time and experience.
External dependencies are some of the most frustrating things you can deal with. Whether its relying on another team to get X service up and running or to merge Y PR before you can implement your changes… never underestimate the additional complexity of a task when relying on external teams or resources outside of your immediate control
Different developers withhold different standards for things like documentation and PRs. One of the key bits of advice I try to follow is to remain consistent. Regardless of the size of a PR I always take time to put in a brief description, link it to an issue, insert a changeling and a flag for whether or not the PR is ready for review. While this can take some additional time its worthwhile remaining consistent and clear with even these basic things.
You will spend what feels like a lifetime waiting for continuous pipelines, code compilation and many other fun processes to complete. The key with this is to plan your time carefully around these potential time sinks.

Git can be a harsh mistress. Its worth getting familiar with using git to do things like rebaseing commits to tidy up your PRs so you dont have long commit history chains with duplicate commit message or worse. Its worth taking the time to read up on the git documentation and practicing working across multiple branches and rebasing commits. Commands like git stash and git stash pop are also invaluable to have at your disposal.

Comparisons can be detrimental. Try not to compare your weekly output to that of your more experienced colleagues. Its a natural thing to try and keep up to speed with the best of the best but starting out you need to a accept that you may need to work at a slightly slower pace, and thats perfectly ok. Developers take some of there knowledge for granted and when passing along work they can often forget that something that they’ve done 101 times is actually pretty tricky the first 4 or 5 times. People won’t know your limitations unless you are able to voice them. This can be tricky depending on your work environment but if you work for a company that has understanding colleagues and a positive work environment you should be in a scenario to feel free shouting out that you can’t complete a task in the time allotted, or that you may need some help with pair programming on a certain task.
The stuff you learned in college, while useful, didn’t cover half of what you most likely will use in work. In my own practical case I never had exposure to technologies like Docker & Kubernetes, Continuous Integration Pipelines, Build Tools and in some cases even the languages I ended up adopting for projects including React/JavaScript and Scala/Play/Akka.
Technology really does change frequently and often. Chances are you will go through using various languages, tech stacks, deployment services. databases and many other resources within a very short time frame. Working in the tech sector is very much a continuous college-esque experience where upskilling is vital to your career. You might have graduated and started a new job but the real learning experiences are only just beginning.

Closing Thoughts

I’m very grateful for having the opportunity to switch careers into the tech industry and gain exposure and knowledge of so many different technologies. I find myself in a very fortunate position to have fantastic work colleagues and an amazing work environment under which I have had room to grow my own skillset. I would be lying if I said it was an easy and smooth ride - working in tech can be challenging but therein lies the fun. If it was easy it would be boring.

I think the first year of working will be one of the toughest for many graduates going into the professional software engineering world. My advice is to keep asking questions, find yourselves as many mentors as possible, continuously request feedback and keep plucking away at your own personal projects. Upskilling can be boring at times relying on online courses and occasionally boring lecturers however nothing can beat the thrill and exposure of building your own applications end to end and deploying them on AWS/Heroku/Firebase or any other service you can access.

I hope there is some knowledge in here that will help others who are moving into the professional software development world. I’d love to here the experiences of others who are approaching the one year mark and get your feedback. Be sure to comment below any advice or experiences you have to share to the community.

Many thanks for reading.
Until next time!

The Inspiration API: A project built with Scala & Play Framework

Daniel McMahon — Tue, 17 Jul 2018 19:43:37 +0000

You don't learn to walk by following rules. You learn by doing, and by falling over. - Richard Branson

The Inspiration API is the start of my journey down the path of trying to improve my Scala skills. Having coded very little in this language I decided to attempt to build a basic API using the language alongside the Play Framework. Rather than take the textbook approach I often find at times you just need to get your hands dirty and learn from your mistakes. I mean... how hard can it be?

Up to recently my primary coding focus has been on Frontend or more JavaScript heavy services. For work and personal purposes I wanted to expand my skillset to encompass a more broad scope of programming practices, and so I've found my way to Scala. I've studied the OOP basics of Java in college and taken some Scala training on the job but being rusty with using the JVM and blind to the more complex usages of the functional programming paradigm I've been ill prepared to adopt it.

The Inspiration API

The Codebase: Available Here!

I guess the name of the API is self evident -> I needed some sort of project to keep me inspired and interested in learning a new language. As many of you are aware it can be difficult at the best of times to find energy coding after a full days work!

So I was a little lazy at this point... I basically googled 'inspirational quotes' and grabbed the first 10 I came across that didn't sound entirely cheesy or ridiculous and decided to use them for the basis of the API's data.

Note that I say data here as my first attempt at making the API was focused on setting up a basic GET endpoint that return a response. Rather than relying on a DB I started with a basic array of JSON objects, I used a similar trick in another project to populate a dropdown menu on a static site instead of relying on a DB. Start small and work my way up I figured...

I set up a bunch of github issues which linked to an overall RESTful API milestone, essentially it was more or less 1 issue per CRUD feature:

When starting out I cloned a Play/Scala seed from the play website -> in retrospect this featured quite a lot of what I'll playfully call Bash bloatware that wasn't necessary for just building the project locally with SBT - however it did the job!

The main considerations for using play was that it should be as simple as modifying your routes configuration to determine your GET, POST, PUT, DELETE endpoints to point to a specific controller.

Handling JSON in Scala

Immediately it became evident that there is a big difference in the level of Google Fu required between searching for how to do something in JS versus how to do something in Scala. And to be fair I can accept that JSON is native to JavaScript... We can all agree that Stack Overflow development isn't an optimal way of programming however it can prove a useful tool when starting out and wanting to build something with a tool you have had very limited use with!

So the basic logic of my JSON solution (and prior to this an array implementation) featured a Play controller which looked a little like this (forgive the placeholder variable names... I never do this in production code... honest!):

Routes:

# Routes
# This file defines all application routes (Higher priority routes first)
# https://www.playframework.com/documentation/latest/ScalaRouting
# ~~~~

# An example controller showing a sample home page
GET     /                           controllers.HomeController.index

# Map static resources from the /public folder to the /assets URL path
GET     /assets/*file               controllers.Assets.versioned(path="/public", file: Asset)

# Inspiration CRUD endpoints
GET     /inspiration                controllers.InspirationController.index

Inspiration Controller:

package controllers

import javax.inject._
import play.api._
import play.api.mvc._
import play.api.libs.json._
import scala.collection.mutable.ArrayBuffer

/**
 * This controller creates an `Action` to handle HTTP requests to the
 * application's home page.
 */
@Singleton
class InspirationController @Inject()(cc: ControllerComponents) extends AbstractController(cc) {
  def index() = Action { implicit request: Request[AnyContent] =>
    Ok(generateQuote(y, scala.util.Random.nextInt(10)))
  }

  // json method of generating quotes
  // (y(scala.util.Random.nextInt(10))\"quote").get

  var y: JsValue = Json.arr(
      Json.obj("quote" -> "Make your life a masterpiece, imagine no limitations on what you can be, have or do.", "author" -> "Brian Tracy"),
      Json.obj("quote" -> "We may encounter many defeats but we must not be defeated.", "author" -> "Maya Angelou"),
      Json.obj("quote" -> "I am not a product of my circumstances. I am a product of my decisions.", "author" -> "Stephen Covey"),
      Json.obj("quote" -> "We must let go of the life we have planned, so as to accept the one that is waiting for us.", "author" -> "Joseph Campbell"),
      Json.obj("quote" -> "Believe you can and you're halfway there.", "author" -> "Theodore Roosevelt"),
      Json.obj("quote" -> "We know what we are, but know not what we may be.", "author" -> "William Shakespeare"),
      Json.obj("quote" -> "We can't help everyone, but everyone can help someone.", "author" -> "Ronald Reagan"),
      Json.obj("quote" -> "When you have a dream, you've got to grab it an never let go.", "author" -> "Carol Burnett"),
      Json.obj("quote" -> "Your present circumstances don't determine where you can go; they merely determine where you start.", "author" -> "Nido Quebein"),
      Json.obj("quote" -> "Thinking: the talking of the soul with itself.", "author" -> "Plato")
    )

  // Function that returns a random string include quote & author
  def generateQuote( quotes:JsValue, random:Int) : String = {
    var quote:JsValue = (quotes(random)\"quote").get
    var author:JsValue = (quotes(random)\"author").get
    return (author.as[String] + ": " + quote.as[String])
  }

  // array method of generating quotes
  // quotes(scala.util.Random.nextInt(10))
  //  var quotes = ArrayBuffer[String]()
  //  quotes += "Make your life a masterpiece, imagine no limitations on what you can be, have or do. - Brian Tracy"
  //  quotes += "We may encounter many defeats but we must not be defeated. - Maya Angelou"
  //  quotes += "I am not a product of my circumstances. I am a product of my decisions. - Stephen Covey"
  //  quotes += "We must let go of the life we have planned, so as to accept the one that is waiting for us. - Joseph Campbell"
  //  quotes += "Believe you can and you're halfway there. - Theodore Roosevelt"
  //  quotes += "We know what we are, but know not what we may be. - William Shakespeare"
  //  quotes += "We can't help everyone, but everyone can help someone. - Ronald Reagan"
  //  quotes += "When you have a dream, you've got to grab it an never let go. - Carol Burnett"
  //  quotes += "Your present circumstances don't determine where you can go; they merely determine where you start. - Nido Quebein"
  //  quotes += "Thinking: the talking of the soul with itself. - Plato"

}

Wahey - just like that we have our GET /inspiration up and running!

The Database Hookup

This was the MOST painful part of the project for several reasons.

Did you know that you can't run Docker natively on Windows 10 Home edition? You have to use Docker Toolbox combined with vagrant or some virtualbox setup?

I wanted to use Docker to run a simple postgres database that my service would connect to. I had used Docker before and figured this would be the easiest solution! This shouldn't have caused me so many issues (and time) but it did. I usually code on a MacBook in work but on a Windows PC at home for personal projects - I find its a good way to stay ambidextrous across platforms. In the end I called it quits and settled with using my trusty MacBook (which worked almost instantly once I cloned the Dockerfile locally).

To try and keep things self contained I included a basic dbsetup.sql file in the repository that users could load into their Docker container using a simple command.

dbsetup.sql (comment below if you have any better default quotes I should include!):

\c inspiration_db

CREATE TABLE quotations(
  index serial,
  author varchar(255) NOT NULL,
  quote varchar(1000) NOT NULL
);

INSERT INTO quotations (author, quote) VALUES ('Brian Tracy', 'Make your life a masterpiece, imagine no limitations on what you can be, have or do.');
INSERT INTO quotations (author, quote) VALUES ('Maya Angelou', 'We may encounter many defeats but we must not be defeated.');
INSERT INTO quotations (author, quote) VALUES ('Stephen Covey', 'I am not a product of my circumstances. I am a product of my decisions.');
INSERT INTO quotations (author, quote) VALUES ('Joseph Campbell', 'We must let go of the life we have planned, so as to accept the one that is waiting for us.');
INSERT INTO quotations (author, quote) VALUES ('Theodore Roosevelt', 'Believe you can and you''re halfway there.');
INSERT INTO quotations (author, quote) VALUES ('William Shakespeare', 'We know what we are, but know not what we may be.');
INSERT INTO quotations (author, quote) VALUES ('Ronald Reagan', 'We can''t help everyone, but everyone can help someone.');
INSERT INTO quotations (author, quote) VALUES ('Carol Burnett', 'When you have a dream, you''ve got to grab it an never let go.');
INSERT INTO quotations (author, quote) VALUES ('Nido Quebein', 'Your present circumstances don''t determine where you can go; they merely determine where you start.');
INSERT INTO quotations (author, quote) VALUES ('Plato', 'Thinking: the talking of the soul with itself.');

Docker Commands for setting up the Docker DB & Inserting above rows:

docker-compose up -d
psql -h localhost -U user inspiration_db -f dbsetup.sql

At this point I had to start the Google searching: 'how to connect to a postgres db with scala' and there are a bunch of different libraries and results that showed up such as jdbc, postgres-scala, doobie and many more... It was a little overwhelming and difficult to just get a few lines of documentation for a super simple implementation. In the end I went with a library called Slick.

After settling on Slick I setup a basic Class to represent the quote entries for the DB. I had to do some tinkering before figuring out how the heck to handle serial (auto incrementing) postgres values but thats a story best unmentioned! Just enjoy the completed functioning code!

Class matching quotes structure form the postgres DB:

// Matches schema of the docker-compose psql DB quotations table
class Quotes(tag: Tag) extends Table[(Int, String, String)](tag, "quotations") {
  def index = column[Int]("index")
  def author = column[String]("author")
  def quote = column[String]("quote")
  def * = (index, author, quote)
}

At this point the rest of the 'hard' work came down to figuring out how to translate postgresSQL to this Slick style syntax. Heres a rough breakdown of how the various endpoints worked:

GET /inspiration

import scala.slick.driver.PostgresDriver.simple._

def index() = Action { implicit request: Request[AnyContent] =>
    Ok(generateQuote(scala.util.Random.nextInt(10)))
  }

val connectionUrl = "jdbc:postgresql://localhost:5432/inspiration_db?user=user"

 def generateQuote(random:Int): String = {
    var output = ""

    // connecting to postgres db for accessing data
    Database.forURL(connectionUrl, driver = "org.postgresql.Driver") withSession {
      implicit session =>
        val quotes = TableQuery[Quotes]

        // SELECT * FROM quotations WHERE id=randomInt
        quotes.filter(_.index === random+1).list foreach { row =>
          output = row._2 + ": " + row._3
        }
    }
    output
  }

POST /inspiration

import scala.slick.driver.PostgresDriver.simple._

def add() = Action { request =>
    val body: AnyContent = request.body
    val json: Option[JsValue] = body.asJson
    val author = json.get("author").toString.stripPrefix("\"").stripSuffix("\"").trim
    val quote = json.get("quote").toString.stripPrefix("\"").stripSuffix("\"").trim
    addQuote(author, quote)
    Ok("Successfully updated quotations DB")
  }

val connectionUrl = "jdbc:postgresql://localhost:5432/inspiration_db?user=user"

def addQuote(author:String, quote:String): Unit ={
    var index = 0
    Database.forURL(connectionUrl, driver = "org.postgresql.Driver") withSession {
      implicit session =>
        val quotes = TableQuery[Quotes]
        // getting id of last element in table
        quotes.sortBy(_.index.desc).take(1).list foreach { row =>
          index = row._1 + 1
        }
        quotes += (index, author, quote)
    }
  }

PUT /inspiration

import scala.slick.driver.PostgresDriver.simple._

def replace() = Action { request =>
    val body: AnyContent = request.body
    val json: Option[JsValue] = body.asJson
    val index: Int = json.get("index").toString.toInt
    val author = json.get("author").toString.stripPrefix("\"").stripSuffix("\"").trim
    val quote = json.get("quote").toString.stripPrefix("\"").stripSuffix("\"").trim
    updateQuote(index, author, quote)
    Ok("Successfully updated quotations DB")
  }

val connectionUrl = "jdbc:postgresql://localhost:5432/inspiration_db?user=user"

 def updateQuote(index:Int, author:String, quote:String) = {
    Database.forURL(connectionUrl, driver = "org.postgresql.Driver") withSession {
      implicit session =>
        val quotes = TableQuery[Quotes]
        quotes.filter(_.index === index).update(index, author, quote)
    }
  }

DELETE /inspiration/:index

import scala.slick.driver.PostgresDriver.simple._

def delete(index: Int) = Action { request =>
    deleteQuote(index)
    Ok(s"Successfully deleted entry $index")
  }

val connectionUrl = "jdbc:postgresql://localhost:5432/inspiration_db?user=user"

 def deleteQuote(index:Int): Unit = {
    Database.forURL(connectionUrl, driver = "org.postgresql.Driver") withSession {
      implicit session =>
        val quotes = TableQuery[Quotes]
        quotes.filter(_.index === index).delete
    }
  }

The routes:

GET     /inspiration                controllers.InspirationController.index
POST    /inspiration                controllers.InspirationController.add
PUT     /inspiration                controllers.InspirationController.replace
DELETE  /inspiration/:index         controllers.InspirationController.delete(index: Int)

And there you have it - a basic RESTful API in Scala & Play!

The Codebase: Available Here!

Lessons Learned

The project was a nice way to go off book and try learn by developing something directly. There were some minor pain points I definitely learned from including:

How to connect to a Postgres DB with scala
How to handle Array & JSON data structures with scala
How to handle routes with the Play Framework
How to setup a basic frontend for paths in the Play Framework
Handling Docker on Windows... this one still irks me for the time wasted!

Improvements

Theres a bunch of additions I plan to make to this project over time including:

Adjusting the GET endpoint to return items in the DB with an index higher than 10 -> currently this is hardcoded but it should be easy to swap it out for a DB count amount
Setup an API swagger definition for the generated API -> this would just be some useful additional experience
Deploy the API somewhere -> Heroku is the lead favourite for now... Once deployed implementing some endpoint tracking & analytics would be interesting -> potentially some oAuth as well but that's generally a headache to setup.
Develop basic SDKs from the generated Swagger using a service like Swagger Codegen
Add Tests
Modify OK section on routes to return correct response i.e. 200, 201, 202 etc.

As always if you have any feedback, suggestions or thoughts feel free to share below.

'Till Next Time!

Building A Portfolio: The Painful Way

Daniel McMahon — Mon, 18 Jun 2018 22:30:55 +0000

The Goal: Lets combine all the JS & React related things I've learned to date in one big project! What could go wrong?

The End Result: Website & Codebase

JavaScript is complicated... sorta

I've learned a lot of things about JavaScript over the past year on the job, things that anybody new to the field will no doubt also go through - here are some questions I faced along the way that some of you can no doubt relate to and will hopefully take some comfort in reading:

What IDE should I use for programming in JS?
What Plugins should I use to optimize my work flow?
Whats a package manager and whats the big fuss between Yarn or NPM?
What the hell is Webpack and why do I already hate it?
ESLINT? Sure its nice in theory but have you tried setting up the config yourself - painful!
Enzyme, Jest & Mocha - are you sure you're still talking about JS?
Whats the difference between CommonJS, ES6 & ES2018?

Over the past year I've come to terms with the peculiarities of the language and can happily work alongside it. It is still very much a love hate relationship but having tackled multiple projects ranging from full-stack web apps, chrome extensions and publishing basic node packages I feel much more confident in my abilities using this language as a coding medium.

I felt so confident that I figured I would try demonstrate my knowledge and skills by creating a portfolio site on my personal GitHub to try out all the techniques I had learned and marvel at what monstrosity would come out.

KISS - Keep It Simple Stupid

Those who work in UX will know that in advance of any project you should ideally do mock up designs and user testing to deliver an optimal product to achieve maximum impact. This was a venture down the perfect road of what not to do... make it up as you go along.

I had an idea (a rough sketch) on a notepad that I would use as the basis for the site structure and figured I would work from there... so how did the process work out?

Coding Setup

First things first I had to setup the project from scratch. I used a basic create-react-app template to start with.

To make the development process streamlined I had setup my Atom IDE with Prettier for handling some automatic line indentation on save, and added my favourite City Lights theme to make the project easier on my eyes.

I set out building some basic reusable React components such as Cards, Headers, Footers and NavBars. In order to allow for multiple page functionality I ended up adding React Router and React DOM - not something I had really done with the create-react-app template before.

I ended up treating the individual pages as 'containers' as opposed to 'components' in order to make the code base more legible. It appeared a good way to segment what should be dumb stateless components from the more intelligent stateful page components.

Next up I wanted to try show off some of the libraries I had been using over the past year so I added in Styled Components & Grid Styled to help structure the individual pages layouts with custom styling.

I decided to add in a tribute to a design library I had encountered during the past year with the inclusion of the Ant Design Carousel. I'd like to build my own Carousel in the near future but for now this felt like a good placeholder and the end result was quite satisfying.

I added in a Donut graph to represent my confidence levels in my top skills so that the landing page of the site would give anybody interested in my skill sets a snapshot of my strengths and weaknesses.

Venturing into the unknown... css media

I had no exposure to using animations or transitions with css and wanted to try something new so I was browsing some other dev.to developers portfolio sites for inspiration. In particular Ali Spittel's site stood out to me. I loved her use of cards to display some samples of work she had done. I always enjoyed the playful nature of the 'about' section on her page with a rotating image and decided to integrate this on my own projects page with the addition of rotating skill icons.

I decided to take this media exploration a step further and ended up including rotating cards on my homepage to reveal the flip of each outlined skill to contain some placeholder text. This works with a hover in browser and with a tap on mobile devices. I had initially seen this technique in a tutorial demonstrating face down poker cards that revealed themselves on mouse hover. The idea interested me so I couldn't resist adding it!

The 'awwwww yes' features

There were some parts of what I had designed that I was oddly proud of and had not tried implementing before. I always struggled with the concept of how to handle with mobile -v- desktop web applications and controlling the content to optimize the user experience. I ended up using some custom divs to hide the display of both the Donut graph and the Carousel when the screen was below a set resolution so as to allow for a beneficial experience on mobile and desktop experiences. While this is not the optimal way to handle this problem, it was something new and creative that I enjoyed implementing.

I had experimenting with using some custom css grayscaling features to allow for hoverable social media icons on the contact page that turned out relatively well but could use refinement at a later date.

The cards I had setup for demonstrating my projects were a personal highlight, especially seeing the interactivity between having custom buttons, rotating skill icons and clear text being displayed in harmony.

The 'above and beyond' pain points

The following features were an absolute pain for me to implement and still require some fine tuning but I'm glad to have gone through the pain from the ground up.

I set out to use some custom eslinting rules and combined them with some pre-commit library to prevent the ability to commit your code unless it passed the linting parameters. Traditionally eslinting is used with a 'eslintrc' file defining individual rules. I had not previously setup one of these from scratch and ended up causing myself an awful headache when trying to handle the setup of an eslinter alongside the xo library. In the end I managed to get a form of the xo library working with some plugins, however as of yet the code is still not 100% eslinted correctly, however good progress has been made so far to counteract this minor issue.

The 'to be completed' list

A lot of work has gone into the basic outline of the portfolio site to date however there are some essential missing features that will need to be included:

Circle CI Integration with Protected Master Branch
Updated landing page card text
Jest Tests

I have some duplicated custom styling used across multiple components that I plan to consolidate to a single file and allow for importing to prevent duplication of styling code.

Other key additions I would like to add to the site include:

The integration of Redux
A switch to using server side rendering with Express to serve the content.
Integration of canvas elements (perhaps demonstrating some p5.js knowledge)

While these would be a good way to showcase some more of my skills I feel that the static nature of the site makes these additional features redundant for the time being.

Final Thoughts

While the site is a continued work in progress here are a few of my ongoing thoughts on the project

It was fun and interesting to build my own reusable responsive components, this might be something I look at expanding into a reusable component library project.
The media-css modifications were fun, while only adding minor animations and interactivity it's definitely something I'll explore for future webdev work
I'll definitely put more effort into the UX considerations prior to starting the project as the site in its current form has already undergone many design changes during its development. I may consider the use of marvelapp or sketch for future wireframe design
Static site generators such as SquareSpace tend to be much more artistic in their site layouts with full screen landing pages, cleaner colors and more layered elements creating a more visually appealing style. I feel like taking this more heavily design oriented approach for a different project will be a fun challenge moving forward.
While it was fun doing all this in React, I couldn't help but reflect on how much faster and easier it could have been creating something similar in Bootstrap. I even stopped to consider the simpler tool sets available to developers ranging from pre-defined site design services like SquareSpace and WordPress

It was overall a fun experiment that I was glad to start, however the end result is not as clean and crisp as I would like it to be. This is a result from a lack of design planning. That said I learned a lot engaging in it and after only a years exposure to JavaScript the finished product isn't too shaby!

If you have any feedback or suggestions in terms of what might be some useful additions, or how I could benefit from adding Redux to the service be sure to let me know in the comments.

If you have worked on a similar project be sure to share below - I'd love to have a look!

Many thanks for reading.
till next time!

Leveling Up: From Create-React-App to Express (& some neat styling libraries)

Daniel McMahon — Mon, 09 Apr 2018 18:10:37 +0000

Two months later - time for my second article! You can read my previous article on Visualizing Fibonacci here.

Link to the Live Site

The Project

I'm a professional software engineer and still to this day I find that saying this out loud scares me a little. To try and concrete my confidence in my skills and abilities I've been engaging in a few personal side-projects where I take the fundamental methods of building an application and apply them to some sort of fun side project. This is the first in a hopeful series of posts about my experiences to date.

I had previously developed a few create-react-app static site generation projects that I hosted via GitHub pages as a simple way of developing my understanding of the React component library. For this project I decided to take the create-react-app boilerplate, tie it into the Express web framework, and host it online somewhere for free (copying a GitHub page type flow) so I can easily have it linked on my GitHub repository for anybody wishing to view my personal portfolio (for the total price of... free!).

I wanted an easy way to communicate with some public API but rather than expose the API I was communicating to on my client side code I wanted to keep it on the server side to simulate communicating to my own APIs running on confidential endpoints. I had found a couple of Cat APIs that returned some random images so I figured I'd roll with those and see what I could come up with. You can check the APIs out here:

The end application simply displays an image of a cat that changes on click/tap by making a call to our backend express server, which calls the cat api, and returns the image url for display on the main web page. I added in an additional Spotify player of some random cat noises as a requested extra feature. You can check out the live version of the site here.

The Initial Setup

I used create-react-app as a starting point for my project. It's a very friendly boilerplate that'll get you setup with everything you need to start making a basic React application. In a more professional environment it has certain drawbacks related to the customization of some of its webpack configuration but for the purposes of this project its more than sufficient. Once installed it's as simple as running three commands to get your app started:

create-react-app my-app
cd my-app
npm start

Awesome so now we have our application running!

To handle some of the styling aspects of the application I installed two additional packages:

styled-components for custom element styling.
grid-styled for responsive design styling with flexbox

I highly recommend checking out both libraries if you are new to React and designing some basic UI frontend work. Heres a little summary of what the main purpose of these libraries are:

Styled Components

The common problem with styling, at least during my own time working in web development, has been thatthe global nature of css app styling (at least in the create-react-app framework) makes it difficult to customize your CSS on a component by component level.

The Styled Components library lets you rely on component-defined styles. An example of this from my own code looks like this:

import styled from "styled-components";

const CatPic = styled.img`
  margin: auto;
  background-color: white;
  max-width: 60vw;
  max-height: 60vh;
  margin: 2em;
  border-radius: 10px;
  &:hover {
    cursor: pointer;
  }
  border: 0.2em solid lightgrey;
`;

When using it in my React Component instead of using an <img/> tag I just have to use a <CatPic/> tag and it will import the custom styling defined above.

Grid Styled

Grid styled offered me an easy solution to allow my main Cat component to take up 3/4s of the screen at full resolution, and my Spotify component to take up 1/4 of the screen. Then when the resolution changed to a phone styled size it would flip and allow a full width for the Cat component and a full width for the spotify component. This is a godsend for easy mobile responsiveness design. No more auto calculating pixel widths!

A simplified version of this layout can be seen below:

<Flex flexWrap="wrap">
  <Box p={3} width={[1, 3 / 4]}>
    ... cat component here
  </Box>
  <Box p={3} width={[1, 1 / 4]}>
    ... spotify component here
  </Box>
</Flex>

Integrating Express

Now onto the more challenging part. After installing Express (npm install express) I created a server.js file in the main project directory which looks like the below

const express = require("express");
const bodyParser = require("body-parser");
const path = require("path");

const app = express();

app.use(express.static(path.join(__dirname, "build")));

app.get("/", function(req, res) {
  res.sendFile(path.join(__dirname, "build", "index.html"));
});

app.listen(process.env.PORT || 8080);

After setting up this file I had to modify some of the scripts in the package.json file

"scripts": {
    "dev": "react-scripts start",
    "start": "node server.js",
    "build": "react-scripts build",
    "test": "react-scripts test --env=jsdom",
    "eject": "react-scripts eject"
  }

These new scripts mean that when the dev runs the command npm start rather than running the standard react-scripts start command it will instead run node server.js. The main thing from the script above is in the app.get("/") section which essentially means that when the dev goes to port 8080 after running npm run start Express will direct the individual to the index.html file of the main react application.

The important thing to note at this point is that in order to have valid index.html created you need to have run npm run build prior to npm run start. Rather than having to wait to run a new build and start it every time you make any local changes, I added an extra script called 'dev' which runs the application directly on port 3000. This is a nice work around for developing on the applications features without caring about any of the other server issues.

Another required change I made to the package.json was the inclusion of "proxy": "http://localhost:8080" which proxies the Express server to port 8080, so when the service is running in production any other endpoints setup (i.e. app.get("any-other-endpoint-here") will be accessible when the app is exposed at a URL online.

Frontend to Backend Communication

I added an additional endpoint to the server.js file:

app.get("/cat", function(req, res) {
  axios
    .get("http://thecatapi.com/api/images/get")
    .then(data => res.send(data.request.res.responseUrl))
    .catch(err => console.log(err));
});

I had setup a Cat component that queried this endpoint and used the parsed responseUrl as the src for an img tag (set via the components state). You'll also see below on the initial page load I have the url state query the cat as a service endpoint - just for some added variety:

import React, { Component } from "react";
import styled from "styled-components";
import axios from "axios";

const CatPic = styled.img`
  margin: auto;
  background-color: white;
  max-width: 60vw;
  max-height: 60vh;
  margin: 2em;
  border-radius: 10px;
  &:hover {
    cursor: pointer;
  }
  border: 0.2em solid lightgrey;
`;

class Cat extends Component {
  constructor() {
    super();
    this.state = {
      url: "https://cataas.com/cat" //default image - calling another cat api!
    };
    this.getCat = this.getCat.bind(this);
  }

  getCat() {
    axios
      .get("/cat")
      .then(data => this.setState({ url: data.data }))
      .catch(data => console.log(data));
  }

  render() {
    return (
      <div>
        <CatPic src={this.state.url} onClick={this.getCat} />
        <div />
      </div>
    );
  }
}

export default Cat;

Heroku Integration

So up to this point when working on the project I was commiting all the changes to my local GitHub repository and had not selected a platform for serving the application. I had several services I wanted to try out including Heroku, Firebase and AWS static hosting. In order to go for an easy and cheap hosting solution I ended up trying out Heroku.

I setup my Heroku project and configuring it so that when I made a new commit to my master branch on my project it would auto deploy a new build and expose the service publically. Ideally in the future I'm going to add in some CircleCI testing and protected master branch settings to prevent directly pushing changes to the master branch. For the purposes of this side project the current setup is fine but any errors pushed will go live on the site almost instantly. Instructions for setting up some of this configuration are available over on Heroku if you have any questions on that process feel free to comment below.

The End Result

The application is currently exposed live over here. The codebase is available here if you're interested in having a look or using it as a boilerplate of sorts for your own create-react-app / Express hybrid application!

Further Improvments

Theres a bunch that can be improved about the current implementation including:

Use of node development environments to auto switch urls being used for express communication when developing the service locally
Alternate between APIs being called in backend
Processing of data returned by APIs -> generally response is an image bytestream or some similar structure that may be processible into an image displayed directly on the main page
Security Cert Bug - seems on some Mac devices Chrome has some issue related so the SSL cert on the page, I suspect this can be fixed with some additional Heroku configuration

Your Thoughts & Questions?

Feel free to drop and thoughts or questions you have below. I'd be interested to see if the above is useful to anybody starting out with react and looking to start using services like express with create-react-app.

There are a variety of boilerplates available out there for use that are better than this setup, however the aim of the project was to show how to go from the basic create-react-app to something a bit more complex using an Express backend. This is a very simple example of calling an API in the backend, but you can imagine how this can be taken to the next level to do more complex calculations or communications to private services via the Express server.

Till next time!

Visualizing Fibonacci: For the Music Lover in You!

Daniel McMahon — Tue, 06 Feb 2018 23:12:38 +0000

First time article so excuse the typos and errors!

A Common Problem: Programming the Fibonacci Series

Throughout my recent computer science conversion course I studied introductions to both Python and Java. Across both languages one of the common challenges presented was to be to write a program to display up to the n-th number of the Fibonacci Series. This seems to be a common stepping stone for most programmers when learning a new language.

Having found myself wanting to apply what I was learning in class to an independent project, I had stumbled across this idea of visualizing the Fibonacci Series as sheet music. I don't quite remember where or when I had heard of the idea - possibly from Matt Parker, Hannah Fry or one of my lecturers in passing. I had googled the topic and didn't find any immediate practical implementation of the Fibonacci series being visualized on sheet music. Therein lay my task. In retrospect perhaps a quick github search would have yielded more results - but again I was a younger and more naive developer then I am today.

Python - Which Libraries?

Ok so I had the overall project idea - generate up to the n-th number of the Fibonacci series and somehow visualize that data on sheet music. After some quick research I had settled on using a Python library called Mingus in combination with the program Lilypond

How not to waste time rebuilding a python2 library to work with python3...

This should go without saying, at the time I was less experienced and prone to these errors. I ended up downloading a version of Mingus that ran on Python2 while I was running Python3. Rather than looking up if a Python 3 package existed, I ended up doing some manual tweaking of the package to make it work. D'oh! For the younger Dev's out there be sure to check the python versions of any installed packages!

The Chromatic Scale

While not the most elegant solution I ended up essentially modifying a previously developed college solution that outputted the Fibonacci series up to the n-th number, and used a modulus of that number to determine which note it would represent. Using a modulus of 12 I was able to associate the various numbers to all the notes in a chromatic scale starting on a base note of C (I only used sharps for convenience sake)

After establishing the connection between the Fibonacci series and the chromatic notes of the scale the only 'difficult part' from there was essentially figuring out how to translate these generated notes into sheet music.

After importing the Mingus library this was actually pretty straight forward!
The code looked a little like this (yes I know its a little inefficient - I haven't gone back and refactored the code since it was originally thrown together):

from mingus.containers import Bar
from mingus.containers import Composition
from mingus.containers import Track
from mingus.containers import NoteContainer
import mingus.extra.LilyPond as LilyPond

x = int(input('Song Length (number of notes): '))

song = []
b = Bar() # creates new bar element for Mingus = has limit of 4/4 timing by default
b.set_meter((4,4))

c = Composition()
c.set_author('Daniel McMahon', 'daniel40392@gmail.com')
c.set_title('Ode to Fibonnachi')
t = Track()
t.add_bar(b)
c.add_track(t)

def fib(n):
    """Function that generates notes based off the Fibonnachi Series"""
    a=0
    b=1
    print('Generating Music.... Please Wait....',end='')

    for i in range (1,n+1):
        fib=a+b
        if fib%12==1:
            song.append('C')
        elif fib%12==2:
            song.append('C#')
        elif fib%12==3:
            song.append('D')
        elif fib%12==4:
            song.append('D#')
        elif fib%12==5:
            song.append('E')
        elif fib%12==6:
            song.append( 'F')
        elif fib%12==7:
            song.append('F#')
        elif fib%12==8:
            song.append('G')
        elif fib%12==9:
            song.append('G#')
        elif fib%12==10:
            song.append( 'A')
        elif fib%12==11:
            song.append('A#')
        elif fib%12==0:
            song.append('B')
        a=b
        b=fib

    #this code will print song in reverse
    #for i in reversed(song):
    #    print (i + ' ,', end='')

    n = NoteContainer(song)

while x != '':
    fib(x)
    print()
    for note in song:
            b + note
            c.add_note(note)
    comp = LilyPond.from_Composition(c)
    LilyPond.to_pdf(comp, "Ode_to _Fibonnachi")
    print("Song completed - check desktop for exported PDF")
    x = int(input('Song Length (number of notes): '))

The Result?

Below is a small snapshot of the result - an interesting pattern emerges for the outputted music showing a repeated pattern every 25 or so notes!

A link to the GitHub repo is available here with the sample output available as a PDF file.

Some additional ideas and experiments...

This sheet music generation approach would work well with other mathematical series - the immediate ones that I thought of adopting include the Catalan and Figurate series. Some sample Fibonacci note generations for these are also included in the linked repo above.

It may also be worth experimenting with starting on a different note of the chromatic scale - however the intervals will still remain the same between the notes.

The next logical step in this project is to get musical output from the generated sheet music. I had initially tried this when working on the project 2 years ago on an old Windows laptop, I will have to go back to it soon and see if I can get some live musical feedback integrated.

Another interesting approach I've taken in the repository is using the NLTK library to generate the frequency of note appearances in different series of Fibonacci numbers. You get some fascinating results using different mathematical series - I recommend experimenting with it yourself!

Some Final Thoughts

The project itself was used as a basic way to integrate my love of music into my coding. The Fibonacci series can be considered quite a boring topic when attempting to implement it in a new language, but when you add in some of these basic additional steps you can generate something really interesting and creative!

I would always encourage new developers to explore the basic building blocks of programming languages in new and creative ways. Personally you may find it more rewarding - plus its a great way to build up your personal portfolio and have something extra to chat about in any future job interviews!

Perhaps you've tried programming something similar in different languages with interesting results - I'd love to hear about it - be sure to leave any feedback or comments below.

Again excuse any typos or formatting issues - its my first of hopefully many posts here!

Warmest Regards,
Daniel McMahon