DEV Community: Michael Seymour

Why Are There So Few Women Programmers?

Michael Seymour — Tue, 27 Sep 2022 10:48:29 +0000

Only 5% of developers are women (source: StackOverflow Developer Survey 2022). Someone recently asked me why there's so few women in tech. "It’s not a physically demanding job so why not?" he said.

I didn't really have a good answer to explain it, so I had a look around on the internet and made a summary of the top reasons I could find.

I tried keeping the article succinct and easy to read so excuse the surface-level analysis.

Stereotypes

How your parents, teachers, media etc. treat you while you’re growing up influences your career choice. Some examples:

If a friend has a broken computer, a parent might tell their daughter to 'stay away and let the man do it'
A parent might buy toy cars, guns, legos etc. for a boy which are much more related to programming/engineering than barbies, cooking toys, etc.

Women Don’t Want to Join a Male-Dominated Career

There are so few women programmers because there are so few women programmers to begin with.

Being in a career dominated by the opposite sex can have it’s problems, such as:

Greater chance of facing sexist comments, harassment etc.
Being singled out or getting special treatment
Masculine terms being used by default

Lack of Role Models

Having role models are important. If a girl has an interest in programming but doesn’t see any other girl programmers, she might see that as a red flag and pursue another interest.

Controversial Reason: Biological Differences

This is a very controversial reason made popular by James Damore who got fired from Google for writing a memo suggesting this.

I’m just mentioning it for the sake of completeness but note that none of his theories have been scientifically proven. The gist is:

The brains of men and women are wired up differently which could explain some of the stereotypical differences in male and female behavior.

I've turned this post into a TikTok video which you can view on my account here if you'd like:
Why are there so few women software developers?

Salary Progression: $4k to $150k per year

Michael Seymour — Fri, 23 Sep 2022 09:22:59 +0000

How a software developer went from $4k/year to $150k/year in 8 years:

Michael

@mikeseym

How a software developer went from $4k/year to $150k/year in 8 years

09:13 AM - 23 Sep 2022

The Ultimate Guide to Reverting Git Merges

Michael Seymour — Sun, 08 May 2022 15:20:33 +0000

How to Undo an Unpushed Merge

If you’ve done a merge and haven’t pushed it yet, undoing the merge is very simple.

Use the git reset command to return to the revision before the merge, thereby effectively undoing it.

$ git reset --hard <commit-hash-before-merge>

If you don't have the hash of the commit before the merge at hand, you can use git log to find it out or you can use the following variation of the reset command:

$ git reset --hard HEAD~1

This way, using HEAD~1, you're telling git to go back to the commit before the current HEAD revision, which should be the commit before the merge.

Note: In both cases, we need to use the --hard option. This means that any local uncommitted changes will be discarded - if you have valuable uncommitted changes, be sure to use git stash before.

After doing the above git reset, you’ll notice that any changes from the merge will be undone and you can continue working as normal.

How to Undo a Pushed Merge

Undoing a pushed merge is a bit more complicated. Before getting to it, we need to learn a bit more about commits and git merges.

What is a commit parent?

Every commit has a parent commit. When you git commit normally, the current commit becomes the parent commit of the new commit that you’re creating.

To view the parent of a commit, use git show --pretty=raw <commit-hash>.

What is a merge commit?

When a branch named feature is merged with master, a new “merge commit” is created on the branch master. A merge commit is the same as a normal commit except that it has 2 parents. In this case, the 2 parents of the merge commit will be the previous head of master and the head of feature.

On running git show, the new commit displays both the parents:

$ git show

commit ae2058cf5cafe807af44114d15bac65fc4efd714 (HEAD -> master)
Merge: bf75d61d8f1 12d62bfa0e0

How does git revert work?

git revert <commit-hash>

The revert command in git takes in a commit hash and compares the changes with the parent. The delta or the diff is calculated and the negation of it is applied as a new commit.

What happens if we want to revert a merge commit? The merge commit has 2 parents and git won’t know automatically which parent was the mainline, and which parent was the branch you want to unmerge.

Reverting a merge commit

$ git revert -m 1 <merge-commit-hash>

Let's take a closer look at what this command will do:

git revert will make sure that a new commit is created to revert the effects of that unwanted merge.
The -m 1 option tells git that we want to keep the parent side of the merge (which is the branch we had merged into).
Finally, make sure to specify the actual merge commit's hash.

The -m option specifies the parent number. This is because a merge commit has more than one parent, and git does not know automatically which parent was the mainline, and which parent was the branch you want to unmerge.

When you view a merge commit in the output of git log, you will see its parents listed on the line that begins with Merge:

commit ae2058cf5cafe807af44114d15bac65fc4efd714
Merge: bf75d61d8f1 12d62bfa0e0
Author: Michael Seymour <michael@example.com>
Date:   Sun May 08 13:52:29 2022 +0100

Merge branch 'my-branch'

In this situation, git revert ae2058cf5ca -m 1 will get you the tree as it was in bf75d61d8f1, and git revert -m 2 will reinstate the tree as it was in 12d62bfa0e0.

To better understand the parent commits, you can run:
git log bf75d61d8f1 or git log 12d62bfa0e0

Why can’t we use git reset this time?

git revert will make sure that a new commit is created to revert the effects of that unwanted merge. This is in contrast to git reset, where we effectively "remove" a commit from the history. Hence, git revert is a better solution in cases where you've already pushed to remote as altering the history of the remote repo can cause problems for the other developers using the repo.

References

The SOLID principles of Object Oriented Design

Michael Seymour — Tue, 10 Nov 2020 00:51:21 +0000

What is SOLID?

SOLID helps you to write code that is easy to maintain, extend and understand.

It is an acronym for the following 5 principles:

S = Single-responsibility principle
O = Open-closed principle
L = Liskov substitution principle
I = Interface segregation principle
D = Dependency inversion principle

Single-responsibility principle

A class/module should only be responsible for one thing.
A class/module should have only one reason to be changed.

Here's an example that violates this principle:

public class Customer {
    public void add(Database db) {
        try {
            db.execute("INSERT INTO...");
        } catch (Exception e) {
            File.writeAllText("/var/log/error.log", e.toString());
        }
    }
}

The Customer class is responsible for both writing to the database and writing to the logfile.

If we want to change the way we log errors, Customer needs to change.
If we want to change the way we write to the DB, Customer needs to change.

This code should be refactored to:

class Customer {
    private FileLogger logger = new FileLogger();
    void add(Database db) {
        try {
            db.execute("INSERT INTO...");
        } catch (Exception e) {
            logger.log(e.toString());
        }
    }
}
class FileLogger {
    void log(String error) {
        File.writeAllText("/var/log/error.log", error);
    }
}

Some other examples where you'd need separate classes are: user input validation, authentication, caching.

Be careful not to over-fragment code (creating too many responsibilities). Remember the whole point of SOLID is to make your code easier to maintain.

Further reading:

NDepend

Open-closed principle

Classes/modules should be open for extension but closed for modification.
Rather extend functionality by adding new code instead of changing existing code.
The goal is to get to a point where you can never break the core of your system.

Here's an example that violates this principle:

public void pay(Request request) {
    Payment payment = new Payment();
    if (request.getType().eq("credit")) {
        payment.payWithCreditCard();
    } else {
        payment.payWithPaypal();
    }
}
public class Payment {
    public void payWithCreditCard() {
        // logic for paying with credit card
    }
    public void payWithPaypal() {
        // logic for paying with paypal
    }
}

What if we wanted to add a new payment method? We would have to modify the Payment class, which violates the open-closed principle.

This code should be refactored to:

public void pay(Request request) {
    PaymentFactory paymentFactory = new PaymentFactory();
    payment = paymentFactory.initialisePayment(request.getType());
    payment.pay();
}
//-----------------------
public class PaymentFactory {
    public Payment intialisePayment(String type) {
        if (type.eq("credit")) {
            return new CreditCardPayment();
        } elseif (type.eq("paypal")) {
            return new PaypalPayment();
        } elseif (type.eq("wire")) {
            return new WirePayment();
        }
        throw new Exception("Unsupported payment method");
    }
}
//-----------------------
interface Payment {
    public void pay();
}
class CreditCardPayment implements Payment {
    public void pay() {
        // logic for paying with credit card
    }
}
class PaypalPayment implements Payment {
    public void pay() {
        // logic for paying with paypal
    }
}
class WirePayment implements Payment {
    public void pay() {
        // logic for paying with wire
    }
}

Now we can add new payment methods by adding new classes, instead of modifying existing classes.

Further reading:

NDepend

Liskov substitution principle

If a class implements an interface, it must be able to substitute any reference that implements that same interface.
e.g. if a class called MySQL implements Database, and another class called MongoDB implements Database, you should be able to substitute MySQL objects for MongoDB objects.

Here's an example that violates this principle:

public abstract class Bird {
    public abstract void Fly();
}
public class Parrot : Bird {
    public override void Fly() {
        // logic for flying
    }
}
public class Ostrich : Bird {
    public override void Fly() {
         // Can't implement as ostriches can't fly
        throw new NotImplementedException();
    }
}

The above is a bad design as Bird assumes all birds can fly.

This code could then be refactored to:

public abstract class Bird {
}
public abstract class FlyingBird : Bird {
    public abstract void Fly();
}
public class Parrot : FlyingBird {
    public override void Fly() {
        // logic for flying
    }
}
public class Ostrich : Bird {
}

The gist of this principle is to be careful when using polymorphism and inheritance.

Here is another, more real-world encounter of this principle:

You have a class called BankAccount with a withdrawal() method. Do all bank accounts allow withdrawals? A fixed deposit account won't allow withdrawals, for example.

Further reading:

NDepend

Interface segregation principle

No client should be forced to depend on methods it does not use.

Here's an example that violates this principle:

public interface Athlete {
    void compete();
    void swim();
    void highJump();
    void longJump();
}
public class JohnDoe implements Athlete {
    @Override
    public void compete() {
        System.out.println("John Doe started competing");
    }
    @Override
    public void swim() {
        System.out.println("John Doe started swimming");
    }
    @Override
    public void highJump() {
    }
    @Override
    public void longJump() {
    }
}

JohnDoe is just a swimmer, but is forced to implement methods like highJump and longJump that he'll never use.

This code could then be refactored to:

public interface Athlete {
    void compete();
}
public interface SwimmingAthlete extends Athlete {
    void swim();
}
public interface JumpingAthlete extends Athlete {
    void highJump();
    void longJump();
}
public class JohnDoe implements SwimmingAthlete {
    @Override
    public void compete() {
        System.out.println("John Doe started competing");
    }
    @Override
    public void swim() {
        System.out.println("John Doe started swimming");
    }
}

Now, JohnDoe does not have to implement actions that he is not capable of performing.

Further reading:

Dependency inversion principle

High-level modules should not depend on low-level modules. They should depend on abstractions.
This allows you to change an implementation easily without altering the high level code.

Here's an example that violates this principle:

public class BackEndDeveloper {
    public void writeJava() {
    }
}
public class FrontEndDeveloper {
    public void writeJavascript() {
    }
}
public class Project {
    private BackEndDeveloper backEndDeveloper = new BackEndDeveloper();
    private FrontEndDeveloper frontEndDeveloper = new FrontEndDeveloper();
    public void implement() {
        backEndDeveloper.writeJava();
        frontEndDeveloper.writeJavascript();
    }
}

The Project class is a high-level module, and it depends on low-level modules such as BackEndDeveloper and FrontEndDeveloper. This violates the principle.

This code should be refactored to:

public interface Developer {
    void develop();
}
public class BackEndDeveloper implements Developer {
    @Override
    public void develop() {
        writeJava();
    }
    private void writeJava() {
    }
}
public class FrontEndDeveloper implements Developer {
    @Override
    public void develop() {
        writeJavascript();
    }
    public void writeJavascript() {
    }
}
//-----------------------
public class Project {
    private List<Developer> developers;
    public Project(List<Developer> developers) {
        this.developers = developers;
    }
    public void implement() {
        developers.forEach(d -> d.develop());
    }
}

Now, the Project class does not depend on lower level modules, but rather abstractions.

Further reading:

Final note: Don't be too strict with SOLID principles

SOLID design principles are principles, not rules.
Always use common sense when applying SOLID (know your trade-offs).
Usually with SOLID, it requires more time writing code, so you can spend less time reading it later.
Finally, remember to use SOLID as a tool, not as a goal.

Types of NoSQL Databases

Michael Seymour — Thu, 05 Nov 2020 22:19:09 +0000

Why do we need this blog post?

Most people have only had experience with relational databases but there are many other types of databases, each with their own pros and cons, that developers should know when to use.

Why are relational databases most popular?

The theory behind non-relational databases has been around since the 1970's, but back then, storage was extremely expensive. Thus, database types that optimised for storage space, such as relational databases, are the ones that became popular. Relational databases store normalised data, meaning no redundant data is stored. Databases that weren't normalised were not financially viable at the time. In the 80-90s, people kept using relational databases and then in the 2000's, NoSQL databases started popping up because we were no longer constrained by storage, but rather, the main bottleneck now is compute. However, despite this rise in popularity, relational databases are still used the most today. This is most likely because people already have a lot of experience with them, it is often the type of database that developers first learn to use and also because many apps are already built with relational databases.

Types of databases

Relational

Characteristics
- Stores data in one or more tables of columns and rows, with a unique key identifying each row.
- Supports referential integrity
- Supports ACID transactions
- Strict schema enforcement (schema-on-write)
- Queried using SQL
Examples
- MySQL, MariaDB, PostgreSQL, Oracle, MSSQL
Pros
- Data integrity is enforced
- Avoids data duplication
- ACID-compliant
- Well-documented, lots of tutorials and stackoverflow answers
Cons
- Difficult to scale writes
- Has row locking
- Not designed to handle unstructured/semi-structured data
Use cases
- An ecommerce platform where you have users linked to orders linked to items etc., and many other use cases

Key-value

Characteristics
- Allows storing arbitrary data via a specific key
- Schemaless (values can be anything)
- It is effectively a dictionary/hash table stored on disk
Examples
- Amazon DynamoDB, RocksDB
Pros
- Low latency reads & writes
- Scales easily and cheaply
Cons
- Can only query/search by key
- No data integrity
Use cases
- Config data, url shortener etc.

In-memory key-value

Characteristics
- Same as key-value, but data is stored in memory (RAM)
- Can be configured to persist data, but not the primary focus
Examples
- Redis, memcached, Amazon ElastiCache
Pros
- Extremely low latency
Cons
- Can be hard to scale
Use cases
- Caching, leaderboards, real-time analytics, etc.

Document (a.k.a. document-orientated)

Characteristics
- Similar to key-value, but value is a document
- MongoDB actually uses a key value store called WiredTiger as their default storage engine
- Documents are typically JSON, BSON, XML
- Schemaless
- Documents are stored in collections (not tables)
- Optimised more for speed and scalability rather than space
Examples
- MongoDB, DocumentDB, Couchbase, DynamoDB, Firebase
Pros
- Fast writes
- Scalability
- Schemaless allows changing data structure without downtime
Cons
- Not ACID compliant
- Relationships/cross references not enforced
- Can't join documents/collections in a single query
- Joins can be slow
- Data is usually duplicated
Use cases
- Storing user metadata, product inventory etc., and many other use cases

Wide-column (a.k.a. column-orientated)

Characteristics
- Somewhere in between a document store and a relational DB
- Uses tables, columns, rows etc.
Examples
- Cassandra, Keyspaces (AWS), HBase, Bigtable
Pros
- Same pros as a document db
- Can query/update individual columns
- More efficient aggregation operations
Cons
- Same cons as a document db
Use cases
- Storing events, messages etc.

Graph

Characteristics
- Based on graph theory
- Data represented as network of nodes, edges and properties
- Data is stored in nodes (schemaless)
Examples
- Neo4j, OrientDB, TitanDB, Neptune (AWS)
Pros
- Optimised for querying data with complex relations
Cons
- Difficult for developers to learn
- Poor performance of aggregation operations
Use cases
- Social networks, recommendation engines, gps navigation

Search

Characteristics
- Stores non-relational, document-based data
- Built to be able to index large volumes of full text (e.g. logs)
- Primary focus is on full text search and fuzzy matching
Examples
- Elasticsearch, Splunk, Apache Solr
Pros
- Optimised full text searching
- Highly scalable
- Schemaless
- Other search options such as suggestions
Cons
- Expensive
- Low durability
- No transaction support
- Not efficient for writing/reading other than searching
- Can be difficult to manage
Use cases
- Search results on ecommerce website, searching logs, input form autosuggest

Time series

Characteristics
- Collect, store and process data sequenced by time
- Heavily write oriented
- Designed to handle constant streams of data
- Typically append-only (on modification after ingestion)
- Aggregation/down sampling features to lower archive data footprint
Examples
- InfluxDB, Prometheus, graphite, Timestream (AWS)
Pros
- Can handle high ingestion rates
- Optimised for time-based aggregation queries (mean, std dev, max etc.)
Cons
- Can only deal with time-series data
- Reads are a lot slower than writes
- Does not support transactions
- Append-only
Use cases
- IoT sensor collection, app metrics, log monitoring, alerting

Ledger

Characteristics
- An immutable database
- If you try to update a value, it creates a new entry with a diff
Examples
- Amazon Quantum Ledger Database
Pros
- Complete, immutable, and verifiable history of all changes to application data
- Also provides cryptographic mechanism to verify integrity of data
Cons
- Can grow large in size
Use cases
- Highly regulated industries e.g. banking, systems of record, supply chain, healthcare, registrations

Bonus Section! (hosting options)

There are 3 database hosting options:

On-premises
- Database fully maintained by organization on servers running within their data centers
- More control, but usually more expensive and time consuming
Cloud hosted
- Servers are maintained by cloud providers
- Organizations maintain database software and operating system running on the machine
- Flexible scaling and no server upkeep, but no control over physical server and potential network limitations
Serverless (database-as-a-service)
- Database maintained by service provider
- Organizations only charged for usage of service (storage, bandwidth etc.)
- Can be cost effective
- Zero upkeep: database maintenance, patching, infrastructure support etc. is outsourced

Video Call Bingo

Michael Seymour — Fri, 27 Mar 2020 22:03:57 +0000

This is a Bingo game that you can play with fellow colleagues/friends during video calls. Click on the squares to mark them as they occur during your video call. First player to mark 5 squares in a row (horizontally, vertically or diagonally) wins.

It's just something quick I made now that I'm doing a lot more video calls while working from home.

https://bingo.seymour.co.za/

MySQL Schema Change Without Downtime

Michael Seymour — Wed, 22 Jan 2020 20:12:53 +0000

When altering large MySQL tables (> 200MB) it can block reads or writes for a long time. In this post, I'll explain how to alter a large table without any downtime.

First, let's look at how MySQL does an ALTER TABLE:

Lock the table
Make a copy of the table
Modify the copy (the "new table")
Copy all the rows into the new table
Swap the old and new tables
Unlock the table

How can we avoid the blocking (steps 1 & 6)? Here are some options:

Do it at 2am on Sunday morning
- Problem: Not fun for developers, lowers developer productivity.
Do a planned failover (aka. "rolling schema upgrade")
- Start with slaves and, once the schema change is applied to the slave, promote one of the slaves to the new master, demote the old master to a slave and execute the schema change on it.
- Problem: Can break replication for some schema changes, requires high availability infrastructure.
Make MySQL use row versioning
- Tell MySQL this table has a new version of the schema. Then, you can still read from old row versions but when writing a new row, it writes in the new row format.
- Problem: MySQL still doesn't support row versioning.
Handle the ALTER process yourself
- Manually perform the same steps the server takes (without locking).
- Problems: discussed below

The last option is my preferred so I'll go ahead and explain that. You perform the ALTER steps manually, with the following changes:

~~Lock the table~~
Make a copy of the table
Modify the copy (the "new table")
Copy all the rows into the new table
1. Add triggers to keep track of changes
Swap the old and new tables
~~Unlock the table~~

As we're not locking the table, your application can continue to read/write to the original table. It will take some time to copy rows to the new table, so we add triggers to the old table that update or insert corresponding rows in the new table.

Some notes on the above process:

Remember to make backups before trying this
Do multiple, smaller chunks of INSERT..SELECT when copying data to the new table
- Helps to avoid deadlocks
- Try make the chunks small enough so each one takes about 0.5s
- Start at the low-end of the primary key
The triggers should only copy columns that are common between the 2 tables.
To swap the old and new tables, we can atomically rename the old table to something else and the new table to the old table.
- Don't forget to alter the foreign keys to point to the new table

Problems with above process:

It's a lot slower than just letting MySQL handle the ALTER
The table you're altering can't have triggers already defined

I hope this post has helped if you find yourself altering some large tables. If you have any questions or corrections, you can message me @mikeseym on twitter.

Solving a MySQL Deadlock

Michael Seymour — Mon, 13 Jan 2020 20:56:15 +0000

A deadlock occurs when you have 2 processes that are both waiting on each other to release a lock on a resource.

Imagine we have 2 threads in a Java application:

Thread 1 acquires a lock on resource A
Thread 2 acquires a lock on resource B
In order to continue execution (and release the lock on resource A), Thread 1 waits until resource B is free
In order to continue execution (and release the lock on resource B), Thread 2 waits until resource A is free

Neither thread will ever finish execution, and our app has reached a deadlock.

You can actually try this for yourself on a MySQL table:

mysql> CREATE TABLE a (id int primary key, value int);
Query OK, 0 rows affected (0.01 sec)

mysql> INSERT INTO a VALUES (1, 0), (2, 0), (3, 0), (4, 0);
Query OK, 4 rows affected (0.01 sec)
Records: 4  Duplicates: 0  Warnings: 0

mysql console 1:
step 1> START TRANSACTION;
step 3> UPDATE a SET value = 1 WHERE id = 2;
step 5> UPDATE a SET value = 1 WHERE id = 1;

mysql console 2:
step 2> START TRANSACTION;
step 4> UPDATE a SET value = 1 WHERE id = 1;
step 6> UPDATE a SET value = 1 WHERE id = 2;

ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction

Recently I was working on a multi-threaded application that had this problem. It was a Java application with 16 threads. Each thread would update a batch of 10 rows in a MySQL table every 1ms. When updating a row in MySQL, an exclusive lock is set on that particular row - blocking any other transaction from updating it. It didn't happen often, but on occasion 2 threads would be waiting on each other to release a row lock, causing a deadlock.

I considered a few different solutions:

Restarting the transaction if a deadlock occured.Problem: I'd be updating my rows with stale data
Restarting the entire process if a deadlock occured.Problem: This could potentially make the app a lot slower.
Make each thread ‘claim' a row before trying to update it. If it's already claimed, find another row to update.Problem: This adds a lot of complexity to the app and slows down performance as well

Before continuing, I took a break and thought more about the cause of a deadlock. All threads were performing 10 updates each on the same MySQL table. The only way a deadlock could occur is if the order and timing matched a very particular pattern. I couldn't control the timing, but I could control the order of updates.

In fact, if I ordered all updates in the same way (e.g. by ascending order of the row id), this would eliminate the possibility of a deadlock.

But why are updates occuring in a random order anyway? It turns out the application fetches 100 rows from the database and picks 10 to update. However, in order to prevent the 16 threads from always updating the same 10 rows, it would shuffle the list of 100 rows first. This means that the 10 updates would always occur in some random order.

I modified the application so that before updating the 10 rows, it would order the updates in ascending order of the row id. I deployed the app and that was the end of the deadlocks.

Deadlocks can occur in many different ways, but in this case, the lesson learned is:

If you're locking resources in a multi-threaded application, make sure all threads are locking in the same order (order based on an attribute of the resource).

In simpler terms:

If you're updating rows in a multi-threaded application, make sure all threads are updating the rows in order of the row id (ascending or descending).