DEV Community: Mykola

When Postgres index meets Bcrypt

Mykola — Fri, 31 Jan 2025 07:45:33 +0000

Hello there! In the previous post “What Okta Bcrypt incident can teach us about designing better APIs”, we discussed the 72-chars limit of the input value of the Bcrypt hashing algorithm that caused quite a big security incident in the industry. That reminded me about another example of Bcrypt misuse that I, personally, came across a few years ago while investigating a quite nasty performance issue with one of the services. Let's jump right into it!

A product manager of one of the neighboring teams approached me and asked if I could help them with the performance degradation that had been experienced lately with their newest feature, when the users were prompted to enter their SSN (social security number), and they'd get a dashboard with the personalized data about them. The simplified architecture looked like this:

As you can see, the flow consists of 3 steps:

The user enters their SSN in the UI, and the UI sends it to the Internal service.
Internal service checks whether the data for the user with such SSN is already present in the DB (Postgres): if yes, that data is returned to the UI
Otherwise, the request is made to the Third-party API to fetch the desired data, then it's being processed, saved into the DB, and returned to the user

The nature of the user data didn't require real-time updates, so the same data was stored and used for the month, after which it was considered to be expired, and was deleted by the cronjob. A pretty straightforward system.

The issue (the team was experiencing) was the fact that it took 10+ seconds to follow the 3-steps flow we discussed above, which led to a terrible UX and a high bounce rate. That made a good sense, as the average attention span of the users nowadays is way less than 10 seconds. But why was it so slow?

The bug issue was reproducible in the production setup, the logs/metrics were not so useful with the clues for the cause. So, I cloned the project code to my laptop and launched a Postgres instance via Docker Compose. Additionally, I started mitmproxy to be able to intercept and inspect HTTP requests on my machine, and created a template of the request to the Internal service API with my own SSN in Postman. My debugging setup was ready, so it was time to run and test the app.

Running it locally

To my great surprise, after sending a request from Postman, I got a response in the matter of tens of milliseconds. mitmproxy showed that there was indeed the request to the Third-party API, but it had a minimal latency.

Hm…interesting. I tried repeating the same request again, and again got a pretty quick answer, but this time without any calls to the third-party API. That made a perfect sense, as my user info was present in Postgres. I tried sending the same requests a few more times, but didn't have any luck with reproducing the issue.

Despite the failure, that gave me a good clue: it seemed like I could exclude the Third-party API from the list of suspects, or at least for now. Of course, it's important to mention that the local setup is different from the production one, so there is always a chance that something might be slowing down the egress traffic. However, for simplicity of the investigation, I decided to make sure that nothing in the app caused the issue before checking possible external factors, like cloud proxies, security filters, etc.

I realized that it's time to take a look at the code. If you'd like to follow along, here are the code samples that reproduce the setup we are discussing, check the README.md file for how to run instructions.

Btw, if you like my blog and don’t want to miss out on new posts, consider subscribing to my newsletter here. You'll receive an email once I publish a new post.

Browsing the code

You might know that there is a saying “it's always DNS”, which applies when some service is down due to no obvious reasons. From my experience, when there is a performance issue with a relatively simple app, “it's often database”. So, even though, I started navigating the source code from the top to bottom (API -> DB), I didn't expect to see anything suspicious within the layers above the one that runs DB queries. And, indeed, the interesting bits of code were 2 functions of the Repo struct responsible for inserting a new user into the DB and selecting a particular user from the DB by SSN. Here how they looked:

func (r *Repo) SelectUserBySsn(ssn string) (*common.User, error) {
    now := time.Now()

    var user common.User
    err := r.dbPool.QueryRow(
        context.Background(),
        "SELECT id, user_info FROM users WHERE ssn_hash = crypt($1, ssn_hash)",
        ssn,
    ).Scan(&user.Id, &user.UserInfo)
    if err != nil {
        if errors.Is(err, pgx.ErrNoRows) {
            return nil, nil
        }
        fmt.Println(err)
        return nil, err
    }

    fmt.Println("Time taken to get user:", time.Since(now))
    return &user, nil
}

func (r *Repo) InsertUser(user common.NewUserEntity) error {
    _, err := r.dbPool.Exec(
        context.Background(),
        "INSERT INTO users (id, ssn_hash, user_info) VALUES ($1, crypt($2, gen_salt('bf')), $3)",
        user.Id, user.Ssn, user.UserInfo,
    )
    if err != nil {
        fmt.Println(err)
        return err
    }
    return nil
}

The crypt($1, ssn_hash) and crypt($2, gen_salt('bf') parts immediately caught my eyes: “aha, so we are not storing plain SSNs in the DB, but hashed versions”. Interesting discovery! This didn't give me any hints on the exact algorithm that was used for hashing, but a quick Google search by the gen_salt('bf') part helped me to discover that it's a Bcrypt one. And indeed, after I did SELECT * FROM users in my local setup, I noticed that the ssn_hash column had this value $2a$06$r4UXr0F80tz57DGzCQBf6uIpQealvM3Rb6E/TvsyFkJJaiLXt9J8W, and the $2a$ prefix was indeed a Bcrypt one. All right, we are into something! The chances are that some of you, who are experts in cryptography and/or databases (or, Postgres, at least), already can see the reason for the poor performance of the app (kudos to you!), but that was not the case for me, so I had to keep digging.

I realized that DB was my best lead at the moment, so I jumped away from the source code to the Postgres console in my IntelliJ IDEA, and ran this query: SELECT * FROM users WHERE ssn_hash = crypt('0852285111', ssn_hash) (0852285111 is my fake SSN in this example). I got the result in 9 milliseconds, which was quite fast, and didn't hint any issues with the performance.

But then I connected to the production DB and ran the same query. It took 15 seconds to get the similar response. Wow! What's more interesting, was the fact that fetching all the existing users via SELECT * FROM users took only 6 milliseconds. Puzzling, isn't it?

Despite being a bit puzzled, that definitely felt like good progress, as I had a strong clue that DB was the bottleneck here. And it became clear that my failures to reproduce that locally was due to the fact that my local table had only 1 row, while the production one had 5000 users as of that moment, and kept growing hour by hour. I could have kept debugging the issue within the production Postgres instance, as it was reproducible there, but doing that is not a good practice due to many reasons, like degrading performance even more, or running some unsafe queries that might do some harm to the data there. So, at first, I wanted to take a snapshot of the production DB and use it locally, but realized that it was not much better idea as well: taking a snapshot might be a costly operation for Postgres. Plus, there were privacy implications of me copying the sensitive data of the customers to my local machine. That's why I went with the third option of inserting a lot of fake data into my local DB.

Fake it until you make it

The fact that our table was so simple and consisted of 3 columns only, made the task very simple. So, I ended up writing this fake data generation code:

func GenRandomUsers(num int) []common.NewUserEntity {
    users := make([]common.NewUserEntity, num)
    for i := 0; i < num; i++ {
        users[i] = GenRandomUser()
    }
    return users
}

func GenRandomUser() common.NewUserEntity {
    ssn := genRandomSsn()
    fmt.Println("Generated SSN:", ssn)
    return common.NewUserEntity{
        Id:       uuid.New().String(),
        Ssn:      ssn,
        UserInfo: GenRandomUserInfo(),
    }
}

func GenRandomUserInfo() string {
    return fmt.Sprintf("Some info %s %s", strconv.FormatInt(time.Now().UnixMilli(), 10), uuid.New().String())
}

func genRandomSsn() string {
    ssn := ""
    for i := 0; i < 10; i++ {
        ssn += strconv.Itoa(rand.IntN(10))
    }
    return ssn
}

And made a use of it within the fillDb function that I triggered on the app start up:

func fillDb(repo *db.Repo) {
    users, err := repo.SelectUsers()
    if err != nil {
        panic(err)
    }
    if len(users) > 0 {
        fmt.Println("users already exist, skipping db fill")
        return
    }

    fmt.Println("generating random users...")
    newUsers := utils.GenRandomUsers(5000)

    fmt.Println("inserting random users...")
    err = repo.InsertUsers(newUsers)
    if err != nil {
        panic(err)
    }
    fmt.Println("inserted random users")
}

Once I ran the app, it took more than 15 seconds to finish inserting the users, which I found weird, as the InsertUsers code made only 1 call to DB:

func (r *Repo) InsertUsers(users []common.NewUserEntity) error {
    now := time.Now()

    query := "INSERT INTO users (id, ssn_hash, user_info) VALUES "
    var args []interface{}
    for i, user := range users {
        query += fmt.Sprintf("($%d, crypt($%d, gen_salt('bf')), $%d),", i*3+1, i*3+2, i*3+3)
        args = append(args, user.Id, user.Ssn, user.UserInfo)
    }
    query = query[:len(query)-1] // remove the trailing comma

    _, err := r.dbPool.Exec(
        context.Background(),
        query,
        args...,
    )
    if err != nil {
        fmt.Println(err)
        return err
    }
    fmt.Println("Time taken to insert users:", time.Since(now))
    return nil
}

It became clearer and clearer to me that the crypt($%d, gen_salt('bf')) part was to blame, but I didn't understand why it was so just yet. Anyway, the intermediate goal was achieved, as at that point of time, I could reproduce the performance issue in my local setup as well, the SELECT * FROM users WHERE ssn_hash = crypt('0852285111', ssn_hash) query became as slow as within the production DB, even though my SSN wasn't among the existing DB records.

I decided to take a pause for a moment and check the facts:

the reasons of the slow performance were queries to the DB
the performance degraded with the growth of the number of the records within the DB table
something is fishy with the crypt() function for both INSERT and SELECT queries

The second fact was a good hint, so I decided to check what Postgres query planner could tell me about the plan for executing the query:

EXPLAIN
SELECT *
FROM users
WHERE ssn_hash = crypt('0852285111', ssn_hash);

The keyword EXPLAIN in this case triggers Postgres to build the plan for the query, but not run it. The results were like this:

Seq Scan on users  (cost=0.00..182.00 rows=25 width=138)
"  Filter: (ssn_hash = crypt('0852285111'::text, ssn_hash))"

Aha, Seq Scan on users gives us a hint that Postgres is about to perform a sequential scan, which means that it will go through every single row of the table to find the matching one - O(n) if we'd like to use the algorithmic complexity here. This could very much explain the correlation between the increase in the number of rows and performance degradation.

Let's add ANALYSE keyword to actually execute the query and get more metrics:

EXPLAIN ANALYSE
SELECT *
FROM users
WHERE ssn_hash = crypt('0852285111', ssn_hash);

The output:

Seq Scan on users  (cost=0.00..182.00 rows=25 width=138) (actual time=15037.772..15037.774 rows=1 loops=1)
"  Filter: (ssn_hash = crypt('0852285111'::text, ssn_hash))"
  Rows Removed by Filter: 5000
Planning Time: 0.048 ms
Execution Time: 15037.788 ms

Which proves what we had just seen above.

So, I thought that I solved the riddle: the ssh_hash column lacked the index. I was almost proud of myself, but that was a very brief moment, as after browsing both the DB table structure and the schema definition queries, I realized that there is actually an index there:

CREATE INDEX users_ssn_hashed_idx ON users (ssn_hash);

But why did Postgres keep ignoring it and performing sequential scans nevertheless? And despite that fact, why it took Postgres 15 seconds to scan 5000 rows? And why select all the rows were very quick (5 ms)? The only difference between the two was the use of the crypto() function within the WHERE statement. That narrowed down the problem. So, I realized that it was the time to get back to basics and do some research on the nature of the Bcrypt algorithm.

Bcrypt

Here is what Wikipedia specifies about Bcrypt:

The input to the bcrypt function is the password string (up to 72 bytes), a numeric cost, and a 16-byte (128-bit) salt value. The salt is typically a random value. The bcrypt function uses these inputs to compute a 24-byte (192-bit) hash. The final output of the bcrypt function is a string of the form:
$2<a/b/x/y>$[cost]$[22 character salt][31 character hash]
For example, with input password abc123xyz, cost 12, and a random salt, the output of bcrypt is the string
$2a$12$R9h/cIPz0gi.URNNX3kh2OPST9/PgBkqquzi.Ss7KIUgO2t0jWMUW
\__/\/ \____________________/\_____________________________/
Alg Cost      Salt                        Hash

If we distill it a bit, we can see that the input of the Bcrypt function is:

password
numeric cost
salt (typically random)

If we take a look at the crypt function within the insert user statement, we'll see this crypt($2, gen_salt('bf'), where SSN is passed as the $2. So, we do have a password (SSN) and salt (gen_salt('bf')) The numeric cost is missing, but let's take a look at the output of the gen_salt('bf') function. You might need to install the pgcrypt extension by CREATE EXTENSION IF NOT EXISTS pgcrypto; if you don't have it to be able to follow along.

SELECT gen_salt('bf');

The output is $2a$06$Qkp8j0vDoCuqNCgyOuIRxO. If we compare with the example from the Wikipedia above, we can see that this salt includes the cost within (06).

Let's run the SELECT gen_salt('bf'); again. We got $2a$06$enmGwTijs9cvlsIv9dcQQe, which is different from the previous result. As Wikipedia warned us

The salt is typically a random value.

And what happens if we are passing random salt to the crypto function? Right, the result will be random as well. Let's double-check:

SELECT crypt('0852285111', gen_salt('bf'));

The first time I ran it, I got this outcome: $2a$06$DQqtgCtvY9GkT3uHpShehuxb3eHD50H6XNxzEyoxZkuDjEt87/6Ce. The second ran got a different result (as we anticipated): $2a$06$nbwOka/0ve5RvM71f35jEOMGA5qHaa0H1I3RtkYq/sNIXW.Z90KAm. If we keep going, we'll keep getting different results. And this is cool, as Bcrypt was created to be used for passwords hashing, which means that even if different users have the same password, the hashed version for them will be different - pretty neat!

But if results are random, how can we check the hash in the DB corresponds to the plain text record we get as the user input? Well, as you can guess, we need to pass the same salt as the input. But where do we get the salt? Wikipedia has already given us an answer: the salt is the part of the hashed output. Let's try it out by copying the first 29 chars of the output we got above: $2a$06$DQqtgCtvY9GkT3uHpShehuxb3eHD50H6XNxzEyoxZkuDjEt87/6Ce -> $2a$06$DQqtgCtvY9GkT3uHpShehu and check if that helps:

SELECT '$2a$06$DQqtgCtvY9GkT3uHpShehuxb3eHD50H6XNxzEyoxZkuDjEt87/6Ce' = crypt('0852285111', '$2a$06$DQqtgCtvY9GkT3uHpShehu');

The result is true. Perfect! But wait a minute: do we need to extract the salt and friends manually each time? Let's take a look at the query we used to select the user by SSN from DB to look for some hints:

SELECT id, user_info FROM users WHERE ssn_hash = crypt($1, ssn_hash)

where $1 is the SSN value. This is a bit different as instead of only salt, we are passing the entire hash value into the crypt function this time. Let's try it out:

SELECT '$2a$06$DQqtgCtvY9GkT3uHpShehuxb3eHD50H6XNxzEyoxZkuDjEt87/6Ce' = crypt('0852285111', '$2a$06$DQqtgCtvY9GkT3uHpShehuxb3eHD50H6XNxzEyoxZkuDjEt87/6Ce');

Still true. How come? The answer is simple: since salt (with the prefix) has a defined length of 29, Postgres retrieves it for us, and drops the rest of it.

The beauty of Postgres (which is one of my favorite tools out there) is the fact that it's open-source. So, let's take a peak and its source code to make sure that our assumptions are correct.

Bcrypt in Postgres source code

Here is the mirror of the official PostgreSQL GIT repository. Let's find the pgcrypto file which corresponds to the extension we installed to be able to run crypto functions. Here is it is. This piece of code is triggered when we do crypt(...) via SQL:

/* SQL function: pg_crypt(psw:text, salt:text) returns text */
PG_FUNCTION_INFO_V1(pg_crypt);

Datum
pg_crypt(PG_FUNCTION_ARGS)
{
    text       *arg0 = PG_GETARG_TEXT_PP(0);
    text       *arg1 = PG_GETARG_TEXT_PP(1);
    char       *buf0,
               *buf1,
               *cres,
               *resbuf;
    text       *res;

    buf0 = text_to_cstring(arg0);
    buf1 = text_to_cstring(arg1);

    resbuf = palloc0(PX_MAX_CRYPT);

    cres = px_crypt(buf0, buf1, resbuf, PX_MAX_CRYPT);

    pfree(buf0);
    pfree(buf1);

    if (cres == NULL)
        ereport(ERROR,
                (errcode(ERRCODE_EXTERNAL_ROUTINE_INVOCATION_EXCEPTION),
                 errmsg("crypt(3) returned NULL")));

    res = cstring_to_text(cres);

    pfree(resbuf);

    PG_FREE_IF_COPY(arg0, 0);
    PG_FREE_IF_COPY(arg1, 1);

    PG_RETURN_TEXT_P(res);
}

If we jump into pg_crypt in this line cres = px_crypt(buf0, buf1, resbuf, PX_MAX_CRYPT);, we'll end up inside the px-crypt.c file here:

char *
px_crypt(const char *psw, const char *salt, char *buf, unsigned len)
{
    const struct px_crypt_algo *c;

    CheckBuiltinCryptoMode();

    for (c = px_crypt_list; c->id; c++)
    {
        if (!c->id_len)
            break;
        if (strncmp(salt, c->id, c->id_len) == 0)
            break;
    }

    if (c->crypt == NULL)
        return NULL;

    return c->crypt(psw, salt, buf, len);
}

As we see, this code navigates through the px_crypt_list while trying to find a match for the provided salt (in our case, it is the entire hashed string) by comparing the id_len number of first characters with something called id. Here is the doc for the strncmp function if you would like to learn more.

Let's see how the px_crypt_list looks like, as it might explain to us the nature of the id_len and id params.

struct px_crypt_algo
{
    char       *id;
    unsigned    id_len;
    char       *(*crypt) (const char *psw, const char *salt,
                          char *buf, unsigned len);
};

static const struct px_crypt_algo
            px_crypt_list[] = {
    {"$2a$", 4, run_crypt_bf},
    {"$2x$", 4, run_crypt_bf},
    {"$2$", 3, NULL},           /* N/A */
    {"$1$", 3, run_crypt_md5},
    {"_", 1, run_crypt_des},
    {"", 0, run_crypt_des},
    {NULL, 0, NULL}
};

Aha, so the id is the salt prefix that Wikipedia called alg ( $2a$ in our case), and the id_len is the length of that prefix. That makes a lot of sense now. And the run_crypt_bf is the actual function that will be invoked in our case. Nice, let's take a look at it here:

static char *
run_crypt_bf(const char *psw, const char *salt,
             char *buf, unsigned len)
{
    char       *res;

    res = _crypt_blowfish_rn(psw, salt, buf, len);
    return res;
}

And let's follow along by jumping into the _crypt_blowfish_rn which leads us to a new file - crypt-blowfish.c. I will not post the entire body of that function here, as it is around 200 lines long. However, this is an interesting part for us:

/*
 * Blowfish salt value must be formatted as follows: "$2a$" or "$2x$", a
 * two digit cost parameter, "$", and 22 digits from the alphabet
 * "./0-9A-Za-z". -- from the PHP crypt docs. Apparently we enforce a few
 * more restrictions on the count in the salt as well.
 */
if (strlen(setting) < 29)
    ereport(ERROR,
        (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
        errmsg("invalid salt")));

where settings is the salt. We can see a number 29 here, which rings a bell, doesn't it? That's precisely the length of the salt prefix we used in one of the examples above. Also, please, note that this validation only checks if the input length is not less than 29, but it doesn't care whether it's above the limit. That gives us a hint that the code below should handle it somehow:

if (setting[0] != '$' ||
        setting[1] != '2' ||
        (setting[2] != 'a' && setting[2] != 'x') ||
        setting[3] != '$' ||
        setting[4] < '0' || setting[4] > '3' ||
        setting[5] < '0' || setting[5] > '9' ||
        (setting[4] == '3' && setting[5] > '1') ||
        setting[6] != '$')
    {
        ereport(ERROR,
                (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
                 errmsg("invalid salt")));
    }

    count = (BF_word) 1 << ((setting[4] - '0') * 10 + (setting[5] - '0'));
    if (count < 16 || BF_decode(data.binary.salt, &setting[7], 16))
    {
        px_memset(data.binary.salt, 0, sizeof(data.binary.salt));
        ereport(ERROR,
                (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
                 errmsg("invalid salt")));
    }

Take a look at this piece: BF_decode(data.binary.salt, &setting[7], 16). Do you remember the example from Wikipedia:

$2a$12$R9h/cIPz0gi.URNNX3kh2OPST9/PgBkqquzi.Ss7KIUgO2t0jWMUW
\__/\/ \____________________/\_____________________________/
Alg Cost      Salt                        Hash

As we can see, the actual salt begins at the 8th char (that's why setting[7]), and the &setting[7] means that all the elements starting from 8th are sliced here. The BF_decode function looks like this:

static int
BF_decode(BF_word *dst, const char *src, int size)
{
    unsigned char *dptr = (unsigned char *) dst;
    unsigned char *end = dptr + size;
    const unsigned char *sptr = (const unsigned char *) src;
    unsigned int tmp,
                c1,
                c2,
                c3,
                c4;

    do
    {
        BF_safe_atoi64(c1, *sptr++);
        BF_safe_atoi64(c2, *sptr++);
        *dptr++ = (c1 << 2) | ((c2 & 0x30) >> 4);
        if (dptr >= end)
            break;

        BF_safe_atoi64(c3, *sptr++);
        *dptr++ = ((c2 & 0x0F) << 4) | ((c3 & 0x3C) >> 2);
        if (dptr >= end)
            break;

        BF_safe_atoi64(c4, *sptr++);
        *dptr++ = ((c3 & 0x03) << 6) | c4;
    } while (dptr < end);

    return 0;
}

While there is a lot of bit manipulation logic here, which I won't dare to explain, but what's important here is that dptr < end will make sure that the result will be 16 bytes long. But the salt should be 22?

In Bcrypt, while the salt is indeed encoded as 22 characters in base64 format, it actually decodes to 16 bytes of raw data. When encoding binary data to base64:

every 3 bytes of binary data becomes 4 characters in base64
this is because each Base64 character represents 6 bits (2^6 = 64 possibilities)
so 3 bytes (24 bits) of binary data maps to 4 base64 chars (4 * 6 = 24 bits)

For the Bcrypt salt:

16 bytes of raw data = 128 bits
when encoded in base64: ⌈(16 * 8) / 6⌉ = ⌈128/6⌉ = 22 characters
the ceiling function is used because we need to round up to the next complete base64 character

This way, we can see that it is safe to pass the entire hashed string as the salt input of the Postgres crypt() function. All right, that was fun, but let's get back to our Bcrypt discussion.

What do we know now?

Before we summarize what we know as of now, let me bring you one more quote about Bcrypt from the Wikipedia article:

There are then a number of rounds in which the standard Blowfish keying algorithm is applied, using alternatively the salt and the password as the key, each round starting with the subkey state from the previous round. In theory, this is no stronger than the standard Blowfish key schedule, but the number of rekeying rounds is configurable; this process can therefore be made arbitrarily slow, which helps deter brute-force attacks upon the hash or salt.

So, the Bcrypt algorithm is relatively slow by design, as the protection from the brute-force attacks.

Let's create a list of the known facts about the Bcrypt algorithm:

requires a salt to generate the hash
ideally, the salt should be random
random salt leads to the different results for the same input to hash
to compare the existing hashed string with the plain one, the original salt should be passed as the input of the crypto function
Postgres allows passing the hashed string as a salt, and takes care of extracting the salt part of it to perform hashing

With all of this information, let's finally try to solve the riddle of the poor performance of the query to fetch the user by SSN.

Postgres index and Bcrypt

So, why does Postgres keep performing sequential scans despite the index for that column? Let's take another look at the query we are trying to perform:

SELECT *
FROM users
WHERE ssn_hash = crypt('0852285111', ssn_hash);

As we already know, to recompute the hash, the crypto() function has to receive the original salt as the input. But, as you can see, the only known thing when we run this query is the plain SSN value. This means that the only possible option that Postgres has is to do the following:

iterate over each existing row in the users table
fetch ssn_hash value
pass the plain SSN and fetched ssn_hash value (as a salt) to the crypt() function and compute the hash
compare the result with the fetched ssn_hash value
if matches, we found what we are looking for, otherwise, we need to proceed to the next item, and so on

Since the algorithm described need to iterate over all the existing items, it makes sense while Postgres doesn't rely on indexes at all. If we modify the query to compare the ssn_hash with the plain string instead of the crypt() function like this:

EXPLAIN ANALYSE
SELECT *
FROM users
WHERE ssn_hash = '$2a$06$DQqtgCtvY9GkT3uHpShehuxb3eHD50H6XNxzEyoxZkuDjEt87/6Ce';

we can see that this time Postgres will use the index:

Index Scan using users_ssn_hashed_idx on users  (cost=0.28..8.30 rows=1 width=138) (actual time=0.015..0.015 rows=0 loops=1)
  Index Cond: (ssn_hash = '$2a$06$DQqtgCtvY9GkT3uHpShehuxb3eHD50H6XNxzEyoxZkuDjEt87/6Ce'::text)
Planning Time: 0.448 ms
Execution Time: 0.033 ms

Also, remember that we mentioned that Bcrypt algorithm is relatively slow? But what if we multiply slow by 5000 (number of table records we have)? That's the actual cause of the performance degradation that we have been investigating all this time!

We can simply verify it by running:

EXPLAIN ANALYSE
SELECT *
FROM users;

The output:

Seq Scan on users  (cost=0.00..157.00 rows=5000 width=138) (actual time=0.009..0.441 rows=5001 loops=1)
Planning Time: 0.094 ms
Execution Time: 0.641 ms

As you can see, even though the query iterated over all the rows, it didn't need to perform Bcrypt hashing at all, the execution time is extremely low.

Ok, so we know now that Bcrypt is the one to blame for the poor performance. But does it mean that Bcrypt is just a bad algorithm performance-wise?

Bcrypt: where to use and where not to use

Similar to any other tool, Bcrypt has its use case: hashing passwords. This means that the expected use case for it is different from the one we have above, as we shouldn't search by it. Let me demonstrate the scenario where Bcrypt can be used:

CREATE TABLE user_credentials
(
    id            UUID PRIMARY KEY DEFAULT gen_random_uuid(),
    login         TEXT NOT NULL UNIQUE,
    password_hash TEXT NOT NULL
);

CREATE INDEX user_credentials_login_idx ON user_credentials (login);

And let's insert a few items:

func main() {
    dbUrl := "postgres://adminU:adminP@localhost:5432/n0rdyblog"
    dbPool, err := pgxpool.New(context.Background(), dbUrl)
    if err != nil {
        panic(err)
    }
    defer dbPool.Close()

    numOfUsers := 5000

    for i := 0; i < numOfUsers; i++ {
        login := randomString(10)
        password := randomString(20)

        dbPool.Exec(context.Background(), "INSERT INTO user_credentials(login, password_hash) VALUES($1, crypt($2, gen_salt('bf')))", login, password)
    }
}

func randomString(length int) string {
    bytes := make([]byte, length)
    _, err := rand.Read(bytes)
    if err != nil {
        panic(err)
    }
    return base64.URLEncoding.EncodeToString(bytes)[:length]
}

The way we are supposed to run queries now is this:

when the user tries to log in, they are passing us their credentials: login and password
here is the query we run to verify the provided credentials are correct:

SELECT *
FROM user_credentials
WHERE login = 'ChCpOSGOCx'
  AND password_hash = crypt('ygEUnWIFIBoJ7ZQQqQ3j', password_hash);

Even though the user_credentials table has 5000 items, this query is very fast, let's see how fast:

EXPLAIN ANALYSE
SELECT *
FROM user_credentials
WHERE login = 'ChCpOSGOCx'
  AND password_hash = crypt('ygEUnWIFIBoJ7ZQQqQ3j', password_hash);

The results:

Index Scan using user_credentials_login_idx on user_credentials  (cost=0.28..8.31 rows=1 width=88) (actual time=4.502..4.513 rows=1 loops=1)
  Index Cond: (login = 'ChCpOSGOCx'::text)
"  Filter: (password_hash = crypt('ygEUnWIFIBoJ7ZQQqQ3j'::text, password_hash))"
Planning Time: 0.067 ms
Execution Time: 4.542 ms

The key difference between this scenario and the one that we had with SSNs, is that here Postgres is able to use indexing due to the WHERE login = 'ChCpOSGOCx' part, and then it computes Bcrypt hash only for the row returned by the index. Since, our login column is unique, it means that Bcrypt computation will happen only 1 time, so O(1) instead of O(n).

That's how Bcrypt is supposed to be used to benefit from its security without paying too high-performance costs.

Nice, but how can we resolve the original problem then?

Let's fix the performance

Of course, there is an obvious technical fix by switching to another faster algorithm with the predictable outcome like SHA-256, and the performance issue will be solved. But there are 2 points worth discussing:

since the input of the SHA-256 (or any other deterministic algorithm) will be SSN, which has a known format of 10 digits, it means that the hash will be easy to crack by the brute-forcing attack, as 10^10 (10 billion) possible options of the SSN are something that is easy to generate for the modern computers

Here is the simple brute-force code to generate all the possible SSN hash values:

const (
    ssnLength = 10
)

func main() {
    start := time.Now()

    // initializes the buffer with ASCII zeros
    current := make([]byte, ssnLength)

    // generates all possible combinations
    generateAllPossibleSSNs(current, 0)

    duration := time.Since(start)
    fmt.Printf("Completed in %v\n", duration)
}

// generateAllPossibleSSNs generates all possible SSNs and hashes them
func generateAllPossibleSSNs(current []byte, position int) {
    if position == ssnLength {
        sha256.Sum256(current)
        return
    }

    // converts 0-9 directly to ASCII bytes ('0' = 48 in ASCII)
    for i := byte(48); i < byte(58); i++ {
        current[position] = i
        generateAllPossibleSSNs(current, position+1)
    }
}

It took 30 minutes to run it on my MacBook Pro 2019, which is definitely an outdated machine as of 2025. If we had another use case, when we, for example, need to store hashes of API keys that are of UUID v4 type, so random, SHA-256 could be a good choice for such a scenario.

This brings us to another point:

why do we hash SSNs in the first place?

Whenever I mentor other software engineers, I always keep saying that writing code is the easy part of our job, and we should validate each problem and requirement first rather than jumping into the coding mode right away. Same here: what do we gain from hashing the SSNs? Is it a privacy requirement or just an assumption by someone that “SSNs are sensitive, so we shouldn't store the plain values for them”? Is there a chance that we already store them in another table / database in our system in a plain way? If not, what are the privacy and legal requirements for the hashed values? As we see, without the salt, SSNs are easy to brute-force, but random salt leads to poor performance.

In some cases, it can appear that, actually, the solution as simple as not hashing the data, if it is not considered too sensitive. In other cases, we might need to have one common salt for all the hashes, which we will store by using Gandalf's rule of thumb:

rather than keeping open/within the hash as in our previous example.

In this case, if the salt is common, we can even stay with Bcrypt, as the query with a known salt will use indexing:

EXPLAIN ANALYSE
SELECT *
FROM users
WHERE ssn_hash = crypt('0852285111', '$2a$06$DQqtgCtvY9GkT3uHpShehu');

The result is:

Index Scan using users_ssn_hashed_idx on users  (cost=0.28..8.30 rows=1 width=138) (actual time=0.038..0.038 rows=0 loops=1)
  Index Cond: (ssn_hash = '$2a$06$DQqtgCtvY9GkT3uHpShehuxb3eHD50H6XNxzEyoxZkuDjEt87/6Ce'::text)
Planning Time: 3.462 ms
Execution Time: 0.118 ms

As you might remember, we started with 15 seconds per query, and now we are down to 3 milliseconds, which is 5000 times faster — an exciting achievement.

Still, something faster like SHA-256, SHA-3, BLAKE2 or BLAKE3 might be better choices if a common salt is given. Of course, this needs benchmarking, consulting your security team and checking Postgres support for your particular case before deciding.

And that's it, the team and the users are happy, the system is healthy again. We saved the day, and learned a thing or two about the Bcrypt algorithm.

That was quite a ride, so thanks a lot for reading my post. I truly hope it was useful, and I'll see you in my next ones, as more to come soon. Have fun! =)

If you liked the post:

subscribe to my newsletter to receive an email once the new post is published
let's follow each other on Bluesky or X

What Okta Bcrypt incident can teach us about designing better APIs

Mykola — Wed, 22 Jan 2025 17:24:38 +0000

Hello there! If you follow tech news, you might have heard about the Okta security incident that was reported on 1st of November. The TLDR of the incident was this:

The Bcrypt algorithm was used to generate the cache key where we hash a combined string of userId + username + password. Under a specific set of conditions, listed below, this could allow users to authenticate by providing the username with the stored cache key of a previous successful authentication.

This means that if the user had a username above 52 chars, any password would suffice to log in. Also, if the username is, let's say, 50 chars long, it means that the bad actor needs to guess only 3 first chars to get in, which is quite a trivial task for the computers these days. Too bad, isn't it?

On the other hand, such long usernames are not very usual, which I agree with. However, some companies like using the entire name of the employee as the email address. So, let's say, Albus Percival Wulfric Brian Dumbledore, a headmaster of Hogwarts, should be concerned, as albus.percival.wulfric.brian.dumbledore@hogwarts.school is 55 chars. Ooops!

This was possible due to the nature of Bcrypt hashing algorithm that has a maximum supported input length of 72 characters (read more here), so in Okta case the characters above the limit were ignored while computing the hash, and therefore, not used in the comparison operation. We can reverse engineer that:

72 - 53 = 19 - user id with separators if any
this way, the password will be outside the 72 chars limit, and, therefore, ignored by the Bcrypt algorithm

However, there was one thing that made me wonder: if there is a known limit of the algorithm, why is it not enforced by the crypto libraries as a form of input validation? A simple if input length > 72 -> return error will do the trick. I assumed that they might have used some custom library for Bcrypt implementation and simply forgotten about the input validation, which can happen. So, I decided to check how other programming languages behave.

Go and Bcrypt

Let's start with Go, and implement the Okta incident-like case with the help of the official golang.org/x/crypto/bcrypt library:

package main

import (
    "crypto/rand"
    "encoding/base64"
    "fmt"
    "golang.org/x/crypto/bcrypt"
)

func main() {
    // 18 + 55 + 1 = 74, so above 72 characters' limit of BCrypt
    userId := randomString(18)
    username := randomString(55)
    password := "super-duper-secure-password"

    combinedString := fmt.Sprintf("%s:%s:%s", userId, username, password)

    combinedHash, err := bcrypt.GenerateFromPassword([]byte(combinedString), bcrypt.DefaultCost)
    if err != nil {
        panic(err)
    }

    // let's try to break it
    wrongPassword := "wrong-password"
    wrongCombinedString := fmt.Sprintf("%s:%s:%s", userId, username, wrongPassword)

    err = bcrypt.CompareHashAndPassword(combinedHash, []byte(wrongCombinedString))
    if err != nil {
        fmt.Println("Password is incorrect")
    } else {
        fmt.Println("Password is correct")
    }
}

func randomString(length int) string {
    bytes := make([]byte, length)
    _, err := rand.Read(bytes)
    if err != nil {
        panic(err)
    }
    return base64.URLEncoding.EncodeToString(bytes)[:length]
}

All the code samples can be found here

What this code does is:

generates 18-chars long userId
generates 55-chars long username
concatenates them with each other and a dummy password super-duper-secure-password with the use of : as a separator
computes Bcrypt hash from the concatenated string
then concatenates the same userId and username with a different password wrong-password
uses bcrypt API to compare whether the 2nd concatenated string matches the hash of the 1st one

Let's run the code and see the result:

panic: bcrypt: password length exceeds 72 bytes

goroutine 1 [running]:
main.main()
    /n0rdy-blog-code-samples/20250121-bcrypt-api/01-bcrypt-in-go/main.go:20 +0x2d1

Good job, Go! If we check the source code of the bcrypt.GenerateFromPassword(...) function, we'll see this piece of code at the very beginning:

if len(password) > 72 {
    return nil, ErrPasswordTooLong
}

Perfect! At this point, I became even more suspicious about the tool Okta used, as it seemed like the industry figured that out based on this example. Spoiler alert: it's not that simple.

Let's proceed with Java.

Java and Bcrypt

Java doesn't support Bcrypt from its core API, but my simple Google search showed that Spring Security library has implemented it. For those who are not into Java ecosystem, Spring is the most used and battle-tested frameworks out there, that has libraries for almost anything: Web, DBs, Cloud, Security, AI, etc. Pretty powerful tool, that I've used a lot in the past, and still sometimes use for my side projects.

Spring Security

So, I added the latest version of Spring Security to the project and reproduced the same scenario, as in Go example above:

import org.apache.commons.lang3.RandomStringUtils;
import org.springframework.security.crypto.bcrypt.BCrypt;

public class BcriptSpringSecurity {
    public static void main(String[] args) {
        // 18 + 55 + 1 = 74, so above 72 characters' limit of BCrypt
        var userId = RandomStringUtils.randomAlphanumeric(18);
        var username = RandomStringUtils.randomAlphanumeric(55);
        var password = "super-duper-secure-password";

        var combinedString = String.format("%s:%s:%s", userId, username, password);

        var combinedHash = BCrypt.hashpw(combinedString, BCrypt.gensalt());

        // let's try to break it
        var wrongPassword = "wrong-password";
        var wrongCombinedString = String.format("%s:%s:%s", userId, username, wrongPassword);

        if (BCrypt.checkpw(wrongCombinedString, combinedHash)) {
            System.out.println("Password is correct");
        } else {
            System.out.println("Password is incorrect");
        }
    }
}

I ran the code, and to my great surprise, saw this outcome:

Password is correct

I took a peak at the implementation code, and was disappointed: even though there are a bunch of checks on salt:

if (saltLength < 28) {
    throw new IllegalArgumentException("Invalid salt");
}
...
if (salt.charAt(0) != '$' || salt.charAt(1) != '2') {
    throw new IllegalArgumentException("Invalid salt version");
}
...
minor = salt.charAt(2);
if ((minor != 'a' && minor != 'x' && minor != 'y' && minor != 'b') || salt.charAt(3) != '$') {
    throw new IllegalArgumentException("Invalid salt revision");
}
...

I didn't see any validation of the input that will be hashed. Hm...

I decided to check other Google results, and the next Java library in the list was bcrypt from Patrick Favre (link to GitHub repo) with 513 starts and the last release version 0.10.2 (so, not stable) from 12th of February 2023 (almost 2 years old). This suggested that I'd not use it in production, but why not to run our tests.

Bcrypt from Patrick Favre

import at.favre.lib.crypto.bcrypt.BCrypt;
import org.apache.commons.lang3.RandomStringUtils;

public class BcryptAtFavre {

    public static void main(String[] args) {
        // 18 + 1 + 55 = 74, so above 72 characters' limit of BCrypt
        var userId = RandomStringUtils.randomAlphanumeric(18);
        var username = RandomStringUtils.randomAlphanumeric(55);
        var password = "super-duper-secure-password";

        var combinedString = String.format("%s:%s:%s", userId, username, password);

        var combinedHash = BCrypt.withDefaults().hashToString(12, combinedString.toCharArray());

        // let's try to break it
        var wrongPassword = "wrong-password";
        var wrongCombinedString = String.format("%s:%s:%s", userId, username, wrongPassword);

        var result = BCrypt.verifyer().verify(combinedHash.toCharArray(), wrongCombinedString);
        if (result.verified) {
            System.out.println("Password is correct");
        } else {
            System.out.println("Password is incorrect");
        }
    }
}

Let's run it:

Exception in thread "main" java.lang.IllegalArgumentException: password must not be longer than 72 bytes plus null terminator encoded in utf-8, was 102
    at at.favre.lib.crypto.bcrypt.LongPasswordStrategy$StrictMaxPasswordLengthStrategy.innerDerive(LongPasswordStrategy.java:50)
    at at.favre.lib.crypto.bcrypt.LongPasswordStrategy$BaseLongPasswordStrategy.derive(LongPasswordStrategy.java:34)
    at at.favre.lib.crypto.bcrypt.BCrypt$Hasher.hashRaw(BCrypt.java:303)
    at at.favre.lib.crypto.bcrypt.BCrypt$Hasher.hash(BCrypt.java:267)
    at at.favre.lib.crypto.bcrypt.BCrypt$Hasher.hash(BCrypt.java:229)
    at at.favre.lib.crypto.bcrypt.BCrypt$Hasher.hashToString(BCrypt.java:205)
    at BcryptAtFavre.main(BcryptAtFavre.java:14)

Nice, good job, Patrick, you saved the day for Java!

After checking the source code, I found this piece:

@Override
public byte[] derive(byte[] rawPassword) {
    if (rawPassword.length >= maxLength) {
        return innerDerive(rawPassword);
    }
    return rawPassword;
}

and the strict strategy that threw the exception we've seen:

final class StrictMaxPasswordLengthStrategy extends BaseLongPasswordStrategy {
    StrictMaxPasswordLengthStrategy(int maxLength) {
        super(maxLength);
    }

    @Override
    public byte[] innerDerive(byte[] rawPassword) {
        throw new IllegalArgumentException("password must not be longer than " + maxLength + " bytes plus null terminator encoded in utf-8, was " + rawPassword.length);
    }
}

We can see that this strict strategy is used as a part of the default configs:

public static Hasher withDefaults() {
    return new Hasher(Version.VERSION_2A, new SecureRandom(), LongPasswordStrategies.strict(Version.VERSION_2A));
}

Cool!

Let's switch to JavaScript.

JavaScript and Bcrypt

Here I used the bcryptjs which has over 2 million weekly downloads based on the NPM stats.

const bcrypt = require('bcryptjs')

function randomString (length) {
  const chars = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
  let result = ''
  for (let i = length; i > 0; --i) {
    result += chars[Math.floor(Math.random() * chars.length)]
  }
  return result
}

function runTest () {
  // 18 + 55 + 1 = 74, so above 72 characters' limit of BCrypt
  const userId = randomString(18)
  const username = randomString(55)
  const password = 'super-duper-secure-password'

  const combinedString = `${userId}:${username}:${password}`

  const combinedHash = bcrypt.hashSync(combinedString)

  // let's try to break it
  const wrongPassword = 'wrong-password'
  const wrongCombinedString = `${userId}:${username}:${wrongPassword}`

  if (bcrypt.compareSync(wrongCombinedString, combinedHash)) {
    console.log('Password is correct')
  } else {
    console.log('Password is wrong')
  }
}

runTest()

The output is:

Password is correct

Not great. The source code reveals that similar to Spring Security, the library validates the salt

if (salt.charAt(0) !== '$' || salt.charAt(1) !== '2') {
     err = Error("Invalid salt version: "+salt.substring(0,2));
     if (callback) {
         nextTick(callback.bind(this, err));
         return;
     }
     else
         throw err;
}
...

but not the input length.

Let's try if Python can do any better.

Python and Bcrypt

Using bcrypt library with 1.3k starts and the latest release in November.

import random
import string

import bcrypt

def random_string(length):
    return ''.join(random.choice(string.ascii_letters) for i in range(length))

if __name__ == '__main__':
    # 18 + 55 + 1 = 74, so above 72 characters' limit of BCrypt
    user_id = random_string(18)
    username = random_string(55)
    password = "super-duper-secure-password"

    combined_string = "{0}:{1}:{2}".format(user_id, username, password)

    combined_hash = bcrypt.hashpw(combined_string.encode('utf-8'), bcrypt.gensalt())

    # let's try to break it
    wrong_password = "wrong-password"
    wrong_combined_string = "{0}:{1}:{2}".format(user_id, username, wrong_password)

    if bcrypt.checkpw(wrong_combined_string.encode('utf-8'), combined_hash):
        print("Password is correct")
    else:
        print("Password is incorrect")

The result is same as we observed for most of our test subjects:

Password is correct

All right, but what about some newer and more safety-oriented language - let's try Rust.

Rust and Bcrypt

Here I need to be honest: since I'm not a Rust expert at all, I used a help of a Claude AI to write this code. So, if you see any issues there, please, let me know in the comments section, so I can fix that.

As a library, I used rust-bcrypt based on my AI friend advice.

use rand::RngCore;
use base64::{Engine as _, engine::general_purpose::URL_SAFE};
use std::error::Error;

fn random_string(length: usize) -> String {
    let mut bytes = vec![0u8; length];
    rand::thread_rng().fill_bytes(&mut bytes);
    URL_SAFE.encode(&bytes)[..length].to_string()
}

fn main() -> Result<(), Box<dyn Error>> {
    // 18 + 55 + 1 = 74, so above 72 characters' limit of BCrypt
    let user_id = random_string(18);
    let username = random_string(55);
    let password = "super-duper-secure-password";

    let combined_string = format!("{}:{}:{}", user_id, username, password);
    let combined_hash = bcrypt::hash(combined_string.as_bytes(), bcrypt::DEFAULT_COST)?;

    // let's try to break it
    let wrong_password = "wrong-password";
    let wrong_combined_string = format!("{}:{}:{}", user_id, username, wrong_password);

    match bcrypt::verify(wrong_combined_string.as_bytes(), &combined_hash) {
        Ok(true) => println!("Password is correct"),
        Ok(false) => println!("Password is incorrect"),
        Err(e) => println!("{}", e),
    }

    Ok(())
}

The output is:

Password is correct

I can see the validation of the cost:

if !(MIN_COST..=MAX_COST).contains(&cost) {
    return Err(BcryptError::CostNotAllowed(cost));
}

but not of the input. And here is the place where the explicit truncation of 72 chars happens (the comment is from the library source code):

// We only consider the first 72 chars; truncate if necessary.
// `bcrypt` below will panic if len > 72
let truncated = if vec.len() > 72 {
    if err_on_truncation {
        return Err(BcryptError::Truncation(vec.len()));
    }
    &vec[..72]
} else {
    &vec
};

let output = bcrypt::bcrypt(cost, salt, truncated);

Why?

That was my first question after seeing that the majority of the tools follow the pattern that leads to the vulnerability. Wikipedia article about Bcrypt gave a hint:

Many implementations of bcrypt truncate the password to the first 72 bytes, following the OpenBSD implementation

Interesting! Let's check the OpenBSD implementation of this algorithm, and here is the link to it. The first point of interest lies here:

/* strlen() returns a size_t, but the function calls
 * below result in implicit casts to a narrower integer
 * type, so cap key_len at the actual maximum supported
 * length here to avoid integer wraparound */
key_len = strlen(key);
if (key_len > 72)
     key_len = 72;
key_len++;

And from that moment on, key_len is used as a limit to iterate over the input string within, for example:

u_int32_t
Blowfish_stream2word(const u_int8_t *data, u_int16_t databytes,
    u_int16_t *current)
{
    u_int8_t i;
    u_int16_t j;
    u_int32_t temp;

    temp = 0x00000000;
    j = *current;

    for (i = 0; i < 4; i++, j++) {
        if (j >= databytes)
            j = 0;
        temp = (temp << 8) | data[j];
    }

    *current = j;
    return temp;
}

Where key_length is passed as a databytes parameter. So this piece of code:

if (j >= databytes)
    j = 0;

will make sure that no chars over the limit (72) will end up being processed.

Git blame shows that the if (key_len > 72) line is 11 years old

while the if (j >= databytes) j = 0; is 28 years old (what were you busy with in 1997, ah?)

So, it's been a while since the API has been reiterated.

Some thoughts on that

Disclaimer

Let me start with a short disclaimer: I have a huge respect for people who spend their free time and mental capacity on maintaining open-source projects. That's a large amount of work, that is not paid, and, unfortunately, quite often not appreciated by the users of the tools. That's why they have all the legal and ethical rights to build the project the way they see them. My opinions below are not targeted towards anyone in particular.

My initial goal was to create issues for each of the mentioned library, but I noticed that this behavior has been already reported to each of them:

Check the discussions and their outcomes by following those links.

Thoughts and lessons

As a guy who spent a few years of my career on building tools and solutions to be used by other software engineers, I understand the frustration: you invested your time and effort into writing a clear documentation and guides, but a certain number of your users don't bother checking it at all, and just use the tool the way they think it should be used. However, that's the reality that I had to accept and started thinking about how can I make my tools handle those use cases. Here are a few principles I came up with in that process.

Don't let the people use your API incorrectly

In my opinion, from the API perspective, the approach when the tool silently cuts the part of the input and processes the remaining one only, it is an extremely poor design choice. What makes things worse is the fact that Bcrypt is used in the domain of security and sensitive data, and, as we can see, most of the tools mentioned above, use password as the name of the input parameter of the hashing method. The good design should explicitly reject the invalid input with the error / exception / any other mechanism the platform uses. So, basically, exactly what Go and Patrick's Java library did. This way, incidents like Okta one would be impossible by design (btw, I'm not shifting the blame away from Okta, considering the domain they operate in).

It is ok, though, to offer the non-default unsafe option, that will let the users pass longer input that will be truncated if the user explicitly asks for that. A prefix/suffix like unsafe, truncated, etc. can be a good addition to the names of the method that expose these options.

Be predictable

If we take a step back from the Bcrypt case, imagine other examples, if such a pattern becomes common in the industry:

We created a new user account on HBO to watch a new season of Rick and Morty, and there is a warning that the max size of the password should not exceed 18 chars. However, the password generator of your password manager tool uses 25 chars as a default length of the produced password. So, the password manager inserts that password while creating an account, but the server cuts the last 7 chars, hashes the rest, and saves the hash to the DB. How easy would it be for us to be able to log in to HBO next time and watch a new episode?
The tech lead of the new project configured a linter tool, and set the max line length as 100 chars. While performing a check, linter removes the chars above the defined limit, and informs that the check has passed. How useful would it be?

A good API design should remember that when it comes to tech, nobody likes surprises.

No ego

While following a few online discussions about the Bcrypt Okta incident, I noticed something else: while the majority of comments agreed that we should design APIs like these better, there were a few folks that took a very defensive stance and exposed their ego: "Read a paper before using anything!", "APIs are only correcting the input after the stupid users!", etc. Based on my experience, ego is a big enemy of engineering. And I wouldn't be surprised if you have a story or two in that regard as well. So, yeah, let's not bring our egos to our APIs.

Be helpful

Don't get me wrong, I do understand the gist that the users should have some basic knowledge before using any tool. But let's get back to the reality: how many different tools, programming languages, databases, protocols, frameworks, libraries, algorithms, data structures, clouds, AI models, etc. does a software engineer use per week these days? I tried to count for my use case, but stopped after the number had reached 30. Is it possible to know all of them deep? To know all the edge cases and limits? For some of them and to some degree is a reasonable ask, as well as having an expertise in 1 or 2, but definitely not all. The hard truth is that on average, the industry today requires the wide spectrum of knowledge over the deep one (check any job opening to verify that claim). Therefore, while designing the tools, why not to help our fellow colleagues? For example, if our tool accepts only positive numbers, let's add if num < 1 -> return error to our solution, and make the life simpler for somebody out there.

Especially, if the tool might be used in the security-sensitive context, where humans are usually the weak point in the thread modelling. The good API can help there.

Be brave

It's not so often that the API we design is something completely new to the world. Most likely, there are other solutions like ours out there. And the chances are that they've been already doing certain things the particular way. However, that doesn't mean that we need to follow the same path. Kudos to the Go team and Patrick's Java library for being brave to do things the different way as the industry does in the Bcrypt example. Let's learn from them.

Reiterate

Regardless of the original design choices and intentions, it's never too late to reiterate on some of them if we see a need or have discovered new information. That's, actually, a place where a lot of us fail due to different reasons, with some of them listed above.

Instead of a conclusion

The Okta incident exposed large security issues out there. Our test showed, even 3 months after the incident, the industry is still vulnerable to the same outcome, so the chances are that more to come. However, we, as software engineers, can learn from that, and apply these lessons while designing APIs to make them predictable and easier to use.

I hope that was useful, and triggered some thoughts. Thanks a lot for reading my post, and see you in the following ones, there are plenty of topics to discuss. Have fun! =)

If you liked the post:

subscribe to my newsletter and receive an email once the new post is published
let’s follow each other on Bluesky or Twitter

TIL: Ghostty — a new and quite promising terminal emulator

Mykola — Sat, 18 Jan 2025 21:40:22 +0000

Hello there! If you've been following my blog, you might have noticed that I'm usually leaning towards longreads as the style of my posts. And while I think such posts are great in general, it takes forever to prepare, write and edit them before publishing, which ends up in the very infrequent and inconsistent blogging ratio on my end.

Lately, I've come across the “What to blog about” article by Simon Willison which, alongside other things, had a great insight about the “Today I Learned” type of short posts to share small bits of recently acquired knowledge with the audience. So, this is my attempt to incorporate that approach alongside the longreads that I will keep writing (I have a few topics planned, so stay tuned and subscribe to my newsletter to receive an email once I publish a new post). Let's gooooooo!

Like many other people in tech, I like trying new apps, libraries, languages, and do my best to stay curious and open-minded. At the same time, there are a few tools that I've been using for a while, and I'm carrying them with me since forever: for example, Firefox, as a default browser, Sublime as a default GUI text editor, etc. The same applies to the terminal app: since I switched to MacBook 6 years ago, I've been using iTerm2 with oh-my-zsh add-on and robbyrussell theme on top, and I've been happy even since. I appreciate the flexibility of its configuration, history-based commands' complete, possibility to scroll with the mouse/touchpad. Additionally, I've accumulated quire a decent number of aliases that I'd need to migrate, should I choose to move to a new terminal.

However, something has happened lately: the SSH-related security vulnerability has been discovered and patched in iTerm2 (if you are the user and haven't updated the version, I'd strongly encourage you to do that asap).

An immediate disclaimer: I have no negative emotions about the security vulnerabilities in the free open-source apps. I have a big respect and appreciation for the people who are willing to dedicate their free time to build and maintain those tools and share them with the world. In my case, it was just a trigger “hey, it's been a while since you checked the state of terminal emulators on the market” that sparked my curiosity.

Wind of change

I remember that Julia Evans, whose blog I follow, mentioned a few time that she uses Fish. Also, some days ago I came across this post about Fish rewrite to Rust from C++, which sounds like a cool thing to do. However, I tried it some time ago, and while pretty neat, I wasn't convinced to switch to it completely.

At the same time, in the internal Slack of the company I work for, my colleague asked the security team whether we have any policies about the apps, as they'd like to start using Ghostty as their terminal emulator. I took a look at it, and it immediately caught my attention: a fresh look, a zero-config setup, platform-native UI (discovered in details in the “Ghostty Is Native—So What?” post by Gregory Anders) and GPU acceleration, and FOSS with very permissive MIT license (here is the GitHub repo). I googled the author (Mitchell Hashimoto), and discovered that he is a co-founder of HashiCorp, that brought Terraform, Vargant, Consult, Vault, and others to the world. That's quite a list. And, last but not the least, Zig as the main programming language was an interesting factor as well.

So, I ran brew install --cask ghostty (the command is from here), and gave it a try.

The first thing that I obviously saw was the design, and it looked minimalistic, distraction-free, and, somewhat, fresh. I had to increase a font size a bit to match my comfort level, though.

While design is an important part to some degree, there is something more that I've become observing and, therefore, liking lately: the reasonable default configs of the apps, which mean that the majority of the users will never need to mess with configs at all. Here is a great post by Arne about this trend which lists such tools like Fish (mentioned above), Helix, Lazygit, Zellij, k9s, etc. And that a very user-friendly approach: install and use right away! I believe that Ghostty would be a good addition to the list. For example:

all of my aliases and environment vars configured within the ~/.zshrc were available right away, as Ghostty appears to be able to load them. So, I can keep typing k if I need to use kubectl, or r for ranger, and dozens of others more complex ones. That's very-very cool, as no copy-pasting required to make it work.
history-based autocomplete is available outside the box
syntax highlighting works by default
familiar shortcuts, like ⌘ + T to open a new tab, ⌘ + W to close it, ⌘ + D to split the window into 2 tabs.
not exactly config-related, but the terminal feels extremely fast without any settings tinkering

There was a tiny detail that I found frustrating in iTerm2: when I pressed ⌥ + Left Arrow (which is in my muscle memory), instead of jumping over the word on the left, the terminal typed D . The same attempt, but in the other direction, resulted in C. No biggies, but annoying when the command consists of many words. In Ghostty this worked as I expected it to: 5 points to Gryffindor!

I've mentioned above that I've been using robbyrussell (from here) for my iTerm2 terminal setup, so I wondered if the same one is available. I was about to start googling the list of available topics, but I've noticed a hint in the Ghostty doc to run ghostty +list-themes in the terminal, and here is what has happened:

Isn't that cool? I haven't found the robbyrussell theme there though, maybe it has a different name now?

One more thing that amazed me was the embedded dev tool, which uses the same key shortcut as Firefox and Chrome DevTools: `⌥ +⌘ + I`:

Honestly, I haven't come up with the use case for this feature in my daily terminal use, but it shows the level of attention to the small details Ghostty has to offer.

Ghostty seems to become a hyped thing lately (well-deserved, imho), as I see more and more content about it. Most of the people appreciate its performance, which I didn't have an opportunity to test properly yet, rather than trying to insert the text of this post into terminal to see how fast it renders it:

Not a rocket speed, but my MacBook is 6 years old, so hard to blame only Ghostty for that. iTerm2 was slightly slower. Please, let me know if you have tried the same on M1 or newer MacBook, I'd be curious to know the results.

Instead of conclusions

As of today, I'm still in the testing Ghostty phase, but haven't had the need to return to the old terminal setup yet, which is a good sign. However, that I might happen, as the terminal is still missing ⌘ + F search functionality, which comes handy from time to time. There is an open issue and a message from the tool creator that this is one of the top priorities, so I hope it's the matter of time when it is coming.

Issues-wise, I haven't noticed any, except that resizing window sometimes has a tiny lag, when the top bar is a few milliseconds behind the main window. Also, something that is outside the terminal standards, but I'd personally love to have is:

⌘ + A to select the current command instead of entire window text (or with any other keybinding for this action)
⌥ + ⇧ + Left/Right Arrow to select the left/right word of the entered but not yet submitted command (another muscle memory thingy), but Ghostty types D instead

I'll update the post if I notice anything or change my mind and go back to the previous setup, so stay tuned. So far, only good vibes, so if you feel like trying something new, that can be a good place to start.

Have fun! =)

P.S. Something obvious, but need to mention it nevertheless: I'm not affiliated with Ghostty anyhow, and nobody asked me to test or review it, that's my honest opinion of the tool which I found worth sharing with the community.

P.P.S. I hope you liked this style of short-ish (still working on the "short" part) posts. If you don't want to miss the future ones:

subscribe to my newsletter to receive an email once I publish a new post
follow me on Bluesky, Mastodon, or Twitter

Demystifying OTPs: the logic behind the offline generation of tokens

Mykola — Tue, 10 Dec 2024 17:53:54 +0000

Hello there! Another evening, on my way back home, I decided to check the mailbox. I don't mean my email inbox, but the old-school actual box where the postman puts the physical letters. And to my great surprise, I found an envelope there with something inside! While opening it, I spent a few moments hoping that it's the decades delayed letter from Hogwarts. But then I had to get back down to Earth, once I noticed that it's a boring "grown-up" letter from the bank. I skimmed through the text and realized that my "digital-only" bank for cool kids had been acquired by the biggest player on the local market. And as a token of the new beginning, they added this to the envelope:

Alongside the instructions on how to use it.

If you are like me, and have never come across such a piece of the tech innovation, let me share what I learned from the letter: new owners decided to enforce the security policies from their company, which means that all the user accounts from now on will have the MFA enabled (kudos for that, btw). And the device you could see above generates one-time tokens 6-digits long that are used as a second factor while logging in to your bank account. Basically, the same way as apps like Authy, Google Authenticator or 2FAS work, but in a physical shape.

So, I gave it a try, and the login process went smoothly: the device showed me a 6-digits code, I entered it in my banking app, and this got me in. Hooray! But then something struck me: how does this thing work? There is no way it is connected to the internet somehow, but it manages to generate correct codes that are accepted by my bank server. Hm... Could it have a SIM-card or something similar inside? No way!

Realizing that my life will never be the same, I began wondering about apps I've mentioned above (Authy and friends)? My inner researcher was awakened, so I switched my phone into the airplane mode and, to my big surprise, realized that they work perfectly fine offline: they keep generating codes that are accepted by the servers of the apps. Interesting!

Not sure about you, but I've always taken the one-time token flow for granted and have never actually given it a proper thought (especially due to the fact that these days it is rare that my phone has no internet unless I'm doing some outdoor adventures), so that was the root cause of my surprise. Otherwise, it makes a perfect sense from the security point of view to work this way, as the generation process is purely local, so safe-ish from the external actors. But how does it work?

Well, modern technologies like Google or ChatGPT make it easy to find the answer easily. But this technical problem seemed fun to me, so decided to give it a try and solve it on my own first.

Requirements

Let's start with what we have:

an offline device that generates 6-digits codes
a server that accepts these codes, validates them and gives a green signal if they are correct

The server validation part hints that the server must be able to generate the same code as the offline device to compare them. Hm..that can be helpful.

My further observations of my new "toy" brought even more discoveries:

if I turn it off and then off, usually I can see the same code as before
however, occasionally, it changes

The only logic explanation I could come up with is that these codes have a certain lifetime. I'd like to tell a story of me trying to count the duration of it in "1-2-3-...-N" fashion, but it won't be true: I got a big hint from the apps like Authy and Co, where I saw the 30 seconds TTL. Good find, let's add this to the list of the known facts.

Let's summarize the requirements we have so far:

predictable (rather than random) code generation in 6 digits format
the generation logic should be reproducible, which allows getting the same result regardless of the platform
the code lifetime is 30 seconds, which means that within this timeframe the generation algorithm produces the same value

The big question

All right, but the main question remains unanswered: how come the offline app can generate the value that will match the one from the other app? What do they have in common?

If you are into the Lord of the Rings universe, you might remember how Bilbo played a riddle game with Gollum, and got this one to solve:

This thing all things devours:
Birds, beasts, trees, flowers;
Gnaws iron, bites steel;
Grinds hard stones to meal;
Slays king, ruins town,
And beats high mountain down.

Spoiler alert, but Mr. Baggins got lucky and came up with the correct answer accidentally - "Time!". Believe it or not, but this is exactly the answer to our riddle as well: any 2 (or more) apps have access to the same time as long as they have an embedded clock within. The latter is not a problem these days, and the device in question is big enough to fit it. Look around, and the chances are that the time on your hand watch, mobile phone, TV, oven, and clock on the wall is the same. We are into something here, it seems like we have found the base for the OTP (one-time password) computing!

Challenges

Relying on time has its own set of challenges:

timezones - which one to use?
clocks tend to get out of sync, and it's a big challenge in the distributed systems

Let's address them one by one:

timezone: the simplest solution here is to make sure that all the devices rely on the same zone, and UTC can be a good location-agnostic candidate
as for clocks getting out of sync: actually, we might not even need to solve it but rather accept as something inevitable, as long as the drift is within the second or two, which might be tolerable considering the 30 seconds TTL. The hardware producer of the device should be able to predict when such drift will be achieved, so then the device will use it as its expiration date, and the bank will simply replace them with the new ones, or will have a way to connect them to the network to calibrate the clock. At least, that's my train of thoughts here.

Implementation

Ok, this is settled, so let's try to implement the very first version of our algorithm using time as the basis. Since we are interested in a 6 digits result, it sounds like a smart choice to rely on a timestamp rather than the human-readable date. Let's start from there:

// gets current timestamp:
current := time.Now().Unix()
fmt.Println("Current timestamp: ", current)

According to the Go docs, the .Unix() returns

the number of seconds elapsed since January 1, 1970 UTC.

This is what printed to the terminal:

Current timestamp:  1733691162

That's a good start, but if we rerun that code, the timestamp value will change, while we'd like to keep it stable for 30 seconds. Well, piece of cake, let's divide it by 30 and use that value as a base:

// gets current timestamp
current := time.Now().Unix()
fmt.Println("Current timestamp: ", current)

// gets a number that is stable within 30 seconds
base := current / 30
fmt.Println("Base: ", base)

Let's run it:

Current timestamp:  1733691545
Base:  57789718

And again:

Current timestamp:  1733691552
Base:  57789718

The base value remains the same. Let's wait for a bit, and run it again:

Current timestamp:  1733691571
Base:  57789719

The base value has changed, as the 30 seconds window has passed - good job!

If the "divide by 30" logic doesn't make sense, let me explain it with a simple example:

imagine that our timestamp returns 1
if we divide 1 by 30, the result will be 0, as when we are using a strictly typed programming language, dividing integer by integer returns another integer, which isn't concerned about the floating-point part
this means that for the next 30 seconds we'll be getting 0 while timestamp is between 0 and 29
once timestamp reaches the value of 30, the result of division is 1, until 60 (where it becomes 2), and so on

I hope it makes more sense now.

However, not all the requirements are satisfied yet, as we need a 6 digits result, while current base has 8 digits as of today, but at some point of time in the future it might reach 9 digits point, and so on. Well, let's use another simple math trick: let divide the base by 1 000 000, and get the remainder, which will always have exactly 6 digits, as the reminder can be any number from 0 to 999 999, but never larger:

// gets current timestamp:
current := time.Now().Unix()
fmt.Println("Current timestamp: ", current)

// gets a number that is stable within 30 seconds:
base := current / 30
fmt.Println("Base: ", base)

// makes sure it has only 6 digits:
code := base % 1_000_000

// adds leading zeros if necessary:
formattedCode := fmt.Sprintf("%06d", code)
fmt.Println("Code: ", formattedCode)

The fmt.Sprintf("%06d", code) part appends leading zeros in case if our code value has less than 6 digits. For example, 1234 will be converted into 001234.
The entire code for this post can be found here.

Let's run this code:

Current timestamp:  1733692423
Base:  57789747
Code:  789747

All right, we get our 6-digits code, hooray! But something doesn't feel right here, does it? If I give you this code, and you will run it at the same time as I do, you'll get the same code, as I. This doesn't make it a secure one-time password, right? Here comes a new requirement:

the result should be different for different users

Of course, some collisions are inevitable, if we have beyond 1 million users, as this is the max possible unique values per 6 digits. But these are rare and technically unavoidable collisions, not the algorithm design flaw like we have now.

I don't think any clever mathematical tricks will help us here on their own: if we need separate results per user, we need a user-specific state to make it happen. As engineers and, at the same time, users of many services, we know that to grant access to their APIs, services rely on private keys, which are unique per user. Let's introduce a private key for our use case as well to distinguish between the users.

Private key

A simple logic for generating private keys as integers between 1 000 000 and 999 999 999:

var pkDb = make(map[int]bool)

func main() {
    prForUser := nextPrivateKey()
    fmt.Println("Private key for user: ", prForUser)
}

func nextPrivateKey() int {
    r := randomPrivateKey()
    for pkDb[r] {
        r = randomPrivateKey()
    }
    pkDb[r] = true
    return r
}

// generates random number from 1 000 000 to 999 999 999
func randomPrivateKey() int {
    return rand.Intn(999_999_999-1_000_000) + 1_000_000
}

We are using pkDb map as a way to prevent duplicates among private keys, and if the duplicate has been detected, we run the generation logic once more until we get a unique result.

Let's run this code to get the private key sample:

Private key for user:  115537846

Let's make a use of this private key within our code generation logic to make sure we are getting different results per private key. Since our private key is of integer type, the simplest thing we can do is to add it to the base value, and keep the remaining algorithm as it is:

func generateCode(pk int64) string {
    // gets current timestamp:
    current := time.Now().Unix()
    fmt.Println("Current timestamp: ", current)

    // gets a number that is stable within 30 seconds:
    base := current / 30 + pk
    fmt.Println("Base: ", base)

    // makes sure it has only 6 digits:
    code := base % 1_000_000

    // adds leading zeros if necessary:
    return fmt.Sprintf("%06d", code)
}

Let's make sure it produces different results for different private keys:

var pkForUser1 int64 = 115537846
var pkForUser2 int64 = 715488689

codeForUser1 := generateCode(pkForUser1)
fmt.Println("Code for user 1: ", codeForUser1)

fmt.Println()

codeForUser2 := generateCode(pkForUser2)
fmt.Println("Code for user 2: ", codeForUser2)

The outcome looks as we wanted and expected:

Current timestamp:  1733695429
Base:  173327693
Code for user 1:  327693

Current timestamp:  1733695429
Base:  773278536
Code for user 2:  278536

Works like charm! This means that the private key should be injected into the device that generates codes before it is sent to the users like me: that should not be a problem at all for the banks.

Are we done now? Well, only if we are satisfied with the artificial scenario we used. If you have ever enabled the MFA for any services / websites you have account on, you might have noticed that the web resource asks you to scan the QR code with your second-factor app of choice (Authy, Google Authenticator, 2FAS, etc.) that will input the secret code into your app and start generating 6-digits code from that moment on. Alternatively, it is possible to enter the code manually.

I'm bringing this up to mention that it is possible to take a peek at the format of the real private keys used in the industry. They are usually 16-32 characters long Base32-encoded strings that look like this:

JBSWY3DPEB2GQZLSMUQQ

As you can see, this is quite different from the integer private keys we used, and the current implementation of our algorithm won't work if we are to switch to this format. How can we adjust our logic?

Private key as a string

Let's start with a simple approach: our code won't compile, because of this line:

base := current/30 + pk

as pk is of type string from now on. So why don't we convert it into integer? While there are way more elegant and performant ways to do that, here is the simplest thing I came up with:

// converts the string pk to a number
func hash(pk string) int64 {
    var num int64 = 0
    for _, char := range pk {
        num = 31*num + int64(char)
    }
    return num
}

This is heavily inspired by the Java hashCode() implementation for the String data type, which makes it good-enough for our scenario.

Here is the adjusted logic:

func main() {
    var pkForUser1 int64 = 115537846
    var pkForUser2 int64 = 715488689

    codeForUser1 := generateCode(pkForUser1)
    fmt.Println("Code for user 1: ", codeForUser1)

    fmt.Println()

    codeForUser2 := generateCode(pkForUser2)
    fmt.Println("Code for user 2: ", codeForUser2)
}

func generateCode(pk int64) string {
    // gets current timestamp:
    current := time.Now().Unix()
    fmt.Println("Current timestamp: ", current)

    // gets a number that is stable within 30 seconds:
    base := current / 30 + pk
    fmt.Println("Base: ", base)

    // makes sure it has only 6 digits:
    code := base % 1_000_000

    // adds leading zeros if necessary:
    return fmt.Sprintf("%06d", code)
}

Here is the terminal output:

Current timestamp:  1733771953
Base:  9056767456302232090
Code:  232090

Nice, 6-digits code is there, good job. Let's wait to get to the next time window and run it again:

Current timestamp:  1733771973
Base:  9056767456302232091
Code:  232091

Hm...it works, but the code is, basically the increment of the previous value, which is not good, as this way OTPs are predictable, and having its value and knowing what time it belongs to, it is very simple to start generating the same values without the need of knowing the private key. Here is the simple pseudocode for this hack:

now = currentTimestamp()
diff = now - oldOtpTimestamp
numOfOtpsIssuedSinceThen = diff / 30
currentOtp = keepWithinSixDigits(oldOtp + numOfOtpsIssuedSinceThen)

where keepWithinSixDigits will make sure that after the 999 999 the next value is 000 000 and so on to keep the value within the 6-digits limit possibilities.

As you can see, it is a serious security flaw. Why does it happen? If we take a look at the base calculation logic, we'll see that it relies on 2 factors:

current timestamp divided by 30
hash of the private key

The hash produces the same value for the same key, so its value is constant. As for the current / 30 , it has the same value for 30 seconds, but once the window has passed, the next value will be the increment of the previous one. Then base % 1_000_000 behaves the way we see. Our previous implementation (with private keys as integers) had the same vulnerability, but we didn't notice that - lack of testing to blame.

We need to transform current / 30 into something to make the change of its value more noticeable.

Distributed OTP values

There are multiple ways to achieve that, and some cool math tricks exist out there, but for the educational purposes let's prioritize the readability of the solution we'll go with: let's extract current / 30 into a separate variable base and include it into the hash computation logic:

...
base := current / 30
...

// converts base and pk into a number
func hash(base int64, pk string) int64 {
    num := base
    for _, char := range pk {
        num = 31*num + int64(char)
    }
    fmt.Println("Hash: ", num)
    return num
}

This way, even though the base will change with 1 every 30 seconds, after being used within the hash() function logic, the weight of the diff will increase due to the series of performed multiplications.

Here is the updated code example:

func main() {
    pk := "JBSWY3DPEB2GQZLSMUQQ"

    codeForUser1 := generateCode(pk)
    fmt.Println("Code: ", codeForUser1)
}

func generateCode(pk string) string {
    // gets current timestamp:
    current := time.Now().Unix()
    fmt.Println("Current timestamp: ", current)

    // gets a number that is stable within 30 seconds:
    base := current / 30

    // combines the base with the private key to get a number unique to the user:
    baseWithPk := hash(base, pk)
    fmt.Println("Base: ", baseWithPk)

    // makes sure it has only 6 digits:
    code := baseWithPk % 1_000_000

    // adds leading zeros if necessary:
    return fmt.Sprintf("%06d", code)
}

// converts base and pk into a number
func hash(base int64, pk string) int64 {
    num := base
    for _, char := range pk {
        num = 31*num + int64(char)
    }
    fmt.Println("Hash: ", num)
    return num
}

Let's run it:

Current timestamp:  1733773490
Hash:  -1073062424175310899
Base:  -1073062424175310899
Code:  -310899

Boom! How come did we get the minus value here? Well, it seems like we ran out of the int64 ranges, so it capped the values to the minus and started over - my Java fellows are familiar with this from the hashCode() behavior. The fix is simple: let's take the absolute value from the result, so then the minus sign is ignored:

// combines the base with the private key to get a number unique to the user:
baseWithPk := hash(base, pk)

// makes sure it is positive:
absBaseWithPk := int64(math.Abs(float64(baseWithPk)))

Here is entire code sample with the fix:

func main() {
    pk := "JBSWY3DPEB2GQZLSMUQQ"

    codeForUser1 := generateCode(pk)
    fmt.Println("Code: ", codeForUser1)
}

func generateCode(pk string) string {
    // gets current timestamp:
    current := time.Now().Unix()
    fmt.Println("Current timestamp: ", current)

    // gets a number that is stable within 30 seconds:
    base := current / 30

    // combines the base with the private key to get a number unique to the user:
    baseWithPk := hash(base, pk)

    // makes sure it is positive:
    absBaseWithPk := int64(math.Abs(float64(baseWithPk)))
    fmt.Println("Base: ", absBaseWithPk)

    // makes sure it has only 6 digits:
    code := absBaseWithPk % 1_000_000

    // adds leading zeros if necessary:
    return fmt.Sprintf("%06d", code)
}

// converts base and pk into a number
func hash(base int64, pk string) int64 {
    num := base
    for _, char := range pk {
        num = 31*num + int64(char)
    }
    fmt.Println("Hash: ", num)
    return num
}

Let's run it:

Current timestamp:  1733774391
Hash:  3581577654419246315
Base:  3581577654419246080
Code:  246080

Let's run it again to make sure that the OTP values are distributed now:

Current timestamp:  1733774402
Hash:  4351623792829383276
Base:  4351623792829383168
Code:  383168

Nice, finally a decent solution!

Actually, that was the moment when I stopped my manual implementation process, as I had my share of fun and learned something new. However, it's neither the best solution nor the one I'd go live with. Among other things, it has a big flaw: as you can see, our logic always deals with big numbers due to the hashing logic and timestamp values, which means that it's highly unlikely for us to be able to generate results that start with 1 or more zeros: e.g., 012345 , 001234, etc., even though they are completely valid. Due to that, we are 100 000 possible values short, which is 10% from the possible number of outcomes of the algorithm - the chances of collisions are higher this way. Not cool!

Where to go from here

I won't go deep into the implementation that is used in the real applications, but for those who are curious, I'll share two RFCs worth taking a look at:

And here is the pseudocode implementation that will work the intended way based on the RFCs above:

function generateTOTP(secretKey):
    current = time.now().unix()
    counter = floor(currentTime / 30)
    hmac = HMAC-SHA1(secretKey, counter)
    offset = last_byte(hmac) & 0x0F
    truncatedHash = hmac[offset:offset+4]  // Extract 4 bytes
    binary = truncatedHash & 0x7FFFFFFF    // Ensure positive number
    otp = binary % 1_000_000               // Reduce to desired length
    return otp

As you can see, we got very close to that, but the original algorithm uses more advanced hashing (HMAC-SHA1 in this example), and performs some bitwise operations to normalize the output.

Security considerations: reuse rather than build yourself

However, there is one more thing I'd like to cover before we call it a day: security. I'd strongly encourage you against implementing the logic of generating OTPs on your own, as there are plenty of libraries out there that have it done for us. The room for error is huge, and it's a short distance to the vulnerability that will be discovered and exploited by the bad actors out there.

Even if you get the generation logic right and will cover it with tests, there are other things that can go wrong. For example, how much do you think it will take to brute-force the 6-digit code? Let's experiment:

func main() {
    pk := "JBSWY3DPEB2GQZLSMUQQ"

    code := generateCode(pk)
    fmt.Println("Code: ", code)

    // brute forces the OTP and measures the time it takes to find it in milliseconds:
    start := time.Now()
    for i := 0; i < 1_000_000; i++ {
        if fmt.Sprintf("%06d", i) == strconv.FormatInt(code, 10) {
            fmt.Println("Found: ", i)
            finish := time.Now()
            fmt.Println("Time: ", finish.Sub(start).Milliseconds(), "ms")
            break
        }
    }
}

func generateCode(pk string) int64 {
    // gets current timestamp:
    current := time.Now().Unix()

    // gets a number that is stable within 30 seconds:
    base := current/30 + hash(pk)

    // makes sure it has only 6 digits:
    return base % 1_000_000
}

// converts the string pk to a number
func hash(pk string) int64 {
    var num int64
    for _, c := range pk {
        num += int64(c)
    }
    return num
}

Let's run this code:

Code:  661504
Found:  661504
Time:  75 ms

And once more:

Code:  524928
Found:  524928
Time:  67 ms

As you can see, it takes around 70 ms to guess the code via simple brute-forcing for-loop. That's 400+ times faster than the lifetime of the OTP! The server of the app/website using OTP mechanism needs to prevent that by, for example, not accepting new codes for the next 5 or 10 seconds after 3 failed attempts. This way the attacker gets only 18 or 9 attempts correspondingly within the 30 seconds window, which is not enough for the pool of 1 million possible values.

And there are other things like this that are easy to overlook. So, let me repeat: don't build this from scratch, but rely on the existing solutions.

Anyway, I hope you learned something new today, and the OTP logic won't be a mystery for you from this point on. Also, if at some point in life, you'll need to make your offline device generate some values using a reproducible algorithm, you have a good idea where to start.

Thanks for the time you spent reading this post, and have fun! =)

P.S. Receive an email once I publish a new post - subscribe here

P.P.S. Like the rest of the cool kids, I have created a Bluesky account lately, so, please, help me to make my feed more fun =)

JWT, JWS, JWE and how to cook them

Mykola — Thu, 28 Mar 2024 19:42:33 +0000

Hello there! It's been a while since I wrote here - all of a sudden, 2024 became way busier than I planned it to be. But hey, it's good to be back!

Similar to my previous post, Understanding CORS, this one has the same backstory: lately, I have had to explain a few times to different people about such concepts as JSON Web Tokens (JWT), their structure, types, use cases, etc., so I realized that it would be smart to write a post about that and use it for the future reference. I hope it will be helpful for someone else out there.

Traditionally, let's start with a real-life example.

Real-life example

Let me introduce you to Craig and Clyde, two buddies from the same class. They like to hang out together after school, but they also don't miss the opportunity to chat during the lessons. It didn't go unnoticed by their teacher, Mr. Garisson, so he made sure they didn't sit next to each other. Well, challenge accepted, there are other ways to contact each other: their classmates use smartphones and different message apps for that, but these two dudes are a bit nerdy, so they came up with something smarter - they pass messages written on a piece of paper. However, the message written on a piece of paper is not plain text but something more complicated. Let me show you a small example.

Imagine that Clyde wants to send a "Hello there!" message to Craig. Then, the message on a piece of paper will look like this:

ewogICJzdWIiOiJDcmFpZyIsCiAgImlhdCI6MTcxMDYxMTk1MywKICAibWVzc2FnZSI6IkhlbGxvIHRoZXJlISIKfQ

Weird, right? How come is the mumbo jumbo above "Hello world!"? And why bother? Too many characters for such a simple phrase! Good questions, my friend! There are 2 good reasons for the following:

Remember, I mentioned that their teacher made sure that they didn't sit next to each other? Well, actually, there is only one desk between them, but the guy sitting on that desk, Eric, is well-known for his curiosity and evil pranks attitude. And since the piece of paper goes through him each time, Craig and Clyde had to come up with some layer of privacy. This leads us to the 2nd reason.
As I stated while introducing the guys, they are nerdy, so they found it to be funny in their way.

Ok, the "why" is clear now, but the "how", or, to be more specific, the "what the hell is that text?" question remains. That's the exact question Eric asked himself each time he passed the piece of paper between Craig and Clyde as his curiosity grew day by day.

Soon, he understood that there was no way he could solve the mystery on his own, so he went to the place where other nerdy folks hang out - the internet! Some "good" people out there, after shaming him for asking such simple questions, gave him a hint that this piece of text is nothing more than a Base64URL-encoded string, and there are plenty of online tools to decode it.

Decoding the message

Once Eric posted the text he copied from the piece of paper to the Base64URL online decoder, he saw this output:

{
  "sub": "Craig",
  "iat": 1710611953,
  "message": "Hello there!"
}

Lucky day, the mystery is not a mystery anymore!

If Eric had some coding experience, he could have easily decoded the text without any online tools by using this code:

func main() {
    encodedMessage := "ewogICJzdWIiOiJDcmFpZyIsCiAgImlhdCI6MTcxMDYxMTk1MywKICAibWVzc2FnZSI6IkhlbGxvIHRoZXJlISIKfQ"
    res, _ := base64.RawURLEncoding.DecodeString(encodedMessage)

    fmt.Println(string(res))
}

All the code examples are available here. If you run this code, you'll see the same JSON printed to the terminal as above.

As you can see the original JSON contains several fields:

sub stays for "subject" and specifies whom the message refers to
iat is the acronym for "issued at" and shows when the message was created in a Unix epoch seconds timestamp fashion - meaning how many seconds have passed since the ~~beginning of times~~ 1st January 1970
message is self-explanatory and the main reason of all this hustle

Not sure about you, but for me all of these fields make sense, as they somewhat mimic the message and associated metadata we can see in WhatsApp/Telegram/Signal/SMS/youNameIt messaging apps.

If you don't know what Base64URL encoding is, let me give you a brief explanation: it is a technique for transforming any text (or binary data) into a sequence of characters. A set of 64 unique characters is used for the resulting text. It means that regardless of whether the input is Latin, Cyrillic, Arabic, Chinese, etc. chars, the output is always generated with the same 64 characters. That's where the "64" name part comes from. The important moment to highlight is that the text can be easily encoded and decoded back and forth by anyone, as Base64 doesn't imply any security for the original input. Check the corresponding Wikipedia page for more details about Base64 and its Base64URL subtype.

As you can see, I highlighted the

the text can be easily encoded and decoded back and forth by anyone, as Base64 doesn't imply any security for the original input

part. As soon as Eric learned this, he immediately saw the opportunity for a cool evil prank there.

The prank

The idea behind the prank was both simple and easy: since now Eric knows how they encode their messages, nothing stops him from decoding the original input, modifying the message field value in some evil way, decoding it back, writing to a piece of paper and passing it forward to the desired destination. Guys don't suspect he knows their secret, so he has an advantage.

Soon enough, he received a piece of paper from Craig that he was supposed to pass forward to Clyde. The text looked like that:

ewogICJzdWIiOiJDbHlkZSIsCiAgImlhdCI6MTcxMDYxNTMzOCwKICAibWVzc2FnZSI6IkxldCdzIHBsYXkgZm9vdGJhbGwgdG9nZXRoZXIgbGF0ZXIgdG9kYXkhIgp9

"An opportunity!" - Eric immediately decoded the message the way he already knew, and saw this:

{
  "sub":"Clyde",
  "iat":1710615338,
  "message":"Let's play football together later today!"
}

"Let me fix this a bit!"

{
  "sub":"Clyde",
  "iat":1710615338,
  "message":"I see that I'm too smart to be friends with a guy like you - don't talk to me, ok?"
}

But how to decode this back? Well, while there are plenty of online tools to help with that, let's achieve the same result with the code:

func main() {
    message := `{
  "sub":"Clyde",
  "iat":1710615338,
  "message":"I see that I'm too smart to be friends with a guy like you - don't talk to me, ok?"
}`
    encodedMessage := base64.RawURLEncoding.EncodeToString([]byte(message))

    fmt.Println(encodedMessage)
}

The result is:

ewogICJzdWIiOiJDbHlkZSIsCiAgImlhdCI6MTcxMDYxNTMzOCwKICAibWVzc2FnZSI6Ikkgc2VlIHRoYXQgSSdtIHRvbyBzbWFydCB0byBiZSBmcmllbmRzIHdpdGggYSBndXkgbGlrZSB5b3UgLSBkb24ndCB0YWxrIHRvIG1lLCBvaz8iCn0

Eric quickly replaced the original text on a piece of paper with a new one and passed it to Clyde. Soon enough, he got Clyde's response which contained the message that if I decided to share here, I'd need to use a "Strong language" warning. "Great success!" - Eric thought and tried hard not to laugh. I told you about his attitude, so don't be surprised!

For the next couple of days, there was no correspondence between the two friends, so Eric felt kinda proud of his smart little trick. But one rainy day (like that wasn't too bad already), Craig gave him a piece of paper and asked to pass it to Clyde. Eric immediately realized that something was wrong with the message:

ewogICJ0eXAiOiJKV1QiLAogICJhbGciOiJIUzI1NiIKfQ.ewogICJzdWIiOiJDbHlkZSIsCiAgImlhdCI6MTcxMDYxNzA5MSwKICAibWVzc2FnZSI6IkhleSwgYnVkZHkhIEJvYXJkIGdhbWVzIHRvbmlnaHQhIgp9.ny29zDJjI-QCbihNyPx7hjj0wxpM3E6Isagktf9U-1o

When he tried to decode it via the online tool, he got this output:

{
  "typ":"JWT",
  "alg":"HS256"
}'7V"#$6ǖFR"&B#scs&W76vR#$W'VFG&&BvRvBFF EI?U6?=W(KtC

which was definitelly off. Trying the same with the Go code didn't change the result much:

{
  "typ":"JWT",
  "alg":"HS256"

After the short moments of panic, Eric calmed himself down and took a closer look at the new message format. Soon enough, he noticed an interesting moment: the message contains 3 parts that are separated by the . symbol. "Ok, we are getting to something!" It seemed logical to try to split the text into 3 chunks and decode them separately:

func main() {
    encodedMessage := "ewogICJ0eXAiOiJKV1QiLAogICJhbGciOiJIUzI1NiIKfQ.ewogICJzdWIiOiJDbHlkZSIsCiAgImlhdCI6MTcxMDYxNzA5MSwKICAibWVzc2FnZSI6IkhleSwgYnVkZHkhIEJvYXJkIGdhbWVzIHRvbmlnaHQhIgp9.ny29zDJjI-QCbihNyPx7hjj0wxpM3E6Isagktf9U-1o"
    chunks := strings.Split(encodedMessage, ".")
    for _, chunk := range chunks {
        res, _ := base64.RawURLEncoding.DecodeString(chunk)
        fmt.Println(string(res) + "\n")
    }
}

The result is:

{
  "typ":"JWT",
  "alg":"HS256"
}

{
  "sub":"Clyde",
  "iat":1710617091,
  "message":"Hey, buddy! Board games tonight!"
}

�-��2c

"Ha! And they call themselves smart!". The first and the third parts looked kinda of weird to Eric, but was sure they tried to confuse him this way, so without further hesitation, he changed the message part of the second JSON to "Hey! Board games tonight for smart guys only, so you are not invited!", encoded it back, appended 1st and 3rd part to where they belong to unchanged and passed the modified piece of paper to Clyde. The answer came back in a few minutes:

ewogICJ0eXAiOiJKV1QiLAogICJhbGciOiJIUzI1NiIKfQ.ewogICJzdWIiOiJFcmljIiwKICAiaWF0IjoxNzEwNjE3MTkxLAogICJtZXNzYWdlIjoiTmljZSB0cnksIEVyaWMsIGJ1dCBnbyBwbGF5IHdpdGggdGhlIGtpZHMgaW5zdGVhZCB3aGlsZSBtZW4gYXJlIHRhbGtpbmcgaGVyZSEiCn0.m1HhjXosZYlEdiOHE8X_34ydYBBNAhG08xgZBigaXtE

Eric started decoding it with great impatience, being twice as proud of himself. However, the result was far from what he had expected:

{
  "typ":"JWT",
  "alg":"HS256"
}

{
  "sub":"Eric",
  "iat":1710617191,
  "message":"Nice try, Eric, but go play with the kids instead while men are talking here!"
}

�Q�z,e�Dv#�

"What's going on?!!"

The anti-prank measures

Little did he know that Craig and Clyde talked to each other to resolve the conflict they thought they had and figured out they had been fooled. And that was easy to guess who did that. The harder part was to come up with an idea of how to prevent this from happening. They realized Eric could read their messages, but that was fine - they didn't share anything sensitive. But the ability to modify them wasn't something they had a piece with. It meant they needed a way to see whether the original input had been changed along the way. And nerdy folks like them (or you and I) know the answer - the signature!

If you are not familiar with this concept, no worries, it is pretty simple and looks like this:

imagine there is a text that we'd like to make sure stays the same along the way
we need depending on the algorithm, either a secret phrase that both sender and receiver know, or the private and public key - only the public key is shared with the receiver in this case
the sender takes the original text and applies a specific signing algorithm on it with the usage of either the secret phrase or the private key - the result looks similar to the third part of the message we saw above: for example, ny29zDJjI-QCbihNyPx7hjj0wxpM3E6Isagktf9U-1o - we'll call this a signature
the signature is passed alongside the original text all the way to the receiver
once receiver gets a message:
- if the secret phrase was used, the receiver calculates the signature for the text they received the same way as the sender did above and compares the two - if they are the same, the original text was not modified; otherwise - it shouldn't be trusted. As long as the secret phrase is kept safe, the man-in-the-middle has no way to both change the text and add a valid signature to it
- if the private key was used, the flow is a bit complicated, and we'll discuss it a bit later, but the high-level idea remains the same

As you can already guess, that's exactly the measure that Craig and Clyde took against Eric's prank, and that's how they noticed when he replaced the original message with the fake one.

Let's see how they achieve that by code. The secret phrase part was easy as they agreed on it before the lesson, so they both knew it.

func main() {
    // sender part:
    secretPhase := "goToHellEric"

    part1 := `{
  "typ":"JWT",
  "alg":"HS256"
}`

    part2 := `{
  "sub":"Eric",
  "iat":1710617191,
  "message":"Nice try, Eric, but go play with the kids instead while men are talking here!"
}`

    part1Encoded := encode(part1)
    part2Encoded := encode(part2)
    signature := sign(part1Encoded, part2Encoded, secretPhase)

    encodedMessageWithSignature := part1Encoded + "." + part2Encoded + "." + signature

    fmt.Println(encodedMessageWithSignature)

    // receiver part:
    parts := strings.Split(encodedMessageWithSignature, ".")

    receiverSignature := sign(parts[0], parts[1], secretPhase)
    senderSignature := parts[2]
    if receiverSignature == senderSignature {
        fmt.Println("The signature matches - the original message hasn't been modified")

        part1Decoded := decode(parts[0])
        part2Decoded := decode(parts[1])

        fmt.Println(part1Decoded)
        fmt.Println()
        fmt.Println(part2Decoded)
    } else {
        fmt.Println("The signature differs - the original message has been modified")
    }
}

func encode(text string) string {
    return base64.RawURLEncoding.EncodeToString([]byte(text))
}

func decode(text string) string {
    decoded, _ := base64.RawURLEncoding.DecodeString(text)
    return string(decoded)
}

func sign(part1 string, part2 string, secretPhrase string) string {
    textToSign := part1 + "." + part2
    signer := hmac.New(sha256.New, []byte(secretPhrase))

    signer.Write([]byte(textToSign))
    binarySignature := signer.Sum(nil)
    return base64.RawURLEncoding.EncodeToString(binarySignature)
}

A disclaimer: the author of this post doesn't know why they used a wish for Eric to go to the specific Norwegian village as their secret phrase.

I believe the code should be pretty straightforward, except maybe the sign function and the first part of the message:

{
  "typ":"JWT",
  "alg":"HS256"
}

Fear not, we'll discuss this soon. But I think now you should have a very good understanding of why Eric's prank didn't work once the signature was introduced - the if receiverSignature == senderSignature will return false in that case, so we won't even bother decoding the message.

I think it's a good moment to pause our interactions with the guys (we'll get back to them soon though), and try to map what we have learned so far to the main topic of this post - JWT.

JWT and JWS

If you have read the post all the way to this very point and began wondering why there is nothing about JWT/JWS/JWE yet, I have to say that it's a valid concern. But, hey, let me tell you this: with the things we have discussed above, you have learned almost everything I wanted to share about JWT and JWS. How come? Let me try to explain.

The (probably) most famous web resource about JWT - https://jwt.io - provides such a definition of JSON Web Tokens:

JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between parties as a JSON object. This information can be verified and trusted because it is digitally signed. JWTs can be signed using a secret (with the HMAC algorithm) or a public/private key pair using RSA or ECDSA.

If we use a Wikipedia article about the JWT as a reference, we'll notice that it specifies the following structure of JWTs:

header
payload
signature

Hmm...3 parts and our message had 3 parts - coincidence? Not really. Let's see what each part usually contains and compare it with the Craig and Clyde approach. A spoiler alert: we'll see soon enough that once guys took the anti-prank measures, they were using JWT/JWS format =)

Header

If you are familiar with the concept of HTTP headers, there is a similarity between the two in a way: the header part is used to pass some metadata. To be more specific: usually, the type of token and the algorithm used to generate the signature are provided there. Here is how it looks in our case:

{
  "typ":"JWT",
  "alg":"HS256"
}

It is clear to us that the type of the token is JWT, and the algorithm used is HS256, which stands for HMAC-SHA256. Algorithm info is very important for the receiver, as they need to know the way signature has been calculated, so they can follow the same algorithm in order to get the same result.

There is some variety of algorithms available for the signature in the JWT specification. They belong to 3 main groups:

HMAC
RSA
ECDSA

The high-level key difference between them is the fact that the HMAC algorithms rely on the secret phrase (like we saw below), while RSA and ECDSA use a private-public keys pair for generating/verifying signature. We won't go deep into these algorithms, as it's way outside the scope of this post, but we'll discuss how to use private-public keys for JWT purposes further.

There are other header fields available in the JWT specification, but the two above are the most commonly used.

Payload

If we stick to the HTTP analogy, we can compare the payload with the request body, as this is the place where the domain-specific information can be found. The fields of the payload are called claims, and there is a list of the standard claims available in the JWT specification. To name a few:

iss - who issued the JWT
sub - the user/application the token belongs to
exp - expiration time for JWT
nbf - not before, the time at which the JWT becomes valid
iat - issued at
jti - the unique identifier of the JWT

Users are free to include their own custom claims on top of these - we saw that in the Craig and Clyde example when they used the message claim for their needs:

{
  "sub":"Clyde",
  "iat":1710617091,
  "message":"Hey, buddy! Board games tonight!"
}

Signature

Well, we have already discussed this part and even saw what the secret phrase flow looks like for it. Let me mention, though, that in the realm of JWT, the signature is called JSON Web Signature (or JWS) - so now you know 2 out of 3 acronyms mentioned in the title of this post - good job! =)

But let's another look at the code we wrote for the JWS:

func sign(part1 string, part2 string, secretPhrase string) string {
    textToSign := part1 + "." + part2
    signer := hmac.New(sha256.New, []byte(secretPhrase))

    signer.Write([]byte(textToSign))
    binarySignature := signer.Sum(nil)
    return base64.RawURLEncoding.EncodeToString(binarySignature)
}

Since we know now that for the signature we used the HMAC-SHA256 algorithm, the line

signer := hmac.New(sha256.New, []byte(secretPhrase))

should make good sense - all we do is initiate the signer that will follow the HMAC-SHA256 algorithm to generate the signature using the secret phrase we provided. The next steps are as simple as that:

combine a header and a payload with the . as a separator - please, note that both header and payload should be Base64URL encoded at that point
create a signature - it has a binary format
apply Base64URL encoding to the binary signature to get it as a string

Well done! As we already know the receiver will follow the same steps and use the same secret phrase. That's the reason why the HMAC algorithm is known as a symmetric one.

For this example, we used HS256 as the algorithm, but quite often, you can see the HS384 or HS512 within the alg JWT header. As you might have guessed so far, it's still the HMAC algorithm, but with different hashing logic applied: SHA-384 and SHA-512, respectively.

HMAC, as any other symmetric algorithm, is a very decent one. However, there are certain cases when there is either no way to share the secret phrase between the parties in advance (remember that it was easy for Craig and Clyde, as they could meet in person before exchanging the JWTs) or the issues of JWT doesn't want to share it, as they want to be the only one who can issue that type of tokens. Imagine that the teacher of the class, Mr. Garisson, would like to send the scores for the math test in a JWT format. In that case, he wants to make sure that only he can issue that kind of token, as otherwise, some folks can try to hack the system and fake their scores. However, it is still important that there is a way for the receivers to verify the signature of the tokens. Here is when asymmetric algorithms like RSA and ECDSA come in handy.

Since this post is not about cryptography, we won't go deep into those algorithms but rather look at how we can use the asymmetric approach for the JWS needs. We'll stick to the RSA one, as I feel it's way more adopted in the industry these days, as it's an older one.

On a high level, the asymmetric algorithms rely on the concepts of the private and public keys. The private key is used for generating the signatures, and it is never shared with anyone by the JWT issues - remember Gandalf's rule of thumb "keep it secret, keep it safe". The public key, as the name suggests, is something that can be freely shared with anyone who needs to verify the signature - it can even be retrieved via the call to the specific public endpoint to simplify the flow. The main moment here is the following:

private key can only generate the signature but not verify it
public key can only verify the signature but not generate it

That's why such algorithms are called assymetric.

Here is how the signature flow looks for the sender - notice that it is very similar to the HMAC one:

combine a header and a payload with the . as a separator - please, note that both header and payload should be Base64URL encoded at that point
hash the resulted string by using the SHA-256 (for RS256) algorithm - it seems to be a new step, but only because the HS256 algorithm did it implicitly due to the Go implementation
sign the hashed value by applying the RSA algorithm with the usage of the private key - the signature has a binary format
apply Base64URL encoding to the binary signature to get it as a string

For the receiver the flow is a bit different:

combine a header and a payload with the . as a separator
hash the resulted string by using the SHA-256 (for RS256) algorithm
verify the signature by applying the RSA verification algorithm with the usage of the private key and the signature attached to the JWT

Let's see how it looks in the code:

func main() {
    // sender part:
    fmt.Println("Sender part:")

    privateKey, _ := rsa.GenerateKey(rand.Reader, 2048)
    publicKey := &privateKey.PublicKey

    header := `{
  "typ":"JWT",
  "alg":"RS256"
}`

    payload := `{
  "sub":"Clyde",
  "iat":1710617191,
  "message":"Hey, buddy! Board games tonight!"
}`

    headerEncoded := encode(header)
    payloadEncoded := encode(payload)
    signature := sign(headerEncoded, payloadEncoded, privateKey)

    encodedMessageWithSignature := headerEncoded + "." + payloadEncoded + "." + signature

    fmt.Println(encodedMessageWithSignature)

    // receiver part:
    fmt.Println("\nReceiver part:")

    parts := strings.Split(encodedMessageWithSignature, ".")
    senderSignature := parts[2]
    if verify(parts[0], parts[1], senderSignature, publicKey) {
        fmt.Println("The signature is valid - the original message hasn't been modified")

        headerDecoded := decode(parts[0])
        payloadDecoded := decode(parts[1])

        fmt.Println(headerDecoded)
        fmt.Println()
        fmt.Println(payloadDecoded)
    } else {
        fmt.Println("The signature differs - the original message has been modified")
    }
}

func encode(text string) string {
    return base64.RawURLEncoding.EncodeToString([]byte(text))
}

func decode(text string) string {
    decoded, _ := base64.RawURLEncoding.DecodeString(text)
    return string(decoded)
}

func sign(header string, payload string, privateKey *rsa.PrivateKey) string {
    textToSign := header + "." + payload
    hashed := sha256.Sum256([]byte(textToSign))

    binarySignature, _ := rsa.SignPKCS1v15(rand.Reader, privateKey, crypto.SHA256, hashed[:])
    return base64.RawURLEncoding.EncodeToString(binarySignature)
}

func verify(header string, payload string, signature string, publicKey *rsa.PublicKey) bool {
    textToSign := header + "." + payload
    hashed := sha256.Sum256([]byte(textToSign))

    binarySignature, _ := base64.RawURLEncoding.DecodeString(signature)
    err := rsa.VerifyPKCS1v15(publicKey, crypto.SHA256, hashed[:], binarySignature)
    return err == nil
}

If we run the code, we'll see the following output:

Sender part:
ewogICJ0eXAiOiJKV1QiLAogICJhbGciOiJSUzI1NiIKfQ.ewogICJzdWIiOiJDbHlkZSIsCiAgImlhdCI6MTcxMDYxNzE5MSwKICAibWVzc2FnZSI6IkhleSwgYnVkZHkhIEJvYXJkIGdhbWVzIHRvbmlnaHQhIgp9.0F0WJ08wQOoXwuqJrvboAlEBOuSuSwFUxH-OZFnPOw7CUqXKVp9l8uYWPtizLuN3_RnrN5AwTIySCh5BQrh6c4TeTtVBgguonKDFYXvxA33DhUeB8Zzpa_1w_NXbS20Xg-ZysE6nOpZDkI9EhUajK9_p3ulJ7wmI_0DIOj1oX7OfjMHrwg7Kj4NrBLgjOPV9cFd6-FysUWJTRfW6OeF6rKK3jacO_Bhtw9dF8Igjt4ZFjHosRKjCth67agFex4SmN_qaRxTW0d0TZSL5c_bp_xWP-gwosNWPctJKzc-AEY51ZHc-7izpOrcQYwM5TGliVDyL1FvDVhXF6qhwtb8jWw

Receiver part:
The signature is valid - the original message hasn't been modified
{
  "typ":"JWT",
  "alg":"RS256"
}

{
  "sub":"Clyde",
  "iat":1710617191,
  "message":"Hey, buddy! Board games tonight!"
}

Hopefully, this should make good sense now. You might have noticed that we are hardcoding the header and the payload part as strings, while the good practice here will be to rely on structs or, at least, a map. Good thinking, but I kept it like this to simplify the post and make it shorter - I believe I failed with the latter, though =)

One important undiscussed point remains: how does the receiver get the public key? The answer is: it depends. It is not wrong to share the public key somehow directly by sending a file or so. But that requires manual actions, and since humans are lazy, and laziness is a good driver of a progress, the industry came up with something smarter for that - JSON Web Key Set (JWKS).

JWKS

JWKS is a set of keys containing the public keys used to verify the JWT signature. JWKS is usually shared as a public endpoint. For example, Auth0 keeps them in https://{auth0CustomerDomain}/.well-known/jwks.json .
It looks something like this (the example is generated by ChatGPT):

{
  "keys": [
    {
      "kty": "RSA",
      "use": "sig",
      "kid": "2",
      "alg": "RS256",
      "n": "vGtXtSE3pPmV...iC5X3kjMz4dVvw",
      "e": "AQAB"
    },
    {
      "kty": "RSA",
      "use": "sig",
      "kid": "1",
      "alg": "RS256",
      "n": "pVMOEtC2ZvYg...kPQ2E7pFCl5LZpM",
      "e": "AQAB"
    }
  ]
}

We'll skip some of these fields, as they are outside the scope of this post, but let's notice the kid which is a unique identifier for the key. This kid is added as a header to the JWT by the issuer, so the sender can find the corresponding public key by its ID within the JWKS.

This approach makes it very easy to rotate keys: once the new private key is issued, the corresponding public key is added to the JWKS with a new kid. Regardless of whether JWT was signed by the old or new private key, thanks to the kid, the sender can fetch the correct public key and successfully verify the signature. Simple as that!

Ok, we know now what JWT is, also we went through its structure and took a very good look at the signature part. But the key part is missing: why do we need JWTs at all? A good question!

Usage

While fellow developers out there might have come up with countless use cases for JWT, there are 2 most common scenarios for these tokens in the industry these days:

authorization
info exchange

Both of them can be perfect candidates for a separate blog, so I'll not only briefly elaborate on these scenarios.

Authorization

This is by far the most common use case for JWT. Usually, it looks like this:

the user visits the website and presses the login button
they enter their credentials (or use any other sign-in method)
as a result, they receive the JWT, that, from now on, will be passed within the request to the server - as a rule of thumb, via the Authorization header

In such a case, JWT can contain user identifiers and some info (like name, email, etc.) within its payload., as well as the list of permissions the user has.

Info exchange

As we have seen before, thanks to the signature mechanism, JWT provides a simple way of ensuring that the information hasn't been modified along the way, which makes it a good candidate for the information exchange format.

However, there is one critical moment to remember: JWT stores information in the open format, meaning that if intercepted, anyone can read it - as our example with the school dudes proved. But what if we'd like to pass something that should remain hidden from the unwanted guests? Shall we abandon the JWT format entirely in that case? These are excellent questions, and let's get back to the classroom to find the answers together with our old friends Craig, Clyde, and Eric.

Real-life example #2

There have been peace and balance for quite a while: Craig and Clyde kept exchanging messages in the JWT format with a signature, and Eric didn't manage to figure out how to bypass the signature protection but was partly satisfied by his possibility to be able to read every single message they sent. Until the day came when two friends needed to discuss something more privately: they had a very cool idea for a new video game, and the last thing they wanted was to share it with anyone else, especially Eric. Should they abandon their ways of messaging and start using mobile phones like the rest of their classmates? That sounded too simple and not nerdy at all, so "nope" was their answer. And they began thinking and asking themselves the right questions:

What is the key problem with the JWT in this case? The fact that the payload is passed in the opened format as the encoding (e.g., Base64) has nothing to do with the data protection.

Are there other ways, then? Of course, encryption! What stops us from encrypting the payload and passing it this way? Nothing. Let's do it then!

To achieve the desired result, the guys need to have one more secret phrase on top of the existing one and use it as the key for the encryption/decryption of the payload - they decided to use the symmetric algorithm again, as it is the most sufficient one for their use case. Once they have the key, they must add one extra step to their existing flow: encrypt the payload. Their signature should be calculated for the encrypted payload.

At first, Craig and Clyde wanted to encrypt headers as well, but then they realized that it was not the best idea, as otherwise, they wouldn't be able to tell each other about the algorithm they used for the encryption.

The same additional step applies to the message receiver flow: they need to decrypt the payload first and only then decode it. Here is how the sender and receiver flow looks on a code level:

func main() {
    // sender part:
    fmt.Println("Sender part:")

    signingSecretPhase := "goToHellEric"
    encryptionSecretPhase := "keepItSecretKeepItSafe"

    jweHeader := `{
  "typ":"JWE",  
  "alg":"dir",
  "enc":"A256GCM",  
}`

    payload := `{
  "sub":"Craig",
  "iat":1710617455,
  "message":"I think we should include elves in our videogame!"
}`

    headersEncoded := encode(jweHeader)
    payloadEncoded := encode(payload)
    payloadEncrypted, _ := encrypt(payloadEncoded, encryptionSecretPhase)
    signature := sign(headersEncoded, payloadEncrypted, signingSecretPhase)

    encodedMessageWithSignature := headersEncoded + ".." + payloadEncrypted + "." + signature

    fmt.Println(encodedMessageWithSignature)

    // receiver part:
    fmt.Println("\nReceiver part:")

    parts := strings.Split(encodedMessageWithSignature, ".")
    headersReceived := parts[0]
    payloadReceived := parts[2] + "." + parts[3]
    signatureReceived := parts[4]

    receiverSignature := sign(headersReceived, payloadReceived, signingSecretPhase)
    if receiverSignature == signatureReceived {
        fmt.Println("The signature matches - the original message hasn't been modified")

        headersDecoded := decode(headersReceived)
        payloadDecrypted, _ := decrypt(payloadReceived, encryptionSecretPhase)
        payloadDecoded := decode(payloadDecrypted)

        fmt.Println(headersDecoded)
        fmt.Println()
        fmt.Println(payloadDecoded)
    } else {
        fmt.Println("The signature differs - the original message has been modified")
    }
}

func encode(text string) string {
    return base64.RawURLEncoding.EncodeToString([]byte(text))
}

func decode(text string) string {
    decoded, _ := base64.RawURLEncoding.DecodeString(text)
    return string(decoded)
}

func encrypt(payload string, secretPhrase string) (string, error) {
    hash := sha256.Sum256([]byte(secretPhrase))
    secretKey := hash[:]

    block, err := aes.NewCipher(secretKey)
    if err != nil {
        return "", err
    }

    gcm, err := cipher.NewGCM(block)
    if err != nil {
        return "", err
    }

    nonce := make([]byte, gcm.NonceSize())
    if _, err = io.ReadFull(rand.Reader, nonce); err != nil {
        return "", err
    }
    encodedNonce := base64.RawURLEncoding.EncodeToString(nonce)

    ciphertext := gcm.Seal(nil, nonce, []byte(payload), nil)
    encodedCiphertext := base64.RawURLEncoding.EncodeToString(ciphertext)

  // The encrypted payload includes the IV and the ciphertext, separated by a dot.
    return fmt.Sprintf("%s.%s", encodedNonce, encodedCiphertext), nil
}

func decrypt(encryptedPayload string, secretPhrase string) (string, error) {
    hash := sha256.Sum256([]byte(secretPhrase))
    secretKey := hash[:]

    // The encryptedPayload includes the IV and the ciphertext, separated by a dot.
    parts := strings.Split(encryptedPayload, ".")
    if len(parts) != 2 {
        return "", fmt.Errorf("invalid encrypted payload format")
    }
    encodedNonce, encodedCiphertext := parts[0], parts[1]

    nonce, err := base64.RawURLEncoding.DecodeString(encodedNonce)
    if err != nil {
        return "", err
    }

    ciphertext, err := base64.RawURLEncoding.DecodeString(encodedCiphertext)
    if err != nil {
        return "", err
    }

    block, err := aes.NewCipher(secretKey)
    if err != nil {
        return "", err
    }

    gcm, err := cipher.NewGCM(block)
    if err != nil {
        return "", err
    }

    decrypted, err := gcm.Open(nil, nonce, ciphertext, nil)
    if err != nil {
        return "", err
    }

    return string(decrypted), nil
}

func sign(part1 string, part2 string, secretPhrase string) string {
    textToSign := part1 + "." + part2
    signer := hmac.New(sha256.New, []byte(secretPhrase))

    signer.Write([]byte(textToSign))
    binarySignature := signer.Sum(nil)
    return base64.RawURLEncoding.EncodeToString(binarySignature)
}

I should warn you here: this code is written for educational purposes only, and while it was sufficient for the boys' case, it is not good enough for the proper production systems - please treat it like that.

If we run this code, we'll see that it works well. The resulting token looks like this:

ewogICJ0eXAiOiJKV0UiLAkKICAiYWxnIjoiZGlyIiwKICAiZW5jIjoiQTI1NkdDTSIsCQp9..4kt2psa8PhmBLfz6.G7zzu7eJRL0x8vomHI6f9mQF4XODm8TbWpo6f13AGlkczA3ceU5OtL1-gF9crSlYdsHIhfe_4Z0U2gwk_tWxf_GWW6MV1XDfDecPFSQJOJhcgLoeLRwdzvMN5SA73WsZdf6atY8tS1TfbZ-mcLLkyfcLdSsig1pBr33dQBRdbp1RQ9fqxyhpszQwRgCbDXBLcXUvFYFF0Y1_pgw.318c4zbUNCAU36s1uBn5H_LbHC_OB12gnd_oC3PRXwE

If you take a closer look, you'll see that its format differs from the JWT header.payload.signature one, as here we can see 4 dots instead of 2. What's going on? To answer this properly, we need to dive deeper into cryptography, and this post is not the right place for it. But let me give you a very short answer: to make sure that the receiver can decrypt the payload, the sender has to include additional parts to the token, so then the format looks like this:

header.encryptedKey.initializationVector.cipherText.cipherText

We know what a header is. The encryptedKey and the initializationVector parts are needed to decrypt the payload, as that's how some encryption/decryption algorithms work. The cipherText contains the encrypted payload, while the cipherText ensures the content hasn't been modified along the way. The code we wrote above mimics this, but with some hacks along the way, that's why I explicitly specified that it shouldn't be used outside this article.

But let's get back to our friends. As we saw above, the format of the message has changed. Eric also noticed it when the message came to him. "Hm, another trick, but I know what to do in such a case!" - he thought and began decoding each part of the message.

The ewogICJ0eXAiOiJKV0UiLAkKICAiYWxnIjoiZGlyIiwKICAiZW5jIjoiQTI1NkdDTSIsCQp9 converted into the

{
  "typ":"JWE",  
  "alg":"dir",
  "enc":"A256GCM",  
}

"A good start!" - he thought. But then the situation got worse:

4kt2psa8PhmBLfz6 -> �Kv�Ƽ>�-��

G7zzu7eJRL0x8vomHI6f9mQF4XODm8TbWpo6f13AGlkczA3ceU5OtL1-gF9crSlYdsHIhfe_4Z0U2gwk_tWxf_GWW6MV1XDfDecPFSQJOJhcgLoeLRwdzvMN5SA73WsZdf6atY8tS1TfbZ-mcLLkyfcLdSsig1pBr33dQBRdbp1RQ9fqxyhpszQwRgCbDXBLcXUvFYFF0Y1_pgw 
-> 
󻷉D1&dsZ:]Y
yNN~_\)Xvȅ$ձ[p
$ 8\-
 ;ku-KTmpu+"ZA}@]nQC(i40F�
pKqu/Eэ

318c4zbUNCAU36s1uBn5H_LbHC_OB12gnd_oC3PRXwE -> _64 ߫5/]s_

"Oh, no! Something is completely off, I can't see their messages!" - that was not his best day at school. And as he learned soon, there was nothing he could do about that - that's the beauty of the encryption.

I bet you have already guessed that we have been talking about a new concept all this time, and you might even have guessed what kind of concept it is - JWE or JSON Web Encryption.

JWE

Finally, we have reached the last term in this post's title, so if you are running out of tea or coffee, fear not - it is fine, as we are getting closer to the end of the post.

Since you have a very good understanding of JWT, it is easy to see why we sometimes need the encrypted version of it. As you have already noticed, the key difference between the two is that the payload is encrypted in the JWE, so only the designated actor can decrypt and read it.

Similar to JWS, JWE can use either symmetric or asymmetric algorithms for the encryption, and the logic is similar to the JWS:

symmetric reuses the same key or pass pharse
asymmetric uses a private key to decrypt the payload and a public key to decrypt it

There is a set of supported algorithms: to name a few, AES-CBC and AES-GCM are symmetric ones, and RSA-OAEP or ECDH-ES are asymmetric options. As with JWT, the algorithms are passed within the alg header. We won't try to reimplement the JWE asymmetric algorithms flow from scratch in this post, as it would be too artificial thing to do, considering the complexity of the cryptography behind it. However, later, I'll show you how to use proper libraries for that purpose - stay tuned!

Remember we talked about the usage scenarios for JWT? The info exchange one is the primary use case for JWE, while it's not that common to use them for authorization (it still happens sometimes, though). Here is one of the possible ways of exchanging the data between the actors through the UI app with the use of the JWE:

Since it doesn't mean that step 4 will be invoked right away after step 3, the UI might need to store the sensitive data within the browser storage, which leaves the data easily accessible by the browser users and the UI app in general. That's why using the JWE can be a good solution here, as the Data consumer can decrypt the JWE and retrieve the data.

There might be one moment that I found confusing when I first came across it: the public key is used to encrypt the payload, and it can be publicly available, while the decryption is done by the private key that the receiver has access to. That's completely opposite from the signature algorithm - why is it so? Well, this is a very good question.

The answer, while confusing, uses logic as reasoning: if one of the keys should be public (so anyone can repeat the same operation), what is more sensitive in the scope of JWE: that anyone can encrypt or decrypt it? The answer is obvious: if anyone can decrypt it, that destroys the purpose of the JWE concept, that's why the decryption should be done with the private key, not the encryption.

There is one more important feature of JWE that I need to mention briefly: it is possible to nest a signed JWT inside the JWE, if there is a need to use the signature.

Nested JWS

As you might have noticed, the JWE structure has no signature:

header.encryptedKey.initializationVector.cipherText.authenticationTag

And while the authentication tag does its job to make sure that the data hasn't been modified by the man-in-the-middle (hey, Eric), there is one thing that signature does on top: verifies the senders' identity - the same way as the signatures we (humans) use to sing the documents. As discussed above, encryption can be performed with the public key if the asymmetric algorithm is used, so anyone can do that and pretend they are the issuer. When this part becomes a problem, both JWS and JWE must be combined by nesting it within. Here is how the structure will look:

JWE {
    Header (encryption algorithm, content type, etc.)
    Encrypted Key (if applicable)
    Initialization Vector
    Ciphertext (Encrypted {
        JWS {
            Header (signature algorithm, etc.)
            Payload (data)
            Signature
        }
    })
    Authentication Tag
}

As you can see, JWS lives within the Ciphertext. In such cases, it is a good practice to pass the cty (content type) header with a value JWT to the JWE headers. This will hint to the receiver that the encrypted payload is not a set of claims, as usual, but rather a signed JWT.

All right, that's all I wanted to share about the JWE, as the remaining parts are the same/similar to the JWT concepts that we discussed quite thoroughly before. But before wrapping it up, The title of the article promised to show how to cook the JWT/JWS/JWE, so let's look at how it is done in real life. We'll be brief =)

How to cook them

I bet that even without me mentioning this, it was crystal clear to you that it's a bad practice to implement the JWT/JWS/JWE-related code from scratch. Good thinking! This is quite a trivial task to solve, and there are tonnes of battle-tested libraries in different programming languages for that matter, here is a quite comprehensive list provided by the Auth0. Anyway, it's always wise to check the library repo to see whether it's supported, what kind of issues are opened, when the last activity was and whether the license is sufficient for your needs. It is also smart to try it with the required scenarios before pulling the library into your project - but I bet you already know that.

Since I used Go for the examples above, let's use the Go JOSE library to show how it can help us.

JWT and JWS

Symmetric algorithm example

Here what the code looks for the symmetric HS256 algorithm:

import (
    "crypto/sha256"
    "fmt"
    "github.com/go-jose/go-jose/v4"
    "github.com/go-jose/go-jose/v4/jwt"

    "time"
)

type Payload struct {
    jwt.Claims
    Message string `json:"message"`
}

func main() {
    // sender part:
    fmt.Println("Sender part:")

    signingSecretPhase := "goToHellEric"
    // to make sure the secret is 256 bits long
  // it's better to use 256 bits long string instead of doing this
    hasher := sha256.New()
    hasher.Write([]byte(signingSecretPhase))
    signingSecret := hasher.Sum(nil)

    payload := Payload{
        Claims: jwt.Claims{
            Subject:  "Craig",
            IssuedAt: jwt.NewNumericDate(time.Now()),
        },
        Message: "Hello there, Craig! What's up, buddy?",
    }

    var signerOpts = jose.SignerOptions{}
    signerOpts.WithType("JWT")

    singer, err := jose.NewSigner(jose.SigningKey{Algorithm: jose.HS256, Key: signingSecret}, &signerOpts)
    if err != nil {
        panic(err)
    }

    signedJwt, err := jwt.Signed(singer).Claims(payload).Serialize()
    if err != nil {
        panic(err)
    }

    fmt.Println(signedJwt)

    // receiver part:
    fmt.Println("\nReceiver part:")

    receiverJws, err := jose.ParseSigned(signedJwt, []jose.SignatureAlgorithm{jose.HS256})
    if err != nil {
        panic(err)
    }

    verifiedPayload, err := receiverJws.Verify(signingSecret)
    if err != nil {
        fmt.Println("The signature differs - the original message has been modified", err)
    } else {
        fmt.Println("The signature matches - the original message hasn't been modified")
        fmt.Println(string(verifiedPayload))
    }
}

If we run the code, we'll get this:

Sender part:
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3MTE2MzEwNjIsIm1lc3NhZ2UiOiJIZWxsbyB0aGVyZSwgQ3JhaWchIFdoYXQncyB1cCwgYnVkZHk_Iiwic3ViIjoiQ3JhaWcifQ.tZ9j19pdM4vO7enkVSKuxoi6k3yB04aRlOKPzp_Rsnc

Receiver part:
The signature matches - the original message hasn't been modified
{"iat":1711631062,"message":"Hello there, Craig! What's up, buddy?","sub":"Craig"}

Asymmetric algorithm example

import (
    "crypto/rand"
    "crypto/rsa"
    "fmt"
    "github.com/go-jose/go-jose/v4"
    "github.com/go-jose/go-jose/v4/jwt"

    "time"
)

type Payload struct {
    jwt.Claims
    Message string `json:"message"`
}

func main() {
    // sender part:
    fmt.Println("Sender part:")

    privateKey, err := rsa.GenerateKey(rand.Reader, 2048)
    if err != nil {
        panic(err)
    }

    payload := Payload{
        Claims: jwt.Claims{
            Subject:  "Craig",
            IssuedAt: jwt.NewNumericDate(time.Now()),
        },
        Message: "Hello there, Craig! What's up, buddy?",
    }

    var signerOpts = jose.SignerOptions{}
    signerOpts.WithType("JWT")

    singer, err := jose.NewSigner(jose.SigningKey{Algorithm: jose.RS256, Key: privateKey}, &signerOpts)
    if err != nil {
        panic(err)
    }

    signedJwt, err := jwt.Signed(singer).Claims(payload).Serialize()
    if err != nil {
        panic(err)
    }

    fmt.Println(signedJwt)

    // receiver part:
    fmt.Println("\nReceiver part:")

    receiverJws, err := jose.ParseSigned(signedJwt, []jose.SignatureAlgorithm{jose.RS256})
    if err != nil {
        panic(err)
    }

    verifiedPayload, err := receiverJws.Verify(&privateKey.PublicKey)
    if err != nil {
        fmt.Println("The signature differs - the original message has been modified", err)
    } else {
        fmt.Println("The signature matches - the original message hasn't been modified")
        fmt.Println(string(verifiedPayload))
    }
}

The only difference here is that we need to generate and use RSA keys instead of a secret phrase. Usually, such keys are not generated by the application but rather stored somewhere, so the application reads them on its startup.

Here is the output of this code:

Sender part:
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpYXQiOjE3MTE2MzEyODYsIm1lc3NhZ2UiOiJIZWxsbyB0aGVyZSwgQ3JhaWchIFdoYXQncyB1cCwgYnVkZHk_Iiwic3ViIjoiQ3JhaWcifQ.jZ4OIgCRyP2dHxa8wV_QqNYweT1p1ntGUfWHAxbHQyonAQAHhHcL75hCfPhxduGKqGU_dsrepKv6epBFGtMpUwlqvDiduQWKut0slqr6OQI_BedoQ8QB2Z9XdPM-smya7f1teEwYF9lDmU2Rz6s5pff_vwuuIQU2JRISg_fCKr0EqQv0Z5kvlxBeWJ3T5G0VBTTQccZZXkPCNovjT7eTXyChIQ8LT6RV4Fc5giLHi_-eydmJIcPCjJR2mjUk7-JI94b3QR2-ZTQcx96fbmMT3kp4RGn3r9SCAGAZQob43zxc2rcwEEEx07t4cuSa1BpJHrkzYA5bzpiRDH1fxWclyw

Receiver part:
The signature matches - the original message hasn't been modified
{"iat":1711631286,"message":"Hello there, Craig! What's up, buddy?","sub":"Craig"}

JWE

Symmetric algorithm example

import (
    "crypto/sha256"
    "fmt"
    "github.com/go-jose/go-jose/v4"
    "github.com/go-jose/go-jose/v4/jwt"

    "time"
)

type Payload struct {
    jwt.Claims
    Message string `json:"message"`
}

func main() {
    // sender part:
    fmt.Println("Sender part:")

    encryptionSecretPhase := "goToHellEric"
    // to make sure the secret is 256 bits long
    hasher := sha256.New()
    hasher.Write([]byte(encryptionSecretPhase))
    encryptionSecret := hasher.Sum(nil)

    payload := Payload{
        Claims: jwt.Claims{
            Subject:  "Craig",
            IssuedAt: jwt.NewNumericDate(time.Now()),
        },
        Message: "Hello there, Craig! What's up, buddy?",
    }

    encrypter, err := jose.NewEncrypter(jose.A256GCM, jose.Recipient{Algorithm: jose.A256GCMKW, Key: encryptionSecret}, nil)
    if err != nil {
        panic(err)
    }

    issuedJwe, err := jwt.Encrypted(encrypter).Claims(payload).Serialize()
    if err != nil {
        panic(err)
    }

    fmt.Println(issuedJwe)

    // receiver part:
    fmt.Println("\nReceiver part:")

    receiverJwe, err := jose.ParseEncrypted(issuedJwe, []jose.KeyAlgorithm{jose.A256GCMKW}, []jose.ContentEncryption{jose.A256GCM})
    if err != nil {
        panic(err)
    }

    verifiedPayload, err := receiverJwe.Decrypt(encryptionSecret)
    if err != nil {
        fmt.Println("Error decrypting the payload", err)
    } else {
        fmt.Println("The payload has been decrypted successfully")
        fmt.Println(string(verifiedPayload))
    }
}

The output:

Sender part:
eyJhbGciOiJBMjU2R0NNS1ciLCJlbmMiOiJBMjU2R0NNIiwiaXYiOiJhS2VNLWctT3BtYzBJeEhoIiwidGFnIjoiZk11aF9wemZ1UmlsNDgtOVJfRzIxZyJ9.1LAUhXXHcfcCHXftex9qVFhHTx3UnCK4R-Mc6jBsmtg.TagB8zrlcBTc2125.YHiktAQ0oPZ_Rdiptp3ok11v-Xbh4fts4ovgrcwqHFsMnVy-766R6Dg4utOGs8NoOW-C7SeQCSWRv5ojMo5ksnyPM-fKGmbXeRYZC32U2vsKxQ.IzBaULb22KI0vjA7NvuwZQ

Receiver part:
The payload has been decrypted successfully
{"iat":1711631810,"message":"Hello there, Craig! What's up, buddy?","sub":"Craig"}

Asymmetric algorithm example

import (
    "crypto/rand"
    "crypto/rsa"
    "fmt"
    "github.com/go-jose/go-jose/v4"
    "github.com/go-jose/go-jose/v4/jwt"

    "time"
)

type Payload struct {
    jwt.Claims
    Message string `json:"message"`
}

func main() {
    // sender part:
    fmt.Println("Sender part:")

    encryptionKey, err := rsa.GenerateKey(rand.Reader, 2048)
    if err != nil {
        panic(err)
    }

    payload := Payload{
        Claims: jwt.Claims{
            Subject:  "Craig",
            IssuedAt: jwt.NewNumericDate(time.Now()),
        },
        Message: "Hello there, Craig! What's up, buddy?",
    }

    encrypter, err := jose.NewEncrypter(jose.A256GCM, jose.Recipient{Algorithm: jose.RSA_OAEP, Key: &encryptionKey.PublicKey}, nil)
    if err != nil {
        panic(err)
    }

    issuedJwe, err := jwt.Encrypted(encrypter).Claims(payload).Serialize()
    if err != nil {
        panic(err)
    }

    fmt.Println(issuedJwe)

    // receiver part:
    fmt.Println("\nReceiver part:")

    receiverJwe, err := jose.ParseEncrypted(issuedJwe, []jose.KeyAlgorithm{jose.RSA_OAEP}, []jose.ContentEncryption{jose.A256GCM})
    if err != nil {
        panic(err)
    }

    verifiedPayload, err := receiverJwe.Decrypt(encryptionKey)
    if err != nil {
        fmt.Println("Error decrypting the payload", err)
    } else {
        fmt.Println("The payload has been decrypted successfully")
        fmt.Println(string(verifiedPayload))
    }
}

Output:

Sender part:
eyJhbGciOiJSU0EtT0FFUCIsImVuYyI6IkEyNTZHQ00ifQ.NNr1KAaT8gus0FEbvDWHBONo-CrXKy23aDQOZChKYm9mkuRHUX7NLj0G3ChuqCIy3jUDPsM_ICWQe4nX_XdJlBUnVN_-hKEPzaZ5cRcsNv_hKrohX657jRxcDxY15J9kVp68VaJsjG0rpPH69u1fDaT2kRNex5CZ54V-5Ywz0SvG0B_82pXDzoHfDxCCaKZFGQA7giBTJgaEsl1DVWB0xYHdZLo33JivSLGsKFUs48o_YSOKpbv5cnzSSj1cvrG5-36ousoPmJgIdkEqru8k_d6cG7T5xZILphYNrRaovBUT4IW2EZS4YpBoRxrZTLuXvN6ZcA-H78484PAqUZfxRA.qNhm_UONfkMr38Bv.BGciOa9CxI5iHou7TabTEhmXlJGr1OU-RPDvTx0FhzsPJUDUxF8cFRlDvIdNw7OdN_iblngtTSnB2hoEKpSfeHLQpqR1NwHfP2WufAnIexrjfQ.1uAbUYQR__j5Xtq4vW2t6g

Receiver part:
The payload has been decrypted successfully
{"iat":1711632447,"message":"Hello there, Craig! What's up, buddy?","sub":"Craig"}

Nested JWS

Asymmetric algorithms are used here for both signing and encrypting.

import (
    "crypto/rand"
    "crypto/rsa"
    "fmt"
    "github.com/go-jose/go-jose/v4"
    "github.com/go-jose/go-jose/v4/jwt"

    "time"
)

type Payload struct {
    jwt.Claims
    Message string `json:"message"`
}

func main() {
    // sender part:
    fmt.Println("Sender part:")

    signaturePrivateKey, err := rsa.GenerateKey(rand.Reader, 2048)
    if err != nil {
        panic(err)
    }
    encryptionPrivateKey, err := rsa.GenerateKey(rand.Reader, 2048)
    if err != nil {
        panic(err)
    }

    signedJwt := issueJws(signaturePrivateKey)
    encryptedJwe := encryptJwe(&encryptionPrivateKey.PublicKey, signedJwt)

    fmt.Println(encryptedJwe)

    // receiver part:
    fmt.Println("\nReceiver part:")

    decryptedJwe := decryptJwe(encryptionPrivateKey, encryptedJwe)
    verifyJws(decryptedJwe, &signaturePrivateKey.PublicKey)
}

func issueJws(privateKey *rsa.PrivateKey) string {
    payload := Payload{
        Claims: jwt.Claims{
            Subject:  "Craig",
            IssuedAt: jwt.NewNumericDate(time.Now()),
        },
        Message: "Hello there, Craig! What's up, buddy?",
    }

    var signerOpts = jose.SignerOptions{}
    signerOpts.WithType("JWT")

    singer, err := jose.NewSigner(jose.SigningKey{Algorithm: jose.RS256, Key: privateKey}, &signerOpts)
    if err != nil {
        panic(err)
    }

    signedJwt, err := jwt.Signed(singer).Claims(payload).Serialize()
    if err != nil {
        panic(err)
    }
    return signedJwt
}

func encryptJwe(publicKey *rsa.PublicKey, signedJwt string) string {
    encryptionOpts := (&jose.EncrypterOptions{}).WithContentType("JWT")

    encrypter, err := jose.NewEncrypter(jose.A256GCM, jose.Recipient{Algorithm: jose.RSA_OAEP, Key: publicKey}, encryptionOpts)
    if err != nil {
        panic(err)
    }

    encryptedJwe, err := encrypter.Encrypt([]byte(signedJwt))
    if err != nil {
        panic(err)
    }

    serialized, err := encryptedJwe.CompactSerialize()
    if err != nil {
        panic(err)
    }
    return serialized
}

func decryptJwe(privateKey *rsa.PrivateKey, encryptedJwe string) string {
    jwe, err := jose.ParseEncrypted(encryptedJwe, []jose.KeyAlgorithm{jose.RSA_OAEP}, []jose.ContentEncryption{jose.A256GCM})
    if err != nil {
        panic(err)
    }

    decryptedJwe, err := jwe.Decrypt(privateKey)
    if err != nil {
        panic(err)
    }

    return string(decryptedJwe)
}

func verifyJws(signedJwt string, publicKey *rsa.PublicKey) {
    receiverJws, err := jose.ParseSigned(signedJwt, []jose.SignatureAlgorithm{jose.RS256})
    if err != nil {
        panic(err)
    }

    verifiedPayload, err := receiverJws.Verify(publicKey)
    if err != nil {
        fmt.Println("The signature differs - the original message has been modified", err)
    } else {
        fmt.Println("The signature matches - the original message hasn't been modified")
        fmt.Println(string(verifiedPayload))
    }
}

Output:

Sender part:
eyJhbGciOiJSU0EtT0FFUCIsImN0eSI6IkpXVCIsImVuYyI6IkEyNTZHQ00ifQ.DiGVfL5PqRL5phPdN-8rxNyeb1orfJTfseJzDzV4lcXurdTiCNJ0jnmNS2bG1yF7NfKSOC6yslasBmvpAQb2OCishTwAvWqIUj9BOByER73sYJKmtBhmLbHkzn19UEgo1ihA7s7LdrqtYCCepX3PVGAhGRsjIZp8BiSGpoc2TfEP4_MknEyEkoaukpqtXiBUUTRDfUNfzJg2gyC_ca1xu-jrwB2AbQyDw7cXQX4dsxpBstCmXnpThjlRrjMB7lE4Upc05aONakv_WLOLBjH1UInywPgihXQ9zpjsejPtWRfjJmwvV0AHKmvWYAZ-0YXg02LzYcmrOzPaQpTOgPC6PQ.dkRdR2yTUt3r3_OY.J8kh2oIqXF0YwDLCj06sPzkTLy9uwbsWE2M2DZYKmJLM5YCAHzuwYSDoXqkAM0cebEmks8zPtMOVMSKC-3wTXZOK3v13GyrcTfHIahDiuu2KwRqZOEB_7yJM0IgbCzLK_M8BWT_aTEtAqxDobUUU9TRsR0W7Zmn4pvO53jMQ3cRq2iIppSWkTR9W8DiW8xspCAxOhYMXymvQVGt4-bF1rThyrhvtXsKdMcIiWbO4MWV4mD9j1sVLVEXQDnLhNBT0wmGXYKRI-MsC0Ka1NupUs7ItcMqfcQnUL2GrjmH27dDtsuCCjkqo5xU-EISpLOx4z1kZna0iv8O9X_S30VtR-T4jAehwGx7aG3vUjBVSKOqkjVcA0YGez8FDehhk5Zlu0-yteMcad6nHjcn_qVRZL64ebTD053jIn4bLBz7YrgukcMp-NX7AwKbZ12AxoSXPWj75vZHie_oFevN3mEVFh6kFHg7I_6xzmXQA5QaLLPnFXtkSkf1x2KZldZ3PMK7NIrUuKYp1_rejqHJxtXL8HE3OnAAMfw_xdbSYDPT0WeEO7CxAZ5EWAfNJ1qZI3cRyDfFFKpeAIWL5u1BJS9Gte9QS1glsb7iZ_N_GZ6_Bjj4VvEfDMjwXidpxJvoQ-xLYNf2O9wKDVOTUaA.mbvor-v6UmO6u7ZGoHxY9Q

Receiver part:
The signature matches - the original message hasn't been modified
{"iat":1711638756,"message":"Hello there, Craig! What's up, buddy?","sub":"Craig"}

Things to be aware of

We spent a lot of time talking about JWT/JWS/JWE and also discussed how to implement them using the dedicated library. Everything sounds cool and logical, but are there any drawbacks to these concepts? Of course!

First of all, it is important to remember that JWS and JWE heavily rely on cryptographic algorithms, as we have seen so far, and that means that they are exposed to the same problems as any other things in that realm:

cryptography is a complex subject, so it is easy to mess things up while implementing the algorithms. It's not rare that flaws are found in the JWT libraries; that's why it's important that the libraries we use are well-maintained and we have automatic ways of detecting and fixing security vulnerabilities in our projects with tools like GitHub Dependabot or similar ones.
some cryptographic algorithms that have been used for a long time, can become obsolete and insecure, like, for example, MD5 these days. And JWT specification sometimes too slow to react on that, so it can still support those algorithms that should be avoided (for example, like this time in the past). It's our, engineers, responsibility to make sure that we are using reliable algorithms. Good linters can help detect this as well.

There is another crucial moment about JWT that many engineers have encountered if they had to work with the authorization domain: there is no way to revoke the JWT after it has been issued. As we discussed before, once the user provides their credentials, the JWT is issued to prove that the user is signed in. We can set up the expiry date for it, but imagine if the user clicked the "log out" button, JWT is still valid per se, but it shouldn't be trusted anymore in the scope of the user session. Some auth-providers handle this on your behalf, but others leave this problem to their users, so we, engineers, need to solve it somehow. One solution for this case is a DB table with the blocklist of revoked JWTs; we should keep them there until they expire.

There are other things to be aware of, but these are the most critical in my opinion.

Farewell

It's been a long journey, and it took me a few evenings to finish this post. So, thanks a lot for reading it, and I hope you have learned a thing or two about JWT/JWS/JWEs today. Before I say, "That's all, folks", let me share further readings if you'd like to dive deeper into the topics. And there is nothing more profound than dedicated RFCs:

RFC 7519 - JSON Web Token (JWT)
RFC 7515 - JSON Web Signature (JWS)
RFC 7516 - JSON Web Encryption (JWE)

Have fun! =)

P.S. Receive an email once I publish a new post - subscribe here

P.P.S. I have created a Twitter account lately, so if you'd like to save my feed from ads, "funny" videos, and posts by Elon Musk, let's follow each other there =)

Understanding CORS

Mykola — Wed, 03 Jan 2024 16:42:34 +0000

Hello there! Happy New Year! I hope you had an opportunity to get some rest during the winter holidays and maybe even made a snowman or two =)

Several days ago, I had a dialog with a friend of mine (let's call her Eowyn) who has recently started her path in software engineering:

Eowyn: Hey, buddy! I'm building a web project that has the frontend and backend parts. Whenever I click a button on the UI (that triggers a DELETE request to the server), my browser blows with these errors:

I tried calling the same endpoint with the curl command, and it works. I did the same via Postman, and it also works. It makes no sense! What the heck?

Me: Hehe, congrats, this is a historical moment in your career - the day you discovered CORS! =)

To her credit, she didn't want my help in fixing the issue, she wanted to understand it. That's why I spent the next half an hour explaining the topic of CORS. And since that was not the first time I had to do that, I realized that it is an excellent opportunity to write a post on this subject and, next time, share the link to it instead of explaining it once again.

A disclaimer: this post is called Understanding CORS, and that's exactly the goal we are going to pursue today. The target audience is the folks who know nothing or very little about the CORS and would like to learn what's happening on a high level behind the errors like above. We won't dive deep and won't cover all the use-/edge-cases of this topic - if you are looking for more advanced reading, here is an excellent longread from Mozilla.

Real-life example

If you are following my blog, you might have noticed that I like to explain technical aspects via real-life scenarios/examples. I believe this is the shortest and easiest path to understanding. We'll do the same today.

Let me introduce you to Geralt, a witcher, one of the best in his craft. Due to his skills and due to the number of dangerous beasts out there, Geralt's services are in high demand. People contact him so often that he has to hire Bob, a secretary, to manage the calls he receives. Thus, once somebody calls Geralt's office, it is Bob who manages the call. It usually works like this:

once somebody calls Geralt's office, Bob picks up the phone, gathers the info about the call (who, why, and where they are calling from), and asks them to wait

Bob calls Geralt, shares the info about the caller, and asks whether Geralt is willing to talk to them

As we can see, Geralt replied that if, during the next 2 hours, people were calling from the place called Ingenheim about hunting, Bob should connect them with him right away.

Bob gets back to the caller and connects them with Geralt

once there is another call within the next 2 hours coming from a different place than Ingenheim, Bob rejects the request to connect them with Geralt:

However, if there is someone (like the witcher's buddy Yarpen) who knows Geralt's direct phone number, they can still call him regardless of the place and reason for making a call:

I think this should make perfect sense. But how come is it related to the CORS?

From the real life to software engineering

If we try to map the real-life example we used to the software engineering concepts, it will look like this:

the person who calls the office - frontend application
Bob - browser
Geralt - backend application

And the step-by-step sequence of events is the following:

once the frontend app tries to send a request to the backend API, a browser makes a so-called pre-flight request to the backend API and asks about the allowed options: who is allowed to call the API and what type of requests can be sent
the API sends a response with such options and (optionally) includes the duration for which the browser should cache those settings and rely on them instead of making another pre-flight request
if the frontend app and the request it tries to make are within the allow list, the browser lets it through
otherwise, the request is rejected with the error we saw at the very beginning of this post

However, this mechanism is easy to bypass by skipping the browser and sending a request directly (like via curl, Postman, or any other HTTP client) - that's exactly what Yarpen did above by calling Geralt's direct phone number instead of the office's one.

Shall we assume that CORS is quite a poor security mechanism, as we can easily bypass it? The answer is "it depends". If we'd like to secure our API in a way that only the allowed services can call it, CORS is not a good idea as a single-on solution, as it doesn't apply to server-to-server communications. CORS' primary use case is CSRF (Cross-Site Request Forgery) attack prevention. Let's discuss what it is.

CSRF

Imagine that today is a payday, so you have just logged in to your online bank account to check the balance, and you can see that the money has arrived - nice! Glad and happy you have opened the social network feed and started scrolling it. And, all of a sudden, there is another "bang": someone has posted a link with the description that there is a big sale on your favorite pickles - that's your lucky day, isn't it? You followed the link, but there are no pickles behind it, just a blank page. "That's unfair! How could someone make such jokes?" Was your thought, after which you closed that page.

Suppose you have monitored your network traffic while visiting the "pickles scam" page. In that case, you'd notice that even though the page was blank, it actually contained a small JavaScript code that made a request to the API of your bank and requested a money transfer to the unknown (for you) account. You've been logged in to your online bank account, so the bank is unaware that you made this transfer without even knowing. But if the bank has CORS configs enabled, the browser will verify whether this "pickles scam" domain is allowed to call the bank's API, and since it's not allowed, the request will be rejected. Even though you are left without pickles, your money is safe.

I asked ChatGPT to provide me with more examples of CSRF attacks, and here is what I got:

Example 1: Changing Email Address

Scenario: Alice is logged into her email account on emailservice.com.
Attack: She then visits a malicious website, malicioussite.com, which contains a hidden form that is automatically submitted by JavaScript. This form is crafted to send a POST request to emailservice.com to change her email settings (like her recovery email address).
Result: If emailservice.com doesn't have proper CSRF protections, it might process this request as if Alice intentionally submitted it, leading to her recovery email being changed without her knowledge.

Example 2: Social Media Post

Scenario: Bob is logged into a social media platform.
Attack: He clicks on a link that leads him to a malicious site. This site contains a script that makes a request to the social media platform to post a message or send a message to all his contacts.
Result: If the social media platform doesn't verify the authenticity of the request, it could result in spam or malicious messages being sent from Bob's account.

Example 3: Changing Password

Scenario: Dana is logged into a forum.
Attack: She receives an email with a link to an interesting article. Clicking the link takes her to a website that secretly contains a form that sends a request to the forum to change her password.
Result: Without CSRF protection, Dana’s password could be changed without her consent, potentially locking her out of her account.

As you can see, all of this could have been avoided if the server had CORS configured. How can we do that, though? Let's finally see some code!

Some code

As we have already determined, there are 3 types of actors here:

browser
frontend
backend

In this example, we'll use Brave, a Chromium-based browser, for UI simplicity purposes. Let's jump into the code, then.

Backend

We are going to build a tiny books API that has 3 endpoints:

get all books
add a new book
delete all books

It's a dummy application, and that's why we'll store all the data in the memory. Here is the entire Go code for our backend:

package main

import (
    "encoding/json"
    "errors"
    "fmt"
    "github.com/go-chi/chi/v5"
    "net/http"
)

var books = []string{"The Lord of the Rings", "The Hobbit", "The Silmarillion"}

type Book struct {
    Title string `json:"title"`
}

func main() {
    err := runServer()
    if err != nil {
        if errors.Is(err, http.ErrServerClosed) {
            fmt.Println("server shutdown")
        } else {
            fmt.Println("server failed", err)
        }
    }
}

func runServer() error {
    httpRouter := chi.NewRouter()

    httpRouter.Route("/api/v1", func(r chi.Router) {
        r.Get("/books", getAllBooks)
        r.Post("/books", addBook)
        r.Delete("/books", deleteAllBooks)
    })

    server := &http.Server{Addr: "localhost:8888", Handler: httpRouter}
    return server.ListenAndServe()
}

func getAllBooks(w http.ResponseWriter, req *http.Request) {
    respBody, err := json.Marshal(books)
    if err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        return
    }

    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write(respBody)
}

func addBook(w http.ResponseWriter, req *http.Request) {
    var book Book
    err := json.NewDecoder(req.Body).Decode(&book)
    if err != nil {
        w.WriteHeader(http.StatusBadRequest)
        return
    }

    books = append(books, book.Title)

    w.WriteHeader(http.StatusCreated)
}

func deleteAllBooks(w http.ResponseWriter, req *http.Request) {
    books = []string{}

    w.WriteHeader(http.StatusNoContent)
}

You can find it in this GitHub repo.

As you can see, I used an external dependency, github.com/go-chi/chi/v5 , for the API. It is completely optional to achieve the same with pure Go, but I did that to improve the readability of the code. Other than that, the code is pretty simple: it reads, writes, or deletes the data from/into the slice of books and sends a successful response.

Let's run it: the server will be running as http://localhost:8888

It's time to define a frontend now:

Frontend

Here we need a simple HTML page with JS scripts to make requests to the backend API, and a tiny Go server to serve the page. Here is the HTML code:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Books</title>
    <link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/css/bootstrap.min.css" rel="stylesheet"
          integrity="sha384-T3c6CoIi6uLrA9TneNEoa7RxnatzjcDSCmG1MXxSR1GAsXEV/Dwwykc2MPK8M2HN" crossorigin="anonymous">
    <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.2/dist/js/bootstrap.bundle.min.js"
            integrity="sha384-C6RzsynM9kWDrMNeT87bh95OGNyZPhcTNXj1NW7RuBCsyN/o0jlpcV8Qyq46cDfL"
            crossorigin="anonymous"></script>
</head>
<body>
<div class="container p-3">
    <button type="button" class="btn btn-primary" id="getBooks">Get books</button>
    <button type="button" class="btn btn-danger" id="deleteAllBooks">Delete all books</button>
    <br>
    <br>

    <form>
        <div class="mb-3">
            <label for="inputBookTitle" class="form-label">Book title</label>
            <input type="text" class="form-control" id="inputBookTitle" aria-describedby="emailHelp">
        </div>
        <button type="submit" class="btn btn-primary">Add</button>
    </form>
</div>

<script>
  function getBooks () {
    fetch('http://localhost:8888/api/v1/books')
      .then(response => response.json())
      .then(data => {
        const booksList = document.querySelector('.books-list')
        if (booksList) {
          booksList.remove()
        }

        const ul = document.createElement('ul')
        ul.classList.add('books-list')
        data.forEach(book => {
          const li = document.createElement('li')
          li.innerText = book
          ul.appendChild(li)
        })
        document.body.appendChild(ul)
      })
  }

  function deleteAllBooks () {
    fetch('http://localhost:8888/api/v1/books', {
      method: 'DELETE'
    })
      .then(response => {
        if (response.status === 204) {
          getBooks()
        } else {
          const div = document.createElement('div')
          div.innerText = 'Something went wrong'
          document.body.appendChild(div)
        }
      })
  }

  const getBooksButton = document.getElementById('getBooks')
  const deleteAllBooksButton = document.getElementById('deleteAllBooks')
  const input = document.querySelector('input')
  const form = document.querySelector('form')

  getBooksButton.addEventListener('click', () => getBooks())
  deleteAllBooksButton.addEventListener('click', () => deleteAllBooks())

  form.addEventListener('submit', (event) => {
    event.preventDefault()

    const title = input.value

    fetch('http://localhost:8888/api/v1/books', {
      method: 'POST',
      headers: {
        'Content-Type': 'application/json'
      },
      body: JSON.stringify({ title })
    })
      .then(response => {
        if (response.status === 201) {
          input.value = ''
          getBooks()
        } else {
          const div = document.createElement('div')
          div.innerText = 'Something wend wrong'
          document.body.appendChild(div)
        }
      })
  })
</script>
</body>
</html>

And a Go server:

package main

import (
    "errors"
    "fmt"
    "github.com/go-chi/chi/v5"
    "net/http"
)

func main() {
    err := runServer()
    if err != nil {
        if errors.Is(err, http.ErrServerClosed) {
            fmt.Println("client server shutdown")
        } else {
            fmt.Println("client server failed", err)
        }
    }
}

func runServer() error {
    httpRouter := chi.NewRouter()

    httpRouter.Get("/", serveIndex)

    server := &http.Server{Addr: "localhost:3333", Handler: httpRouter}
    return server.ListenAndServe()
}

func serveIndex(w http.ResponseWriter, req *http.Request) {
    http.ServeFile(w, req, "20240103-cors/01-no-cors/client/index.html")
}

If we run the Go code, it will serve the HTML page to http://localhost:3333

Based on my drawings, you might have already noticed that I have an exceptional talent in design that's the UI is another masterpiece of mine:

Let's test what we have now.

Testing time

Navigate to http://localhost:3333/ in your browser, and open a DevTools there: it's Option+Command+I on MacOS or View -> Developer -> Developer Tools . Ideally, we should be able to see the Network tab and the Console like this:

Let's try to add a new book - "Harry Potter", and click "Add". Boom!

The error looks somewhat familiar, doesn't it? It's not exactly the same as at the beginning of this post, but it's very similar. Let's try to understand what it actually means.

You might have noticed that our backend code doesn't mention CORS at all. It is indeed true, we haven't implemented any CORS configs as of now. But that doesn't matter for the browser: it tried to make a preflight request anyway

If we click on it, it will expand some details, and we can see that the browser tried to make an OPTIONS request to the same path as the add book endpoint, and received a 405 Method Not Allowed response, which makes sense as we haven't defined the OPTIONS endpoint in our backend.

If we get back to our real-life example for a moment, what happened here is the following:

somebody calls Geralt's office, so Bob picks up the phone, gathers the info about the call (who, why, and where they are calling from), asks them to wait, and calls Geralt to double-check
but "Houston, we have a problem": Geralt's phone is off, so there's no way Bob can get Geralt's preferences for today

Will Bob connect the caller with Geralt somehow? Or will he agree on the services required without talking to the witcher? Of course, no, Bob has to reject the customer for now. The same happens in our case: since the browser has no idea about the backend API CORS settings, it simply refuses to make any requests there - safety first!

Let's fix that!

Fixing time

The frontend app remains the same, but as for the backend, we need to make a few changes:

introduce a new function to enable CORS:

func enableCors(w http.ResponseWriter) {
    // specifies which domains are allowed to access this API
    w.Header().Set("Access-Control-Allow-Origin", "http://localhost:3333")

  // specifies which methods are allowed to access this API
    w.Header().Set("Access-Control-Allow-Methods", "GET, POST, DELETE")

  // specifies which headers are allowed to access this API
    w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type")

  // specifies for how long the browser can cache the results of a preflight request (in seconds)
    w.Header().Set("Access-Control-Max-Age", strconv.Itoa(60*60*2))
}

As you can see, we have introduced 4 CORS settings:

domain from which it is allowed to call our API - http://localhost:3333
HTTP methods that are allowed to be used with our API - GET, POST, DELETE
request headers that are allowed to be passed to our API - Accept, Content-Type
time for which the browser can remember and cache these settings - 2 hours in seconds

A short disclaimer: there are more CORS-related headers, but these are enough for understanding. I'll share the links to dive deeper at the end of this post.

introduce an OPTIONS endpoint alongside the existing ones and a function to handle it:

...
    httpRouter.Route("/api/v1", func(r chi.Router) {
        r.Options("/books", corsOptions)
        r.Get("/books", getAllBooks)
        r.Post("/books", addBook)
        r.Delete("/books", deleteAllBooks)
    })
...

func corsOptions(w http.ResponseWriter, req *http.Request) {
    enableCors(w)
    w.WriteHeader(http.StatusOK)
}

add enableCors invocation to the existing functions of other endpoints, for example:

func getAllBooks(w http.ResponseWriter, req *http.Request) {
    respBody, err := json.Marshal(books)
    if err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        return
    }

    enableCors(w)
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write(respBody)
}

The final code looks like this:

package main

import (
    "encoding/json"
    "errors"
    "fmt"
    "github.com/go-chi/chi/v5"
    "net/http"
    "strconv"
)

var books = []string{"The Lord of the Rings", "The Hobbit", "The Silmarillion"}

type Book struct {
    Title string `json:"title"`
}

func main() {
    err := runServer()
    if err != nil {
        if errors.Is(err, http.ErrServerClosed) {
            fmt.Println("server shutdown")
        } else {
            fmt.Println("server failed", err)
        }
    }
}

func runServer() error {
    httpRouter := chi.NewRouter()

    httpRouter.Route("/api/v1", func(r chi.Router) {
        r.Options("/books", corsOptions)
        r.Get("/books", getAllBooks)
        r.Post("/books", addBook)
        r.Delete("/books", deleteAllBooks)
    })

    server := &http.Server{Addr: "localhost:8888", Handler: httpRouter}
    return server.ListenAndServe()
}

func corsOptions(w http.ResponseWriter, req *http.Request) {
    enableCors(w)
    w.WriteHeader(http.StatusOK)
}

func getAllBooks(w http.ResponseWriter, req *http.Request) {
    respBody, err := json.Marshal(books)
    if err != nil {
        w.WriteHeader(http.StatusInternalServerError)
        return
    }

    enableCors(w)
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    w.Write(respBody)
}

func addBook(w http.ResponseWriter, req *http.Request) {
    var book Book
    err := json.NewDecoder(req.Body).Decode(&book)
    if err != nil {
        w.WriteHeader(http.StatusBadRequest)
        return
    }

    books = append(books, book.Title)

    enableCors(w)
    w.WriteHeader(http.StatusCreated)
}

func deleteAllBooks(w http.ResponseWriter, req *http.Request) {
    books = []string{}

    enableCors(w)
    w.WriteHeader(http.StatusNoContent)
}

func enableCors(w http.ResponseWriter) {
    // specifies which domains are allowed to access this API
    w.Header().Set("Access-Control-Allow-Origin", "http://localhost:3333")

    // specifies which methods are allowed to access this API (GET is allowed by default)
    w.Header().Set("Access-Control-Allow-Methods", "POST, DELETE")

    // specifies which headers are allowed to access this API
    w.Header().Set("Access-Control-Allow-Headers", "Content-Type")

    // specifies for how long the browser can cache the results of a preflight request (in seconds)
    w.Header().Set("Access-Control-Max-Age", strconv.Itoa(60*60*2))
}

Let's run this code to see whether it works. It should - if you experience any issues, try to restart both frontend and backend apps, and even access http://localhost:3333 via incognito mode, as there might be some issues due to browser caching.

If you click all the buttons a few times by trying to add some books and get/delete all of them, you'll see that it works as expected. Even more, if we take a look at the Network tab, we'll see that there is only 1 preflight request in total:

Of course, we are not surprised about that, as we have just configured the Access-Control-Max-Age header for those purposes.

I believe you should already have a good understanding of CORS. But let's take a final step and try to mess with CORS configs to see how it breaks the flow again.

Breaking (purposely) time

We'll do that in a one-by-one fashion by breaking (and reverting) each of the CORS configs. Please remember to restart both frontend and backend apps once we apply a change, as otherwise, we won't see any effect. Let's gooooo!

Access-Control-Allow-Origin

Our frontend app runs on http://localhost:3333, and that's exactly the value we have within the Access-Control-Allow-Origin header. Let's change it to something else:

func enableCors(w http.ResponseWriter) {
    // specifies which domains are allowed to access this API
    w.Header().Set("Access-Control-Allow-Origin", "http://example.com")

    // specifies which methods are allowed to access this API
    w.Header().Set("Access-Control-Allow-Methods", "GET, POST, DELETE")

    // specifies which headers are allowed to access this API
    w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type")

    // specifies for how long the browser can cache the results of a preflight request (in seconds)
    w.Header().Set("Access-Control-Max-Age", strconv.Itoa(60*60*2))
}

Let's restart the apps and see what will happen if we try to use them.

Well, the outcome is as expected: only http://example.com is allowed to call the API. What's even more interesting is the fact that if we put the localhost value there but with a different port, we'll still get an error:

The same applies for http vs https - CORS is very strict:

There is a possibility, though, of allowing anyone to call your API - by using * as a value for the Access-Control-Allow-Origin header:

func enableCors(w http.ResponseWriter) {
    // specifies which domains are allowed to access this API
    w.Header().Set("Access-Control-Allow-Origin", "*")

    // specifies which methods are allowed to access this API
    w.Header().Set("Access-Control-Allow-Methods", "GET, POST, DELETE")

    // specifies which headers are allowed to access this API
    w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type")

    // specifies for how long the browser can cache the results of a preflight request (in seconds)
    w.Header().Set("Access-Control-Max-Age", strconv.Itoa(60*60*2))
}

Even if we try to run our frontend app on different ports, it will pass the CORS step successfully. Use this only if you know what you are doing!

Ok, let's change the Access-Control-Allow-Origin to the original one and jump to the next one.

Access-Control-Allow-Methods

Before started playing with this header, let me share an important gotcha with you: GET and POST methods are allowed by default regardless the settings. It means that w.Header().Set("Access-Control-Allow-Methods", "GET, POST, DELETE") has the same effect as w.Header().Set("Access-Control-Allow-Methods", "DELETE")

That's why there is no need to delete them and wonder why the application still works. However, let's try to get rid of the DELETE one:

func enableCors(w http.ResponseWriter) {
    // specifies which domains are allowed to access this API
    w.Header().Set("Access-Control-Allow-Origin", "http://localhost:3333")

    // specifies which methods are allowed to access this API
    w.Header().Set("Access-Control-Allow-Methods", "GET, POST")

    // specifies which headers are allowed to access this API
    w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type")

    // specifies for how long the browser can cache the results of a preflight request (in seconds)
    w.Header().Set("Access-Control-Max-Age", strconv.Itoa(60*60*2))
}

Once we restart both apps, we'll see that there are no issues with getting the books or adding a new one, but trying to delete them fires an error.

If the flow works for you with no issues, it is the browser caching (the 2nd hardest thing in the computer science) who spoils the fun. To fix that, try either:

ticking Disable cache box under the Network tab
opening a new incognito window

As we can see, the errors clearly states the Method DELETE is not allowed by Access-Control-Allow-Methods in preflight response - we knew that already, didn't we?

Let's revert the values and proceed to the next header.

Access-Control-Allow-Headers

We don't explicitly use Accept header in our code, that's why let's keep it there. But we do use the Content-Type one when we make a POST request. You know what to do with it =)

func enableCors(w http.ResponseWriter) {
    // specifies which domains are allowed to access this API
    w.Header().Set("Access-Control-Allow-Origin", "http://localhost:3333")

    // specifies which methods are allowed to access this API
    w.Header().Set("Access-Control-Allow-Methods", "GET, POST, DELETE")

    // specifies which headers are allowed to access this API
    w.Header().Set("Access-Control-Allow-Headers", "Accept")

    // specifies for how long the browser can cache the results of a preflight request (in seconds)
    w.Header().Set("Access-Control-Max-Age", strconv.Itoa(60*60*2))
}

If we click all the buttons we have, we'll see that getting and deleting books work, but adding a new one fails as expected:

Request header field content-type is not allowed by Access-Control-Allow-Headers in preflight response. makes sense.

Production applications use way more headers, that's why review them carefully in order to configure CORS properly.

Time to revert the changes and proceed to the last header in out list.

Access-Control-Max-Age

If not set, the default value is 0 which means that browser shouldn't cache the prefligh request data at all. Let's comment the header out and see what happens next:

func enableCors(w http.ResponseWriter) {
    // specifies which domains are allowed to access this API
    w.Header().Set("Access-Control-Allow-Origin", "http://localhost:3333")

    // specifies which methods are allowed to access this API
    w.Header().Set("Access-Control-Allow-Methods", "GET, POST, DELETE")

    // specifies which headers are allowed to access this API
    w.Header().Set("Access-Control-Allow-Headers", "Accept, Content-Type")

    // specifies for how long the browser can cache the results of a preflight request (in seconds)
    //w.Header().Set("Access-Control-Max-Age", strconv.Itoa(60*60*2))
}

Observe the Network tab after restarting the app:

No cache policy forces browser to make a preflight request each time there is a POST or DELETE call (there are no preflight requests for GET calls - it's a rule). It's acceptable while doing testing, but it leads to the undesired load to your servers if there is no Access-Control-Max-Age header provided, that's why the rule of thumb is to have it. There is no ideal value for it though, it depends on your situation and requirements.

And that's basically all I wanted to show you today. I'm sure you have a good understanding about CORS now, and can dive deeper on your own to learn even more on that topic.

Where to go from here

As I mentioned on the disclaimer section, there is an excellent longread from Mozilla on that topic - I know that now you are well-equipped for it.

If you are more into the RFC types of read, here is the one that covers CORS as well - the link leads exactly to the CORS section, but feel free to read all of it, if you have time and inspiration.

Anyway, it's getting late in my time zone, so I need to call it a day and get some sleep. I hope you learn some new today and get a good grasp on what CORS is and how it works under the hood. See you in the next posts!

Have fun =)

P.S. Receive an email once I publish a new post - subscribe here

P.P.S. I have created a Twitter account lately, so if you'd like to save my feed from ads, "funny" videos, and posts by Elon Musk, let's follow each other there =)

AI tools for software engineers: 5 things to be careful about/aware of

Mykola — Tue, 26 Dec 2023 16:01:26 +0000

Hello there! 2023 is about to end soon, so it's a good time to sit and reflect on it. If we think about the tech industry, in general, it's easy to define which topic was the hottest this year - AI. And the chances are that in 2024, this area will grow even bigger alongside the growing investments and the new astonishing scientific discoveries in this field.

Unlike previous "big things" in tech, like blockchain, crypto, NFT, Web3, etc., AI is way more accessible and covers more use cases than the rest:

I can guess with a high probability that you have already embedded some of the AI tools in your daily life or career-related activities
the chances are that the backlog of your project(s) has/had quite a few tasks related to introducing/integrating AI capabilities, and some of them are pretty damn good features!

If you answer "no" to these bullet points, you are missing out a lot, my friend. In my case, I'm actively using GitHub Copilot for coding assistance and ChatGPT for brainstorming ideas. I believe this list is very familiar to my fellow software engineers out there, and it's not only me who experienced a significant productivity boost thanks to these tools.

Of course, we will see what will happen with AI development once the hype is gone (and some investments with it) and there is a new "big thing" out there, like quantum computing or something less useful/practical. But for me, personally, the tools I started to use in 2023 are here to stay, as now they are a part of the essential toolset alongside Git, Docker, Excel/Google Spreadsheets, note-taking apps, etc.

As you can see, I'm pretty happy about the possibilities AI brought to us this year. And many articles out there talking about the cool features those tools have and the game-changing UX they bring. Sometimes, they create a feeling that those are silver bullets. However, it's not common for those posts to explore the problems, limits, and challenges of those tools. "Know your tool!" - someone wise once said. That's why today we are going to discuss some of these "dark parts" of the AI tools we use to be aware.

A short disclaimer: the message of this post is not "don't use the AI tools," but instead "use the AI tools mindfully." - the author is not one of the Luddites, I can assure you =)

Contextless advises

Let me introduce you to Dave. Dave is a friendly and very talkative guy, the social bee, if you will. But you should know this about him: Dave likes to share his opinions and, even more, give his advice regarding whether he has been asked to or not. Usually, he doesn't bother diving deep into your situation, but instead, he prefers to share his thoughts right away: "Do you want to buy a car? Buy Tesla!", "Are you considering upgrading your laptop? MacBook Pro is all you need, I assure you!". Here is him:

Do you have any Dave-s in your life? I wouldn't be surprised if you do.

As you can see from Dave's answers, he is not a bad guy, he really wants to help, and his advice is not dumb, but it's not something we should rely upon, at least at the very beginning of the conversation.

Why did I mention Dave at all? The post was supposed to be about AI tools. Well, let me tell you that GitHub Copilot is one of the "Dave-s", especially once you have created a new project. It's easy to reproduce this behavior as of the day of writing this post:

open your favorite IDE (I use IntelliJ IDEA from JetBrains)
create a new project
create a new file "test.md" there
open it and start typing "My name is "

Not sure about you, but in my case, I got this suggestion from GitHub Copilot:

Ok, let's accept the suggestion and add and I'm from - I got this:

Hey, wait, but it's not a code! I hear you, let's try the same with a code:

create a main.go file in the root of your project
make sure it has the package main on top
type/copy-paste this func pickLast() string { and wait for the suggestion

I got this on my machine:

It's a fair suggestion, and it compiles, but is this something you, as a software engineer, would suggest? Would you suggest anything at this moment of project development if, let's say, you are assisting your colleague via the pair programming session, but you knew nothing about this project before now? It's a rhetorical question.

What should we do about it?

Well, the same as with Dave, we should be aware of this way of working with GitHub Copilot and be careful with the suggestions it provides at the very early stage of the project or before its model has enough time and data to learn about your project.

Also, I'd recommend enabling native IDE suggestions alongside the GitHub Copilot ones to get the best of the 2 worlds: AI-based and docs/code-based, meaning that the latter doesn't suffer from hallucinations. You can do that:

in JetBrains IDEs by: Settings -> Languages & Frameworks -> GitHub Copilot -> enable Show IDE completions side-by-side option under the Editor settings chapter

Once enabled, it works like this:

I'm not a heavy VS Code/Vim/Neovim user, that's why I'm not sure how to achieve the same there - if you know, please, let me and others know in the comments section - pay it forward =)

XY problem

Have you ever heard of XY problem? If not, here is how Wikipedia defines it:

The XY problem is a communication problem encountered in help desk, technical support, software engineering, or customer service situations where the question is about an end user's attempted solution (X) rather than the root problem itself (Y or Why?).

The XY problem obscures the real issues and may even introduce secondary problems that lead to miscommunication, resource mismanagement, and sub-par solutions. The solution for the support personnel is to ask probing questions as to why the information is needed in order to identify the root problem Y and redirect the end user away from an unproductive path of inquiry.

If that doesn't make sense, here is a good example from "The XY Problem" website:

<bob> How can I echo the last three characters in a filename?
<feline> If they're in a variable: echo ${foo: -3}
<feline> Why 3 characters? What do you REALLY want?
<feline> Do you want the extension?
<bob> Yes.
<feline> There's no guarantee that every filename will have a three-letter extension,
<feline> so blindly grabbing three characters does not solve the problem.
<feline> echo ${foo##*.}

As you can see from this example, the person who answers the question should have a certain level of both professionalism and soft skills to get to the root of the challenge the other person is facing. That's not an easy task, I have to admit!

What about our AI friends? Lately, I have been brainstorming an idea for creating a tool that will:

fetch a list of GitHub projects within different organizations I have access to
do specific manipulations within the package.json of each repo

And I used ChatGPT to be my buddy in that process. I provided a high-level context for the tool and said that I'm considering writing a script that will clone each of these repositories, read the content of the package.json file via the IO operation, make the desired change, and write the modified context back to the file. My AI friend replied that it was a good idea and explained how to do that with some pseudocode. However, I felt that I was reinventing the wheel with all of that, so I decided to double-check:

and then ChatGPT suggested a list of alternative approaches, one of which was to use the existing GET /repos/:owner/:repo/contents/:path endpoint of the GitHub API. That was good advice and exactly the way I went forward (successfully if you are wondering). But would I have gotten it if I hadn't explicitly asked about more elegant ways of achieving the desired result? Nope.

What should we do about it?

That's something that I'd like you to be aware of while using tools like ChatGPT: they are falling (still?) into the XY problem, so be mindful and explicitly double-check whether the way you are going right now is the best one and whether there are any better alternatives.

The code that looks damn realistic but has some hard-to-notice issues

Some time ago, I was working on my simple terminal reminder app remindme (btw, check it out, it's pretty good and might be useful for you, and it's FOSS), and it was one of the first times when I could feel the power of the GitHub Copilot assistance: it basically wrote all of the SQL-related code in my project. And I have to say that the queries were pretty good: here is what one of the functions looked like:

func (repo *sqliteReminderRepo) List() ([]common.Reminder, error) {
    rows, err := repo.db.Query(`
        SELECT id, message, remind_at FROM reminders;
    `)

    if err != nil {
        return nil, err
    }

    reminders := make([]common.Reminder, 0)
    for rows.Next() {
        var id int64
        var message string
        var remindAt int64

        err := rows.Scan(&id, &message, &remindAt)
        if err != nil {
            return nil, err
        }
        reminders = append(reminders, common.Reminder{
            ID:       id,
            Message:  message,
            RemindAt: time.Unix(remindAt, 0),
        })
    }
    return reminders, nil
}

It's a decent code, but there is one catch that is easy to miss while doing a code review, especially if the pull request is quite big: a rows variable created in line 2 of the code is of type *sql.Rows. And it should be closed to release the resources once the processing is completed, that's why we need to add defer rows.Close() after the 1st if err != nil { ... } statement like this:

func (repo *sqliteReminderRepo) List() ([]common.Reminder, error) {
    rows, err := repo.db.Query(`
        SELECT id, message, remind_at FROM reminders;
    `)

    if err != nil {
        return nil, err
    }
  defer rows.Close()

    reminders := make([]common.Reminder, 0)
    for rows.Next() {
        var id int64
        var message string
        var remindAt int64

        err := rows.Scan(&id, &message, &remindAt)
        if err != nil {
            return nil, err
        }
        reminders = append(reminders, common.Reminder{
            ID:       id,
            Message:  message,
            RemindAt: time.Unix(remindAt, 0),
        })
    }
    return reminders, nil
}

In this case, leaving rows open won't cause huge problems because the app is small and is not exposed to the big scale. However, not closing resources (especially the IO or network ones) in the production code can lead to severe issues and wake the on-call fellas up in the middle of the night.

I believe you'll agree that it's easier to miss this line while reading the code written by someone (the colleague or AI assistant) than to make the same mistake while writing this function manually.

Let me show the same "realistic but not entirely correct code" problem but from a different angle. Some time ago, a company that I was collaborating with got a new junior engineer - let's call them Knut. It was the guy's first job ever, and I heard from the other colleagues (who worked full-time on the same project as Knut) that the new hire was doing great.

Some weeks later, I got an email that Knut requested my review of their pull request. I opened the link and saw Files changed 70 . "Wow!" was my first reaction, and then I grabbed a cup of coffee and decided to allocate an hour or so to do a deep review. I left a bunch of comments, nothing biggie, but there was one moment I couldn't comprehend: I noticed that Knut introduced a new dependency by adding a pretty big library (one of those "Chuck Norris" tools that can do everything). I briefly went through the list of code changes one more time, but I couldn't see where this library was used. I left a comment about that change as well, and on top of that, I sent a private message to Knut and said that if there is a need to jump into the call to clarify some of the comments or explain some code-/domain-wise moments, I could help with that.

Pretty soon, we jumped into the call and had a good discussion on the places to improve within the PR. Finally, I had an opportunity to ask about that new dependency introduced by the pull request. Knut explained that otherwise, the project wouldn't compile, so the guy asked ChatGPT about how to fix those errors, and that's the solution the tool suggested using. And it helped!

Let me step aside for a moment here to share some context. Suppose you have experience with the Java language and a Spring framework. In that case, you might be familiar with the situation when there is a library, something-cool-for-web, that, for example, brings you a bunch of tools for web development, but it might provide only the interface for some parts, like HTTP Client. And now, it's up to you to provide the implementation or the explicit dependency for the HTTP Client to make the tool work. Otherwise, the compiler won't be happy, as the interface without the implementation is not a way to go.

So, the correct solution is to use the some-cool-and-tiny-http-client library and provide the missing implementation to the something-cool-for-web configs. Instead, ChatGPT suggested using the huge-and-heavy-all-inclusive library, which has our some-cool-and-tiny-http-client within its sub-dependencies but also hundreds of others on top. Yes, this approach fixed the issue and made the compiler happy, but do we need those 100+ new libraries within the project? Of course not, as it creates a lot of risks, from unexpected bugs/behaviors to security vulnerabilities. But it was easy to miss this part while doing a code review of the pull request with 70 files changed.

Don't get me wrong, the pre-AI tools we used (and still use), like StackOverflow, are also exposed to the same issue, but that doesn't mean we shouldn't mention it then.

What should we do about it?

As with other AI hallucinations, we should be extra careful with all the output provided by those tools and do our best to verify it. In the examples above, I mentioned only the manual verifications, but it's a good practice to rely on the automation tools as well, like different kinds of tests (unit, integration, end-to-end, etc.), linters, dependency restrictions, pro-commit hooks, code-formatters, etc.

On top of that, we should aim to introduce/advocate a practice of smaller PRs (whenever possible) to make it easier to review and increase the chances of finding issues, if any.

And my personal reflection: since AI assistants are getting pretty good at writing code, we should invest more time and effort into the skill of reading/reviewing code, as it becomes more and more important these days.

The sunk cost fallacy

This one is not a problem of the AI tools, but rather of us humans. And it's the most questionable point of this post, so please feel free to disagree. But let me explain first what I mean here.

According to this article on Scribbr:

The sunk cost fallacy is the tendency for people to continue an endeavor or course of action even when abandoning it would be more beneficial. Because we have invested our time, energy, or other resources, we feel that it would all have been for nothing if we quit.

Sunk cost fallacy example
You are watching a movie, and after 30 minutes you realize it’s not what you expected. Instead of finding another movie, you convince yourself to continue. You think to yourself that you have already invested half an hour and the whole movie is just an hour and a half. If you quit now, you will have wasted your time, so you decide to stick it out.

I'm not sure about you, but it's something that has happened to me. Here is one of the examples in the scope of the AI tools.

These days, whenever I have time, I work on my new FOSS pet project: a service to ping your APIs/websites that is configurable via a simple YAML file and should support various queries/selectors (CSS, jq, etc.) and different protocols (HTTP(S), TCP, UDP, GRPS, GraphQL, etc.). One of the most challenging (and boring, IMHO) parts was to define the structure of the YAML config file, as it feels like creating a new DSL, so it's essential to keep it simple, extendable, and as short as possible. For that reason, I postponed the YAML part until the stage when I created a skeleton for the business logic and implemented a happy path for one of the protocols (HTTP). Once the time came, I decided to create a YAML config example with all the possible options before making the specifications and schema.

I started with the first lines like these (pingheim is the name of the tool):

pingheim: 0.0.1
description: A sample of a full YAML file

pings:

To my great surprise, GitHub Copilot immediately offered me a completion that created the structure of the ping with the desired feature I had introduced in my business logic. I was quite happy about it, as it seemed like I could outsource the most boring part to my AI friend. So I accepted all the suggestions (there were dozens of lines of YAML code), quickly implemented the code of deserialization of this file, plugged it into the existing business logic, and decided to call it a day.

I didn't have time to get back to this project for the next couple of days, and when I finally could, I opened my IDE and saw the generated YAML configurations with a fresh look. Well, it was clear that while there were a lot of valid points and even interesting ideas I had foreseen, the result itself was definitely far from being perfect and production-ready. What's worse is that it required proper surgery rather than a few minor changes to make it the way it should be, considering the abovementioned requirements. Did I mention that I found this part boring? Well, that was the moment when my inner resistance struck the most: "Come on, man, it's not that bad!", "It will require changing the entire codebase if we modify this part", "Let's keep it for now and change it in the future", "It's a tool for your own purposes, why bother?", "You invested so much time and effort to go back and reiterate!" and so on and so forth.

As you can see, I became a victim of the sunk cost fallacy. I even had to interrupt the coding session and go for a short walk to fix the frustration part. At the end of the day, I made the right choice and spent another couple of evenings changing everything the way it should be, but that was mentally hard to start/keep doing. Otherwise, my tool won't be the one I wanted to build, but rather the one that generated code pointed me to.

Again, it's not a problem of the AI tool, but rather my problem of non-ideal human-being - according to the fact that such effect has a separate term and About 1,920,000 results in Google search, I'm not alone here. The thing is that AI tools can help create the feeling of a massive investment by generating tons of code that is not that easy to refactor once accepted as it is. In the pre-AI era, the same situations could have happened if folks were copy-pasting large chunks of code from StackOverflow or similar places, but AI helps make it faster and in larger volumes.

While doing a couple of pair programming sessions with other engineers of different levels (from intern to staff), I realized that I'm definitely not alone there: some people are being caught in the same trap I mentioned above, and then it's up to their level of discipline and professionalism whether they can overcome it. Have you ever seen/experienced something similar?

What should we do about it?

The answer is simple to define but sometimes hard to follow: be aware of such cognitive bias and always remain professional and disciplined. If you feel tired, call it a day or go for a short walk/physical activity to let the brain relax, as it's way easier to be hooked by this bias once you are tired, and, therefore, you might be considering shortcuts to finish the task.

Privacy and Legal

This is the most boring and overlooked (by us engineers) challenge AI tools introduce: privacy and legal compliance. I remember that as soon as GitHub Copilot was released (in Beta?) for individual users, some of my friends and colleagues got access to it and started using it both for their pet projects and at work. It's only another tool, right? Well, it's not that simple.

There is an interesting discussion about "Is Github Copilot safe privacy wise?" on GitHub. The short conclusion is that it's not, meaning that you should not use your individual license with a code that belongs to the company, as the tool collects metrics, telemetry, and, probably, some code/suggestions/prompts as well. I can see that the same conclusion came from the privacy and legal teams of the different tech companies out there, so the company started the enterprise agreements' negotiations with GitHub to obtain the business licenses instead, which are more privacy-friendly.

For example, here is what GitHub states in the "About GitHub Copilot Individual" article:

As you can see, they are pretty transparent about the telemetry they collect from the individual users, but not for the business ones.

Many companies applied similar restrictions for the use of ChatGPT. The same approach goes for the other AI tools out there.

What should we do about it?

If you are using your private license with the production data, stop it right away! Talk to the lawyers in your company before sharing any company-owned data with the AI tools. Treat no approval as forbidden. We didn't share the company code on the internet forums and other services like StackOverflow, so we should treat AI tools the same way.

Final words

I know there are other issues and challenges with the tools we discussed above, especially related to the current AI limitations, but I am no expert in this field, so I'd like to let scientists discuss this.

I hope this post has triggered some thoughts, and it's more than OK to disagree with some. or all the points above (except the last one, as it's a law) - I'm open to meaningful discussion. As I said at the beginning of the article, there is no message to stop using these tools, but rather to be aware of their limits and use them in a better and more productive way.

Merry Christmas and a Happy New Year to those who celebrate. See you in 2024!

Have fun =)

P.S. If you liked my post and don’t want to miss out on my future publications, consider subscribing to my newsletter to stay tuned and get my new posts delivered right to your email inbox.

Go concurrency simplified. Part 4: Post office as a data pipeline

Mykola — Thu, 21 Dec 2023 23:23:58 +0000

Hello there! The main part of my moving to a new place adventures seems to be behind. Since I'm still waiting for a furniture delivery, I'm writing this post while lying on the floor using my foam camping mat as a sofa. It's not the most ideal setup, but it works. Anyway, I feel like today is the right time to start working on Part 4 of the "Go concurrency simplified" series. So far, we have learned and explored the key Go concurrency concepts, such as goroutines, channels, and the ways to synchronize and manage them. And today, we are going to combine all of our knowledge to make this world a better place to live (not really) or at least to help the post office (and our good old friend postman Bob) by making the queue handling process more effective. In the meantime, let's briefly recap where we stopped in Part 3 (if you missed it, here is the link).

Recap

The queue situation is still alarming, to say the least. On top of that, the management decided to assign a new responsibility to a postman in addition to handling the queue - answering the phone calls from the customers. Here is what it looks like:

This is the code that backs this setup:

type Customer struct {
    Name string
    Item string
}

func (c *Customer) GiveAway() string {
    item := c.Item
    fmt.Printf("%s gives away %s\n", c.Name, item)
    c.Item = ""
    return item
}

type Worker struct {
    Name string
}

func (w *Worker) StartWorkingDay(deskChan chan string, phoneChan chan string, shutdownChan chan struct{}) {
    for {
        select {
        case item := <-deskChan:
            w.Process(item)
        case call := <-phoneChan:
            fmt.Printf("Worker %s received a call: %s\n", w.Name, call)
        case <-shutdownChan:
            fmt.Println("the desk is closed - time to go home")
            return
        }
    }
}

func (w *Worker) Process(item string) {
    fmt.Printf("Worker %s received %s\n", w.Name, item)
    fmt.Printf("Worker %s started processing %s...\n", w.Name, item)

    // to simulate long processing
    time.Sleep(1 * time.Second)

    fmt.Printf("Worker %s processed %s\n\n", w.Name, item)
}

func main() {
    deskChan := make(chan string)
    phoneChan := make(chan string)
    shutdownChan := make(chan struct{})
    wg := &sync.WaitGroup{}

    bobWorker := Worker{Name: "Bob"}

    wg.Add(1)
    go func() {
        bobWorker.StartWorkingDay(deskChan, phoneChan, shutdownChan)
        wg.Done()
    }()

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    go func() {
        phoneChan <- "Has my package arrived?"
        time.Sleep(1 * time.Second)
        phoneChan <- "What about now?"
    }()

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    shutdownChan <- struct{}{}

    close(deskChan)
    close(phoneChan)
    close(shutdownChan)

    wg.Wait()
}

This is a decent solution for the tutorial, but I'd not deploy it to the actual production server. Why? Let me ask you first: How much time would it take for this program to complete? Feel free to ignore the phone calls for now. The approximate answer is at least 5 seconds. But why 5? Well, we have 5 customers in the queue, and we have this piece of code that runs for each customer:

func (w *Worker) Process(item string) {
    fmt.Printf("Worker %s received %s\n", w.Name, item)
    fmt.Printf("Worker %s started processing %s...\n", w.Name, item)

    // to simulate long processing
    time.Sleep(1 * time.Second)

    fmt.Printf("Worker %s processed %s\n\n", w.Name, item)
}

Did you notice the time.Sleep(1 * time.Second) part? This is our bottleneck, as each customer will need to wait for at least this amount of time. Let's add a simple timer to see whether our estimate is correct:

func main() {
    start := time.Now()

    deskChan := make(chan string)
    phoneChan := make(chan string)
    shutdownChan := make(chan struct{})
    wg := &sync.WaitGroup{}

    bobWorker := Worker{Name: "Bob"}

    wg.Add(1)
    go func() {
        bobWorker.StartWorkingDay(deskChan, phoneChan, shutdownChan)
        wg.Done()
    }()

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    go func() {
        phoneChan <- "Has my package arrived?"
        time.Sleep(1 * time.Second)
        phoneChan <- "What about now?"
    }()

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    shutdownChan <- struct{}{}

    close(deskChan)
    close(phoneChan)
    close(shutdownChan)

    wg.Wait()

    end := time.Now()
  fmt.Println("\n=====================================")
    fmt.Printf("Execution time in milliseconds: %v\n", end.Sub(start).Milliseconds())
}

You can find the code samples for the post in this GitHub repo.

The idea is pretty straightforward here:

the first line of the main function determines the start time: start := time.Now()
once the business logic is completed, we determine the end time: end := time.Now()
we subtract one from another, convert the result to milliseconds, and print it to see the total execution time

Here is what I can see on my machine:

=====================================
Execution time in milliseconds: 5002

So, our estimate of 5 seconds was entirely accurate. But is 1 second per customer a realistic number? If you have ever been to the post office, you know that it takes way more than 1 second to send the parcel: depending on where you live, it can take 1 to 10+ minutes to fix that. That's why if we apply the 10 seconds timing (still too little, but tolerable to wait while running the program) to our code like this:

func (w *Worker) Process(item string) {
    fmt.Printf("Worker %s received %s\n", w.Name, item)
    fmt.Printf("Worker %s started processing %s...\n", w.Name, item)

    // to simulate long processing
    time.Sleep(1 * time.Minute)

    fmt.Printf("Worker %s processed %s\n\n", w.Name, item)
}

it will change the total execution time from 5 seconds to 50 seconds. But what if we have more than 5 customers in the queue? Since Christmas is coming, here where I live the queues are pretty long these days, especially right after working day hours.

I hope it's clear now that we need to come up with something even better to improve the efficiency of the post office business and make the customers happier.

Back to the post office

It's clear to us that the main problem here is that it takes a lot of time for the postman to process a parcel, and the customers have to wait in the meantime. Luckily, the same is clear to Triss - a post office manager. She understands that the postman Bob needs help. And since this post office branch is very important, it is possible to reallocate some workers from the other places here, so Oda, Robert, and Martha will join. However, instead of opening 3 new desks to serve customers, Triss came up with something more innovative: the idea is to split the tasks in a way that Bob will still handle the customers, but instead of performing all the parcel clearance routines, from now on he is going to check the ID of the customers, take the parcel from them and pass it forward. The customer can leave right away, so the next in the queue can proceed. In the meantime, either of the 3 new employees takes the parcel and performs the rest of the actions with it.

If you have ever taken the algorithms classes, you might have heard of the "divide and conquer" approach to solving problems, where the large task is divided into smaller subtasks that are easier to solve. Triss studied management at the university, so she didn't have the algorithms in her curriculum - nevertheless, that's exactly the approach she took here.

Let's see what the new setup will look like:

Once Bob receives the parcel from the customer, he passes it forward right away:

One of the new colleagues picks it up right away. At the same time, the customer can leave, so the queue is moving:

This is quite an efficient process, as customers won't need to wait for the entire parcel clearance process to finish. Let's think about how we can code this.

A new way of working as a code

Until now, we had only one type of worker responsible for everything post-office-related. That's why we have 1 struct that defines this type of employee:

type Worker struct {
    Name string
}

From now on, we have 2 types of workers:

the one who stands at the desk and handles the customers: Bob
the ones who are in the back and handle the parcels once Bob passes them - our 3 new colleagues: Oda, Robert, and Martha

Let's rename the existing Worker struct to DeskWorker and agree that it represents the 1st type. Also, let's change the Process function a bit to specify the activities that Bob will do:

func (dw *DeskWorker) Process(item string) {
    fmt.Printf("Desk worker %s received %s\n", dw.Name, item)
    fmt.Printf("Desk worker %s started checking ID of the customer with the %s item...\n", dw.Name, item)

    // to simulate long processing
    time.Sleep(1 * time.Second)

    fmt.Printf("Desk worker %s finished checking ID of the customer with the %s item\n", dw.Name, item)

    fmt.Printf("Desk worker %s started passing %s to the back office...\n", dw.Name, item)
    dw.BackOfficeDeskChan <- item
    fmt.Printf("Desk worker %s passed %s to the back office\n\n", dw.Name, item)
}

We have yet to define the logic of passing the parcel to the back office, but we'll get back to this soon.

We need a new struct for the 2nd type - BackOfficeWorker :

type BackOfficeWorker struct {
    Name string
}

Similar to the existing struct, BackOfficeWorker needs 2 functions: StartWorkingDay and Process:

func (bow *BackOfficeWorker) StartWorkingDay(backOfficeDeskChan chan string, shutdownChan chan struct{}) {
    for {
        select {
        case item := <-backOfficeDeskChan:
            bow.Process(item)
        case <-shutdownChan:
            fmt.Printf("the back office is closed - time to go home, %s\n", bow.Name)
            return
        }
    }
}

func (bow *BackOfficeWorker) Process(item string) {
    fmt.Printf("Back office worker %s received %s\n", bow.Name, item)
    fmt.Printf("Back office worker %s started processing %s...\n", bow.Name, item)

    // to simulate long processing
    time.Sleep(10 * time.Second)

    fmt.Printf("Back office worker %s finished processing %s\n", bow.Name, item)
}

Unlike DeskWorker, these should not worry about the phone calls (poor Bob). Also, for the simulation purposes, there is a difference in processing time:

1 second for the desk worker
10 seconds for the desk office workers

These numbers are unrealistic, but the scale might be good enough for us.

Let's define 3 new workers in the main function next to the Bob declaration:

bobWorker := DeskWorker{Name: "Bob"}
odaWorker := BackOfficeWorker{Name: "Oda"}
robertWorker := BackOfficeWorker{Name: "Robert"}
marthaWorker := BackOfficeWorker{Name: "Martha"}

Since we have 2 types of employees now instead of 1, let's have 2 shutdown channels for each:

deskShutdownChan := make(chan struct{})
backOfficeDeskShutdownChan := make(chan struct{})

We need to make sure we send a shutdown signal 3 times to the backOfficeDeskShutdownChan:

deskShutdownChan <- struct{}{}
// 3 stands for the number of back office workers
for i := 0; i < 3; i++ {
    backOfficeDeskShutdownChan <- struct{}{}
}

The channel for the back office is missing as well:

deskChan := make(chan string)
backOfficeDeskChan := make(chan string)

Let's make sure to start the working day for every back office employee the same way as for the desk worker:

wg.Add(1)
go func() {
    bobWorker.StartWorkingDay(deskChan, phoneChan, deskShutdownChan)
    wg.Done()
}()

wg.Add(1)
go func() {
    odaWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
    wg.Done()
}()

wg.Add(1)
go func() {
    robertWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
    wg.Done()
}()

wg.Add(1)
go func() {
    marthaWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
    wg.Done()
}()

We can refactor this code by introducing a private function to start the employees' working day, but I'll keep it as it is for readability purposes - feel free to change the code on your own, as it's a good exercise!

The only missing part is when Bob passes the parcels to the back office. Let's extend the DeskWorker struct with a new field for the back office channel:

type DeskWorker struct {
    Name               string
    BackOfficeDeskChan chan string
}

We can use this channel now to pass the parcel into it within the Process function:

func (dw *DeskWorker) Process(item string) {
    fmt.Printf("Desk worker %s received %s\n", dw.Name, item)
    fmt.Printf("Desk worker %s started checking ID of the customer with the %s item...\n", dw.Name, item)

    // to simulate long processing
    time.Sleep(1 * time.Second)

    fmt.Printf("Desk worker %s finished checking ID of the customer with the %s item\n", dw.Name, item)

  fmt.Printf("Desk worker %s started passing %s to the back office...\n", dw.Name, item)
    dw.BackOfficeDeskChan <- item
    fmt.Printf("Desk worker %s passed %s to the back office\n\n", dw.Name, item)
}

Please, don't forget to initialize this field in the place where we declare the bobWorker variable:

bobWorker := DeskWorker{Name: "Bob", BackOfficeDeskChan: backOfficeDeskChan}

The code after all of these changes looks like this:

type Customer struct {
    Name string
    Item string
}

func (c *Customer) GiveAway() string {
    item := c.Item
    fmt.Printf("%s gives away %s\n", c.Name, item)
    c.Item = ""
    return item
}

type DeskWorker struct {
    Name               string
    BackOfficeDeskChan chan string
}

func (dw *DeskWorker) StartWorkingDay(deskChan chan string, phoneChan chan string, shutdownChan chan struct{}) {
    for {
        select {
        case item := <-deskChan:
            dw.Process(item)
        case call := <-phoneChan:
            fmt.Printf("Desk worker %s received a call: %s\n", dw.Name, call)
        case <-shutdownChan:
            fmt.Println("the desk is closed - time to go home")
            return
        }
    }
}

func (dw *DeskWorker) Process(item string) {
    fmt.Printf("Desk worker %s received %s\n", dw.Name, item)
    fmt.Printf("Desk worker %s started checking ID of the customer with the %s item...\n", dw.Name, item)

    // to simulate long processing
    time.Sleep(1 * time.Second)

    fmt.Printf("Desk worker %s finished checking ID of the customer with the %s item\n", dw.Name, item)

    fmt.Printf("Desk worker %s started passing %s to the back office...\n", dw.Name, item)
    dw.BackOfficeDeskChan <- item
    fmt.Printf("Desk worker %s passed %s to the back office\n\n", dw.Name, item)
}

type BackOfficeWorker struct {
    Name string
}

func (bow *BackOfficeWorker) StartWorkingDay(backOfficeDeskChan chan string, shutdownChan chan struct{}) {
    for {
        select {
        case item := <-backOfficeDeskChan:
            bow.Process(item)
        case <-shutdownChan:
            fmt.Printf("the back office is closed - time to go home, %s\n", bow.Name)
            return
        }
    }
}

func (bow *BackOfficeWorker) Process(item string) {
    fmt.Printf("Back office worker %s received %s\n", bow.Name, item)
    fmt.Printf("Back office worker %s started processing %s...\n", bow.Name, item)

    // to simulate long processing
    time.Sleep(10 * time.Second)

    fmt.Printf("Back office worker %s finished processing %s\n", bow.Name, item)
}

func main() {
    start := time.Now()

    deskChan := make(chan string)
    backOfficeDeskChan := make(chan string)
    phoneChan := make(chan string)
    deskShutdownChan := make(chan struct{})
    backOfficeDeskShutdownChan := make(chan struct{})
    wg := &sync.WaitGroup{}

    bobWorker := DeskWorker{Name: "Bob", BackOfficeDeskChan: backOfficeDeskChan}
    odaWorker := BackOfficeWorker{Name: "Oda"}
    robertWorker := BackOfficeWorker{Name: "Robert"}
    marthaWorker := BackOfficeWorker{Name: "Martha"}

    wg.Add(1)
    go func() {
        bobWorker.StartWorkingDay(deskChan, phoneChan, deskShutdownChan)
        wg.Done()
    }()

    wg.Add(1)
    go func() {
        odaWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
        wg.Done()
    }()

    wg.Add(1)
    go func() {
        robertWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
        wg.Done()
    }()

    wg.Add(1)
    go func() {
        marthaWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
        wg.Done()
    }()

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    go func() {
        phoneChan <- "Has my package arrived?"
        time.Sleep(1 * time.Second)
        phoneChan <- "What about now?"
    }()

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    deskShutdownChan <- struct{}{}
    // 3 stands for the number of back office workers
    for i := 0; i < 3; i++ {
        backOfficeDeskShutdownChan <- struct{}{}
    }

    close(phoneChan)
    close(deskChan)
    close(backOfficeDeskChan)
    close(deskShutdownChan)

    wg.Wait()

    end := time.Now()
    fmt.Printf("Execution time in milliseconds: %v\n", end.Sub(start).Milliseconds())
}

Let's run the code and see whether it works:

Lisa gives away basketball
Desk worker Bob received a call: Has my package arrived?
Desk worker Bob received basketball
Desk worker Bob started checking ID of the customer with the basketball item...
Eric gives away teddy bear
Desk worker Bob finished checking ID of the customer with the basketball item
Desk worker Bob started passing basketball to the back office...
Desk worker Bob passed basketball to the back office

Back office worker Oda received basketball
Back office worker Oda started processing basketball...
Desk worker Bob received teddy bear
Desk worker Bob started checking ID of the customer with the teddy bear item...
Jenny gives away watermelon
Desk worker Bob finished checking ID of the customer with the teddy bear item
Desk worker Bob started passing teddy bear to the back office...
Desk worker Bob passed teddy bear to the back office

Desk worker Bob received watermelon
Desk worker Bob started checking ID of the customer with the watermelon item...
Ben gives away box
Back office worker Martha received teddy bear
Back office worker Martha started processing teddy bear...
Desk worker Bob finished checking ID of the customer with the watermelon item
Desk worker Bob started passing watermelon to the back office...
Desk worker Bob passed watermelon to the back office

Desk worker Bob received box
Desk worker Bob started checking ID of the customer with the box item...
Zlatan gives away football
Back office worker Robert received watermelon
Back office worker Robert started processing watermelon...
Desk worker Bob finished checking ID of the customer with the box item
Desk worker Bob started passing box to the back office...
Back office worker Oda finished processing basketball
Back office worker Oda received box
Back office worker Oda started processing box...
Desk worker Bob passed box to the back office

Desk worker Bob received football
Desk worker Bob started checking ID of the customer with the football item...
Back office worker Martha finished processing teddy bear
Desk worker Bob finished checking ID of the customer with the football item
Desk worker Bob started passing football to the back office...
Desk worker Bob passed football to the back office

the desk is closed - time to go home
Back office worker Martha received football
Back office worker Martha started processing football...
Back office worker Robert finished processing watermelon
the back office is closed - time to go home, Robert
Back office worker Oda finished processing box
the back office is closed - time to go home, Oda
Back office worker Martha finished processing football
the back office is closed - time to go home, Martha

=====================================
Execution time in milliseconds: 22003

The application works as expected, which is nice. On top of that, we can see quite a dramatic increase in the performance from 50 to 22 seconds. Exciting! But can we do better than that?

Better than that

Triss is impressed with such results. But she is still able to see the room for improvement. She noticed that after serving 3 customers and picking up the 4th parcel, Bob hung for quite some time, as there were no free back-office workers to pass the parcels to. If you ran the code above, you might have noticed the same:

the Desk worker Bob started passing box to the back office... message was printed
the execution paused
in +-10 seconds, the flow resumed

Well, nothing unexpected has happened, as there are only 3 back-office employees for 5 customers. This makes Triss think: "OK, if I push hard enough during the next meeting with the executives, I'll make them allocate 2 more employees to this branch". And since she is an extremely good negotiator, Carl and Luna joined the post office crew in no time.

Let's change our code to represent this:

func main() {
    start := time.Now()

    deskChan := make(chan string)
    backOfficeDeskChan := make(chan string)
    phoneChan := make(chan string)
    deskShutdownChan := make(chan struct{})
    backOfficeDeskShutdownChan := make(chan struct{})
    wg := &sync.WaitGroup{}

    bobWorker := DeskWorker{Name: "Bob", BackOfficeDeskChan: backOfficeDeskChan}
    odaWorker := BackOfficeWorker{Name: "Oda"}
    robertWorker := BackOfficeWorker{Name: "Robert"}
    marthaWorker := BackOfficeWorker{Name: "Martha"}
    carlWorker := BackOfficeWorker{Name: "Carl"}
    lunaWorker := BackOfficeWorker{Name: "Luna"}

    wg.Add(1)
    go func() {
        bobWorker.StartWorkingDay(deskChan, phoneChan, deskShutdownChan)
        wg.Done()
    }()

    wg.Add(1)
    go func() {
        odaWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
        wg.Done()
    }()

    wg.Add(1)
    go func() {
        robertWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
        wg.Done()
    }()

    wg.Add(1)
    go func() {
        marthaWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
        wg.Done()
    }()

    wg.Add(1)
    go func() {
        carlWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
        wg.Done()
    }()

    wg.Add(1)
    go func() {
        lunaWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
        wg.Done()
    }()

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    go func() {
        phoneChan <- "Has my package arrived?"
        time.Sleep(1 * time.Second)
        phoneChan <- "What about now?"
    }()

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    deskShutdownChan <- struct{}{}
    // 5 stands for the number of back office workers
    for i := 0; i < 5; i++ {
        backOfficeDeskShutdownChan <- struct{}{}
    }

    close(phoneChan)
    close(deskChan)
    close(backOfficeDeskChan)
    close(deskShutdownChan)

    wg.Wait()

    end := time.Now()
    fmt.Println("\n=====================================")
    fmt.Printf("Execution time in milliseconds: %v\n", end.Sub(start).Milliseconds())
}

Let's run it:

Lisa gives away basketball
Eric gives away teddy bear
Desk worker Bob received basketball
Desk worker Bob started checking ID of the customer with the basketball item...
Desk worker Bob finished checking ID of the customer with the basketball item
Desk worker Bob started passing basketball to the back office...
Desk worker Bob passed basketball to the back office

Desk worker Bob received teddy bear
Desk worker Bob started checking ID of the customer with the teddy bear item...
Jenny gives away watermelon
Back office worker Robert received basketball
Back office worker Robert started processing basketball...
Desk worker Bob finished checking ID of the customer with the teddy bear item
Desk worker Bob started passing teddy bear to the back office...
Desk worker Bob passed teddy bear to the back office

Desk worker Bob received a call: Has my package arrived?
Desk worker Bob received watermelon
Desk worker Bob started checking ID of the customer with the watermelon item...
Back office worker Oda received teddy bear
Ben gives away box
Back office worker Oda started processing teddy bear...
Desk worker Bob finished checking ID of the customer with the watermelon item
Desk worker Bob started passing watermelon to the back office...
Desk worker Bob passed watermelon to the back office

Desk worker Bob received box
Desk worker Bob started checking ID of the customer with the box item...
Zlatan gives away football
Back office worker Carl received watermelon
Back office worker Carl started processing watermelon...
Desk worker Bob finished checking ID of the customer with the box item
Desk worker Bob started passing box to the back office...
Back office worker Martha received box
Back office worker Martha started processing box...
Desk worker Bob passed box to the back office

Desk worker Bob received a call: What about now?
Desk worker Bob received football
Desk worker Bob started checking ID of the customer with the football item...
Desk worker Bob finished checking ID of the customer with the football item
Desk worker Bob started passing football to the back office...
Desk worker Bob passed football to the back office

the desk is closed - time to go home
Back office worker Luna received football
Back office worker Luna started processing football...
Back office worker Robert finished processing basketball
the back office is closed - time to go home, Robert
Back office worker Oda finished processing teddy bear
the back office is closed - time to go home, Oda
Back office worker Carl finished processing watermelon
the back office is closed - time to go home, Carl
Back office worker Martha finished processing box
the back office is closed - time to go home, Martha
Back office worker Luna finished processing football
the back office is closed - time to go home, Luna

=====================================
Execution time in milliseconds: 15003

15 seconds - that's impressive, isn't it? And I'll tell you what: we can't make it faster with current constraints. Let's see why:

Bob spends 1 second on each parcel before passing it further -> 5 seconds
each back office worker spends 10 seconds on parcel -> 10 seconds
total: 5 + 10 = 15 seconds

If 50 -> 15 seconds of progress doesn't sound like a significant boost for you, please remember that we are using artificially simplified time numbers. However, if we use more realistic ones (let's say, 5 minutes per customer), we'll see:

the original code (from the Recap section): 25 minutes
the latest code (with the 1 minute for DeskWorker and 4 minutes for BackOfficeWorker setup): 9 minutes

That's quite a difference, especially from the customer side. For a queue of 10 people, it will be even more significant if we have 10 workers: 50 vs. 19 minutes.

Does it mean we have just solved the post office issue we've discussed since Part 1 of this series? Yes, for this setup. But there is one more thing that we can improve here to make the code more realistic to the real post office.

A realistic post office as a data pipeline

While it's fair to be satisfied with the work we have done so far, there is 1 hardcoded thing left that prevents us from being 100% happy: currently, we have hardcoded 5 customers into a list:

zlatan := Customer{Name: "Zlatan", Item: "football"}
ben := Customer{Name: "Ben", Item: "box"}
jenny := Customer{Name: "Jenny", Item: "watermelon"}
eric := Customer{Name: "Eric", Item: "teddy bear"}
lisa := Customer{Name: "Lisa", Item: "basketball"}

queue := []Customer{lisa, eric, jenny, ben, zlatan}

But it's not realistic, as in real life the customers come and go during working hours rather than the queue being static. What should we use instead of the list to make things more dynamic? Hmm... I'm sure you know the answer - a channel! It's pretty easy to imagine the post office as a channel:

the channel is initiated when the entry door to the post office has been opened - this means that the working day has begun
once the customer enters the building through the entry door - it means that the new item has been written to the channel
once the working day is over, the entry door is closed - the channel is closed too

Wait a minute! With the new approach, the number of customers is uncertain, so how many back office workers do we need? That's an excellent question! And from this moment, it becomes harder to explain this by using a real-life example. Luckily, we are not limited by the reality here, that's why:

if you are into Sci-Fi, imagine that our back-office worker is a robot that can immediately clone itself once there is a new item to process in a way that the copy does all the clearance routine and disappears once it finishes its work. Basically, all the "original" robot has to do is create a clone and pass the parcel to them
if you are more into fantasy (like me), imagine that our back-office worker is a wizard, and once there is a new item to process, they cast a spell so the clearance routine for this particular parcel is completed on its own
or if you are more into Xmas mood these days, imagine that our back-office worker is Santa, who has an unlimited number of elves that are willing to help and take a parcel each

Regardless of the example, the idea I'm describing is the situation when there is a way to have an unlimited number of back-office workers spawned once there is a new parcel and released once it's processed. It's hard to imagine this in real life, as it's nearly impossible to have an unlimited number of employees due to such constraints as money and space. Unlike real life, it's different in software engineering: we can easily spawn a massive amount of goroutines even on a local machine because goroutines are cheap. And if we have a super powerful machine, their maximum number can be almost indefinite.

Let's use this knowledge to modify our code. As a big fan of fantasy, I'll go with the wizard scenario.

We need to replace the existing back-office worker with a wizard one. Let's start by renaming BackOfficeWorker to WizardBackOfficeWorker:

type WizardBackOfficeWorker struct {
    Name string
}

Within the StartWorkingDay function the BackOfficeWorker (the muggle one) has this logic for handling items:

func (bow *BackOfficeWorker) StartWorkingDay(backOfficeDeskChan chan string, shutdownChan chan struct{}) {
    for {
        select {
        case item := <-backOfficeDeskChan:
            bow.Process(item)
        case <-shutdownChan:
            fmt.Printf("the back office is closed - time to go home, %s\n", bow.Name)
            return
        }
    }
}

This means that the workers processed the items themselves. As discussed above, the wizard won't do this but instead cast a spell (create a new worker) to delegate the processing process to it. On a code level, it means that we'll have to create a new goroutine for each item and let it do the job:

func (bow *WizardBackOfficeWorker) StartWorkingDay(backOfficeDeskChan chan string, shutdownChan chan struct{}) {
    wg := &sync.WaitGroup{}

    for {
        select {
        case item := <-backOfficeDeskChan:
            fmt.Printf("Wizard %s received %s\n", bow.Name, item)

            wg.Add(1)
            go func(item string) {
                defer wg.Done()

                fmt.Printf("Wizard %s casted a spell to process %s\n", bow.Name, item)
                bow.Process(item)
            }(item)
        case <-shutdownChan:
            fmt.Printf("the back office is closed - time to go home, %s\n", bow.Name)
            wg.Wait()
            return
        }
    }
}

As you see, we had to introduce an instance of the sync.WaitGroup here to ensure that once the shutdown signal is received, the function won't quit before all the goroutines (spawned by it) are completed. That's why we do wg.Wait() before the return part. If you are not familiar with the concept of syncing, please refer to Part 2 of this series - I dedicated the entire post to this.

The powerful part here is that once the new goroutine is launched by the go func(item string) { code, the wizard is immediately available for the next parcel - no need to be blocked for the 10 seconds (processing time) like in the previous implementation.

There is one part that might be a bit confusing to you if you are new to Go:

go func(item string) {
    defer wg.Done()

    fmt.Printf("Wizard %s casted a spell to process %s\n", bow.Name, item)
    bow.Process(item)
}(item)

In Part 2, we touched on a similar concept that looked like this:

func() {
    // do something
}()

We said that what happens here is that the func(){} part declares a new anonymous function while the () part invokes it right away.

What we have here is similar, but this time, our anonymous function accepts a parameter item of a type string:

func(item string) {
    // do something
}

And to invoke it, we have to pass a value for this parameter - basically, the same as for the regular function, that's why we have the (item) part:

func(item string) {
    // do something
}(item)

item is the variable defined above in the code: case item := <-backOfficeDeskChan:

But ok, let's get back to the original topic, as I believe you are more than equipped to understand the rest of the code there. Let's modify the Process function of the WizardBackOfficeWorker to specify that it is a cast who does the processing, not the wizard themselves:

func (bow *WizardBackOfficeWorker) Process(item string) {
    fmt.Printf("Wizard %s's spell started processing %s...\n", bow.Name, item)

    // to simulate long processing
    time.Sleep(10 * time.Second)

    fmt.Printf("Wizard %s's spell finished processing %s\n", bow.Name, item)
}

As you can see, the time.Sleep(10 * time.Second) part still remains, as it takes the same time to do the magic parcel handling as the regular one.

The next step should be to replace the slice with a channel to define the customers' queue. But how should we get the customers to put in that channel? Another good question, you are on fire today! We need a function to generate random customers and, ideally, with random waiting to make it more realistic, as customers come to the post office at a different frequency. For that, I asked ChatGPT to generate a list of 50 random names and 50 random Xmas presents that fit into the postal package. I won't provide these lists here to save space, but feel free to check them out on a GitHub repo. Once we have it, the rest of the generator code is trivial:

var (
  names = {
    // a list of 50 names generated by ChatGPT
  }

  items = {
    // a list of 50 items generated by ChatGPT
  }
)

func generateCustomerWithRandomWait() Customer {
    // to simulate a random customer arrival while the post office is open
    time.Sleep(time.Duration(rand.Intn(5)) * time.Second)
    return randomCustomer()
}

func randomCustomer() Customer {
    return Customer{
        Name: randomName(),
        Item: randomItem(),
    }
}

func randomName() string {
    return names[rand.Intn(len(names))]
}

func randomItem() string {
    return items[rand.Intn(len(items))]
}

Now, we have everything to start using a channel for the queue representation.

// a new channel to send a signal to once it's time to close the post office
postOfficeShutdownChan := make(chan struct{})

// ...............
// some other code
// ...............

queueChan := make(chan Customer)
go func() {
    for {
        select {
        case <-postOfficeShutdownChan:
            fmt.Println("the post office is closed - time to go home")
            close(queueChan)
            return
        default:
            // to simulate a random customer arrival while the post office is open
            customer := generateCustomerWithRandomWait()
            fmt.Printf("%s enters the post office\n", customer.Name)
            queueChan <- customer
        }
    }
}()

for customer := range queueChan {
    deskChan <- customer.GiveAway()
}

As you can see, inside the go func(), the code will generate a new customer and put it into the queueChan within the default section until there is a signal to shutdown the post office. If the shutdown signal is received, the queueChan will be closed immediately. A for loop below consumes items from the queueChan while it is open. We don't have the code for sending the signal to the postOfficeShutdownChan, so let's add it:

postOfficeShutdownChan := make(chan struct{})

// to simulate a long working day
time.AfterFunc(5*time.Minute, func() {
    postOfficeShutdownChan <- struct{}{}
})

time.AfterFunc is a new concept for us, but it's pretty straightforward: it accepts a time duration and a function and invokes this function after the provided duration has passed. In our case, it will send the shutdown signal to the postOfficeShutdownChan in 5 minutes. This way, we are going to simulate a working day - feel free to change it to 8 hours if you have such a lot of time to wait for the program completion =)

Here is the entire code after our changes:

type Customer struct {
    Name string
    Item string
}

func (c *Customer) GiveAway() string {
    item := c.Item
    fmt.Printf("%s gives away %s\n", c.Name, item)
    c.Item = ""
    return item
}

type DeskWorker struct {
    Name               string
    BackOfficeDeskChan chan string
}

func (dw *DeskWorker) StartWorkingDay(deskChan chan string, phoneChan chan string, shutdownChan chan struct{}) {
    for {
        select {
        case item := <-deskChan:
            dw.Process(item)
        case call := <-phoneChan:
            fmt.Printf("Desk worker %s received a call: %s\n", dw.Name, call)
        case <-shutdownChan:
            fmt.Println("the desk is closed - time to go home")
            return
        }
    }
}

func (dw *DeskWorker) Process(item string) {
    fmt.Printf("Desk worker %s received %s\n", dw.Name, item)
    fmt.Printf("Desk worker %s started checking ID of the customer with the %s item...\n", dw.Name, item)

    // to simulate long processing
    time.Sleep(1 * time.Second)

    fmt.Printf("Desk worker %s finished checking ID of the customer with the %s item\n", dw.Name, item)

    fmt.Printf("Desk worker %s started passing %s to the back office...\n", dw.Name, item)
    dw.BackOfficeDeskChan <- item
    fmt.Printf("Desk worker %s passed %s to the back office\n", dw.Name, item)
}

type WizardBackOfficeWorker struct {
    Name string
}

func (bow *WizardBackOfficeWorker) StartWorkingDay(backOfficeDeskChan chan string, shutdownChan chan struct{}) {
    wg := &sync.WaitGroup{}

    for {
        select {
        case item := <-backOfficeDeskChan:
            fmt.Printf("Wizard %s received %s\n", bow.Name, item)

            wg.Add(1)
            go func(item string) {
                defer wg.Done()

                fmt.Printf("Wizard %s casted a spell to process %s\n", bow.Name, item)
                bow.Process(item)
            }(item)
        case <-shutdownChan:
            fmt.Printf("the back office is closed - time to go home, %s\n", bow.Name)
            wg.Wait()
            return
        }
    }
}

func (bow *WizardBackOfficeWorker) Process(item string) {
    fmt.Printf("Wizard %s's spell started processing %s...\n", bow.Name, item)

    // to simulate long processing
    time.Sleep(10 * time.Second)

    fmt.Printf("Wizard %s's spell finished processing %s\n", bow.Name, item)
}

func main() {
    start := time.Now()

    deskChan := make(chan string)
    backOfficeDeskChan := make(chan string)
    phoneChan := make(chan string)
    deskShutdownChan := make(chan struct{})
    backOfficeDeskShutdownChan := make(chan struct{})
    postOfficeShutdownChan := make(chan struct{})

    // to simulate a long working day
    time.AfterFunc(5*time.Minute, func() {
        postOfficeShutdownChan <- struct{}{}
    })

    wg := &sync.WaitGroup{}

    bobWorker := DeskWorker{Name: "Bob", BackOfficeDeskChan: backOfficeDeskChan}
    wizardBackOfficeWorker := WizardBackOfficeWorker{Name: "Radagast"}

    wg.Add(1)
    go func() {
        bobWorker.StartWorkingDay(deskChan, phoneChan, deskShutdownChan)
        wg.Done()
    }()

    wg.Add(1)
    go func() {
        wizardBackOfficeWorker.StartWorkingDay(backOfficeDeskChan, backOfficeDeskShutdownChan)
        wg.Done()
    }()

    go func() {
        time.Sleep(5 * time.Second)
        phoneChan <- "Has my package arrived?"
        time.Sleep(1 * time.Second)
        phoneChan <- "What about now?"
    }()

    queueChan := make(chan Customer)
    go func() {
        for {
            select {
            case <-postOfficeShutdownChan:
                fmt.Println("the post office is closed - time to go home")
                close(queueChan)
                return
            default:
                // to simulate a random customer arrival while the post office is open
                customer := generateCustomerWithRandomWait()
                fmt.Printf("%s enters the post office\n", customer.Name)
                queueChan <- customer
            }
        }
    }()

    for customer := range queueChan {
        deskChan <- customer.GiveAway()
    }

    deskShutdownChan <- struct{}{}
    backOfficeDeskShutdownChan <- struct{}{}

    wg.Wait()

    close(phoneChan)
    close(deskChan)
    close(backOfficeDeskChan)
    close(deskShutdownChan)
    close(backOfficeDeskShutdownChan)
    close(postOfficeShutdownChan)

    end := time.Now()
    fmt.Println("\n=====================================")
    fmt.Printf("Execution time in milliseconds: %v\n", end.Sub(start).Milliseconds())
}

Let's run this code. It will take 5 minutes to complete, so please feel free to pour a cup of tea or coffee in the meantime. You should be able to see a lot of activity printed into the terminal that ends with something like this:

............................................................................................
the post office is closed - time to go home
Anika gives away handwritten holiday card
Desk worker Bob received handwritten holiday card
Desk worker Bob started checking ID of the customer with the handwritten holiday card item...
Wizard Radagast's spell finished processing miniature snowman kit
Desk worker Bob finished checking ID of the customer with the handwritten holiday card item
Desk worker Bob started passing handwritten holiday card to the back office...
Desk worker Bob passed handwritten holiday card to the back office
the desk is closed - time to go home
Wizard Radagast received handwritten holiday card
the back office is closed - time to go home, Radagast
Wizard Radagast casted a spell to process handwritten holiday card
Wizard Radagast's spell started processing handwritten holiday card...
Wizard Radagast's spell finished processing teddy bear
Wizard Radagast's spell finished processing football
Wizard Radagast's spell finished processing pocket-sized board games
Wizard Radagast's spell finished processing festive cocktail mixers
Wizard Radagast's spell finished processing wine sampler
Wizard Radagast's spell finished processing handmade jewelry
Wizard Radagast's spell finished processing handwritten holiday card

=====================================
Execution time in milliseconds: 311080

This is a perfect post office implementation:

we have a start of the working day
we have a dynamic customers that come and go
we have a fast track to serve the customers
we have an end of the working day that gracefully shuts down the execution by allowing already started work to finish

And this is exactly how the data pipelines look like out there: if we drop the technical part, what is left at the end of the day is the data flow pipeline, where we can see a data transformation happening on each step with a clear producer-consumer pattern:

Good job, my friend! Triss is happy, so should we be, as we can officially claim that we have just solved the inefficient post office queue handling issue we've been discussing since Part 1.

Where to go from here

Before wrapping the "Go concurrency simplified" series up, let me give you some hints on where you can go from here:

try to play with a code we implemented and change it: for example, introduce a new type of worker - a delivery worker that is responsible for picking up the parcels processed by the back office and delivering them to the destination; then try adding a new desk, so there are 2 desk workers to handle the customers, etc.
implement your own data pipeline: for example, you can create a search engine that will receive a path to a directory to scan, then read all the text files from this directory & subdirectories and search for some word you choose within those files - it should be doable with the knowledge you've learned so far; feel free to share links your GitHub repos with the program you've implemented in the comments section
take a look at the concurrent code written by other devs out there: for example, feel free to check the internals of my library Pippin, but I bet there are many better projects out there to learn from - Google/Bing/DuckDuckGo/Kagi and ChatGPT can help to find the right one
check the Go documentation, especially the sections about the concurrency
check the Go documentation for the sync package to see what else Go has to offer for the synchronization routines
check the Go documentation for the x/sync package to see the beta version of the features and data structures that Go devs are experimenting with - one day, they might (or might not) be a part of the language core
watch a great talk by Rob Pike - one of the Go creators on the "Go Concurrency Patterns" topic
if you understood the "A realistic post office as a data pipeline" chapter of this post, you have a good foundation to learn about the event-driven architecture as it uses the similar principles that we applied here
also, this knowledge applies to learning more about data engineering, as this field of software engineering relies heavily on the event-driven approach via tools like Spark, Flink, Kafka, etc.

Farewell

It is time to wrap up the "Go concurrency simplified" series. It was quite a journey, and I'm more than happy that we've made it through together. I hope you've learned a lot and are ready to write a concurrent program on your own using the knowledge from this series.

Even though it's the end of this series, there will be more posts about different concepts of software engineering and tech, including (but not limited to) different parts of the Go concurrency concepts. If you don't want to miss out on my future publications, consider subscribing to my newsletter to stay tuned and get my new posts delivered right to your email inbox.

Also, you can find me on X/Twitter, Mastodon, and Bluesky.

I wish you a Merry Christmas and a Happy New Year for those who celebrate - careful with the amount of "Last Christmas" and "All I want for Christmas" songs you listen to =)

Have fun! =)

P.S. If you like my post and don't want to miss out the future ones, consider:

subscribing to my newsletter
following me on Twitter/X

Go concurrency simplified. Part 3: Managing channels with `for` loops and `select` statements

Mykola — Thu, 14 Dec 2023 22:59:39 +0000

Hello there! I feel like I got my covid under control and will be back to daily life soon. In the meantime, I'm sitting at my desk in a nearly empty apartment (I'm moving soon) and wondering whether it's possible to produce an echo if I scream loud enough 🤔 Anyway, I feel like it's the right time to start working on Part 3 of the "Go concurrency simplified" series. Today, we'll move on and explore the ways Go offers us to sync goroutines - it will get us closer to solving the queue situation in the post office we discussed last time. But let's start with a short recap of where we stopped in the previous post (if you missed it, here is the link).

Recap

Last time, we refactored our post office code by replacing the ugly time.Sleep() workaround with a sync.WaitGroup approach. Here is the entire code for that:

type Customer struct {
    Name string
    Item string
}

func (c *Customer) GiveAway() string {
    item := c.Item
    fmt.Printf("%s gives away %s\n", c.Name, item)
    c.Item = ""
    return item
}

type Worker struct {
    Name string
}

func (w *Worker) StartWorkingDay(deskChan chan string) {
    for {
        item, ok := <-deskChan
        if !ok {
            break
        }
        w.Process(item)
    }

    fmt.Println("the desk is closed - time to go home")
}

func (w *Worker) Process(item string) {
    fmt.Printf("Worker %s received %s\n", w.Name, item)
    fmt.Printf("Worker %s started processing %s...\n", w.Name, item)

    // to simulate long processing
    time.Sleep(1 * time.Second)

    fmt.Printf("Worker %s processed %s\n\n", w.Name, item)
}

func main() {
    deskChan := make(chan string)
    wg := &sync.WaitGroup{}

    bobWorker := Worker{Name: "Bob"}

    wg.Add(1)
    go func() {
        bobWorker.StartWorkingDay(deskChan)
        wg.Done()
    }()

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    close(deskChan)

    wg.Wait()
}

If we take a closer look at this code using the experience of the refactoring we did in Part 2, it becomes clear that there is one part that looks hacky - I mean this one:

for {
    item, ok := <-deskChan
    if !ok {
        break
    }
    w.Process(item)
}

A quick reminder of what's happening there: the code is supposed to fetch and process items from the channel until it's open. Once the channel is closed, the code should finish. Even though our solution does its job, our engineering experience tells us that it feels kinda weird that the designers of Go implemented channels but made us reinvent the wheel to consume items from them. And that's a good way of thinking! What if I tell you that actually there is a way to use for loops with channels? Let's jump into it!

Managing channels with `for` loops

If we summarize what the code above does, it will be:

reads items from the channel while it's open
if the channel is closed, finishes the execution

And that's the exact description of how for loop works with the channel. Let me show you how we can rewrite the piece of code above:

for item := range deskChan {
    w.Process(item)
}

And that's it - as simple as that. The line for item := range deskChan blocks the execution until there is a new item in the channel or until the channel is closed.

Let's apply this change to the StartWorkingDay function:

func (w *Worker) StartWorkingDay(deskChan chan string) {
    for item := range deskChan {
        w.Process(item)
    }

    fmt.Println("the desk is closed - time to go home")
}

You can find the code samples for the post in this GitHub repo.

And run the code:

Lisa gives away basketball
Eric gives away teddy bear
Worker Bob received basketball
Worker Bob started processing basketball...
Worker Bob processed basketball

Worker Bob received teddy bear
Worker Bob started processing teddy bear...
Jenny gives away watermelon
Worker Bob processed teddy bear

Worker Bob received watermelon
Worker Bob started processing watermelon...
Ben gives away box
Worker Bob processed watermelon

Worker Bob received box
Worker Bob started processing box...
Zlatan gives away football
Worker Bob processed box

Worker Bob received football
Worker Bob started processing football...
Worker Bob processed football

the desk is closed - time to go home

Works like a charm!

This straightforward approach opens endless opportunities. Let's imagine that NASA asks us to build a data pipeline that has the following stages:

Mars rover sends metrics to the International Space Station (ISS)
ISS enriches the data and forwards it to the NASA data center
the NASA data center processes the data

Let's help NASA

This might sound like a challenging task, but since we'll mock the business logic of processing data and focus on the data pipeline part only, you'll see the beauty and the simplicity of Go here:

import (
    "fmt"
    "math/rand"
    "strconv"
)

type MarsRover struct{}

func (mr *MarsRover) GatherMetrics() int {
    fmt.Println("Mars rover: gathering metrics...")
    return rand.Int()
}

type Iss struct{}

func (i *Iss) Enrich(metrics int) string {
    fmt.Printf("ISS: enriching metrics [%d]...\n", metrics)
    return "ISS" + strconv.Itoa(metrics)
}

type NasaDataCenter struct{}

func (ndc *NasaDataCenter) Process(data string) {
    fmt.Printf("Nasa data center: processing data [%s]...\n", data)
}

func main() {
    marsRover := MarsRover{}
    iss := Iss{}
    nasaDataCenter := NasaDataCenter{}

    issChan := make(chan int)
    nasaChan := make(chan string)

    go func() {
        for metrics := range issChan {
            nasaChan <- iss.Enrich(metrics)
        }
    }()

    go func() {
        for data := range nasaChan {
            nasaDataCenter.Process(data)
        }
    }()

    for {
        issChan <- marsRover.GatherMetrics()
    }
}

If you are new to Go and don't know what rand.Int() does
math/rand package offers ways to generate random data. rand.Int() returns a random integer value.
Please, note that if you need a secure random data, use crypto/rand package instead.

If you are new to Go and don't know what strconv.Itoa(metrics) does
strconv.Itoa is function to convert integer value to string. So if you pass 42 there, you'll get "42" as a result of it.
ITOA stands for "Integer to ASCII".
This StackOverFlow answer (https://stackoverflow.com/a/2909772) shows that the itoa (or atoi - so, the opposite action) was mentioned in the manuals in 1971 - that's quite a while ago!

If you run this code, it will be printing data like this:

Mars rover: gathering metrics...
ISS: enriching metrics [7508100157780980863]...
ISS: enriching metrics [8542411963209765260]...
Mars rover: gathering metrics...
Nasa data center: processing data [ISS7508100157780980863]...
Nasa data center: processing data [ISS8542411963209765260]...
ISS: enriching metrics [6666611277333029651]...
Nasa data center: processing data [ISS6666611277333029651]...

Please terminate the execution manually by either clicking a stop button in your IDE or pressing CTRL+C or CTRL+Din your terminal. Otherwise, it will run forever and keep producing logs like I showed above.

As you can see, the solution is super simple and uses concurrency concepts we have learned so far: goroutines, channels, and for-loops. Next time you need to do a task that reminds a sequence of steps that you need to repeat many times, you can reuse the example above and tweak it to your needs.

So, are we done with the Go concurrency? Have we already learned all we need? I wish! Let's leave the space for now and get back to the post office, as we have a situation there.

Back to the post office

Our good old friend postman Ben has just received a call from his manager, Triss. She has a new business idea that should increase customer satisfaction - install a phone so the customers can call it when they have questions. This means that Ben will have a new responsibility on top of handling the parcels that clients bring. This is the new setup:

Hmm...it means that Ben needs to handle 2 tasks instead of 1 from now on. But what is he supposed to do once there are both customers and a phone call? Well, he is free to choose which one to handle - "We have a lot of freedom at work," as Triss likes to say.

But how should we represent the phone with the code? It wouldn't be surprising that you already know the answer - another channel.

func main() {
    phoneChan := make(chan string)

    // listen to the phone calls:
    go func() {
        for call := range phoneChan {
            fmt.Println("Got a call: " + call)
        }
    }()

    // somebody calls the phone:
    phoneChan <- "Yo! What's up?"
    close(phoneChan)
}

The output is:

Got a call: Yo! What's up?

Let's take another look at the code that we wrote to represent the working day of the postman:

func (w *Worker) StartWorkingDay(deskChan chan string) {
    for item := range deskChan {
        w.Process(item)
    }

    fmt.Println("the desk is closed - time to go home")
}

As we can see, the code is perfectly fit to handle the data from 1 channel. But we have just mentioned that from now on, there will be 2 channels instead. Shall we come up with another workaround? It's a good time to introduce a new Go concept for managing multiple channels simultaneously - select statement.

Managing multiple channels with `select` statements

As we see now, the for loop is a handy construction when we need to query only 1 channel. And this might cover a lot of cases that you are going to face sooner or later. However, once this limit is reached, it's time to onboard the select statement.

The syntax for this construct is pretty straightforward and (my guess) inspired by the switch statement:

select {
case f := <-firstChan:
  handleFirst(f)
case s := <-secondChan:
  handleSecond(s)
case t := <-thirdChan:
  handleThird(t)
}

f := <-firstChan is exactly the same syntax we used to fetch data from the channel once we first met the concept of channels at all. This means it is possible to do f, ok := <-firstChan as well if we'd like to know whether the firstChan is opened.

Let's replace our for loop with the select statement and see how it works:

func (w *Worker) StartWorkingDay(deskChan chan string, phoneChan chan string) {
    select {
    case item := <-deskChan:
        w.Process(item)
    case call := <-phoneChan:
        fmt.Printf("Worker %s received a call: %s\n", w.Name, call)
    }

    fmt.Println("the desk is closed - time to go home")
}

We also need to apply changes to the main function to setup the phone and add the code to make calls to it with a provided interval:

func main() {
    deskChan := make(chan string)
    phoneChan := make(chan string)
    wg := &sync.WaitGroup{}

    bobWorker := Worker{Name: "Bob"}

    wg.Add(1)
    go func() {
        bobWorker.StartWorkingDay(deskChan, phoneChan)
        wg.Done()
    }()

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    go func() {
        phoneChan <- "Has my package arrived?"
        time.Sleep(1 * time.Second)
        phoneChan <- "What about now?"
    }()

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    close(deskChan)
    close(phoneChan)

    wg.Wait()
}

Let's run the code and see whether it works or not:

Lisa gives away basketball
Worker Bob received a call: Has my package arrived?
the desk is closed - time to go home
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.main()
        /n0rdy-blog-code-samples/20231214-go-concurrency-with-for-and-select/04-select-without-loop/select.go:74 +0x2b8

goroutine 19 [chan send]:
main.main.func2()
        /n0rdy-blog-code-samples/20231214-go-concurrency-with-for-and-select/04-select-without-loop/select.go:70 +0x45
created by main.main in goroutine 1
        /n0rdy-blog-code-samples/20231214-go-concurrency-with-for-and-select/04-select-without-loop/select.go:67 +0x25f

Wait a minute, that's not the output we were supposed to get! How come there is a deadlock? And why the the desk is closed - time to go home message

Actually, there is a tiny detail that I have entirely forgotten to mention - unlike for loops, the select statements are not loops but rather 1-time actions, the same as the switch operator. That's why this part

    select {
    case item := <-deskChan:
        w.Process(item)
    case call := <-phoneChan:
        fmt.Printf("Worker %s received a call: %s\n", w.Name, call)
    }

was executed once, and then the code proceeded to the fmt.Println("the desk is closed - time to go home") line, so Ben called it a day and went home. Since the queue of customers was still there, and there was nobody to handle them, we ended up in a deadlock situation. A nasty issue to have!

I think it's clear that we need to make our select statement run as a loop somehow. Without playing hide and seek, let me tell you that the Go-idiomatic way of solving this is wrapping it with a for loop on top like this:

for {
  select {
    case item := <-deskChan:
        w.Process(item)
    case call := <-phoneChan:
        fmt.Printf("Worker %s received a call: %s\n", w.Name, call)
    }
}

However, there is a clear issue: we don't even need to run code to spot that this loop will run indefinitely, and there is no way to stop it. And let me tell you that Go has no dedicated language-level features to solve this. But fear not, we can fix this with what we already know.

Actually, there are 2 (or maybe even more, but I usually use these 2) idiomatic ways to solve this:

Create a boolean flag that will keep the for loop running while it is true. Once either of the channels is closed, set this flag to false and terminate the loop this way.
In the main function, create a new channel that will be notified once the select statement has to finish its execution. Pass this channel to the method with the select statement and introduce a new case for it. If there is an input to that channel, break the loop. A real-life representation of this approach is like setting a timer/an alarm: once it rings, the postman knows it's time to call it a day.

The 1st approach (the boolean flag one) looks like this:

func (w *Worker) StartWorkingDay(deskChan chan string, phoneChan chan string) {
    keepRunning := true
    for keepRunning {
        select {
        case item, ok := <-deskChan:
            if ok {
                w.Process(item)
            } else {
                keepRunning = false
            }
        case call, ok := <-phoneChan:
            if ok {
                fmt.Printf("Worker %s received a call: %s\n", w.Name, call)
            } else {
                keepRunning = false
            }
        }
    }

    fmt.Println("the desk is closed - time to go home")
}

If we run our application now, we'll see that the deadlock issue is gone, and the output is the way it should be:

Lisa gives away basketball
Worker Bob received a call: Has my package arrived?
Worker Bob received basketball
Worker Bob started processing basketball...
Eric gives away teddy bear
Worker Bob processed basketball

Worker Bob received teddy bear
Worker Bob started processing teddy bear...
Jenny gives away watermelon
Worker Bob processed teddy bear

Worker Bob received a call: What about now?
Worker Bob received watermelon
Worker Bob started processing watermelon...
Ben gives away box
Worker Bob processed watermelon

Worker Bob received box
Worker Bob started processing box...
Zlatan gives away football
Worker Bob processed box

Worker Bob received football
Worker Bob started processing football...
Worker Bob processed football

the desk is closed - time to go home

The 2nd approach (a new channel) will look a bit different code-wise:

func (w *Worker) StartWorkingDay(deskChan chan string, phoneChan chan string, shutdownChan chan struct{}) {
    for {
        select {
        case item := <-deskChan:
            w.Process(item)
        case call := <-phoneChan:
            fmt.Printf("Worker %s received a call: %s\n", w.Name, call)
        case <-shutdownChan:
            fmt.Println("the desk is closed - time to go home")
            return
        }
    }
}

We also need to introduce shutdownChan in the main function and send a signal to it at some point:

func main() {
    deskChan := make(chan string)
    phoneChan := make(chan string)
    shutdownChan := make(chan struct{})
    wg := &sync.WaitGroup{}

    bobWorker := Worker{Name: "Bob"}

    wg.Add(1)
    go func() {
        bobWorker.StartWorkingDay(deskChan, phoneChan, shutdownChan)
        wg.Done()
    }()

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    go func() {
        phoneChan <- "Has my package arrived?"
        time.Sleep(1 * time.Second)
        phoneChan <- "What about now?"
    }()

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    shutdownChan <- struct{}{}

    close(deskChan)
    close(phoneChan)
    close(shutdownChan)

    wg.Wait()
}

If you find the type of the shutdownChan a bit unusual - struct{} - this is a Go style of specifying the channel in which data will be ignored. As in our case, we don't care about the data being sent to this channel - all we need is that someone triggered it so we can use it as a signal to shut down the execution.

If we run this code, we'll see that it works as expected - good job! Triss is happy, and our customers are happy, but what about the postman, Ben? Well, we can't make everyone happy, can we?!

I believe this is a perfect moment to stop. We have learned a lot today, and now you know how to manage channels in a simple yet powerful way. I'm confident that you are more than ready to solve the post office issue finally we started talking about in Part 1 of this series. And that's what we are going to do in the next post, as well as turning our post office into a proper data streaming-like pipeline with respect to the working day hours.

Stay tuned, as you don't want to miss out on the end of this story. See you in Part 4, and in the meantime, have fun! =)

P.S. If you like my post and don't want to miss out the future ones (part 4 is coming soon), please, consider:

subscribing to my newsletter
following me on Dev.to
following me on Twitter/X
following me on Mastodon

Go concurrency simplified. Part 2: Syncing goroutines with `sync.WaitGroup`

Mykola — Mon, 11 Dec 2023 21:51:49 +0000

Hello there! Despite the beautiful snowy weather outside, I'm at home these days with covid, so I can dedicate some additional time to blogging.

Last time, we discussed the very basic concepts of Go concurrency: goroutines and channels. If you missed that post, please check it out here, it has some cool drawings =) Today, we'll move on and explore the ways Go offers us to sync goroutines - it will help us get rid of some hacky workarounds we have used so far.

Recap

Have you already forgotten about it? My silly drawing will help you to remember:

Also, while discussing channels, we realized that there are the following associations with the Go language here:

a post office desk -> Go channel
a basketball -> a value written to the channel
a customer -> a code that writes to the channel
a postman -> a code that reads from the channel

Here is the summary of that with another beautiful drawing authored by me:

We seem to have a good theoretical foundation of the goroutines and channels. But our post office code still relies on the sequential approach with a for-loop and synchronous processing within. Here is the recap of how it looks:

type Customer struct {
    Name string
    Item string
}

func (c *Customer) GiveAway() string {
    item := c.Item
    fmt.Printf("%s gives away %s\n", c.Name, item)
    c.Item = ""
    return item
}

type Worker struct {
    Name string
}

func (w *Worker) Process(item string) {
    fmt.Printf("Worker %s received %s\n", w.Name, item)
    fmt.Printf("Worker %s started processing %s...\n", w.Name, item)

  // UPDATED: switched from 1 minute to 1 second to reduce the execution time
    // to simulate long processing
    time.Sleep(1 * time.Second)

    fmt.Printf("Worker %s processed %s\n\n", w.Name, item)
}

func main() {
    bobWorker := Worker{Name: "Bob"}

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    for _, customer := range queue {
        item := customer.GiveAway()
        bobWorker.Process(item)
    }
}

This and other code examples are available in this GitHub repo.

Running this code gives the following output:

Lisa gives away basketball
Worker Bob received basketball
Worker Bob started processing basketball...
Worker Bob processed basketball

Eric gives away teddy bear
Worker Bob received teddy bear
Worker Bob started processing teddy bear...
Worker Bob processed teddy bear

Jenny gives away watermelon
Worker Bob received watermelon
Worker Bob started processing watermelon...
Worker Bob processed watermelon

Ben gives away box
Worker Bob received box
Worker Bob started processing box...
Worker Bob processed box

Zlatan gives away football
Worker Bob received football
Worker Bob started processing football...
Worker Bob processed football

Nice! We are in a good position to start improving our code.

Concurrency time

That was supposed to sound like "Adventure Time", if you were wondering =)

As we can see, the sequential parcels' processing algorithm is pretty straightforward:

customer gives away an item which is persisted into the variable: item := customer.GiveAway()
this item is passed to the postman for processing: bobWorker.Process(item)

As a first step to the concurrent approach, let's introduce a few changes in between those two:

customer will put the item into the channel instead of a variable
postman will get the item from the channel rather than directly from the variable

The only missing part is the channel, but we know how to create it from the previous article in this series. Here is what the modified main method looks like:

func main() {
    deskChan := make(chan string, 1)

    bobWorker := Worker{Name: "Bob"}

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    for _, customer := range queue {
        deskChan <- customer.GiveAway()

        item := <-deskChan
        bobWorker.Process(item)
    }

    close(deskChan)
}

The same result as before is printed to the terminal.

One interesting moment to notice is make(chan string, 1). As you see, we explicitly specified the capacity for the channel as 1. This is an important moment I skipped in Part 1 of this series for simplicity's sake, but it's the right time to discuss this now.

Go channels capacity

In the Go language, the channel capacity shows the number of elements you can send to the channel without blocking the send operation. Basically, this means the following:

if the capacity is 0, the send operation will block the current goroutine until the other goroutine reads from this channel
if the capacity is 1, it is possible to send 1 element into the channel without blocking the goroutine
if the capacity is n, it is possible to send n elements into the channel without blocking the goroutine

The default capacity is 0, meaning that make(chan int) creates a channel with a capacity of 0.

Let's try to show this in action with a piece of code to make it even more clear. Let's start with the 0 capacity channel:

func main() {
    zeroCapacityChan := make(chan string)

    zeroCapacityChan <- "a"

    received := <-zeroCapacityChan
    fmt.Println(received)
}

Before running this code, let's try to predict what the result will be. Let's read the above statement about the 0-capacity channel again: the send operation will block the current goroutine until the other goroutine reads from this channel . Let's see what we have here: all the code runs in 1 (main) goroutine, which means that the zeroCapacityChan <- "a" will block it. So the only goroutine we have will be blocked forever. As we discussed in the previous post, this is a deadlock situation, and kudos to the Go compiler developers that make it detect situations like this one.

Let's run the code to see whether we were right or not:

fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan send]:
main.main()
        /n0rdy-blog-code-samples/20231210-go-concurrency-with-for-and-select/03-channels-capacity/capacity.go:8 +0x36

Good job! Let's change the code by providing capacity as 1 to fix the error we've experienced:

func main() {
    oneCapacityChan := make(chan string, 1)

    oneCapacityChan <- "a"

    received := <-oneCapacityChan
    fmt.Println(received)
}

And run it:

Works like a charm! I hope it makes sense now why we did deskChan := make(chan string, 1) in the post office example above. Let's get back to it.

Back to the post office

So, now our code uses a channel, so can we call it concurrent and add "Go concurrency" to our CV? Well, not really, it is still a sequential code that runs in one goroutine. However, since we are almost experts in goroutines now, let's tweak our code the way that:

we'll remove the postman code from the door loop and create a new function that will constantly be listening to the desk channel and receive items from it once available - we'll run this function into a separate goroutine
inside a for loop, the customer will the way it does now: deskChan <- customer.GiveAway()
since we are introducing a new goroutine, we can create a channel with the default capacity, as we are safe from the deadlock from now on

While steps 2 and 3 are pretty straightforward, let's focus on the 1st one. How come and why should the postman listen constantly to the service desk? Since we have already allowed ourselves to apply a lot of simplifications to the post office way of working, let's agree that the only responsibility of the postman there is to stand at the desk the whole day and serve the customers, if any. If there are no customers, the working day looks like this:

And we have already seen the drawings of the working day with the queue of clients.

Since we have established the responsibilities for the postman, let's add a new function StartWorkingDay to the Worker struct that receives a desk channel and constantly listens to it:

func (w *Worker) StartWorkingDay(deskChan chan string) {
    for {
        item := <-deskChan
        w.Process(item)
    }
  fmt.Println("the desk is closed - time to go home")
}

Since the Go language doesn't have a concept of while loops, for {} is a direct equivalent of the while (true) {} from other programming languages.

One problem with this code, though: this for loop will never finish, and the desk is closed - time to go home part will never be printed. This means that our postman's working day will last forever 😱

Let's try to help the poor guy, as I bet he definitely has something else to do outside the working hours. But how can we do that? Hmm...what do we know about the closed channels so far? In the previous part of the series, we mentioned this:

if the channel is closed, it is impossible to write into it (panic: send on closed channel), but the reader can get the existing value from the channel. If the reader keeps reading from the channel, it will receive the default values (e.g. 0 for int)

If the reader keeps reading from the channel, it will receive the default values is something that can be pretty useful for our use case. Empty string ("") is a default value for the string data type. What if we add a simple check there: if the received item is an empty string, can we assume that the desk channel is closed? This is what it's going to look like:

func (w *Worker) StartWorkingDay(deskChan chan string) {
    for {
        item := <-deskChan
        if item == "" {
            break
        }

        w.Process(item)
    }
    fmt.Println("the desk is closed - time to go home")
}

That will definitely help, so our postman won't need to work endlessly anymore. But let's stop and think for a moment: can we think of any edge cases here? For example, how are we supposed to distinguish the empty string sent by the closed channel from the empty string sent by the customer? Once any customer wants to send an empty parcel, our postman will treat it as a signal to call it a day - I don't think this will be that good for the post office business. We need a more comprehensive solution here.

There is one trick that I'd like to show you that can help us here. When I introduced channels to you in my previous post, I mentioned that this is the way to read a value from the channel:

value := <- ch

While this is true, there is one tiny addition we can make to this code if we'd like to know whether the channel was opened or closed the moment we read from it:

value, ok := <- ch

ok is a boolean value that is true if the channel is still open or false if it has already been closed. This will become very handy soon enough.

Let's use that knowledge to replace our error-prone workaround:

func (w *Worker) StartWorkingDay(deskChan chan string) {
    for {
        item, ok := <-deskChan
        if !ok {
            break
        }
        w.Process(item)
    }
    fmt.Println("the desk is closed - time to go home")
}

Time to adjust the main method and see whether this new approach works - we'll run the StartWorkingDay as a new goroutine:

func main() {
    deskChan := make(chan string)

    bobWorker := Worker{Name: "Bob"}
    go bobWorker.StartWorkingDay(deskChan)

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    close(deskChan)
}

If we run the code, we'll see the following output:

Lisa gives away basketball
Eric gives away teddy bear
Worker Bob received basketball
Worker Bob started processing basketball...
Worker Bob processed basketball

Worker Bob received teddy bear
Worker Bob started processing teddy bear...
Jenny gives away watermelon
Worker Bob processed teddy bear

Worker Bob received watermelon
Worker Bob started processing watermelon...
Ben gives away box
Worker Bob processed watermelon

Worker Bob received box
Worker Bob started processing box...
Zlatan gives away football
Worker Bob processed box

Worker Bob received football
Worker Bob started processing football...

That doesn't look right at all, as our application had exited before Bob managed to finish processing the football. And the the desk is closed - time to go home message has never been printed. For those who followed along with the previous post, it shouldn't be a big surprise, as we had exactly the same situation there when the main goroutine finished before the child one, and the execution was interrupted. Here is how it looked and how we fixed it back then:

func main() {
    go printNTimes("Hello there.", 5)
    printNTimes("General Kenobi. You are a bold one.", 5)

    // to prevent the program from exiting before goroutine finishes
    time.Sleep(1 * time.Second)
}

If you have a feeling deep within that the time.Sleep(1 * time.Second) way of fixing this issue is somewhat a dirty hack - good intuition, my friend! But let's use it anyway to make sure it solves the issue, but increase the waiting to 2 seconds, so it is enough:

func main() {
    deskChan := make(chan string)

    bobWorker := Worker{Name: "Bob"}
    go bobWorker.StartWorkingDay(deskChan)

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    close(deskChan)

    // to make sure that the worker has finished processing the last item
    time.Sleep(2 * time.Second)
}

The result looks as expected now:

Lisa gives away basketball
Eric gives away teddy bear
Worker Bob received basketball
Worker Bob started processing basketball...
Worker Bob processed basketball

Worker Bob received teddy bear
Worker Bob started processing teddy bear...
Jenny gives away watermelon
Worker Bob processed teddy bear

Worker Bob received watermelon
Worker Bob started processing watermelon...
Ben gives away box
Worker Bob processed watermelon

Worker Bob received box
Worker Bob started processing box...
Zlatan gives away football
Worker Bob processed box

Worker Bob received football
Worker Bob started processing football...
Worker Bob processed football

the desk is closed - time to go home

This is satisfying indeed, but not 100% for sure. As we mentioned above, the time.Sleep(2 * time.Second) part is a temporary workaround, not a proper solution. Can we improve that part? I bet we can! Before jumping into a solution mode, let's take a moment to think about why we need that workaround in the first place at all.

As you remember, the problem we solved by introducing this sleeping interval was the fact that the main goroutine finishes before the worker one, so the postman doesn't get enough time to complete his work. Like in real life: even if the working day ends at 17:00, it doesn't mean we should turn the lights on in the entire building immediately. This means that if there was a way to know whether the postman has done his work, we would keep the application running as long as needed. The most straightforward approach I can think of here is introducing a boolean flag that will be true if the postman work is finished and false otherwise. Let's do that.

type Worker struct {
    Name           string
    isDoneForToday bool
}

If you are new to Go and don't know the difference between the lowercase and uppercase fields' namings
If the name is uppercase - it means that the field is public and can be accessible from the outside of the current package, while the lowercase stands for private visibility.
By the way, the same approach applies to the functions' namings.

So far, so good! Let's make sure to set it to false once we call the StartWorkingDay and change it to true. Here is how it looks now:

func (w *Worker) StartWorkingDay(deskChan chan string) {
    w.isDoneForToday = false

    for {
        item, ok := <-deskChan
        if !ok {
            break
        }
        w.Process(item)
    }

    fmt.Println("the desk is closed - time to go home")
    w.isDoneForToday = true
}

With this in place, we need a way to use the isDoneForToday field to let the postman finish his job. Sounds like a room for a new function for the Worker struct:

func (w *Worker) WaitToFinish() {
    for !w.isDoneForToday {
    }

    fmt.Printf("Worker %s has finished work for today\n", w.Name)
}

As you can see, we have a for loop with the empty body that keeps running until the worker is done for today. We can safely do that, as another goroutine runs the StartWorkingDay function, which will change the isDoneForToday to true once the work has been completed - we have just implemented this part. This change will trigger the for loop to be over, so then the execution will be unblocked.

Time to replace the time.Sleep workaround with the call to our new function in a main function:

func main() {
    deskChan := make(chan string)

    bobWorker := Worker{Name: "Bob"}
    go bobWorker.StartWorkingDay(deskChan)

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    close(deskChan)

    bobWorker.WaitToFinish()
}

Running this code returns the same result as before - good job. However, it still feels a bit hacky, and if your intuition tells you that we might have just reinvented the wheel - that's a good intuition (or an experience)! The for !w.isDoneForToday {} part is definitely a workaround to make the code wait for a signal to proceed.

If you are ever in a similar need, before starting to google third-party libraries that might have solved this, a good rule of thumb is to check what Go has to offer as a part of the language. If your problem lies within the concurrency domain, start checking the sync package. And while there are several canonical ways to solve our problem, we'll focus on one exact solution for educational purposes - relying on sync.WaitGroup.

`sync.WaitGroup` - one tool to sync them all

So, what kind of creature is sync.WaitGroup? Here is what the official Go docs say about it:

A WaitGroup waits for a collection of goroutines to finish. The main goroutine calls Add to set the number of goroutines to wait for. Then each of the goroutines runs and calls Done when finished. At the same time, Wait can be used to block until all goroutines have finished.
A WaitGroup must not be copied after first use.

If we split it into the step-by-step instructions, it will look like this:

create a new wait group wg in the main goroutine
before starting each new goroutine, do wg.Add(1) to let the wait group know about the number of running goroutines in a group
pass a pointer to the wait group to each goroutine - I've highlighted the word "pointer" as this is a crucial moment, as otherwise, we'll pass the copy, which goes against the rule A WaitGroup must not be copied after first use. from the docs - it will simply not work the way it should
in the main goroutine call wg.Wait() in the place where you'd like to wait for the child goroutines to finish their tasks
in each of the child goroutines call wg.Done() once the work is done - it will reduce the counter of the running goroutines within the wait group

Basically, wg.Wait() will keep the main goroutine in the waiting stage until the counter is 0. That's why we should be careful while using sync.WaitGroup, as we might hang the program forever if we forgot to call wg.Done() for each wg.Add(1).

We'll use the printNTimes example from the previous article to try this with a code. Let's recap what it looks like:

func main() {
    go printNTimes("Hello there.", 5)
    printNTimes("General Kenobi. You are a bold one.", 5)

    // to prevent the program from exiting before goroutine finishes
    time.Sleep(1 * time.Second)
}

func printNTimes(s string, n int) {
    for i := 0; i < n; i++ {
        fmt.Println(s)
    }
}

The aim here is to get rid of the time.Sleep(1 * time.Second) part by replacing it with the sync.WaitGroup following the instructions above. We'll need a new function, printNTimesAsync, that will accept the pointer to the sync.WaitGroup on top of the other params. Also, we'll create an instance of the sync.WaitGroup in the main function and do the required actions with it:

func main() {
    wg := &sync.WaitGroup{}

    wg.Add(1)
    go printNTimesAsync("Hello there.", 5, wg)
    printNTimes("General Kenobi. You are a bold one.", 5)

    // to prevent the program from exiting before goroutine finishes
    wg.Wait()
}

func printNTimes(s string, n int) {
    for i := 0; i < n; i++ {
        fmt.Println(s)
    }
}

func printNTimesAsync(s string, n int, wg *sync.WaitGroup) {
    for i := 0; i < n; i++ {
        fmt.Println(s)
    }
    wg.Done()
}

If we run it, it works the way it should:

General Kenobi. You are a bold one.
Hello there.
Hello there.
Hello there.
Hello there.
Hello there.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.

Let's make this example a bit more complex and say that we'd like to repeat the printing part

go printNTimesAsync("Hello there.", 5, wg)
printNTimes("General Kenobi. You are a bold one.", 5)

10 times to show the same in scale.

func main() {
    wg := &sync.WaitGroup{}

    for i := 0; i < 10; i++ {
        wg.Add(1)
        go printNTimesAsync("Hello there.", 5, wg)
        printNTimes("General Kenobi. You are a bold one.", 5)
    }

    // to prevent the program from exiting before goroutine finishes
    wg.Wait()
}

func printNTimes(s string, n int) {
    for i := 0; i < n; i++ {
        fmt.Println(s)
    }
}

func printNTimesAsync(s string, n int, wg *sync.WaitGroup) {
    for i := 0; i < n; i++ {
        fmt.Println(s)
    }
    wg.Done()
}

I will not show the result of running this code here, as it will take too many lines of text, but you can try it yourself and see that it works well. Which makes sense, of course!

As of now, you should be pretty familiar with the concept of sync.WaitGroup, so let's apply it to our post office example.

These are the steps we are going to take here:

remove WaitToFinish function
create an instance of sync.WaitGroup in the main function and use it to control the execution of the worker goroutine
use wg.Wait() instead of the removed WaitToFinish function at the end of the main function.

This is what the updated main function looks like:

func main() {
    deskChan := make(chan string)
    wg := &sync.WaitGroup{}

    bobWorker := Worker{Name: "Bob"}

    wg.Add(1)
    go func() {
        bobWorker.StartWorkingDay(deskChan)
        wg.Done()
    }()

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    for _, customer := range queue {
        deskChan <- customer.GiveAway()
    }

    close(deskChan)

    wg.Wait()
}

If you are new to Go and don't understand the go func() {...}() construction
The func() {...} part declares the anonymous function - in this context, the way to invoke multiple lines of code as a part of the same goroutine.
The () part calls this function immediately.
We could have achieved a similar result by extracting the body of this function into a separate function and calling it here. But since we don't need that function for our business logic, I used this approach.

Running this code gives the same result as the previous version - nice work!

I believe this is a good place to stop for today. We learned a pretty important concept of Go today - sync.WaitGroup - that you are going to use a lot once you start writing concurrent code on a daily basis. And we have managed to eliminate some hacks in our code - always a good achievement.

In Part 3 of the series, we'll talk about the ways to manage channels by using for and select statements, which will get us closer to the solution to the queue situation in the post office. That's why stay tuned, and in the meantime, have fun! =)

P.S. If you like my post and don't want to miss out the future ones (parts 3 and 4 are coming soon), please, consider:

subscribing to my newsletter
following me on Dev.to
following me on Twitter/X
following me on Mastodon

Go concurrency simplified. Part 1: Channels and goroutines

Mykola — Fri, 08 Dec 2023 19:17:43 +0000

Christmas season is around the corner, that's why another evening I was standing in a long queue at the post office with some Xmas presents packed inside the box. The line moved pretty slowly, as there was only one postman for the whole crowd of customers. The guy was running back and forth, and I felt really sorry for him. Not sure why, either out of boredom or because of several long evenings I spent working on my open-source library for managing asynchronous pipelines, but my brain turned engineering mode on and tried to optimize the process of handling parcels.

Suppose we imagine the post office as an application. In that case, it becomes clear that we are dealing with the classical "consumer-producer" problem, where customers like me are producers (because we bring boxes, letters, etc.), and the postman is a consumer of all these. And it's pretty easy to see that the system bottleneck is that there is only one consumer for N producers.

So, how can we improve this? Shall we increase the amount of consumers? Or should we open more postal windows? A little bit of both? And how can we map this into the application and code after all? These are good questions, so let's try to answer them in this series of blog posts.

A step aside

Let's take a short break before diving deep into the technical topic and discuss the context for this series of blog posts. As you might have noticed from the post title, the main idea is to examine the concurrency in the Go programming language. And even though I'd love to jump into such powerful concepts like select statements, for loops with channels, WaitGroup, etc., this will be unfair to the readers who are not that familiar with Go but still would like to follow along to explore and learn the power and beauty of it. That's why we'll take it slowly here and start with the basics.

Wait a minute! Why the heck are we talking about Go in the first place? Well, this is a fair question. I promise you that my blog won't be only about any specific language, framework, or tool but rather about software engineering and tech in general. However, to discuss some topics/concepts, I'll need to pick a language or a tool to use, and I believe that this time Go is the right choice. Why? Google designed this language to be both powerful and simple, and while I wouldn't claim it to be the best language out there (I don't believe in the concept of "the best" at all, by the way), it is a good one to have in the toolbox - for instance, it's my language of choice for writing the CLI applications. Also, it has a clear syntax that should be easy to understand if you are familiar with any C-like language.

However, while Go is pretty straightforward, some parts of the language might not "click" from day 1. And concurrency is one of those. Therefore, I'll use real-life examples alongside the code to guide you all the way to the "aha" moment. Let's jump into it!

Back to the post office

Welcome back to the post office! The queue is still here, so the problem hasn't solved itself - surprise-surprise. But you know what - it's time to represent the situation we have with a code. Let's start with the models' definitions:

type Customer struct {
    Name string
    Item string
}

type Worker struct {
    Name string
}

This and other code examples are available in this GitHub repo.

If you are new to Go and don't know what struct is
Struct is a Go way to represent user-defined types in the form of a collection of fields. Thinks of them as Java/Python/Kotlin/C#/many other languages classes. If the field name starts with the uppercase letter, the field is public, while the lowercase ones are private.

Let's add some actions that those actors can perform:

func (c *Customer) GiveAway() string {
    item := c.Item
    fmt.Printf("%s gives away %s\n", c.Name, item)
    c.Item = ""
    return item
}

func (w *Worker) Process(item string) {
    fmt.Printf("Worker %s received %s\n", w.Name, item)
    fmt.Printf("Worker %s started processing %s...\n", w.Name, item)
    fmt.Printf("Worker %s processed %s\n", w.Name, item)
}

If you are new to Go and don't know what (c *Customer) is
It's a Go way to say: "This function belongs to the Customer struct. Use c to access a current instance of the struct within the function." In this context, think of c as Java/Kotlin/C# this or Python self keywords.

And now, it's time to combine all of this to recreate the post-office situation:

func main() {
    bobWorker := Worker{Name: "Bob"}

    zlatan := Customer{Name: "Zlatan", Item: "football"}
    ben := Customer{Name: "Ben", Item: "box"}
    jenny := Customer{Name: "Jenny", Item: "watermelon"}
    eric := Customer{Name: "Eric", Item: "teddy bear"}
    lisa := Customer{Name: "Lisa", Item: "basketball"}

    queue := []Customer{lisa, eric, jenny, ben, zlatan}

    for _, customer := range queue {
        item := customer.GiveAway()
        bobWorker.Process(item)
    }
}

If you are new to Go and don't know what []Customer is
It's a Go way to specify a slice of Customers. Thinks of slice as Java/Kotlin/C# ArrayList, Python list, or dynamic arrays in general.

If you are new to Go and don't know what _ is
It's a Go way to ignore the result of the function. Go compiler is pretty strict and throws a compilation error if there are unused variables within the codebase.
range returns two results: the current index and the current element of the slice. In this context, we don't need an index. That's why we use _ to make the compiler happy.

If, after looking at this code, you find it silly - it is silly indeed. However, it is done this way to simplify it and focus on the post's main topic.

Once we run it, the result will be like this:

Lisa gives away basketball
Worker Bob received basketball
Worker Bob started processing basketball...
Worker Bob processed basketball

Eric gives away teddy bear
Worker Bob received teddy bear
Worker Bob started processing teddy bear...
Worker Bob processed teddy bear

Jenny gives away watermelon
Worker Bob received watermelon
Worker Bob started processing watermelon...
Worker Bob processed watermelon

Ben gives away box
Worker Bob received box
Worker Bob started processing box...
Worker Bob processed box

Zlatan gives away football
Worker Bob received football
Worker Bob started processing football...
Worker Bob processed football

If you followed along and ran this program, you might have noticed that the execution was done in no time. So, no problem then, right? Well, not really. I bet you have visited the post office in your life. And the chances are that you stayed in the queue, and maybe even in a quite long one. Then you know that the real-life processing of the parcels the customer would like to send or receive is way more complicated than this piece of code:

func (w *Worker) Process(item string) {
    fmt.Printf("Worker %s received %s\n", w.Name, item)
    fmt.Printf("Worker %s started processing %s...\n", w.Name, item)
    fmt.Printf("Worker %s processed %s\n", w.Name, item)
}

Depending on the digitalization level of the post offices in the country of your residence, once you pass the parcel to the worker there, the following might happen:

they might request your ID
they might need to type your and the parcel's info into their system
they might scan the QR code attached to the parcel
they might need to "parse" your handwriting on a parcel and type it into the system
if you brought the item without the package, they might need to put it into a box or an envelope
you might need to sign some documents
you might need to pay for their service
you might have some questions for them
and so on and so forth

I'm trying to say here that the processing of one customer might take a while, so if we want to be 100% precise in the code, it will be fair to add some waiting time between the "started processing" and "processed" stages. Let's assume that the average processing time is 1 minute, then the code would look like this:

func (w *Worker) Process(item string) {
    fmt.Printf("Worker %s received %s\n", w.Name, item)
    fmt.Printf("Worker %s started processing %s...\n", w.Name, item)

    // to simulate long processing
    time.Sleep(1 * time.Minute)

    fmt.Printf("Worker %s processed %s\n\n", w.Name, item)
}

As you can see, the only change we made was adding a time.Sleep(1 * time.Minute) part, which suspends the function execution for 1 minute. Obviously, if we run our code once again, it will take around 5 minutes for it to complete. And it is despite the fact that 1 minute is quite a generous estimate, as we could have easily picked up 5 minutes, and that would still be fair - the total execution would be around 25 minutes. That's far too long!

I hope the challenge we have is crystal clear as of now. How can we solve it?

Possible solutions

In an ideal world, the solution will be super simple: to have 1 worker per 1 customer. Then, in our scenario, if the average processing time is 1 minute, it will take exactly 1 minute to handle the entire queue. Impressive, right?

Well, it's pretty doable with 5 customers, like in our example, but what if the queue is 10 people long? What about 50? 100? It's obvious that in the real world, there are limits: money, time, space, etc., that's why the post office (or any other business, for that matter) won't hire that many employees to have a 1-1 ratio with the customers. What are the alternatives?

If we are limited by the number of workers, let's consider improving the quality of work. What if we try to reduce the amount of work the worker needs to do with the customer by doing the rest of the work later or by delegating to colleagues? Let's imagine that all the worker has to do is scan the QR code on the parcel, compare the photo from the system with the person standing in front, and pass the parcel to the other colleague - the customer can leave right away, as the rest of the activities will be done without their presence. Cool!

If you have an experience in programming, I bet you have already recognized this pattern - asynchronous execution. The beauty of such an approach is that the resources (the customer in our case) can be released fast, as the processing happens in a background fashion.

Concurrency vs Parallelism
If you want to dive deeper into this topic, there is no way that I can present it better than Rob Pike (one of the Go creators) did: https://go.dev/blog/waza-talk

We now clearly understand how to solve the challenge we faced, but before jumping into the code, we need to discuss Go concepts for asynchronous/concurrent programming. As the post title suggests, we'll explore goroutines and channels today.

Goroutines

A Tour of Go defines goroutines as

a lightweight thread managed by the Go runtime

The word "thread" suggests that if some part of the code is wrapped into the goroutines, the execution won't be sequential.

If this doesn't make sense - fear not; let's try to see that via the simple code example. Here is a printNTimes function that does what the name suggests: prints a string the number of requested times:

func printNTimes(s string, n int) {
    for i := 0; i < n; i++ {
        fmt.Println(s)
    }
}

And let's call it from the main function to see how it works:

func main() {
    printNTimes("Hello there.", 5)
    printNTimes("General Kenobi. You are a bold one.", 5)
}

If we run this code, the output is as expected:

Hello there.
Hello there.
Hello there.
Hello there.
Hello there.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.

This is the sequential order of execution we discussed above, meaning that there is a defined order:

printNTimes("Hello there.", 5) runs and finishes its execution
printNTimes("General Kenobi. You are a bold one.", 5) runs and finishes its execution
the application finishes the execution

Let's try to run one of the printNTimes as a goroutine to see a difference. But wait a minute, how can we do that? Luckily, it's super-duper simple in Go - the only thing we have to do is to add a go keyword before the function call:

func main() {
    go printNTimes("Hello there.", 5)
    printNTimes("General Kenobi. You are a bold one.", 5)
}

If you try to run this code, you might get different results. For instance, my first 5 attempts to run got this output:

General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.

I tried to rerun it and got this:

General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
Hello there.

And even this:

General Kenobi. You are a bold one.
Hello there.
Hello there.
Hello there.
Hello there.
Hello there.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.

What's going on here? Why is the code wrapped into a goroutine so unpredictable and sometimes even not executed? These are good questions, and if you are familiar with concurrency/parallelism/multithreading from any other programming languages, you know the answer already, so feel free to scroll down ahead. For the rest, let's dive into this.

We discussed above that the goroutines are the lightweight threads managed by the Go runtime, and we need to use the go keyword to start a new one. However, there is one edge case that we didn't mention: if you run the Go application (or rather the main function, to be more specific), the Go runtime creates 1 goroutine and runs the app within it - it is called a main goroutine. Then, if we do go printNTimes("Hello there.", 5), the Go runtime creates a new goroutine which is a child (or a branch) from the main one. And here is the catch:

if the child goroutine is done before the main one - it's OK, the app execution goes on
if the main goroutine is done, the app doesn't wait for the child ones to be completed, but exits the execution

If this sounds too complex, let me use a simple example from real life. I'm a happy owner of the PlayStation game console and a remote controller for it. When I turn my console on (like running a main function and a main goroutine), PS starts. Then I have to explicitly turn my remote controller on and connect it to the console (like running a child goroutine). If I turn my remote controller off in the middle of the game (or it gets disconnected/runs out of power), the console will keep running as before (see the 1st statement above). But if I turn my PS off, my remote controller will stop working properly immediately (see the 2nd statement).

And this is precisely what happens in our code:

when we run the app, the main goroutine starts
go printNTimes("Hello there.", 5) starts a new goroutine - please, note it takes some time to start it
while the Go runtime creates, initiates, and runs a new goroutine, the execution goes on, and the printNTimes("General Kenobi. You are a bold one.", 5)
the rest depends on many OS and hardware factors, but the possible outcomes are the following:
- the printNTimes("General Kenobi. You are a bold one.", 5) (and the app itself) finishes before the new goroutine starts - then there is no "Hello there" printed to the console
- the printNTimes("General Kenobi. You are a bold one.", 5) (and the app itself) finishes in the middle of the child goroutine execution - then there are less than 5 "Hello there" messages printed to the console.
- the printNTimes("General Kenobi. You are a bold one.", 5) (and the app itself) finishes after the child goroutine has finished its execution - then all the "Hello there" messages are printed to the console.

I believe it should make good sense now.

How can we fix that, though? We need to make the main goroutine wait a bit for the child to finish. There are elegant ways to achieve that, which we'll discuss in future posts. But we'll start with the simple one by forcing the main goroutine to wait for a dedicated amount of time (for example, 1 second) before exiting.

func main() {
    go printNTimes("Hello there.", 5)
    printNTimes("General Kenobi. You are a bold one.", 5)

    // to prevent the program from exiting before goroutine finishes
    time.Sleep(1 * time.Second)
}

time.Sleep(1 * time.Second) is the code that will do nothing (sleep) for 1 second.

If we run this code, we should see that all the 10 messages are printed:

General Kenobi. You are a bold one.
Hello there.
Hello there.
Hello there.
Hello there.
Hello there.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.

The chances are that you might get a different order of messages printed on your machine. And if you rerun the app again, the order might or might not change. This is the nature of the concurrent execution when the order is not guaranteed but decided by the Go runtime.

It is possible to compare it with real life: imagine you got a task to write "Hello there" 5 times (starting each from a new line) in a Google Doc / Microsoft Word document. And your colleague got a similar task, but with the "General Kenobi. You are a bold one." message. Both of you will have to use the same document to do that. Most likely, the result won't be like that if you both start writing at the same time:

Hello there.
Hello there.
Hello there.
Hello there.
Hello there.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.

But rather this:

Hello there.
General Kenobi. You are a bold one.
Hello there.
Hello there.
General Kenobi. You are a bold one.
Hello there.
Hello there.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.
General Kenobi. You are a bold one.

Or any other non-sequential combination depends on many factors like typing speed, copy-paste skills, internet connection speed/stability, etc. The same applies to the concurrent code.

Now we have learned the very basics about the goroutines, and we can create them. It is a good time to jump to another important Go concept - channels.

Channels

Why do we need a new concept alongside the goroutines? The thing is, while goroutines are pretty powerful, they have some limits. And one of them is that it is impossible to return any value from the goroutine. This means that the code like this won't compile:

func main() {
    result := go add(1, 2)
    fmt.Println(result)
}

func add(a int, b int) int {
    return a + b
}

The error is:

./main.go:6:12: syntax error: unexpected go, expected expression

This limit makes sense if we think about it for a moment: the code inside the goroutine is executed asynchronously, meaning that the main program continues its execution. In that case, when the go add(1, 2) code runs, there is no result right away - it will be but some time in the future. And there is nothing to assign to the variable result := and nothing to print in the following line.

Different languages come up with various workaround for that: JVM ones (Java, Scala) introduced a concept of Future-s, while JavaScript/TypeScript operates with Promise-s and C# has Task-s. The idea behind these approaches is the following:

the asynchronous code returns an instance of Future with the desired type within (int in our case)
the Future doesn't contain the value right away, but it will once the async execution is completed
it is possible to request the value from the future by calling its get() method - this will stop the program execution and wait until the result is ready

Our example in Java will look like this:

public class Main {
    public static void main(String[] args) throws ExecutionException, InterruptedException {
        Future<Integer> future = CompletableFuture.supplyAsync(
                () -> add(1, 2)
        );

        System.out.println(future.get());
    }

    private static int add(int a, int b) {
        return a + b;
    }
}

And 3 is printed to the console.

Go creators decided to choose another path by introducing a concept of channels, as a place to put the value once the async execution is completed.

If we reference A Tour of Go once again, it says that channels are

a typed conduit through which you can send and receive values with the channel operator, <-.

and the following example is provided:

ch <- v    // Send v to channel ch.
v := <-ch  // Receive from ch, and assign value to v.

The only missing part is how to create channels. Luckily, it's pretty straightforward:

ch := make(chan int)

where:

chan the built-in channel data type
int the data type of the elements that will be put into the channel

If we rewrite our code, it will look like this:

func main() {
    ch := make(chan int)
  defer close(ch)

    go add(ch, 1, 2)

    result := <-ch
    fmt.Println(result)
}

func add(ch chan int, a int, b int) {
    ch <- a + b
}

As expected, 3 is printed into the console.

But how does this code work? To answer this question, let me take a step aside for the moment and go back to the post-office example to explain this. I hope you remember my drawing (or should I say a masterpiece) from there:

Such a beauty! But let me get back to business: we are interested now in the rightmost side of it, where the customer puts a basketball onto the post office desk, and the post worker takes it from there. This is a good representation of the channels, and here is the same part of the picture but close with the arrows and some text descriptions:

While this is a straightforward transaction, there are some important concepts behind it:

if there is nothing on the post office desk, the postman has nothing to take from it, so they just wait
there is a room for only 1 basketball on the post office desk
if there is something on the post office desk, the customer can't put their stuff there, so they just wait
once the customer puts the basketball on the desk, the postman picks it up reasonably fast - but it's not guaranteed that this will happen right away
if the post office desk is closed, it is not possible to put new items onto it, but the postman can get the item from the desk if there is something left

The same rules apply to the Go channels:

if the channel is empty and the code tries to read from it, the execution is blocked, and the program waits until there is an item in the channel
make(chan int) creates a channel that has room for only 1 item of type int (we'll discuss further how to create channels with larger capacity)
if the channel has an item within, the code that tries to put a new value into it is blocked until the channel is empty again
once the item is in the channel, the code that reads from it will pick it up soon
if the channel is closed, it is impossible to write into it (panic: send on closed channel), but the reader can get the existing value from the channel. If the reader keeps reading from the channel, it will receive the default values (e.g. 0 for int)

I hope this part is clear, so now we are ready to take a look at the code from the above again:

func main() {
    ch := make(chan int)
  defer close(ch)

    go add(ch, 1, 2)

    result := <-ch
    fmt.Println(result)
}

func add(ch chan int, a int, b int) {
    ch <- a + b
}

Let's go through it step-by-step to see what happens here:

ch := make(chan int) creates a new channel of the type int with the default capacity of 1
defer close(ch) tells the program to close the channel once the main function is completed - that's a bit outside of the scope of this post, but TLDR: even though not necessary, it is a good practice to close the channels after they are not needed anymore.
go add(ch, 1, 2) wraps a call to the add function into a new goroutine and passes the channel from the previous step into the function
inside the add function, the result of a + b is written to the channel via the ch <- piece of code
result := <-ch code blocks the execution until the go add(ch, 1, 2) is completed and the result is passed to the channel; once the channel has the result from the add function, the result := <-ch piece of code reads it and assigns to the result variable
fmt.Println(result) prints the result to the console.

Unlike the code from the section where we introduced goroutines, this one doesn't need any tricks like time.Sleep() to get the result before the app is finished. This is the beauty of the <-ch reading from the channel operation that waits for the result.

There is one last thing I'd like to discuss here before I wrap this post up - channels with a capacity larger than 1.

Buffered channels

That's what they called. To create such a channel with, let's say, capacity for 2 items, you have to do this:

ch := make(chan int, 2)

This is how we can use them:

func main() {
    ch := make(chan int, 2)
    defer close(ch)

    ch <- 1
    ch <- 2

    result1 := <-ch
    result2 := <-ch

    fmt.Println(result1)
    fmt.Println(result2)
}

You should see such results in the terminal:

1
2

As you can see, since the channel has capacity for 2 elements, we could do this:

ch <- 1
ch <- 2

without having anyone to read from there. If we try to do the same with the channel of the default capacity, the code will never finish but will rather wait forever:

func main() {
    ch := make(chan int)
    defer close(ch)

    ch <- 1
    ch <- 2

    result1 := <-ch
    result2 := <-ch

    fmt.Println(result1)
    fmt.Println(result2)
}

so the Go compiler will throw an error:

fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan send]:
main.main()
        /n0rdy-blog-code-samples/20231204-go-channels-and-goroutines/05-buffered-channels/bufferedchans.go:23 +0x71

By the way, kudos to Go that its compiler detects deadlocks and fails on them rather than silently keeping the code in the blocked state forever.

fatal error: all goroutines are asleep - deadlock! means exactly what we have predicted: ch <- 2 tries to write to the channel that is already full -> the code waits -> the goroutine falls asleep -> since we have only 1 goroutine (main), it means that all the goroutines are asleep -> deadlock

To understand the concept of the buffered channels, let's get back to our post-office example again and take a look at this picture:

As you can see, the post office desk has become larger and has room for 3 items simultaneously. It means 3 customers can put their stuff onto the desk and leave, so the next ones in the queue can proceed. However, there is still only 1 post office worker, so that they will take the items one by one in the first-in-first-out (FIFO) order.

The similar rules apply here as for the channel with the default capacity:

if the channel is empty and the code tries to read from it, the execution is blocked, and the program waits until there is an item in the channel
make(chan int, n) creates a channel that has room for only n items of type int
if the channel has n items within, the code that tries to put a new value into it is blocked until the channel has a free capacity again
once the item is in the channel, the code that reads from it will pick it up soon in the FIFO order
if the channel is closed, it is impossible to write into it (panic: send on closed channel), but the reader can get the existing values from the channel. If the reader keeps reading from the channel, it will receive the default values (e.g. 0 for int)

So, does this mean that the buffered channels are better than the default ones? Well, not really: as usual in software engineering (or in life in general), the answer is "it depends". We'll dive deeper into that later. However, even the add example we used today clearly shows that there is no need to have a buffered channel there since we expect to get only 1 value.

We did a good job today, and there is quite a lot of info to digest. Still, you should have a good understanding of the goroutines and channels (on top of my drawing skills) now, so we'll dive deeper into this topic next time and go through the built-in constructs to operate channels and goroutines, and get closer to the solution of the post-office long queue problem.

In the meantime, thank you for your time (that was quite a journey), and have fun! =)

P.S. If you like my post and don't want to miss out the future ones (part 4 is coming soon), please, consider:

subscribing to my newsletter
following me on Dev.to
following me on Twitter/X
following me on Mastodon