DEV Community: Joseph Matthias Goh

Kubernetes in Five Minutes

Joseph Matthias Goh — Mon, 30 Nov 2020 13:59:09 +0000

Also available on my Medium

A self-inflicted challenge because I heard people dig Kubernetes in 2020

“If you can’t explain it simply enough, you don’t understand it well enough”.

Challenge accepted. Here’s my attempt to explain native components of Kubernetes in five minutes with the objective of covering concepts and resource types that will enable you to ~~talk about Kubernetes like you understand it~~ understand Kubernetes and deploy applications at a basic level.

As a pretext, I’ve found it useful to think of Kubernetes-related work not as a collection of verbose YAML files defining strange kinds of resources, but as a set of abstractions representing common design patterns in application deployment.

To me, the areas that Kubernetes natively accounts for in logical — imo — order are:

Hardware – Node
Orchestration – Deployment, Job, CronJob, StatefulSet, DaemonSet
Configuration – ConfigMap, Secret
Persistence — PersistentVolumeClaim, PersistentVolume
Execution – Pod, ReplicaSet
Access Control – Namespace, ServiceAccount, Role, ClusterRole
Exposure – Service, Ingress

Hardware

All software has one fundamental dependency: a machine to run on. We need hardware whether it’s our own computers or someone else’s and this abstraction comes in the form of a Node resource type.

Node: Represents a logical machine/VM (eg. your computer, ec2 instance, a droplet).

Orchestration

After we’ve defined our hardware, we need a plan on how to deploy our application. Will it be a single instance? Do we need maybe two? When we need to update our application, should it be done one by one or all at once? Do we want it to be deployed across all our Nodes? Should it be run once every day at 3 AM? We need to orchestrate the deployment.

Deployment: Long-standing workloads (eg. a server)
Job: One-off workload (eg. a shell script/database migration task)
CronJob: Periodically-run one-off workload (eg. data sync task)
StatefulSet: Workloads that require a readable/writable data volume (hard disk) that persists beyond restarts and is not shared amongst other instances of the same application. Use cases include services that implement eventual consistency (eg. transactional databases)
DaemonSet: Workloads that should be distributed across all targeted Nodes (eg. log collectors, system monitors, security software)

Configuration

So we’ve got a plan, but applications tend to be a fickle bunch and also need to be told certain things at runtime instead of plan-time; Things such as which network interface to bind to, which port to listen on, which API keys to use et cetera. While these things can technically be hard-coded, best-practices of 2020 generally suggest they should not. So we need a way to configure our application at runtime.

ConfigMap: Stores configuration for mounting as environment variables or read-only files (eg. SERVER_PORT=8080)
Secret: Fundamentally the same but stored in Base64 encoded text and used for values that should not be stored in plaintext (eg. GITHUB_API_TOKEN=😱)

Persistence

Now what if our application handles files and requires a hard disk that it can read and write from across different versions and instances? We need our data to persist by providing our application’s system with a hard disk (AKA volume in technical terms).

PersistentVolume: A logical “hard drive” (eg. Seagate 2TB, EBS)
PersistentVolumeClaim: Binds a PersistentVolume to a Pod. This is more of a virtual construct: think of it as the act of mounting a hard drive. A PVC defines that intention to mount a hard drive and expose its filesystem for an application to use.

Execution

So we’ve defined how our application will be deployed and configured, and we’ve given it some hard disk space to use. All that’s left now is to execute the application.

Pod: One instance of your application (eg. npm start, go run ./...)
ReplicaSet: Maintains the desired count of your application instances.

Access Control

But what if someone accidentally introduced a nasty virus into our application (somehow)? In an enterprise environment with compliance teams nagging at the whole DevOps thingy, we need to ensure that applications can only access resources that it needs to access. We on the other hand, need access control.

Namespace: Defines a virtual boundary within a cluster for access control mechanisms to be implemented on top of. Think about these like browser tabs: an open Facebook tab shouldn’t know what you’re googling in another tab.
ServiceAccount: Defines a virtual user that can be assigned a Role or ClusterRole which contains a set of permissions scoped to a set of resources (eg. your user login on your machine)
Role: Defines namespace-scoped resource access permissions. Linked to a ServiceAccount with namespace-scoped access via a RoleBinding resource
ClusterRole: Defines cluster-wide resource access permissions. Linked to a ServiceAccount with cluster-wide access via a ClusterRoleBinding resource

Exposure

Finally, there’s no use of an application that no one can access. After we’ve made the application happy and it’s running as expected, we expose it to the rest of the network/public internet for users and other services to access it.

Service: Exposes a workload to the cluster. Think running npm start on a server application and your application being available on localhost:3000. Behind the scenes, your application is binding to a port on your computer. A Service defines that binding.
Ingress: Exposes a Service to outside the cluster. Ever tried asking your co-worker to access your localhost:3000? Chances are you’d have ended up using ngrok or a similar tunneling software. An Ingress is basically an ngrok that exposes your localhost:3000 so it becomes accessible to a larger network.

Linking It All Up —Some Common Deployment Patterns

They said a picture is worth a thousand words. So here’s some diagrams to visualise how all the above links up.

An HTTP-based API server

A periodic job

An eventually consistent database system

A cloud-monitoring agent

Whew!

I hope that was somehow useful for you. If it was, ❤️s are always appreciated for reach, and feel free to follow to get pinged when I publish similar-ish pieces.

If you would also like to support my writing with actual cash and if you have a paid Medium account, consider reading this article on Medium and giving some 👏s there <3

How we use Google Sheets API to Manage Our Notification Banners

Joseph Matthias Goh — Fri, 24 Apr 2020 10:52:39 +0000

If you've visited our site before, you'd have noticed that banner at the top of our page:

Or you might have noticed the inline banner linking to the #SGUnitedJobs virtual career fair site:

There was a time when we didn't have any of these, so what happened?

The Problem Domain

The header banner has been in-place for some time now, and it was created due to some needs:

We wanted a way to notify users about new features we've released
We wanted a way to notify users of scheduled downtimes because of systems that we're integrating with

More recently, to support whole-of-government efforts to reduce the impact on our workforce, the agency we're servicing - Workforce Singapore - decided to launch an #SGUnitedJobs virtual career fair. And there was a slight issue.

In the media release, the press were quoted the words "MyCareersFuture". This led to the general public landing on our site instead of the virtual career fair's site, resulting in a need to redirect users to the correct site.

This meant some updates to our landing page notification/updates elements:

The header banner needed to be clickable
An in-page element needed to be created

In addition to addressing users' problems, our solution also needed to accomodate use cases by our business elements, and they needed to be able to enable text-changes to be made without a new system deployment.

The Solution

We begin with the end.

Google sheets for us was a nice interface between business and technical elements. Business users could enter in information as it came along without interfering with the code-level development using a spreadsheet, with the spreadsheet being able to run custom-code to generate a JSON output that our system could consume.

"Wait, can Google sheets do that?" I hear you ask.

Setting up Google Sheets as a JSON API

Let's set up a simple Google Sheets that you can use as an API that's similar in nature to what we've done.

In MCF's backend, we consume data from Google Sheets, run validations on it before throwing it back out as a .json file which we push to our CDNs so that these can be consumed by our users without placing a load on our servers.

1. Create a new spreadsheet

Go to https://drive.google.com and create a new spreadsheet. Maybe create a table like:

ID	Name	Username	Email
1	Joseph	joseph	j@seph.com
2	Matthias	matthias	m@tthias.com
3	Goh	goh	g@h.com

PROTIP: The table above works like copypasta with Google Sheets if you copy it properly

2. Publish to the web

Go to the top navigation bar, and access File > Publish to the web. Confirm that Link is selected and select Sheet1. Change the type to Comma-separated values (.csv). Click the Publish button and say OK.

A link should be provided to you. Test it out by pasting it into the address bar of your page. A .csv file should be downloaded. Opening it up should reveal (if you've entered in the information as-if from above):

ID,Name,Username,Email
1,Joseph,joseph,j@seph.com
2,Matthias,matthias,m@tthias.com
3,Goh,goh,g@h.com

3. Making it JSON

We'll be inserting the JSON as values in the empty JSON Output column you've copied above using the Script Editor. In the header navigation menu, go to Tools* > **Script editor. A new Code.gs should be waiting for you.

Overwrite the generated code and paste in the following code:

function onEdit(e){
  const editedRow = e.range.getRow();
  const activeSheet = SpreadsheetApp.getActiveSheet()
  const sheetName = activeSheet.getName();
  switch (sheetName) {
    case "Sheet1":
      const jsonData = createUserJSON(activeSheet, editedRow);
      const cell = activeSheet.getRange(editedRow, 5);
      cell.setValue(jsonData);
  } 
}

function createUserJSON(sheet, editedRow) { 
  const userJSON = {
    id: sheet.getRange(editedRow, 1).getValue(),
    name: sheet.getRange(editedRow, 2).getValue(),
    username: sheet.getRange(editedRow, 3).getValue(),
    email: sheet.getRange(editedRow, 4).getValue(),
  }
  return JSON.stringify(userJSON);
}

Google Script is essentially JavaScript. Also, if you've changed your sheet name, you might want to replace the "Sheet1" in the switch-case branch to the name of your sheet.

The above code is triggered on an edit to the sheet. It checks whether the edited sheet is named Sheet1 and if it is, generates a JSON string and pastes it into the 5th column of the row being edited. Save it by going to File > Save. The name shouldn't matter.

Go back to your sheets and edit one of the existing values. On removing your focus from that cell, you should see a JSON value appear in the 5th column.

4. Consuming the JSON

Retrieve the link from the Publish to the web stage. We're going to use that to retrieve our JSON outputs. Your original link should look like:

https://docs.google.com/spreadsheets/d/e/2PACX-XXX/pub?gid=YYY&single=true&output=csv

Confirm you've done an edit so that the JSON appears and then do a curl to see what it looks like now. In your terminal:

curl 'https://docs.google.com/spreadsheets/d/e/2PACX-XXX/pub?gid=YYY&single=true&output=csv'

You should receive something similar to the following as your response:

ID,Name,Username,Email,JSON Output
1,Joseph,joseph,j@seph.com,"{""id"":1,""name"":""Joseph"",""username"":""joseph"",""password"":""j@seph.com""}"
2,Matthias,matthias,m@tthias.com,
3,Goh,goh,g@h.com,

We're almost there!

5. Tying It All Up

Now that we can generate the JSON, go ahead and update the other rows so that all of them have a value in the JSON Output column.

In this step, we'll link all the JSON Output values into a new sheet which we can use as the data source in a service.

Add a New Sheet by clicking the plus (+) symbol at the bottom left of the existing spreadsheet. Name it Output. We'll be writing the aggregated JSON Output to row 1 column 1 of this sheet.

Go to Tools > Script editor once again and modify the script there so that it looks like:

function onEdit(e){
  const editedRow = e.range.getRow();
  const activeSheet = SpreadsheetApp.getActiveSheet()
  const sheetName = activeSheet.getName();
  switch (sheetName) {
    case "Sheet1":
      const jsonData = createUserJSON(activeSheet, editedRow);
      const cell = activeSheet.getRange(editedRow, 5);
      cell.setValue(jsonData);
  }

  // > diff starts here
  const jsonOutputs = activeSheet
    .getRange("E2:E")
    .getValues()
    .filter((val) => val[0].length !== 0);
  const aggergatedUsers = [];
  jsonOutputs.forEach(function(value) {
    const user = value[0];
    aggergatedUsers.push(JSON.parse(user));
  });
  const outputSheet = SpreadsheetApp.getActive().getSheetByName('Output');
  const outputCell = outputSheet.getRange(1, 1);
  outputCell.setValue(JSON.stringify(aggergatedUsers, null, 2));
  // / diff ends here
}

function createUserJSON(sheet, editedRow) { 
  const userJSON = {
    id: sheet.getRange(editedRow, 1).getValue(),
    name: sheet.getRange(editedRow, 2).getValue(),
    username: sheet.getRange(editedRow, 3).getValue(),
    email: sheet.getRange(editedRow, 4).getValue(),
  }
  return JSON.stringify(userJSON);
}

What we just added basically retrieves the value of all rows in column E which in the sample sheet we created corresponds to the column JSON Output. It then transforms the retrieved values into an array object before converting it back into a string using JSON.stringify. The null, 2 arguments in the JSON.stringify call basically indicates to format the JSON in a human readable way (see JSON.stringify documentation on MDN if you're interested in what exactly it does)

After you've made the change, save the GoogleScripts project, head back to your sheet and go ahead make a modification to one of the columns to trigger an output to the Output sheet in your spreadsheet.

Go to File > Publish to the web once again, this time selecting the Output sheet and selecting Tab-separated values (.tsv) as the type. In the Published content and settings section, ensure that Output is also being published and that the checkbox with Automatically republish when changes are made is also checked.

Copy the provided link which look like (sensitive values are masked with XXX)

https://docs.google.com/spreadsheets/d/e/2PACX-XXX/pub?gid=YYY&single=true&output=tsv

To modify this such that it will always return just the first row and column, append a &range=A1 so that the final link looks like:

https://docs.google.com/spreadsheets/d/e/2PACX-XXX/pub?gid=YYY&single=true&output=tsv&range=A1

Let's test this out by running a curl with it which would be what your Request module would be doing if it calls this URL:

curl -vv 'https://docs.google.com/spreadsheets/d/e/2PACX-XXX/pub?gid=YYY&single=true&output=tsv&range=A1'

To test that it's valid JSON, we can pipe it to a tool called jq that can help us validate it:

curl -vv 'https://docs.google.com/spreadsheets/d/e/2PACX-XXX/pub?gid=YYY&single=true&output=tsv&range=A1' | jq '.'

We're done. Aren't we?

6. What if something goes wrong?

As with all code, things can and will go wrong during development, but the awesome news is that Google provides us with a page where we can view errors in our script and this can be found on the Code.gs page through the file navigation menu via View > Executions which should open the script's Google Apps Script dashboard.

Note that only calls to console.error go through to this dashboard.

Looks like we're done for real!

If you like what you just read, don't forget to leave some reactions/comments so we know this has been interesting for you- and do consider following us for more insights into tools we use and our development processes.

Cheers and till next time!

Conducting The Technical Interview: What does it take to conduct an awesome technical interview?

Joseph Matthias Goh — Fri, 17 Apr 2020 04:12:54 +0000

I've been conducting interviews for software engineers together with a couple of like-minded colleagues for some time now; and I thought it'd be nice to share my learnings and experiences from this journey of trying to be both a nice human, as well as a reliable gatekeeper for the team.

This piece that follows will try to provide some insights to an ever-improving attempt to answer the question "what does it take to conduct an awesome technical interview?" I'll be covering how the interview process for software engineers at my tribe and product team have evolved over the years, followed by some principles and methodologies which I've found to be constructive when on the interviewing panel.

The context of what follows will be within my experience interviewing software engineers at the Agile Consulting & Engineering (ACE) tribe, and eventually the MyCareersFuture (MCF) product team. We're all under Government Digital Services (GDS) aka Hive, the "digital transformation" department of GovTech, which is where you should be applying to should you want in (my direct email also follows after the article 😊)

This post is also published on Medium at this URL

And since I'm kinda-sorta-ish a fan of Simon Sinek, let's start with why I became interested in improving the interview process for engineers-

🤔 Starting with Why

"In organisations, people are our greatest assets."

Wrong.

People are not assets. People are literally the organisation. And how do people get into an organisation? The interview, of course!

The interview process when done with intent presents opportunities for both the applicant as well as the organisation:

The interview is an applicant's only reliable signal as to what they're really signing up for beyond the foosball tables and free-flow bars
The interview is the organisation's best-attempt at making an informed decision

With recent shifts in perception of technical expertise from being a cost-centre to being an enabler, the interview is as important as the fancy perks organisations seem to offer these days to attract the best-of-the-best.

First impressions go both ways, and it'd be a pity if we had a unicorn sitting in front of us, and they were the ones saying no thanks. It'd also be a greater pity if we had that unicorn sitting in front of us, and we couldn't recognise them.

To me, it was the initial culture at GDS which enabled the larger organisation to be able to reach the level of capability and community standing we are at today (just ask any software guy from the 80s what they thought about IDA); and sustaining culture requires dedicated intent: intent to hire for a team that inspires and challenges each other, intent to hire not just to fill a headcount or seem full-strength on paper.

I want to work with awesome people, and being part of the interviews and giving it my best-shot was one way I believed would help to sustain the culture I was enamoured with.

TODO: image

And we begin with how it all started…

🐒 Evolution of The Technical Interview

Episode I: The Algorithm Menace (circa 2016)

When I first came onboard, our interview process looked like something out of a 2010s tech-giant in Singapore. There was the pre-invitation dance with the hiring manager where I was like "government *cannot* be cool," and he was like "oh come see it for yourself!"; so I did one fine Tuesday at 9:30AM and I was like "oh indeed it is cool these days," and the rest is history.

"Please set aside three hours for this assignment," came the e-mail which contained a link to HackerEarth and one of those algorithmic technical assessments.

This was followed by a face-to-face chat where I met more senior members from the team that I'd be working with, a live-programming session to make sure I could actually code, a team-lunch for assessing culture-fit, and finally the boss-stage interview with the hiring manager and the HR team.
Fortunately I was deemed worthy and joined in September of 2016.

This interview format worked back then when we were at about 30 people and not in any rush to scale up.

Problems were soon exposed when the need to scale came upon us.

Episode II: Attack of the Applicants (Scaling up – circa '17-'18)

The GDS department started gaining exponential traction within the government with project requests coming in quicker than we could handle. We had to scale; and our hiring manager knew it. This resulted in many candidates being dropped into our Trello board "for (y)our consideration".

As we continued with the interviews as we'd always have, we observed three things that influenced us to want to make changes to the process:

Being proficient at algorithms wasn't indicative of overall engineering talent. While the algorithmic technical assessment was a good filter for technical and conceptual depth, it didn't assess for the kind of skills we needed on a daily basis which was more engineer-ish/systems-level type of work. We needed a more relevant way to assess how someone would fare doing what we do on a daily basis.
More experienced candidates tend to be filtered out by algorithmic technical challenges. As the team grew with the existing interview format, we noticed more junior-level developers and few seniors. This was perceived to be a problem because most brilliant and self-aware juniors seek growth - and there were few who had 'been there done that' to provide that guidance in our ranks. Also, who were we kidding if we were to claim that we needed these experienced people to know how to detect and reverse palindromes under data type constraints? We wanted to unlock capabilities that an experienced software engineer would expose the team to.
The answer for the problem was already online and one candidate had copied most of it. This became a problem because the candidates would stumble at the live-programming session after we had already invested a couple or more hours in them. And time wasn't on our side. We needed a methodology that would allow us to invest less time while providing a worthy challenge; and every algorithmic question that could possibly be discussed on coding sites was already out there.

One assessment I proposed and implemented (and which a few poor souls were subject to) was done in an attempt to change the strategy from testing to assessing. This came about in the form of a multi-page timed questionnaire that had questions across engineering fields that were applicable to us - stuff like how HTTP works, how backends/frontends interact, some trivia questions on JavaScript and Ruby (our only two languages back then) - with the objective of seeing which fields a candidate scored better in and assessing whether they would fit our team vis-a-vis be good enough.

It was a disaster.

People read things at different speeds; people felt a need to score 100% in all fields and felt stressed when they couldn't. Candidates on the overall gave us feedback that it was anxiety-inducing. Not the kind of assessment that would prove effective in screening given that we weren't allowing people to demonstrate their best efforts under reasonable stress.

Episode III: A Solution Awakens (circa '19)

After multiple other experiments like a room-booking system and a URL shortener (spoiler: both didn't last very long), we arrived at a palatable technical assessment that wasn't stressful for candidates, wasn't time consuming for us, involved more concepts we use on a daily basis, and which still retained the crux of an important aspect of algorithmic thinking.

Presenting, our TODO-finder:

What I personally like about this technical challenge is that it solves most - if not all - of the problems that surfaced as we scaled:

The challenge is more similar to work we do on an everyday basis and selects for expertise in areas that's relevant to us. It's more like a fizz-buzz for developers on a product team.
More experienced candidates who've forgotten all their CS algorithms would still be able to complete it since it relies on creative expression as much as technical expertise.
Assessment lies in the how and why, not in the what. This enables a spectrum of correctness over a graded one-dimensional score, removing the pressure to "get it right" and consequently the need to copy a "right" answer.

For me, point (3) was especially important because as with most problems we need to solve, there is no "right" answer. It's always about trade-offs and this allows for nice things like discussions about base assumptions made, system design based on past experiences, how/why code was structured in a certain way, why tests were designed this way et cetera - all of which are more useful to know than the thought process behind an algorithm that's probably been covered multiple times on "land that developer job"-type sites.

🚦 The Current Interview Process at MCF

Throughout the process of revising the technical assessment, we'd been more-or-less consistent with the end-to-end interview process, and here are the 4 stages of our interview process you'd go through if you're applying to MCF (hint hint):

#1: Courtship

Candidates typically come in through our tribe lead, Steven Koh (he has written a post on his processes), recommendations from/connections with our colleagues, and sometimes via the HR department.

Interesting resumes are floated to a Trello board where hiring representatives from the various product teams typically filter based on their current needs. After an applicant is selected for, contact is made by any member of the team who's interested.

#2: Technical Chops

For MCF, we use the afore-mentioned technical assessment. The team member initiating contact with the applicant confirms their desire to apply to us, and following which sends the link to them. Upon receiving the response, he/she will float it internally to the team to assess the quality of the assignment. Should it be of sufficient quality, the applicant gets invited to an onsite interview (virtual these days, but you get the idea).

Note: The technical assessment segment differs from team to team depending on contextual and perceived needs so don't be surprised if you're applying to a different product team and the process doesn't seem familiar!

#3: The Face-to-Face

Ah, the scary one with people.

Should an applicant reach this stage, we will arrange a 2–3 hour face-to-face interview with 3–4 working-level members of the team that ideally spans from junior to senior. This stage allows us to assess and be assessed by a candidate for culture-fit and primarily serves to introduce the team and its dynamics to them.

This is followed by a live-programming section where we work with applicants on their solution to the technical assessment. It's pretty much a creative process that involves exploration of the code architecture and design, and working with the candidate to change features/add new ones.

#4: The Final Countdown

Should a candidate be passed by the working-level members of the team, a final round is arranged with the hiring manager/HR/product team lead together with 1–2 other working-level members who were not in the initial face-to-face interview. This stage further assesses the candidate's culture fit and tends to introduce the larger organisation, GovTech, to them.
Once a candidate is through, we wait for them to join us about a month later-

And that concludes the hard process. The subsequent sections focus on the intricacies of the interview and what I've personally experienced and learnt from being part of the interviewers' panel.

⚓️ Interview Objectives

As with any interview, the objective of our interview process is to enable both sides of the table to make an informed decision. The optimisations we can undertake then follows as a (non-)trivial constraint problem: How can we spend as little time as possible to gain/provide as much data as possible so that we can make an informed decision?

🔦 Interview Heuristics

When it comes to the culture-fit chat with candidates, the mental model I've found most constructive considers three aspects of a candidate:

Culture
Capability
Cause

While definitely not a strict three-bucket rule, being able to broadly categorise conversations into these aspects has helped me with identifying areas that have/have not been covered so I could influence the conversation to expose data.

Keeping track of data exposed by either applicant or interviewers also provides me with a way to reason about the good vibes (or otherwise) that I'm getting about the person in front of me.

Culture

Culture can be said to be a collective and unspoken modus operandi in any organisation of humans. In the context of an interview, assessing for culture is similar to assessing a person's personality with the objective of making a best-guess effort at how someone would interact and grow with the existing team and how the team dynamics might change with his/her presence.

Some examples of questions/statements that expose data related to culture:

Tell us something you're passionate about.
What do you do during your free-time?
How do you think your ex-colleagues/project-mates would describe you?
A superior comes to you at 6pm telling you she needs something done by that day. How would you decide on what to do? (Editor note: this doesn't happen at MCF, but it wouldn't hurt to know!)
Assuming someone on the team tells you your recent code contributions suck. How would you address the situation? (Editor note: again, just so you know, this doesn't happen at MCF)

Capability

I hear you say, "but we already have a technical assessment?" Capability in the context of the non-technical segment of the interview refers to non-technical "special powers" a candidate might have that cannot be easily expressed on a resume.

These could be things such as level of self-awareness, clarity of communication, breadth of knowledge, cross-domain areas of knowledge, experience handling difficult situations/people et cetera.

Some examples of questions/statements that expose data related to capability:

How would you handle (conjure a socially difficult situation up, like...) two colleagues who couldn't agree on spaces VS tabs?
Could you explain the concept of (for example only...) "Public Key Infrastructure" like we were 5?
(Compare two things, like...) JavaScript VS Python, go.
Assuming you were a (some role, like...) solution architect, how would you begin designing an e-commerce system that (introduce quirky business contraint, like...) prides itself on having the "quickest delivery times"?

Cause

Cause in a nutshell is best expressed as a question, "what keeps you awake at 3am?". Beyond doing a good job and getting paid well, I think it's important to know why an applicant is doing what they do.

Knowing why an applicant is in the game enables a better decision to be made by allowing the organisation to get a sensing of the duration a person will stick around in a particular role, what career needs they might have, what challenges we can provide them, and assess how that all ties in with our overall capability needs at that point in time.

Some examples of questions/statements that expose data related to cause:

What problems do you want to solve?
Why do you specifically want to join (in our context...) GovTech instead of other tech companies?
You're applying to be a (the position, for example...) DevOps engineer, how do you see yourself growing in this role?
Imagine it's 2050, what would a typical day in your life be like?

💡 Interview Principles

Fear begets Obscurity

Interviews are anxiety-inducing by their inherent nature. There's a human in front of you who doesn't know anything about you at that point in time. Yet after an hour or two, they will need to decide whether you get to join the party. How obnoxious of them to think they know you!

Despite the injustice of it all, interviews are a necessary evil and probably the only way of attempting an informed decision on whether or not to bring a person on board. As interviewers, it is up to us to make the best of it with the objective of spending as little time as possible to gain as much data as possible, to make an informed decision.

Helping a candidate to feel at ease is one way to do so, and the trick lies in getting them to feel safe enough to be themselves.

Try to recall the last time you felt scared. Even if you're not the type to lie, it's likely that the desire to stay out of trouble would have caused the thought to cross your mind. Fear makes it conducive for people to lie/hide, so don't give the human in front of you a reason to fear.

In safety, we express more. In safety, we learn more about each other.

On implementing this:

Shake the interview process up. Anticipation and hope tends to lead to interviewers being imagined as firing squads. When an interview is "like an interview", this affirms the fear. Shake things up by introducing elements to the interview that applicants wouldn't expect and which would take their minds off the session being "an interview". One example I'll always remember is how Steven Koh engaged me in conversation about achieving parallel read-write accesses to relational databases over ping-pong (whut? ikr).
Show vulnerability. It's normal (even admirable) to want to show your best side to the interviewee. However, I've found that injecting conversations of how the team can fail and has failed (c'mon, no team is perfect), empowers the applicant and triggers off most engineers' problem-solving nature which take their minds off "the interview".
Shift the conversation from answers to opinions. I've found that phrasing questions in a way that requests for opinions over facts opens people up. Anyway, the point of the interview is to assess the candidate, not their knowledge. There's Google for cold hard knowledge.

Culture > Capability || Cause

I quote a recent message my colleague sent to me while discussing soft skills in engineering which I felt was expressed in a pretty eloquent manner.

Describing one of his previous professional stints (might sound a little like bragging but I pinky-promise it's not, and that this is what was said),

"MCF seems to be interested in getting the job done, whether the job is actual delivery feature or sorting out internal processes. XYZ however, feels like people are just trying to please whoever has authority. we have delivery managers accepting any deadline thrown at them without discussing with the team, we have tech leads choosing tech stack because the product owner wanted it. having bad culture doesnt [sic] just make it depressing to work but it actually makes work less effective"

What makes culture so important to get right is that it's based on the collective, perceived and unspoken modus operandi of the individuals in any group of humans.

Maintaining a constructive culture is a tough balancing act, with one unsavoury apple potentially spoiling the lot (whatever 'spoil' means in your context) either through direct or indirect interventions.

As an interviewer, I've found it's also important to note that what works for one team may not work for another and it's crucial that interviewers assess this before the interview based on their own context. The afore-mentioned description of the other team didn't work for my colleague, but it might have worked for others who enjoy the pressure/prefer to leave the decision of technologies to others.

There is no right or wrong in culture, only constructive to the objective, or otherwise.

On implementing this:

Start conversation topics with culture-category questions. Capabilities and cause can be exposed through continuing the conversation on how they achieved it and why they decided to pursue it.
Get them to talk about themselves. Assessing a person's personality is only possible when they feel safe to be who they truly are over who they are in interviews, and what better way to do that then getting candidates to talk about what they know best - themselves?
Respond with kindness and acceptance. A continuation from the previous point: our response as interviewers will decide whether or not a candidate feels comfortable with us. In anyways, we shouldn't be judging people without living their lives.

Assess not Test

We begin with the distinction: the output of a test is a measurement; the output of an assessment is data.

Testing by its very nature of desiring a result/measurement, assumes that there exists a "best" (or "only") way resulting in applicants being graded on a single scale that "is ideal". But the truth is in a constantly evolving environment such as the one we are in today, no one truly knows a "best" or "only" or "ideal" way. Not even leaders of organisations.

Assessing on the other hand comes with the premise that the interviewer acknowledges that there exists variations and different dimensions that require interpretation within a larger context and objective. This leads to a process that exposes and collects information without interpretation which I argue is more constructive than processes derived with testing in mind.

While it's always easier to test rather than assess, it's much more constructive in interviews to assess; and especially so in our context of hiring for a product team that in my opinion requires a skilfully crafted level of diversity in terms of opinions and technical expertise in order to excel.

On implementing this:

Be child-like curious. Changing our mindset to one that's curious opens up the possibility for collaborative exploration which exposes data that will remain hidden if we choose to affirm instead (for example by asking something, and then going "right?")
Know your team. In order to be able to interpret data collected, we need to know what we're looking for. What aspects of culture/capability/cause seems to be lacking in the team? This hints at the types of questions we should be asking to gain more data in specific aspects and this also means that especially so in teams whose value lies in their collective technical expertise, the HR department should try to avoid being the gatekeeper as far as company policy allows for it.

🧸 Improving Skills Related to Interviewing

So you're an engineer reading this and your forward-thinking HR department has tasked you to interview an applicant. Here are some ways that I found I could improve on myself during interviews:

Ask better questions. An answer cannot exist without a question, so it follows that the quality of your questions will impact the quality of your answers. Here's a nice piece on asking questions that I came across recently - 7 ways leaders can ask better questions. Some easy rules that I've found to work in most contexts would be to choose to ask "how" and "why" over "what"/"who"/"when".
Improving somatic expression. Communication is only 7% verbal and 93% non-verbal. How safe someone feels to express themselves has much less to do with what was said, and more to do with how it was said. How do people perceive your accent? What does your body language say about you? Are your body language, emotions, and use of language congruent? In the area of somatics, there's no one dimensional scale that enables us to "become better". Becoming "better" at our somatic expression is a matter of building range. Can I appear confident if it's constructive to be? Can I appear loving if it's constructive to be?
Letting go. It's normal to want to feel in control of your environment. In interviews however, I've found that trying to maintain my control over my environment, resulted in me depriving control from the candidate. This often leads to a conversation that reaffirms my beliefs rather than enabling the candidate to express theirs. This tended to deprive me of data that would've been pivotal. Similar to the previous point on somatics, letting go is about knowing yourself. What emotions are in play – for you – when you desire to be in control of the environment. One excellent book I've read on this is Emotional Agility by Susan David.

🌈 A Conclusion Because I Heard People Like Structure

Conducting an awesome technical interview involves much more than just 'having a chat with' or 'testing' an applicant. When done well, the technical interview should expose enough data for both the applicant as well as the organisation to know their answer.

Having a clear definite "yes" or "no" should ultimately be the objective, and an interview that enables either party to be completely sure of the other, is really what defines a good interview.

🔌 THAT Shameless Plug

Interested to do #techforpublicgood?

We've been expanding as an organisation and there never seems to be enough people.

If you're gifted/talented/passionate about software development and wish to apply yourself to change how technology is being delivered in the public sector for public good, feel free to hit me up at joseph_goh@tech.gov.sg.

Acknowledgements

Shoutout to the partner-in-crime, Ryan Goh, and (more recently) Sze Ying, Dickson Tan, and Wynn, for walking this journey with me and driving me to improve how things are with our technical assessments.

Also, thanks to Steven Koh who's provided sagely guidance over the years to the ACE Hiring (not firing) Squad.

P.S. Thank you to Sze Ying, Dickson, and Wynn for being unpaid editors

New Features on 19th March

Joseph Matthias Goh — Fri, 20 Mar 2020 06:33:55 +0000

We've just completed Sprint 75 and here's the start of many posts that will document shiny new stuff from our weekly deployments (hopefully these can become more frequent, but policies are policies!):

Here's what's new this time:

Features

New section header in resume analysis

If you've been using our new resume analyzer feature (powered by JobScan), you'd notice that we have a new section to provide work experience recommendations for your uploaded resumes.

If you haven't been, why haven't you?

Updated UI for nudges

Our sister team had also worked on some of the digital nudges that we've had from a few deployments ago, adding a Learn More button and making it arguably more user-friendly than before by having both a mobile and desktop version:

Bugs Fixed

When all skills in the skills box are deleted, the nudge box no longer disappears.

Till next time! Got feedback/flames/feature requests? Drop us a note in the comments section and we'll get back to you-

Hello Job.

Joseph Matthias Goh — Tue, 17 Mar 2020 05:53:20 +0000

You've probably seen us in the news, read about us from other places, or even the factsheet if that's your kinda thing. Now here's what goes on behind the scenes.

If you hadn't found the willpower to click any of the links above... (that's dissapointing, but,) We're the technical team behind the MyCareersFuture platform. MyCareersFuture is a jobs portal product created in collaboration between Workforce Singapore and GovTech to support the careers of Singapore residents.

Our initial focus was on Professionals, Managers, Executives, and Technicians (PMETs) in later stages of their careers, and we're gradually widening that focus to support any resident in any stage of their careers.

It's often easy to see the glamourous side of things as the media tends to like to portray it (sponsored or otherwise), but behind every awesome product - not saying we're there yet, but we will be - is a team in the depths of the engine room that truly believes in the product and who are often very invisible.

This blog was created to tell that story (and more), so if tech's your kinda thing, MyCareersFuture happens to be built in-house, and you should (please?) follow us if you'd like to get the latest from real humans on the ground.

If you're interested, other pieces by the technical team can be found at:

See you in a post soon!

P.S. We're also hiring, if you consider yourself talented enough, discounting any Dunning-Kruger effects you're aware of, and would like to join us/GovTech in general, you may hit me up at joseph_goh@tech.gov.sg 😊

Hello from the other side

Joseph Matthias Goh — Sun, 09 Feb 2020 05:21:03 +0000

I began writing technical pieces more seriously about four years ago. Medium was on the rise and I joined a company that encouraged writing technical pieces, and it began - the journey towards not writing. Why?

So why did I slow down/stop? The pressure.

The pressure of having ~500 followers on Medium and the self-imposed expectation of continuing to write awesome articles. I began to see this as being a geek-likewhore of sorts and I began to resent it. Starting a new article usually began with joy and a desire to share some newfound insight, to provide an alternative point-of-view. This quickly turned into self-doubt (common thoughts: "anyone who put in enough thought would be able to discover this for themselves too", "somebody has already written on this, am I wasting my time?", "it's never going to outdo your most famous pieces"), and then into fear ("what if this story sucked?", "what if I can't hit the same applause as my previous articles?"), effectively bringing my writing to a halt.

To be honest, I haven't exactly found a remedy to all that self-doubt, but I do get feedback now and then on having helped people to understand certain concepts which is heartening and makes me want to write again. So I'm going to do just that here on dev.to. No followers, no likes, I'll start again from fresh, writing a little each day on my thoughts and reflections about technology creation and stuff related to it.

If you're reading till this far, thanks for reading!

A Kubectl Alias Dotfile & Other Life-Enhancing Kubectl Tools

Joseph Matthias Goh — Sun, 23 Jun 2019 08:59:43 +0000

TL;DR

It's here: https://gist.github.com/zephinzer/f0a321e9bcb204debd45160b890eb6a3

The backstory

I began actively using Kubernetes some time back, and the commands were getting too lengthy to type out. Imagine having a pod on fire and you're still typing kubectl get pods -n somenamespace | grep deploymentname, finding the pod with the dreaded CrashLoopBackOff, and finally running kubectl log -f -n somenamespace podname.

So. I created a set of aliases that worked using abbreviations of Kubernetes's inbuilt verbs and resource types:

Verbs

apply (a)
config (conf)
describe (d)
delete (del)
edit (e)
exec (exec)
expose (exp)
get (g)
logs (l)
port-forward (pf)
run (r)
top (t)

Resources

config (c)
configmap (cm)
cronjob (cj)
clusterrole (cr)
clusterrolebinding (crb)
deployment (d)
daemonset (ds)
endpoint (e)
ingress (i)
job (j)
pod (p)
persistentvolume (pv)
persistentvolumeclaim (pvc)
node (n)
namespace (ns)
role (r)
rolebinding (rb)
secret (sec)
service (s)
serviceaccount (sa)

Lazy Utility Exports

This set of aliases also exports two functions I've found really useful to avoid typing the -n <NAMESPACE> which happens quite often if you've correctly set up your cluster. Presenting..

kcsetns ${YOUR_NAMESPACE} - this sets your namespace to a string value which all future kubectl commands will respect.

kcgetns - this retrieves the current namespace context of your kubectl commands.

OK, so I promised some other life-enhancing tools as well. I've found these extremely useful when administering a Kubenretes cluster without a UI:

derailed/k9s - a really cool curses-based interface that imo works better than the Kubernetes Dashboard for administering a cluster
wercker/stern - a logs collater that allows you to view logs from all selected pods at once

Till next time!

First thoughts on Apollo GraphQL

Joseph Matthias Goh — Tue, 15 Jan 2019 14:35:20 +0000

It Begins

While I've been working on more operations related technologies, my team at work has been busy with pushing forward in a new service using Apollo GraphQL. So I decided to take a look at what the hype's all about.

Setting Up

Setting up was pretty seamless. Install the dependencies and it's good to go. I followed this guide from the official docs

Apollo GraphQL includes a handy GraphQL Playground which seems better than the GraphiQL which was the way to go when I previously touched GraphQL.

To get started, one simply runs:

npm i apollo-graphql

And then in the code:

const {ApolloServer, gql} = require('apollo-graphql');

const staticData = [{a: 1, b: 2,},{a: 2, b: 3}];
const typeDefs = gql(`
  type Data {
    a: Number
    b: Number
  }
  type Query {
    data: [Data]
  }
`);
const resolvers = {
  Query: {
    data: () => staticData,
  },
};

const server = new ApolloServer({typeDefs, resolvers);
server.listen().then(({url}) => console.info(`Listening on ${url}`));

Monitoring

As an ops engineer, the first question I usually ask is "how do I monitor this thing?"

The official Apollo GraphQL documentation recommends the Apollo Engine, but that requires an API key, and it also meant that I would likely have different sources that I'd need to monitor when it comes to assessing the performance of the service. I'm looking for things such as response time, remote IP address, size of payload, request body structure et cetera

It would be nice if there could be some way to use Prometheus with this.

Option 1: Use the Context

So I explored a little and found the context option which could be passed into the ApolloServer constructor. It exposes a function that exposes the properties from both the incoming request and outgoing response. Could the context be used to inject a Prometheus metrics listener?

Option 2: Use the ApolloServer as a middleware

One nice thing about Apollo GraphQL is that it can be used as a middleware. Changing the dependency to apollo-graphql-express allowd me to use an Express server with express-prom-bundle to monitor the requests coming in. However, there's a problem: All paths are /graphql which doesn't really make sense.

Code Structuring

With the basic example, it's easy to see how the string passed to the gql function could be extended modularly.

Given that we have a directory structure that mimics our data model, each of these models could export a GraphQL type definition which can then be merged into a master schema of sorts. Took reference from this blog post.

Final Thoughts

GraphQL still seems pretty intriguing to me. According to what I've learnt about it so far, Apollo GraphQL also offers React Apollo GraphQL which seems to make writing queries easier and removes the need for using Redux, which is the real benefit I'm looking for.

The ability to define response schemas is pretty interesting to me and allows us to easily test compatibility with other consuming/providing services with contract driven testing.
Despite the benefit, how can we monitor it? A quick Google search doesn't turn up much on monitoring GraphQL queries. I'd rather have an unoptimised service that can be monitored that's written RESTfully, over an optimised one using GraphQL that cannot.