DEV Community: JTK

The dirty secrets of the IT Staffing industry

JTK — Mon, 12 Dec 2022 20:12:47 +0000

In 2013 during a relative low point of both my life and career, I got a job that I had no idea would alter the course of my entire life. It was an entry-level recruiting role paying $13 an hour with paltry benefits at a now-defunct IT Staffing firm in a small, nothing town in Maryland.

I know many tech people hate recruiters, and if you are one of them, I will tell you the exact people you hate most are the kind created by this type of staffing agency.

I was inspired to write this post after hearing a junior developer lament the unpredictable and confusing behavior of recruiters hounding her, then going silent. For anyone else new to the industry who may not be familiar with how these shops work, I'd like to pull back the curtain on how (some, many) IT staffing firms work and why I avoid them.

The basics

Some of you may know this, but it is worth spelling out why staffing agencies exist and how they function. I'm going to be pretty critical in this post, so it may surprise you that I don't think all staffing agencies are necessarily evil or that they don't have a purpose.

Some of the things I will criticize about staffing agencies are also more accurately laid at the feet of the companies that use staffing agencies to keep a perpetual underclass of lesser-paid contractors alongside their cushy salaried workers (looking at you FAANG). Staffing agencies react to what their customers profess to want, so in some ways their behavior is just a reflection of the state of the IT industry at large.

But let's back up and take a 10000 foot view for a minute.

In the most simple terms possible, staffing agencies exist to complement traditional in-house recruiters. If a business's own recruiting department is understaffed, sometimes they'll need help finding and vetting candidates and in theory a staffing agency is a great way to fill these fluid hiring demands. A company may not want to keep 10 in-house recruiters on staff if their demand is seasonal and half of them wouldn't have much work in non-peak season. Using that logic, it makes sense to partner with a staffing firm who can handle the more fluid and unpredictable surges in demand that a business may encounter in their hiring.

In some cases, hiring IT talent is so hard that companies have indefinite agreements to work with a staffing agency. Again, on paper, it makes sense. An in-house recruiter may have to cover roles from HR, accounting, engineering, marketing, on and on. If you use an IT staffing firm where all the recruiters specialize in tech talent, you ought to get better results, right? (Often no, but more on that later).

In general, staffing agencies invest a lot of up-front work into calling candidates, vetting them, and then getting resumes to the business to review. The staffing agency will coach the candidate and confirm their interviews, and if the candidate does well they typically will be hired as a contractor under the staffing firm and receive any benefits the staffing firm offers.

Usually, this work is structured as a contract or contract to hire vs permanent opportunity. Staffing agencies generally make their money in "billable hours", where they charge a surplus on top of whatever hourly rate the hired worker earns that goes to the staffing agency. It is sometimes called the burden rate, the formula to determine how much the staffing agency has to charge on top of the hourly rate going back to the worker for that placement to be profitable to the staffing company.

Adding a middleman between employee <> employer and giving someone else a cut of the budget for a certain job can have the effect of depressing the wages the worker/contractor receives. If you've ever noticed spammy recruiters talking about contract work at abysmal rates, eyyo. That's how that happens.

Beyond the basics: System integrators

It will sound ridiculous to say this, but on top of contract workers losing out on some wages because of the staffing agency middleman, there is often a SECOND middleman in the mix. If you have seen jobs from KForce, TEKsystems, Robert Half, etc etc - in general all the big guys like that as well as smaller staffing firms partner with something called System Integrators.

The hugely popular one I dealt with in my staffing job was called Atos, and their volume of business in the US was enormous. They covered Coca-Cola, Siemens (Atos used to be Siemens IT Services if I remember correctly), Nike, JP Morgan Chase, on and on. I have limited insight since I got out of agency recruiting as soon as I could, but they may be the big dog of all system integrators in the US, they seemed that large.

How the flow of activity worked at my staffing agency (and many others) was that we actually did not deal directly with any company and instead were one of many dogs waiting for table scraps from Atos. Atos would receive a 'req' for the various businesses they served (Coca Cola, Nike) and redistribute those out to numerous staffing agencies simultaneously.

If you have ever annoyingly gotten four different voicemails from recruiters for what sounds like the same job, guess what! It probably was. Recruiters would mad dash to call all the best candidates after receiving a req, since only the first company to submit a certain candidate could get credit for them. Atos would also give us a slap on the wrist if we ever accidentally submitted someone another agency had already sent. Sometimes candidates would get so inundated with recruiter phone calls that they'd legitimately not be able to keep track of what they were submitted for. Some agencies try to scare candidates by saying if they don't reveal if they were already submitted they will get put on a do not hire blocklist. In my experience, that is entirely made up and was not something I ever saw done. Maybe it happens elsewhere, but I never saw it. Candidates are the staffing agency's meal ticket, so you can see why they wouldn't want to "bite the hand that feeds", blocklist someone and miss out on future earnings.

The last thing I'll say about Atos that leaves me with a very bad taste in my mouth about them is that the quality of their job descriptions was absolutely laughable. They were so bad that people would cuss me out for sending them something so useless. I remember it so vividly I was able to find what I'm confident is an Atos-related listing in about a second by searching "IT Support Specialist hardware software peripherals". I got this gem of a job listing:

Its a thumbs down, from me, on Atos.

Other foolishness within the agency day-to-day

Moving away from Atos and back to life in the staffing agency, there are a number of things that I think result in the relatively poor candidate experience we see with agencies.

Early into this role I asked my boss how to handle it when IT guys would be angry that I sent them our Atos job description that was little more than two lines about "support computer hardware, software and peripherals. Troubleshoot and debug end user issues". My boss confidently told me that it wasn't important, that our job was just sales. 90% of people were going to tell you to go to hell and that it just is what it is.

I (of course) ignored him and set about learning more about technology so I could be a better recruiter.

Other troubling things we were asked or forced to do included using a massive, primitive web scraping resume harvesting service called JobDiva. We could set up harvesting jobs to run overnight using various keywords and search terms. When asked what to search, my boss told me hardware AND software. I later went on to be certified in advanced boolean techniques but even without that background yet, it was obvious to me that those search terms were childishly inadequate.

Job Diva would dutifully send "email merges" to the million, jillion, trillion people identified each night unlucky enough to have the words hardware and software in their resume, and each morning myself and the other recruiters would sift through the mountain of angry emails about how off base our outreach efforts were.

Staffing agencies also have some of the most inane, useless metrics of any type of business I've ever seen. Recruiters are commonly judged on metrics for number of phone calls placed, number of candidates submitted, in addition to more meaningful metrics like number of interviews per week, time to fill a role, and number of placements made within a month. My staffing agency didn't have this but some have "lunch quotas", where recruiters have goals to take out a certain amount of candidates for lunch each month. A lot of it is utter nonsense.

The incentives are aligned in such a way that you are so busy you can't follow up with candidates. The focus on placements means that staffing agency recruiters are really not incentivized to spend their time doing things like, updating candidates who weren't hired. The submittal metrics daily meant that we'd take half-qualified candidates and lob them over the fence just to hit our numbers, a waste of time for everyone involved. Alternately, we would have to edit and heavily wordsmith resumes for people, disguising gaps between jobs, generalizing titles, making people add keywords lest Atos reject the submission as "not qualified".

All in all, I would describe staffing as a vehicle of enormous churn. Sure, it can generate a bunch of resumes for a company but when I think about all the waste involved in the process, it is hard to see it as a good investment.

Other miscellaneous bad behavior

I know I've said a lot at this point bad about agencies, and I know there are some out there that are more boutique and don't do the things I describe. But for me honestly, it was clear very quickly that staffing was not the place for me. I was critiqued for caring too much about the technology, spending too much time doing research, spending too much time on the phone with each candidate. Basically, acting like a human being who cared about my job.

The prevailing attitude was definitely in the vein of "it's all just sales, these people are all just numbers. Keep it brief and focus on throwing bodies at Atos". Although I generally met my metrics and wasn't under-performing, I was an office oddity for caring about candidate experience and trying to learn the technology to do better for the people I talked to.

There ARE some things from this time that I am glad to have learned.

Seeing how agencies work was enough for me to steer clear of them, for starters. I saw that agency recruiters had no pull or influence to get an excellent but slightly nontraditional candidate anywhere, so I realized me, without a degree was never going to have success getting jobs through staffing firms.

I do stand by some of the resume edits I learned there, dropping months off resumes to disguise any gaps (you don't need to know that I took a month of funemployment in 2021 because I felt burnt out, Random Hiring Manager). Genericizing my title is something I've done and will continue to do if it serves me. But other than that?

I left staffing as soon as I could, and entered the world of boutique, white-glove internal recruiting. I know a lot of y'all hate recruiters and have only encountered ones that are more of the staffing agency caliber, but I was different. My candidates used to compliment me with astonishment on how much technology I knew. I had printouts all over my office like an FBI agent on a manhunt except it was cross diagrams of programming languages and network engineering terms. I took tech certs, I started coding in my free time, I got certified in advanced boolean searches so I could find people more effectively, I wanted to be as surgical as a scalpel.

Aaaaaand here I find myself, almost ten years since my stint in staffing, as a Senior Software Engineer. My quest to be a better recruiter took me too far over to the dark side, it seems. C'est la vie.

Filtering out the noise

If you didn't already do this, it is possible to largely live out your tech career ignoring staffing agencies. I use a separate email while jobhunting and dutifully unsubscribe from all of the staffing agencies who find me. Their emails are normally:

contract for a client who won't be named (Atos won't allow it)
extremely generic description
pay, if listed, is bad
despite offering little to no info, recruiter asks for your resume and has a heavy bias towards getting you on the phone
(potentially) you get a string of similar emails at one time that all have identical bodies as all the various staffing firms scurry to submit people first

If you are a somewhat traditional candidate and for whatever reason in a tough spot and need a job, no judgment of course if you take a contract through companies like these.

If you have a LESS traditional background, I will tell you that your odds of getting anything meaningful from a staffing agency encounter are very low.

If you are a person who gets one of these jobs, I would also focus on trying to get away from contract work as soon as you can. With great hypocrisy, Atos has a noted bias against serial contract workers. They were seen as inferior, the subtext being "why hasn't anyone wanted to bring you on full time?" What's worse is that in addition to the lower pay, lower level of security than a salaried employee, etc is that on the job contractors may be siloed and kept at arm's length from interesting work. In recessions, contractors are often the first let go. So in exchange for making it through the wringer that is staff agency hiring, it isn't like you get much in return.

My advice if you really need a job and get what you suspect is a staffing agency email is to try to reverse engineer the role from the location/description and apply to the job directly on the company website. It certainly won't hurt your chances and if you succeed, you're going to be getting a lot of benefits a contractor wouldn't.

In conclusion

So that's it! That is the deep, dark, hidden dysfunction that goes on in staffing agencies and the reason I have never gotten a job through one since my own experience recruiting for that type of business.

I was (luckily) able to get away from staffing fairly fast, and loved that in subsequent recruiting roles all the behaviors I got mocked for were appreciated. Learning the technology, being responsive to candidates, giving them a good experience: this stuff matters in my opinion, even if staffing agencies act like it doesn't.

If you have any questions I didn't address or your own experience with staffing agencies, feel free to share in the comments!

The road to staff engineer

JTK — Thu, 01 Dec 2022 19:15:13 +0000

I'm not qualified to write this post, yet. I'm not a Staff Engineer yet. Although of the things I've ever really set my sights on, my track record is pretty impeccable, so I'd estimate my odds are good. I also hope to have another few decades in this field, so judging on my trajectory so far, I can't think of any reason it shouldn't happen.

I wanted to write about this topic though, because I feel like it is shrouded in far more mystery than I would have expected. What is a Staff Engineer? What's the big deal about it?

When I think of my current belief of "what's so special about Staff Engineers" I think if early-career me had heard it, I wouldn't even have understood what it meant. It is not always but can be a lot more complicated than "just a super duper senior engineer".

What's so special about Staff Engineers?

This is a complicated question to answer, and like any title different places it will mean different things. Like any job, different people have different strengths and weaknesses, and you could take two very successful Staff Engineers and find the reason for each of their success to be totally different attributes.

Further complicating things, as my understanding of Staff Engineer grows, I see that it can but may not always share elements of other roles: leadership roles, technical lead, IC. All of these things make it harder than I would have imagined to pin down.

Apparently I'm not alone. Will Larson (who has great content and a book out on this topic) identifies that there are, in fact, staff archetypes people may tend to fall into, which can somewhat explain the disparity you see between person to person who may hold the title. Borrowed completely from the linked resource, they are:

The Tech Lead guides the approach and execution of a particular team. They partner closely with a single manager, but sometimes they partner with two or three managers within a focused area. Some companies also have a Tech Lead Manager role, which is similar to the Tech Lead archetype but exists on the engineering manager ladder and includes people management responsibilities.

The Architect is responsible for the direction, quality, and approach within a critical area. They combine in-depth knowledge of technical constraints, user needs, and organization level leadership.

The Solver digs deep into arbitrarily complex problems and finds an appropriate path forward. Some focus on a given area for long periods. Others bounce from hotspot to hotspot as guided by organizational leadership.

The Right Hand extends an executive's attention, borrowing their scope and authority to operate particularly complex organizations. They provide additional leadership bandwidth to leaders of large-scale organizations.

So these descriptions advance me somewhat in my understanding. I have been called Tech Lead and Dev Lead before, so I can connect the dots and recognize what traits let me do that well. I've also situationally been my team's "Solver" or "Architect". Probably the least relevant to me is "The Right Hand Man" as I don't really like politics, although calling it "politics" grossly simplifies that archetype I realize.

Although there is some overlap, my own understanding of Staff Engineer has been an evolving attempt to put words to people I've known but couldn't have described. There's one at my work who I started with before their promotion to Staff, and I remember just thinking "wow this guy is on another level".

When I tried to articulate my goal of a Staff Eng promotion to my current work, I finally came up with at least a halfassed description of my newly-promoted colleague. "He sees the problem underneath the stated problem, he sees the solution no one was asking for but they actually desperately needed". In addition to being a solid coder, organized, and t-shaped, this person basically has x-ray vision, was I realized the heart of my compliment.

I think the part junior-engineer me wouldn't have been able to conceptualize is how a Staff Engineer puts it all together. It's a mix of things, gravitas, communication skills, consensus building, deep as well as specialized technical knowledge, and the ability to transform all those elements into masterpieces of impact. It is hard to say that in a non-cheesy way that expresses the extent of it. I think as a junior dev, I also would not have understood how deep "depth" can be. My impression of operating at the staff engineer level is that there is a depth borne of deep, deep experience that can't be faked or rushed.

What I want is not an easy path

I was made Dev Lead with about three years of software engineering experience and at the time I remember feeling like it was an inflated title for my capabilities. Not wanting to miss out on any opportunities it netted me, I obviously didn't object to the title bump but inside it felt sort of meaningless.

At that phase in my career, I was a solid communicator, obviously good coder, well-rounded in some ways although not in others, but I think mostly just working crazy hours was enough to get me to Lead.

Since I've been in tech, it has not been unusual for me to see colleagues have a six year path to startup CTO (maybe title inflation, but its happening) or a two-year path to Senior Engineer. Not to detract from anyone else's achievement, but for me senior came too easy - it was meaningless. I didn't feel like I earned it and although I enjoyed the salary that came along I did not in my heart of hearts feel senior.

Staff Engineer is a different beast. While I think a lot of companies slotted people into senior roles in the past few years to make salaries work in a competitive hiring climate, Staff Engineer is a title that carries with it a weight that it isn't going to be given away like that.

In fact, many to most engineers may never make it past a senior title. One resource I found but am now struggling to locate describes Senior Engineer as an often "terminal" title in the progression spectrum of a software engineering career.

Putting my own version of a roadmap together

I'm still a work in progress. I write to you awaiting my first annual review at my job, so that in the event there are critical comments I can integrate them into my personal roadmap. If the comments are good, I'll have additional food for thought about what archetype I want to pursue.

As always, I reach something of a fork in the road in relation to my natural talents versus my interests. People seem to find me likable, I get told I am articulate or a good communicator. Being organized, creating documentation and charts and guides, those are all things I was doing from childhood when I'd put together PowerPoint presentations lobbying my parents for different privileges.

Maybe because it comes easier to me, or because they are skills I used in my past careers, I have always gravitated more towards the hardcore technical work since becoming an engineer. People probably would assume me to be an extrovert but I'm not. All that "likability" costs me, below the surface. The lack of control also fatigues me.

In one of my appointments as Tech Lead we had an under-performing teammate and I was somewhat alarmed to have nebulous aims for that person's growth laid at my feet. Ultimately, I could help, I could coach, I would give them the tools to succeed but had no control over whether the struggling engineer improved. That kind of thing terrifies me, and is why I'm not considering people management, although people sometimes tell me I'd be good at that.

Adding another layer of complication, because Senior Engineer came too easy and didn't mean much to me, I have my own arbitrary thresholds of knowledge I want to conquer in the process of pursuing this title. For my entire dev career up until graduating this past March I have been a half-time to full-time student. Cranking out school assignments kept me so busy that there are many "fundamental" code areas I feel I never got to learn. I've never properly learned to do cool manipulations of tree data structures, if asked to do back of the napkin math about device storage capabilties I know almost none of that offhand.

As someone working with high-level languages to be fair neither thing comes up all that often, but it has still been the voice in the back of my head saying you aren't really senior, you got there through luck and likability.

My managers asked me why I wanted a Staff promotion and I answered, "Not to sound like a real hippie but it's partly about the journey. I feel like its a chance to learn all these things I never got to to become my own idea of a deep, well-rounded engineer".

My plan from here

I will never stop being surprised at the number of times in life being the so-called "squeaky wheel" or asking for what I want has netted me outrageous advantages. Since casually talking to folks about this a number of extremely lucky circumstances have fallen into my lap:

I got to interview a talented Staff Engineer about their path to the title
My old and new boss have both been getting me involved in more database work, which I want to be the crux of my special area of knowledge
I got invited to a recurring Staff Engineer working group where a mix of people who currently hold the title and want it chat together about what it means

With all those things cooking, the next BIG thing I need to do is get my review from work and translate that into specific goals and milestones to attain the promotion in a year.

The year timeline is to be honest with you, maybe a moon shot. I have six years of experience as of last month. I'm seeing people attaining Staff Engineer for the first time after 15 years of experience, some of it far more impressive than mine.

Even so, all I can think is that having this focus so early might let me leapfrog a few years of just cruising at a senior title. The focus, the work, learning from others, can only help right? While a year may be crazy, I'd say my more honest goal would be to make Staff Eng before having ten years of experience. I would be perfectly happy with that outcome.

In closing

While it is nerve-wracking, having this new goal is exciting. It is the first time I've really cared about my title in this field, and I feel like the journey to even trying to get it will be transformative.

It IS about the journey, y'know? I wasn't just saying that. Even if I never get there, taking this road trip through all the topics I always felt I missed out on, developing leadership skills and impact, I'm looking forward to it all.

Thanks for reading, and feel free to share any of your own thoughts on pursuing this title in the comments!

Data modeling crash course

JTK — Sun, 18 Sep 2022 19:42:41 +0000

Intro

Data modeling questions are a fairly normal part of the interview process, and it is worth knowing some basic guidelines for how to organize data. Although in this piece I will primarily talk about this at the database level, it is good knowledge to have across the stack - knowing when a piece of logic should be broken out into a reusable component is something frontend engineers will frequently run into, and it gets into similar logical territory because we are thinking about how it makes sense to organize our information, what belongs together, and what should be broken up.

Because it is the predominant type of database, we are mainly going to talk about relational databases and how to model data using good practices for relational databases specifically. But it is worth knowing that there are many other types of database out there, and what count as “good practices” can be different between different database systems. In addition to relational databases, there are:

NoSQL
Key/value stores
Graph databases
Network databases
Object oriented databases
Columnar databases

Among others. Before you feel pressure to learn all that, it is worth saying that I’ve been doing this a long time and have really only ever significantly dealt with relational databases, key value stores, and NoSQL. There is also some crossover in functionality between databases, at times. One of my favorite things about MongoDB is the ability to store data structures similar to JSON, but PostgreSQL, a relational database, also allows you to store JSON if you need to. Some NoSQL databases also support query languages like SQL (structured query language), the language of relational databases.

Basic ideas

I feel like data modeling is one of those terms that can sound intimidating before you know what it is. I had several of these as a junior developer where when I would hear the term, for some reason my mind would go blank. (”Scripting language” was one of these, even though all I knew was scripting languages 🤷‍♀️).

If hearing data modeling makes you think of training AI models, or complex statistical models like they do for political elections, you can put that to the side. You should be happy to know, data modeling for relational databases more or less follows common sense.

Relational databases and SQL have actually come to be a favorite technology of mine, because the principles so closely echo common sense. It is also somewhat unique for a technology as old as relational databases to have survived the rapidly changing technological landscape since the theories behind them were created by by E.F. Codd between the 1960s-1980s. It is somewhat rare for a technology so old to still work so well and be so well liked by the people using it. The history behind this technology is interesting, and you can read more about it here.

If I could summarize the “common sense” ideas that govern good data modeling for a relational database for someone with zero background, I’d probably highlight the following:

We want to follow separation of concerns by breaking up data as much as possible into distinct tables
We want to avoid duplicating data, because this makes more work for us if we need to delete or update anything
We want the components of our tables to be clear and independently meaningful (the value in column B should not change in meaning in relation to column A, it should stand on its own)
We want to structure our data to have safeguards in place to avoid “junk data” - for instance, do we want to allow a user with a typo on a state abbreviation to send the value “NZ” instead of “NC” to the database? Probably not, because it may cause our application not to work as expected, and we may have to clean up bad data later

Not too crazy, right?

Quick detour into SQL-specific lingo

Some of the things in our toolkit to make the above guidelines happen are indexes, keys and constraints. You can think of these like rules. When you set up these rules, if data comes to your database that violates the rules, instead of writing that data an error will be thrown. This is actually what you want. Bad data can cause all kinds of headaches.

Let’s quickly go into these concepts before we head into some examples and rules of thumb.

Primary Keys

Primary keys are a way for us to uniquely identify our records. This is typically done by creating a numeric ID field. Databases have logic around primary keys so that for each new record, the ID field is incremented by 1 for every new record. They also enforce logic so that you could not have two records with id 2, for instance. This makes dealing with our data much easier, because it is far more convenient for your application to be built around thing with ID 2 than thing with name x and address y and state z and… every time you need to pull up a record

Foreign Keys

It will be no surprise that in a relational database, we have representation of the relationships between our data. In simplest possible terms, a foreign key is when one table references another. In addition to describing the relationships between tables, foreign keys can be used to enforce data quality.

Lets consider that we have a table that collects address info, like the following:

Address	State	Zip code
123 tree street	AZ	87678
4 Cary Ct	NC	27617

What happens if the user inputs a typo in their state? We would have bad data that might cause our application not to work as expected. However if we have an address table and a state table, we could use foreign keys to say that whenever we try to add a new address, we will check that the state exists in the state table, indicating it is valid input.

Indexes

When you create a primary key field on a table, the database also creates what is called an index. The idea is a little like an index in a book, and makes finding things easier. Indexes make searching faster for the database engine, the underlying logic that carries out SQL queries. If you have a table with ID fields like this:

ID
1
2
…
1000000

And you are searching for record with ID 200, the database can use binary search to get to the correct record with great efficiency. How this works is the database might select the middle ID value of all the records, and see that it is 500,000. That’s way too high, so we can ignore the second half of those results, anything from 500001 onwards. We can pick from the middle again, and get ID 250,000. Still way too high, so we can ignore the higher half of those results and pick from the middle again of the remaining lower half of ID numbers. So on and so forth. Binary search is a very efficient algorithm, so database indexes are an important tool for fast searches and good database performance.

Other constraints and keys

We have other ways of applying rules to our database to ensure we have only good data, and no repeated data. We can use something called UNIQUE constraints to create a rule that every record must have values in a certain column that don’t already exist in the database. If you had a database with usernames for instance, you could prevent someone registering with a username value that someone else had. (Side note, when you create a primary key, in addition to it automatically being indexed, it automatically also has a unique constraint out of the gate).

Sometimes, we also have situations where a traditional number ID field does not fit our needs. So we can also make what is called a composite key, and make a primary key that is the combination of two columns.

There are many other types of keys, constraints, and indexes, but it is out of the scope of this article which is for beginners.

Data modeling interviews: brainstorming

Data modeling is something that comes up from time to time in interviews. It may be part of some system design questions, or it may be an exercise of its own. In my experience, data modeling interview discussions are actually one of my favorites types of interviews. They are almost always common sense, straightforward, no trick questions, and I like data so I wind up enjoying them. They are also usually pretty collaborative. If you have gotten an interview like this, I don’t want to call something “easy” but I will say data modeling interviews are among the more pleasant interviews I’ve been part of, so go in with good vibes knowing you won’t be grilled on something random or asked some obscure factoid (ime).

In my experience, brainstorming is an important stage of ALL interviews that ask you to approach a technical problem, and these are no different. With data modeling, there are some guiding questions that reflect good relational data modeling principles, and are good to move through to show that you understand this space and can reason about data and how data is related.

Good brainstorming questions to ask yourself (and say out loud in interviews!):

What are the main entities in this problem? (Often, what you would have as a model in backend code directly corresponds to what you would have as a table or “entity” in the database)
When determining “data entities”, think about whether data is meaningful on its own, or only in the context of other data. It is generally good to split data up - good separation of concerns, a good way to keep tables smaller and more granular, good all around - but if data is not meaningful on its own, it may belong under the umbrella of another entity
In school learning backend code, sometimes we learn to develop models thinking about “is a…” and “has a…”. We can say, a car is a vehicle, and a motorcycle is a vehicle. Maybe we build a Vehicle class for that reason. We can say a motorcycle has a engine, a car has a engine. So maybe we need to track engine data and relate it back to our Vehicles. These concepts translate to the database too, and are worth thinking about planning your models
When you have related data, it is good to ask yourself what entity it is most related to. Sometimes data is equally related to multiple entities. In those cases, sometime it is worth breaking out a separate table reflecting the relationship back to both of two equally related entities
Relational databases deal with, you guessed it, relationships! Some relationships are hierarchical, like a boss with many reports. Some relationships are 1:1, like a website user would probably only have one account ID. Some relationships are many to many, like how a student can be in multiple classes and a class contains multiple students
When you have your main entities figured out, it is worth thinking about the smallest appropriate data type you can use for your data. Booleans and integers generally take up less space than text and can be faster to search. You have to be careful though, since people can run into trouble using INTEGER ID fields for databases with too many records, it is possible to hit the limit of integer values. For this reason sometimes there is another type, BIGINT to allow greater flexibility. There are rarely trick questions here, but it is worth demonstrating you understand the tradeoffs of various data types

Rules of thumb with examples

One thing I’ll cover quickly is that in some of the examples I’ll show you next I borrow concepts from the rules of database normalization. There are various “levels” of database normalization, and usually the higher the level your data conforms to, the more it is seen to follow good relational database practices. You can read more about normalization here.

💡Side note: I have never been asked to remember the definition of a specific form of normalization in an interview, and probably couldn’t if asked. Since most of the rules are common sense, I apply them constantly, but it is intuitive rather than the result of memorization. While it may come up in some rare, rare circumstance, I think it is worth mentioning that I have never been asked to regurgitate definitions of normalization. So if I were you, especially as a new coder, I would not worry about that tbh 🤷

Rule 1: No multi-value records

Consider we have an app like GoodReads where users can “favorite” books. What do you think if we have a table structured like what’s below?

user_id	favorites
1	The Snow Child, Pride and Prejudice, Eleni, The Poisonwood Bible
2	The Revenant, Ender’s Game, The Hobbit

With SQL syntax, if the user “unfavorited” a book in this list, you would have to re-write the entire record to apply that update. There is also not an easy way to insert a new favorite without re-writing this entire comma-separated list. Searching also gets slower, since instead of looking for an exact match if you want to find users who favorited a certain book, you would have to essentially loop through all the favorites of each record and see if the book title is among them.

The way to structure this data with better practices might involve splitting it up, and taking advantage of foreign keys which we chatted about before.

To summarize what we did here, and why it is good:

We can now do a simple search for user favorites with logic like where user = x and book = y on the user favorite table
If a user adds a new favorite, we add a tiny new record to the user favorite table
If there was a typo on the book title, instead of having to go through that giant list of favorites we had in the previous design and updating each record, we can simply update the title field of one single row in the books table, which is much simpler, and the user favorite table remains completely correct
Because we use foreign keys, we won’t have any “junk data” made for users that don’t exist or books that don’t exist as the result of application or user error

There’s also something cool you can do with SQL called CASCADE ON DELETE which means that if a user is deleted or a book is deleted, all the records related to it in the user favorite table also get deleted. There is also something called CASCADE ON UPDATE, so in the scenario of a typo potentially you could correct it in one place, and have it reflected everywhere. Cool, right!?

Rule 2: Deduplicate data

You might see the example above and think, “I don’t really feel like querying three tables, why don’t I add more user info to the user favorite table because we need it all to display on the favorites page”. You might wind up with a table something like this:

user_id	book_id	user_name	book_name
1	2	jtkaufman737	Homegoing
1	5	jtkaufman737	Shadow and Bone
1	40	jtkaufman737	The Ninth House
1	3	jtkaufman737	Infidel
1	24	jtkaufman737	The Big Machine

Look at how many records we have repeating that username! What happens if I want to update my username? What happens if there was a typo in the book name?

The reason it is better to stick to the ID numbers and keep this table small are:

there is less to update
IDs/key values don’t really change, ever, so we don’t care so much if they are repeated, its not as if it will ever lead to a bunch of updates
Integers are a small data type, so it is better for storage and quicker for searching

Overall, it is almost always the case that tables are not the correct place for combining your data into a convenient format. Tables need to be a pure source of truth. (SQL does give you an option for combining your tables “virtually” though, with something called a view).

Your data needs to be really, REALLY independent

At the level of a single record, and the value of a single column, we really need our data to stand on its own two feet for good data modeling. If you make a change in column A, it should not change the meaning of column B. Nor should the meaning of columns be different if you were to change their order. Either of these things is a sign that your data is not structured properly. You should not need to worry about having to update field a just because something was changed in field b. In that case, those data are not truly independent, and some logical concession needs to be made so that they can be treated like the related pieces of data they are.

Let’s look at another variant of our GoodReads data. Say we had a table like this:

book_id	genre_id	genre_type	author	published
1	2	Historical Fiction	Jane Eyre	1800
2	8	Sci-Fi	Bill Murray	2001

A change in the genre_id would also require a change to genre_type. These columns have a dependency, which means you cannot cleanly update one field, which in a perfect world you would be able to do no problem. For this reason, it is again, better to break up our data:

book table

book_id	genre_id	author	published
1	2	Jane Eyre	1800
2	8	Bill Murray	2001

genre table

genre_id	genre_name
1	Historical Fiction
8	Sci-Fi

Your data should be as regulated as possible

This is something that bites me from time to time in application code. The database permits more values than I would expect, so we have unexpected data variance. Maybe that means there are a mix of abbreviations and full state names in a table pertaining to addresses. Maybe the falsey value for a column can be either {} or NULL, so a condition I wrote looking for NULL is not behaving as expected, since it was really supposed to catch all falsey values (that one happened to me recently). Good ways to keep your data quality high:

Rely on lookup tables as much as possible to act as enums, forcing only data conforming to valid options to be created
Pick column data types that enforce quality: should an email field ever allow NULL values when users register for your site? Probably not. You can tell the database not to allow NULL values. There’s also no reason to pick a data type that allows text characters if the column should only ever expect numeric input.
You should use CASCADE options so that there is never “orphan” data, child data to some parent entity left over after the parent entity is deleted

In closing

There is much more to get into on this topic, but I sat down to write this for a newer frontend engineer with an upcoming data modeling interview and I wondered to myself “what are the bare basics that I’d want them to know?”

Data modeling, database design, and performance are deep, fascinating subjects in their own right and if you liked this article, there is a ton more to read and learn. But for someone new in the field juggling learning on multiple fronts, what I’ve outlined here is a perfectly solid foundation for a data modeling discussion while interviewing.

If you want to know more about the building blocks of SQL, I also gave a workshop which you can find here that discusses SQL basics and terminology more in depth, check it out! 🤓 I also highly recommend this tutorial on SQL commands and syntax if you would like some practice learning structured query language.

Surviving live code interviews for newbies

JTK — Fri, 02 Sep 2022 19:15:57 +0000

One of the questions I get a lot is about live coding interviews. Like interviews, there is a HUGE variety in what you may be asked to do during live coding. Some of it is totally open-ended, no trick questions, just writing basic methods. Some of it is done in a live collaborative environment like repl.it. Others are the standard “is this string a palindrome” type riddles.

There is good news and bad news in that, you can’t prepare perfectly for everything, so there is no point driving yourself crazy. However you can stick to some simple guidelines to make a good impression while live coding, even if you make mistakes, or don’t finish the exercise.

While I personally avoided this topic for a long time out of being intimidated, I have done my best to come up with goofy, memorable, and painfully straightforward examples in this section below about Big O notation. Although optional reading, I hope you’ll give it a try and feedback is welcome 👇

A brief conversation about big O

If you are not ready to dive into the big O conversation yet, I would summarize it (in terms of actionable things to focus on in coding interviews) with the following guidelines:

Hash data structures are superheroes (python dict, JS object, etc)
Getting loop-requiring data structures (like arrays) into hashable data structures will be your friend
Getting from nested loops to singular loops will also be your friend

Part I: Getting started

You are at an interview. You may or may not have known there was going to be live coding. In the first few minutes, I would say you have a few key priorities:

Take a minute to quietly read the problem to yourself 📓

Take a minute to read, then reread the problem to yourself. Don’t think of solutions yet, make sure you really understand it. I have definitely read too fast and gone off half-cocked with a solution that missed edge cases before.

Pick the language you are best with - seriously 👨‍💻

If given the option to pick a programming language, don’t worry about picking what the company uses or picking a language to make “a better impression”. It is already a high-stress scenario, pick what you are most comfortable with. If it isn’t what they use, in the past I’ve offered a breezy rationale like “I wind up doing most leetcode problems with x, so guess I’ll go with that”. If they offer the options, they intend to let people use them. Don’t worry about picking JS over Ruby looking bad or anything like that.

Make sure their code environment is working (run hello world) 👋🏽

Believe it or not, I have had the collaboration tool break! Check that you are in good shape by just running a quick log statement.

Set the tone with your interviewer for narration and conversation 🗣️

Let the interviewer know you plan to ask them for feedback and chat with them through your approach, like you would with a colleague in the real world. It feels much less like public speaking when there is back and forth.

Have your interviewer collaborate on some test cases 🏁

It is easy to miss things in the heat of the moment, so this is a great area to get extra eyes.

Bringing some of the above points together, this is something close to what I think I say at the beginning of code challenge interviews:

Me: Great, so first I'm going to take a minute and read over the problem to
myself if that's alright

Interviewer: Of course! Have at it

...

Me: Great, so before I get too far into this I want to run a quick hello
world from this function and make sure everything is cooperating.
...
Awesome, looks good.

Me: So now to get started, I'm going to try to explain what I'm doing and my thinking to you as I go. Feel free to jump in as I'm coding, I think it will be great if this is collaborative and gives us a sense of how we'd pair
together if I joined the team. I'll probably have questions for you as we go also.

Interviewer: Yeah that sounds great! And I will chime in periodically to remind us of where we are with time, so if we run out there's some room to discuss what you'd do next.

Me: Sure, sounds perfect. So to start with, I don't want to miss any edge cases. I was going to put together some test cases and whether they should pass or fail. If there is anything I miss as I'm doing these, please don't hesitate to
let me know. Being on the spot for coding interviews, sometimes can be nerve-wracking and I don't want to accidentally miss anything.

(etc etc)

Probably if you ask 10 people how they like to start code challenges, you’ll get ten different answers. Some people may not like my last comment, that I admit I’m on the spot and may miss something. In my experience interviewers have responded to that in a pretty human way, I even feel that it has been helpful? Sometimes they have supplied a test case I would not have thought of, and sometimes it does illustrate some aspect of the code challenge I hadn’t realized yet and would have wasted time dealing with later.

Especially when you are new to code challenges and extra nervous, you can go quiet for a second thinking about your approach, and the seconds spiral into minutes, and you are just sitting there silently panicking. It is good to get the conversational rapport going early. For a junior coder, it is honestly pretty normal to get stuck on some aspect of a code challenge, but if you are already in a conversational rhythm with interviewer it is much easier to say “I feel a bit torn but think my approach should be this…” and open the door for feedback and direction. If you have been sitting there silently for several minutes, it feels way more awkward to suddenly ask for help.

It varies place to place, but in many interviews the code working is ABSOLUTELY not the only outcome they are looking at. They are looking at collaboration (especially for juniors!) they are looking for how you deal with ambiguity, that you ask for help and don’t get stuck and spiral into panic. I would say almost always in code challenges, if you can convey what you are or were trying to do you can get “partial” credit for that alone, even if you were not close to a successful implementation.

Part II: In the thick of it

So you have your test cases set out, you have the code running. The pleasantries and housekeeping items from the last step are sort of the “easy part” when I do code challenges, and when you are through all that just staring at the code that’s the moment where it is easy to lose your head.

💡Listed below are the BIGGEST priorities while doing the main task.

Comment out the steps/phases of your solution immediately (”comment skeleton”) 🦴

For me, that looks something like this early into a code challenge:

// previously empty function where I did my hello world 
function myCoolAlgo(sortedList, target) {
   // first, I'm going to build a hash from my list items so that I can look
   // them up more efficiently 

   // next, I want to build a separate validation method to check <blah>

   // third, I want to <blahblahblah> 

   // next, I'll take the result indices and see if they are a prime number

   // last, I'll return the result * 23
}

^ by the way, the comment bodies are totally fictitious, no clue what kind of code challenge that would be. But that’s along the lines of stuff I would write.

I like to comment out the stages of what I think I need to do early. A) to get feedback, and B) so if I get panicked and lose my train of thought, I have some reminders. For places that may save the code, it also provides a trail of bread crumbs into what your plans were if you had more time.

Many places are more understanding than you might think about not finishing. Others, they have a laundry list of items that they would add on to the challenge indefinitely, so there is no real “end”, but you may be able to make an impression even on parts of the task you didn’t have time for by leaving some comments with your ideas.

Giving my code a “comment skeleton” is maybe one of my more helpful habits ever in code challenges. I mean it, they do so much good. They keep you from losing your train of thought, for starters. It is also great to see a new coder comment their code, since sometimes junior software engineers do not write the most clear code anyway it is particularly needed for the work of an early career dev, and I know for me seeing an interview-ee has good comment practices would be a huge plus.

Talk about why and how you are making your decisions 📰

if you read the Big O thing linked earlier, or have performance considerations that led you to your solution, this is a great time to talk about your thinking on performance (so you get partial credit for that, even if you don’t finish getting them working). It is also a chance to reveal your knowledge passively outside of the specific code you are writing.

For each increment of new code, run some of your test cases 🧪

If you have commented your code out into “sections” sort of as I suggest, finishing each section is a great time to run some of your test cases and log what is happening so that you can guarantee you don’t go farther on nonworking code

Keep the conversation going, solicit feedback 💬

It is also SO SO important to keep the conversation going here! You really can get “partial credit” for things you know but are not implementing. I know I’ve said things like:

I have a feeling I could get the performance better here, I’d like to get this working first and circle back to it at the end with time permitting
For some reason I have a suspicion there might be edge cases that could bite us at this point, but I can’t put my finger on one…do you have any thoughts?
I had hoped that I could keep this in a single loop for performance, but at this point am not seeing a way around adding a second. If I have time at the end, I would definitely come back and tighten up some things, just as I would with real code before submitting a PR

The subtext, or literal commentary I sometimes have at those points is along the lines of “YOLO code interviews are stressful”. I find owning it relaxes me somehow? Just saying you know what, I have limited time here, some of it is setting off my spider senses and I’d like to revisit it, but out of respect for the time limit I’m going to move on for now.

Some other things to keep in mind 🧠

Ctrl + f can be your friend, misremembering my own variable names is maybe my biggest cause of errors/debugging in code challenges 🥴 usually you can ctrl + f down a page to track something and at least rule that out
When torn between a longer, descriptive name and a shorter, ambiguous one pick the longer one
Even though you are in a rush, take time to make sure your code is reasonably formatted and your variables are named reasonable things
Although often code challenges are in the scope of a single function, if your function gets too big it is worth either mentioning or implementing breaking the logic out into a separate function just to show you understand separation of concerns. If you are tight on time, its fine to just add that in running commentary “normally I would break this out” etc
Sometimes the unexpected happens! I have had the collaboration tool fail on us unexpectedly. It is good to have a backup in mind. I think when that happened to me I recommend we switch to repl.it and screenshare, and I wound up getting that job haha

Part III: Wrapping up

I think every code challenge I have done has been a unique adventure. The first one I did was a train wreck and after that I avoided them for quite a while, I got so lost in my code that I was basically just rambling nonsense by the end. Others, I’ve felt I did nothing special and got job offers. Others, I’ve felt I nailed but got shot down.

Code challenges are kind of a terrible way of assessing coders because of the fact that public speaking is one of the most common phobias people have, and even without a phobia there’s still plenty of other rational anxiety to have about them.

Although they always feel stressful, I think I can count on one hand the live coding challenges I’ve felt were truly awful:

I asked them how to prepare, they said NOTHING about live coding, then sprung it on me before we even had done basic chitchat. I was totally flummoxed (this was that first one I mentioned). I couldn’t remember the difference between slice and splice in JavaScript and just absolutely spiralled over my inability to remember. It sucked monumentally in the moment, but I very quickly moved to finding it kind of hilarious. By the end I knew I bombed so bad I was just giving random database hot takes because it was like? Who cares
They had one of those code editors where you couldn’t run the code. I probably would never even do an interview with those again. It is SO UNREALISTIC! It is not how we write code, and yeah it really did a number on me, I think I did very poorly with that premise

Over time, I’ve kind of come to even like coding challenges just because they are so relatively fast and over with so quickly. The more I did, I realized how much you get so-called “partial credit” for things you didn’t finish, so long as you are communicating. I realized people weren’t really watching me just looking out for any dumb mistake as I’d feared. It really is a lot about how you think, how you collaborate, and it seems like many that I’ve done have found finishing on time secondary.

But as you are wrapping up your code challenge, here are my final pieces of advice for you:

Reiterate anything you would change/continue working on in the current code (this is a nice point to mention you would of course add tests to any functionality you write)
Thank the interviewer for their time, and emphasize that you feel you can definitely thrive in a collaborative coding environment
You can offer a brief defense or acknowledgement of anything that went wrong, and how/why it unfolded and either what you learned, or how you might avoid it in the future. But definitely thread the needle on keeping it positive/brief here

And voila, that’s it!

Optionally afterwards, you can send a follow-up note. It won’t hurt anything to add “I feel so silly that I did not think of this while on the spot, chalk it up to interview nerves, but realized immediately after the interview ended I could have used x to do y. I wish I would have thought of it during our session, but appreciate the opportunity and would look forward to any next steps” to whoever you paired with. In my experience, it is unlikely to hurt anything and shows some self-awareness, if you don’t feel you had time to address it in the interview already.

Conclusion

The biggest thing I would want anyone to know for live coding interviews is that you can’t perfectly prepare for them. Even doing this a long time now, some days I bomb. I’ve had recent interviews where for whatever reason, I could not sleep a wink the night before. My last interview cycle actually I had the bad luck to have this happen TWICE! I felt that I completely flubbed both, one resulted in an offer and the other a rejection.

Live coding is inherently stressful. For me, it has become hugely more manageable over time, so maybe there is something to practicing. You can definitely arrange ways to practice with friends, alternating Leetcode problems while screensharing over Zoom or whatever.

At this point in my life, I don’t know if I will ever stop being nervous to live code, but at this point it is much more like going to the dentist or some other mundane trial than the terrifying gauntlet I once found it to be.

So at the end of the day, chin up, it isn’t the easiest thing in the world but the experience is helpful. You also never know when you will get lucky. I once live coded a problem where I had recently received the exact same on Leetcode and had a solution perfectly ready to go.

There is always the unpleasant possibility that live coding is going to result in you feeling silly or even stupid for an hour, but over the years I’ve started to look at it like “Am I willing to feel stupid for an hour in exchange for a $50k raise?” because more than once, that’s been at stake and it has been secured through live coding. So at this point in my life, I say hell, sign me up to feel stupid for an hour, what’s the worst that can happen 😎

Big O notation basics made dead simple

JTK — Fri, 02 Sep 2022 19:06:38 +0000

A brief conversation about big O

To be honest with you, when I first got into programming I procrastinated learning Big O for quite a while out of intimidation, and this article is really for anyone out there who feels similar. I don’t know if it will help for me to say that most of it is common sense and the math doesn’t get much harder than multiplication. But just in case, I will add some visuals to make this as non-miserable as possible. The graphics are super silly, but honestly that helps me remember things.

If I were to give you the simplest version possible of the idea behind Big O notation, or the time complexity of how your code runs, it would be something like this:

basic operations are the most efficient
repeated operations (loops) can be more of a worry for efficiency
nested loops are among the biggest worries for efficiency

Honestly, that’s the basics. Not too terrible right? There’s also a whole category of things you never have to think about when figuring out the performance of your code. You can consider them “Freebies”. They include:

basic math, addition and subtraction
assigning variables
if/else conditions
declaring a function
calling a function (at face value)
changing variables

If I had to sum the concept of Big O up in simplest terms imaginable, loops are pretty much all we are worried about. I know when I realized that, I felt a lot less intimidated.

💡A note before we begin: This is meant to be the absolute basics, for people who think Big O notation sounds like archaic rituals. There are more nuances, and more descriptors, than I’ve included here. Whenever there is a chance to take clarity vs greater detail, I have chosen clarity. This post is for folks a few months into programming who need basic, actionable info. This is part of a larger series I’m writing on live coding interviews so give that a read as well 🤓

To explain some of these, let’s pretend that in my free time, I like to go on scavenger hunts. My friends make the rules for these hunts and some are less challenging than others. (Sidebar, some people really do this! There is a hobby called geocaching that is pretty much real life treasure hunting. You can read about that here if you are curious. My examples are too silly to conform to really geocaching though so I’ll stick with the term scavenger hunt).

Part I: Constant time or O(1)

🏴‍☠️ Jamie sends me on a treasure hunt to retrieve a pen from a certain address on Chestnut street. I have one address so only need to go to one house, easy peasy.

This is an example of what we would call “constant time”. Whether the scavenger hunt was for a pen or something else, I’m only knocking on one door in this scenario and it will not change. This is considered a very positive state for algorithm performance, the most optimal actually. An example of a constant operation would be retrieving a value from a JavaScript object.



const person = {
  name: "Ann", 
  hobbies: ["drawing", "programming"], 
  pet: "dog"
}

console.log(person.name)

Because JavaScript objects are an example of a hash, it is very performant to look up some value in an object. It is a perfect analog to having an exact address and being able to go straight to it without searching further. Regardless of what is inside, the step of retrieving the item is considered to run in constant time.

💡For this reason, objects/hash data structures are often your friend in code challenges.

Sometimes getting arrays into an object format can be hugely helpful for good performance in these types of challenges. We call this constant time O(1), in Big O notation.

Part 2: Logarithmic time, or O(log N)

🎁 My friend Jake has set me on a trickier task. There is a new development being built near me but they have not added visible street signs. If I walk up to the houses though, They have notated the address on beams by the front door.

I don’t really want to have to walk up to each house individually, but after thinking about it I realize Jake has shared that the home address numbers increase as you progress down the street.

I don’t know the starting number as the road continues a ways past this cul de sac in the opposite direction, so I’m not sure what the lowest house number is. But I do know I’m looking for #22. I realize that if I start in the middle of the street, I’ll see if the address is lower or higher than what I’m looking for, and can knock out half the houses on the block, so I don’t have to search those.

I see the house in the middle is #26, so all the houses to the right of this one would have house numbers too high for what I’m looking for, #22. So I can cross off the house I walked up to, and all the houses to the right because they would have even higher numbers.

Now in a normal world, these houses would all be regular and I could just count my way to the correct one, but I see some empty dirt lots and can’t tell if those will also become homes or are for landscaping or parks. Because I’m not entirely sure what are lots, I am not sure just counting my way down the street to the correct house would work.

So I decide to go with my “pick from the middle” approach again, and that way can at least cross of half of the remaining section of the street based on if the house is numbered too high or low. For this reason, I pick the middle house again of my remaining three houses.

Whaddya know! That’s the correct one. If you want to think of my “treasure hunt algorithm” the end result is pretty good. I wound up only having to walk up to two houses even though there were seven on the street. In terms of how much work I had to do to succeed at this task, my approach meant that the remaining houses to search would have been cut in half each time I walked up to one.

We call the performance when you cut the result set each time logarithmic performance, and after constant time (O(1)) it is as ideal as you can get. Even though you have a result set of 7, the operations you perform could be as low as 2. If you don’t remember much about logarithms from math, that’s honestly fine - all you need to know is that its a GREAT performance for an algorithm to have, because you are cutting your result set down significantly each time.

In Big O language, we write logarithmic performance as O(log n). If you are worried about memorizing the big o descriptor, simply calling it “logarithmic time” in an interview is perfectly ok. And if all you need to know is that logarithmic time means that each pass you cut your results down potentially by half, it is pretty easy to recognize this when you are doing it.

As a side bar: my scavenger hunt example is an example of a binary search algorithm. Not too bad, right? Just pick from the middle over and over. It is a great algorithm to use if you know your results are already in some type of order.

Part 3: Linear time, or O(n)

🍬My friend Jess, on hearing how much work I’ve been putting into these scavenger hunts, takes pity on me. Lucky for me it happens to be October 31st and she assigns me the task of trick or treating! My treasure hunt is to get one piece of candy from every house on Sycamore street.

In this somewhat deliberately messy graphic, you can see how much work this is compared to my last task.

This is what we call in Big O “linear time”, aka a loop, aka simple multiplication. My task is multiplied by the number of houses. On a bigger street, that might be 50 trips. My work increases in a linear fashion based on what street I’m on and how many houses it has.

For big O, linear time is a good algorithm performance in many cases. Not as good as the first two we covered, but normally a code challenge you would receive in an interview is complex enough that there is a reasonable chance loops will be required. We write this as O(n), with “n” being basically what you are multiplying by, or how many loops you’d have to do.

Part 4: Quadratic and cubic time

👿 My last pal, Andrew, is really devious and sets me on the hardest treasure hunt yet. He sets me off on main street and says that for every house on the street, I need to see if I can find an item at each house duplicated at each other house.

Very quickly I’m up to 25 individual tasks, since I have five things to do for each of five houses. You can easily see if we put this to numbers I have 5 x 5 tasks, or 5^2. This is what we call quadratic time.

In programming, this means one thing: nested loops! Each nested layer is adding an exponent. If my friend had said I want you to do this for more than one street, the math becomes something like neighborhood x houses x items, and could easily get into cubic time, or O(n^3), often presented in a third nested loop in real life.

My example is pretty manageable still, but in real life programming we are looking at way larger numbers than this. See how the tasks balloon:

Base number	Tasks ^2	Tasks ^ 3	Tasks ^ 4
100	10000	1000000	100000000
1000	1000000	1000000000	1000000000000
100000	10000000000	1000000000000000	100000000000000000000
1000000	1000000000000	1000000000000000000	1000000000000000000000000

This is maybe the most important area I can talk to you about. It comes up the most in my job and the most programming. You can quickly see why it gets important fast.

You will notice that that first row where your base number is 100 can quickly get into more operations than that last row, with a starting number of a million, if they are not careful about nested looping. Someone with a data set a million large can have a program doing less work than someone with that data set of 100 records, all depending on how careful they are with nested looping.

Most of the finer points that come up for me in code challenges are if I can get something from a loop to constant time, and then if I can get something from a nested loop to a single loop. (So from cubic time O(n^3) to quadratic time O(n^2)).*

We also have a more general word to represent situations where the exponent is going to be to the power of n: *exponential time (O(2^n)).

Part 5: What this may look like

Let’s say we got the following code challenge.



Given an array of unique integers and a target number, determine if any two 
numbers in the array when added together would equal the target. If that 
condition is met, return the array position of the two numbers. You cannot 
reuse a number at a single index position (So array [4,2] could make
target number 6, but array [3] could not since it would require reusing the
array value at position array[0]).

Example: given array [1, 2, 3, 4, 5, 6] and target number 10, you would return 
an answer [3, 5] since at array[3] you have number 4, at array[5] you have 
number 6, and added together they make 10. 

Example: given array [1,2,4] and target number 50, you would return false 
because the condition can not be met.

This is a paraphrase of a Leetcode problem, where below it asks follow up question, “Can you get the time complexity below O(n^2)”. The fact that they ask is a good hint that we might. But for a very new programmer, the obvious way to solve this problem might be something like this. (You can also run the code yourself, here).

You can see that very quickly, the exponential logic catches up with us and we have quite a few individual task calls in this method.

Going back to what I said before about trying to get things into shape where you can take advantage of constant time and linear time, I’m going to use all the same test cases but rewrite this function in a way with better Big O. (BTW, this is illustrative, maybe not precisely how I would solve this problem so don't come for me in the comments lol).

Hashes are really your friend in these kind of problems. So when I iterate over my list, I’m going to make a JavaScript object where each key is what the list item would need to be added with to reach the target number, and the value would be the index I found this item at.



Example: given target number 8 and list [1,2,3,4,9,6] my JS object would 
look like this:
{
   // first array item is 1, we'd need to find a 7 to make 8, its at index 0
   7: 0, 
   // second array item is 2, we'd need to find a 6 to make 8, its at index 1
   // so on and so forth
   6: 1, 
   5: 2, 
   4: 3,
   6: 5
}

So with this object, it means I don’t need to do a nested loop. I can do another regular loop (so linear time) and just check if the current list item is one of the keys for this Javascript object. If it is, I’ve found a matching pair whose sum is the target number. I have saved the index where the first number was located as the key value so I already have that handy, and then can just return the index of the list item I am on in my second loop. You can also play with this code here.

I used the same test cases, and you can see got the same results. But with my nested loop approach the worst performance we saw was > 200 operations, and by not nesting my loops it is only THIRTY! That’s pretty amazing. And it wasn’t too hard, once you get used to thinking this way, to extract the nested loop behavior and find another way.

Visualizing what we’ve learned

One great way to drive home how these perform is to look at a graph of performance, I grabbed this one from this website.

Another fun note

You may wonder when looking at the problem above, is the Big O notation O(2(n)) since we are looping twice? If a loop is O(n), we have done that twice: once to build the JavaScript object, and a second time to process the list again and see if we have two numbers that will sum to the target number.

If you remember me saying that all we care about is loops, mainly, add an asterisk here to say we care about the greatest magnitude of operation present in the problem. What does that mean?

If you have a loop in your program with performance O(n), and later in the code you have a nested loop with performance O(n^2), we call the big O simply O(n^2) - the exponent is the behavior of the highest order of magnitude in the program, aka the biggest worry. So we drop the lesser one
If you have a nested loop running at O(n^2), and later on a triple-nested loop running O(n^3), we would simplify that to cubic time. That triple nested loop is the highest order of magnitude thing going on in the code so we simplify to call it O(n^3).

If you think about it, it is more fair to do it this way. Almost everything you will write will have some operations that are O(1), but it is almost never accurate to say the overall performance of a program is O(1). If it has a single loop, that statement will not be accurate.

Conclusion

There are some other types of Big O notation we could cover, but these are the ones I see most commonly and they should get a new programmer pretty far in code challenges. Even if you don’t feel super into math concepts, remembering that loops are the biggest thing you need to worry about is I think easy enough advice that anyone can keep in mind, and it really does help you in keeping performance in mind no matter what level of coding expertise you have.

Also, here are some other great resources:

Turbocharged Candidate Search With Algolia

JTK — Tue, 26 Jul 2022 22:38:39 +0000

Thinking back to the stone ages of recruiting, before AI

Something you all may or may not know about me is that I found my way to tech through a very unconventional route: recruiting! No, I wasn’t recruited, I was the recruiter.

Back in the day, my claim to fame was that I was certified in Boolean logic searches. This came in handy when I used to work in government contracting. Very specific and rare combinations of a security clearance and one or many certifications were required for many of our hires. When I first heard about Algolia’s tools for AI-powered search, ranking, and recommendation, I thought back to those days I spent arduously searching for candidates. Naturally, I was curious to delve into what might be possible now that AI has the ability to superpower tasks that were once long and tedious.

With that in mind, join me as I explore some of the capabilities of Algolia! I chose to do this with VueJS, but that was nearly at random considering the vast number of language SDKs available, which you can see more here.

Just a brief note before we begin: the team at Algolia is paying me to explore the tool and jot down my unbiased experiences, so all of the opinions below are my own.

Building out some candidate data

Before taking a look at how Algolia tooling might facilitate a candidate search, I wanted to approximate some kind of candidate data. I came up with some logic that would represent a mix of skills and keywords that someone might search for. (I started this work with this boilerplate Vue project if you would like to follow along). I almost expected there to already be something like tech-ipsum or professional-ipsum that I could lean on for this task, but sadly I didn’t really find anything!

So to start with, at the root of my Vue project I created a small seed.js file with the following logic to generate some “candidates” for us.

// seed.js

const fs = require("fs");

const seedData = {
    weights: [2, 3, 5, 8],
    roles: ["backend", "frontend", "fullstack"],
    termMap: {
        backend: ["back-end", "python", "pytest", "django", "sql", "nosql", "redis", "AWS", "api", "ci/cd"],
        fullstack: ["full-stack", "python", "javascript", "css", "sql", "api", "ui", "html","databases", "typescript", "react"],
        frontend: ["front-end", "ui", "ux", "design", "accessibility", "css", "tailwind", "javascript", "react", "typescript" ]
    },
    titles: [
        "software engineer", 
        "senior software engineer", 
        "software developer",  
        "programmer", 
        "software development engineer"
    ],
    randomAchievements: [
        "Led adoption of ",
        "Increased throughput of systems using ",
        "Introduced quality standards for ",
        "Enhanced testing of ",
        "Coached more junior employees in ",
        "Recognized subject matter expert for ",
        "Led research effort on improvements to our ",
        "Decreased bugs by 50 percent related to ", 
        "Created platform with ", 
        "Spearheaded workshops to train colleagues in "
    ],
    candidateList: []
}

function getRandomArrayItems(items, quantity) {
    let randomItems = [] 

    for(let i=0; i < quantity; i++) {
      randomItems.push(items[Math.floor(Math.random() * items.length)]);
    }

    return randomItems;
}

function generateCandidateData() {
    // Silly little function to generate some candidate profiles 
    let candidates = [];

    for(let i=0; i <= 10; i++) {
      const weight = getRandomArrayItems(seedData.weights, 1)[0];

      seedData.roles.forEach(role => {
        const achievementList = getRandomArrayItems(seedData.randomAchievements, weight);

        let profile = { 
          title: `${role} ${getRandomArrayItems(seedData.titles, 1)[0]}` 
        }
        let achievementsWithTechKeywords = []

        achievementList.map(achievement => {
          achievementsWithTechKeywords.push(
            `${achievement} ${getRandomArrayItems(seedData.termMap[role], weight).join(", ")}`
          );
        })

        profile["resume"] = `${achievementsWithTechKeywords.join(". ")}.`;
        candidates.push(profile);
      })
    }
    seedData.candidateList = candidates;
};

function writeCandidateData() {
    fs.writeFile("./candidates.json", JSON.stringify(seedData.candidateList), err => {
        if(err) console.error(err);
    })
};

generateCandidateData();
writeCandidateData();

Moving over to my terminal, I’m going to run this logic quickly to generate a file of JSON we can then use with Algolia by running node seed.js .

Looking over to my code editor, I now have a candidates.json file full of slightly nonsensical but vaguely tech-resume-esque data, with a small sample pictured below.

Not necessarily anyone I’m going to hire onto my team tomorrow but a fair enough approximation to start with. On to the star of our show, Algolia!

Getting started with Algolia

If following along, go ahead and create a free account with Algolia for what comes next. We’ll walk through their easy “getting started” wizard and first create an index called candidates.

I’m going to plug in the JSON file we’d previously created, although Algolia also offers an API client that would make this task also super easy programmatically too.

On submitting, we are redirected to the dashboard where we can now see our data listed.

Exploring some of the fun options we get with Algolia

When you get to this point some getting-started-tips in an overlay will quickly prompt you to explore some of the cooler options that this tool makes available, including the customization options. To start with, we can choose to ignore or include attributes. For a start, I’m including both resume and title.

Why is this option cool? Well, I would definitely have benefited from the option to ignore title on occasion in my candidate searches. In my first recruiting job we worked for this terrible client that had all of their IT support roles under an extremely bland title like Support Services Technician or something equally nondescript. It didn’t take me long to learn that the resume keywords were going to be a much better proxy for fit than anything related to titles. This level of control would have helped me there. You also could potentially configure this type of search as part of an Applicant Tracking System (ATS) to assist with anonymizing parts of candidate profiles, to reduce any element of bias. One instance might be that if our data were structured to include fields for Education and Year Completed, you could ignore the year in to reduce the potential for ageism. Algolia explains more about the benefits of customized attributes here, if you’re curious!

We also get the option to customize the fields most relevant in ranking. Since I’m experimenting here, I’m going to set ranking to be exclusively tied to that resume field. My hypothetical scenario here is that I would be more interested in a match on Python skills than getting hung up on whether someone was a Software Developer vs Software Engineer. This side-steps the title field we created for our data.

One last option here that I would have KILLED for when I recruited is best understood through a little backstory.

When I did government recruiting it was extremely common for people to have “synonymous” credentials, or certifications or status indicators that made a HUGE difference in their hirability but were not always expressed clearly. One example is the CompTIA certification Security +, which many people working in government contracting get trained for and certified in on the job through a Department of Defense directive (8750, specifically). Some of these people are so neck-deep in their world that they do not optimize their resume for search, so you might only find them list the directive that required their certification. Algolia has the option to create a list of synonym keywords that could have saved me from long arduous searches like security AND (engineer OR specialist OR analyst) and (8750 or "Security +" or "Security+) and so on and so forth. The synonym option is super easy to use, pictured below:

A mini candidate search prototype

To apply some of these cool options, I went through building a simple approximation of a candidate search page on that sparse skeleton of a Vue app I linked earlier. Feel free to skip this, but I hacked together a small amount of CSS to make this look reasonable, in the <style></style> tags of our App.vue file:

// App.vue

<style>
button.ais-SearchBox-submit,
button.ais-SearchBox-reset {
  display:none;
}

input {
  margin:2em;
}

html {
  background-color:#edeff5;
  margin:4em;
}

h3 {
  color: #3c4fe0;
  letter-spacing:.04ch;
}

h5 {
  font-family: Arial, Helvetica, sans-serif;
  letter-spacing:.03ch;
  color:#5a5e9a;
  line-height: 0.4rem;
  margin:2.5rem;
}

span#header {
  display:flex;
  justify-content:space-between;
}

span#header img {
  height:25px;
  width:25px;
}

div.searchResults {
  background-color:white;
  padding:3rem;
  margin:2rem;
}

.ais-SearchBox-input {
  /* background-color: white; */
  height:45px;
  width:100%;
  margin:0px;
}

.ais-InstantSearch {
  width:50%;
  margin:auto;
}

.ais-Hits {
  margin-top:1rem;
}

.ais-Hits-item p{
  font-size: 1.5rem;
  margin:0rem 2.5rem 1rem 2.5rem;
  line-height: 2.25rem;  
}

.ais-SearchBox-form {
  background-color: inherit;
  margin-bottom:2rem;
  height:8rem;
}
</style>

And in the <head></head> section of our public/index.html, linked this minimal CSS framework I like, Wing:

<link rel="stylesheet" href="https://unpkg.com/wingcss"/>

Adding Algolia’s UI elements

I wish I could make my effort on this part sound more interesting, but truly the options here made connecting this minimal UI to Algolia relatively effortless. For search, Algolia offers a number of UI libraries including one for Vue. You can check out some of the others here.

🛑 ✋ A quick note: We’re about to stick our Algolia API key in a .env file. That’s completely alright if you’re confident that you haven’t given this particular API key any dangerous permissions. It’s a good thing to check before we start, if you haven’t already.

With that said, to get these values, head to the main dashboard menu tab for API Keys. Here, we’ll want to grab our application ID and search-only API key.

Heading back to the terminal, we’re going to run the following commands to install our needed Algolia packages.

npm install algoliasearch vue-instantsearch --save

We then are going to bind that to our Vue instance, over in main.js.

// main.js

import Vue from 'vue'
import App from './App.vue'
import './registerServiceWorker'
import InstantSearch from 'vue-instantsearch'; // new import

Vue.use(InstantSearch); // now bound to Vue

new Vue({
  render: function (h) { return h(App) }
}).$mount('#app')

Now let’s introduce some changes to App.vue to accomplish a few things:

We are going to use are markdown section to use pre-made search widgets available through this library which are super convenient
We’re going to use the JavaScript logic to connect to our previously uploaded data and supply our records

First, let’s update the <template></template> section of App.vue to now include the search and display widget Algolia has handy for Vue.

// App.vue 

<template>
  <div>
    <center><h3>Turbocharged searches with Algolia</h3></center>
    <ais-instant-search :search-client="searchClient" index-name="candidates">
      <ais-search-box id="searchInput"/>   
      <ais-hits>
        <template v-slot="{ items }">
          <div 
            :id="item.objectID"
            class="searchResults" 
            v-for="item in items" 
            :key="item.objectID"
          >
            <span id="header">
              <h5>{{ item.title }}</h5>
            </span>  
            <br/>
            <p>{{ item.resume }}</p>
          </div>
        </template>
      </ais-hits>
    </ais-instant-search>
  </div>
</template>

Next, in the <script></script> section, let’s add the connection to our Algolia index.

<script>
import algoliasearch from 'algoliasearch/lite';

export default {
  data() {
    return {
      searchClient: algoliasearch(
          process.env.VUE_APP_ALGOLIA_APPLICATION_ID,
          process.env.VUE_APP_ALGOLIA_SEARCH_API_KEY
        ),
    };
  },
};
</script>

Then finally, back in the terminal, if you kick off npm run serve and head to the browser, you should now get a simple search UI like seen below, tying all these pieces together:

We can confirm the search and filtering capabilities are doing their thing by entering some criteria, you can see how mine changed when I typed “frontend”.

And that’s it! With truly, a few simple steps and maybe less than twenty minutes work, we have made a prototype which — simple as it looks — is leveraging the search configuration options we selected earlier and the underlying artificial intelligence that comes baked in to the Algolia search product.

Bonus: add event insights!

If I imagine myself using this tool for this purpose, I realize that there also might be a scientific way to analyze the usage of my fictitious candidate search engine. For that to work, we might want some kind of tracking related to what candidates a recruiter drills down into for further review, or many candidate search applications will also let you “favorite” candidates.

To approximate that behavior, I decided to also make my search work with Algolia event insights. This area of functionality lets you track numerous events, be it clicks or favorites, that could further improve relevance of the search results shown to users. For more background on how to get started with events and Algolia, you can find that here.

To start, we are going to make some updates in App.vue which do a few things:

import the event/insight library from Algolia
connect the API client to our API credentials, so events will be sent back to the correct data respository, the one housing our candidate records
attaches the insight client to our search widget from a previous step
create simple logic to display icons, where clicking an icon “favorites” a candidate, and kicks off an event being sent

The revised sections of App.vue are as seen below:

// App.vue 

<template>
  <div>
    <center><h3>Turbocharged searches with Algolia</h3></center>
    <!-- new :middlewares argument connects the insight client to our search widget -->
    <ais-instant-search :search-client="searchClient" index-name="candidates" :middlewares="middlewares">
      <ais-search-box id="searchInput"/>   
      <ais-hits>
        <template v-slot="{ items, sendEvent }">
          <div 
            :id="item.objectID"
            class="searchResults" 
            v-for="item in items" 
            :key="item.objectID"
            @click="sendEvent('click', item, 'Item Starred')"
          >
            <span id="header">
              <h5>{{ item.title }}</h5>
              <img 
                :id="`img-${item.objectID}`" 
                src="https://i.imgur.com/5Je2dmA.png" 
                @click="favorite(`img-${item.objectID}`)" 
                alt="thumbs up"
              />
            </span>  
            <br/>
            <p>{{ item.resume }}</p>
          </div>
        </template>
      </ais-hits>
    </ais-instant-search>
  </div>
</template>

<script>
import algoliasearch from 'algoliasearch/lite';
// new imports contain what we need to support sending events
import { createInsightsMiddleware } from 'instantsearch.js/es/middlewares';
import aa from 'search-insights';

// reconnecting to the API, so our events/insights go to the right place
aa('init', {
  appId: process.env.VUE_APP_ALGOLIA_APPLICATION_ID,
  apiKey: process.env.VUE_APP_ALGOLIA_SEARCH_API_KEY
});

// fictitious user token lets our "favorites" be tracked to me
aa('setUserToken', '32f32sfds94s032dfjskal')

const insightsMiddleware = createInsightsMiddleware({
  insightsClient: aa,
});

export default {
  data() {
    return {
      searchClient: algoliasearch(
        process.env.VUE_APP_ALGOLIA_APPLICATION_ID,
        process.env.VUE_APP_ALGOLIA_SEARCH_API_KEY
      ),
      middlewares: [insightsMiddleware]
    };
  },
  methods: {
    favorite(id) {
      // simple logic flips the icon, for a visual indicator of "favorited" candidates
      const icon = document.getElementById(id);
      icon.src = "https://i.imgur.com/j635pgy.png";
    }
  }
};
</script>

If we return to our browser now, we see the following:

If we were to simulate doing a real search, and “favoriting” some results, we’d see hearts on our selected candidates (instead of the thumbs-up icon).

And if we return to our Algolia dashboard (the URL is https://www.algolia.com/apps/<your application id>/events/debugger) we can now see events being sent back!

In closing

Honestly, between learning how to use web scrapers and now the knowledge of how easy it is to set up a robust, customizable search UI with Algolia, maybe some parallel-universe version of me is still in recruiting and specializes in building in-house candidate search tools. But the truth is, even out of the tiny niche of recruiting, I can think of limitless applications for search tools this convenient that come AI-powered by default. Thanks for making it this far, and I’d love to hear your ideas for applications of Algolia in the comments! 🤓

The emotional gauntlet of learning to code

JTK — Thu, 21 Jul 2022 22:30:51 +0000

In a recent post I wrote about my long journey getting into programming, it made me remember some funny moments from the most intense period of my programming education which took place Spring/Summer 2016.

I attended a coding bootcamp, and to say that it was a challenge would be an understatement. There was so much going on. I had just moved states, recently lost my dad, and kept getting really bad headaches (whoops, I needed glasses! Go figure).

The specific memory that sticks out is, my anxiety during this period was through the roof. I didn't have a particularly fluid transition to coding, I wasn't great at it. To attempt to deal with my nerves/stress I remember trying at various points to meditate. Ultimately, if I could clear my mind at all I would immediately wind up stress-napping. What a mood, as they say.

"Stress napping" I think is a response perfectly indicative of all the chaos involved with learning to code: it is mentally exhausting, emotionally bruising, and in my case an ego-shredding experience. So today I want to unpack a little bit of why that is, how it manifests, and most importantly some things I would recommend to help with it. (Help, not solve - ultimately this is a hard thing to do and in some ways there is no path but experiencing the difficulty).

What makes learning to code so grueling

If I had to distil the difficulty of learning to code into main areas or themes, I'd probably organize them as follows:

Mentally challenging: I had always been a good student. Not just in one thing, across the board. A-B student, quick learner, and in some cases didn't need to try particularly hard to do well. Coding changed all of that. I could not let my focus drift a single bit. Applying every bit of determination and attention I could muster, still things were going over my head entirely. Things that looked "simple" as other people did them immediately became mystifying hieroglyphics the moment I was on my own.

Distractions and conflicting information: Learning to code alone is tough. Learning to code while learning Github while learning about IDEs while learning about the inspect browser options while learning about Trello while also being a new Mac user was BANANAS.

This gets arguably worse at the point when you have a working knowledge of what I listed above and think "ok now to get a job".

One person tells you to focus on open source contributions on Github, which you still don't know well. One person tells you to focus on algorithms and data structures, which are bafflingly abstract. Another tells you to build a portfolio. Another tells you to do FreeCodeCamp or Codeacademy or Code Wars. Another tells you to practice Hackerrank. Someone recommends Eloquent JavaScript, someone else recommends a Youtuber, someone else recommends Cracking the Coding Interview. You are drowning in material.

To be clear, none of these things will HURT you if you are able to work on them and make progress and through that progress gain experience and practice. These things may hurt or hinder you if they are so overwhelming that you don't know what to do, or if you nervously rotate among them and are context switching so much that you aren't getting traction or practice with code. (More on that and approaches to manage it later). I don't know if this is a real term but I would like to coin it if not: panic-cycling. I see a lot of new programmers panic-cycling through approaches and due to the churn, feeling stalled.

Exhausting hits to self esteem: Neither of the two previous points are particularly helpful to feeling good about yourself. People have told you five million different things you are supposed to do to become employed, and doing any of them feels like a herculean feat. The interview process is also bruising. If you are in dev communities, you may start to pick up on that developers can be dismissive and rude about beginners, Stack Overflow answers are downright hostile sometimes. Trying to use Stack Overflow literally the messaging you receive is often "This question is too dumb to ask" in the form of "this was asked elsewhere" or "this question doesn't meet our criteria". (More on this later too).

Weird industry cliqueyness: In the background of all these other things, as you start coding you start to pick up on the social strata, which might seem bizarre to an outsider. Why is frontend or PHP perceived as less prestigious? Is this thing I'm about to learn "dead"!? (People loooove declaring various technologies "Dead!", its bizarre honestly). So although you are in a poor position to do so, most new coders I know feel immense pressure to pick the correct "path" before they realistically know what it should be, an added worry on everything else.

Phew. I was exhausted just WRITING this list, just REMEMBERING these feelings even though they were six years ago!

How these stressors manifest

Something I find funny that writing this post made me think of is related to travel, something I love. I started traveling solo when I was 17 and have been to sixteen other countries total. Would it surprise you to know, travel is so tiring that on almost every trip I take I have a moment in the first 24 hours where I think "this trip was a mistake". Whether it is a malfunctioning GPS in Iceland, getting stuck in a bathroom in France (door broke!), there is always some logistical mishap that makes me doubt all my life choices.

Invariably, after my "freak out moment" I get a good night's sleep. Wake up the next day. And am a new person. I start to feel the vibe of the city, and stop feeling self conscious about my language barrier and start enjoying myself and just taking it all in. Even knowing this will happen to me, it happens every single time without fail.

What does this have in common with learning to code? Exhaustion.

Ways that stress and exhaustion manifest learning to code:

Panic-cycling: (talked about this a bit already). I'm no doctor but whenever I think of panic cycling that new devs do, I can't help but also think of decision fatigue. New devs feel pressured to make so many decisions that I think their thinking gets frenzied and they wind up trying to do it all and even more exhausted by context switching in a nasty cycle
Emotionally charged relationship with coding: There was one person in my bootcamp who I will never forget who really struggled with this. In talking through debugging, you could hear the person's voice get noticeably emotional with frustration as they felt like they were trying everything and nothing was working. Debugging conversations would turn into frustration conversations, and it was obvious that the person would not be able to productively continue until they calmed down
Fatigue errors: It has been studied that being sleep deprived can have an impairment effect comparable to being actually drunk. For me, this is absolutely true. I get physically clumsy when sleep deprived very noticeably. I also miss things. Tired junior coders are very prone to making errors in my experience
Thought distortions: If you haven't heard this term, a thought distortion is what happens when a logical thought becomes emotionally charged and then spirals. Learning to code, it is the difference between I'm struggling with JavaScript and I'm not cut out for this. One of those things is a logical thought. One of them is a projection spiral based on that thought. Other common ones in this vein: I don't have what it takes, I'll never be able to do this, I'm not smart enough for this.
Procrastination/avoidance: When I first learned to code, like many people out there, I initially assumed backend was out of reach for me. I thought my artistic skill could carry me through on frontend if I could pad my work with design related activities. I avoided exploring my options or trying to fairly evaluate my interests because I was operating from a place of fear and doubt. Which leads me to the next point --
Limiting beliefs: These show up in a lot of ways. Delaying the job hunt because you think you aren't ready and no one will hire you. Blanket judgment of certain topics as "hard" and beyond you. Heck, in my bootcamp there was someone in the same area who wanted us to be friends. I initially avoided her because she was too good a coder and made me feel bad about myself. (Happy ending, I joke that she "bullied" me into us being friends, and here we are six years later good pals. She is a CTO!)

So what actually helps?

So first of all, I discussed the fatigue thing. I will say one thing I did differently than the people in my program was, I never stayed up even close to 3am working on projects (some peers did!). I'm a person that needs my sleep, and I was self-aware enough to immediately see that coding tired just made me break my code even worse. Draw from that what you will. For me, I just couldn't. I was getting killer headaches (needed glasses, lol) and was just very quickly like, no. I'm not going to do it. I'm not going to stay up all night.

Second, I will address the context switching thing. Would it surprise you to know that I have never meaningfully learned code from a specific book or video series? I don't like videos and for whatever reason never got into O'Reilly books or anything else either. I've never done open source. I've done Hackerrank maybe 3-4 times ever for interviews. I don't have a presence answering questions on Stack Overflow. I didn't meaningfully network early in my dev career. I didn't bother trying to learn algorithms meaningfully until I was a couple years in. So take a deep breath. This knowledge isn't going anywhere. If at some point you are unhappy with your learning materials, you always have room to change them.

"Changing them" is different than panic-cycling or feeling like you HAVE to do them all, immediately, right away. When evaluating switching resources (let's assume you are in a program to learn to code or have a specific goal that means you have some basic learning resources as core materials).

In picking your materials and practices, learning to code is a lot like developing a fitness regimen: what method is the best? Generally the one you are willing to keep doing on a daily basis. Pick something you can deal with. Focus on being consistent. Focus on showing up.

I also am a huge proponent of arguing with your limiting beliefs and thought distortions. As I left bootcamp and prepared to get a coding job, I would often talk out loud to myself in the car preparing answers to imagined interview questions. Maybe since I was already talking to myself aloud more than normal, I started also talking out loud against my negative thoughts:

Negative thought: You aren't cut out for this
Me: Hey wait a minute, you just started! This isn't magic, people are learning this all around you. Why NOT you?

Negative thought: You're not going to get this job
Me: Fine, but I'm going to apply anyway. I have the interest, so if someone is going to say no its going to be them, not me.

Negative thought: You can't hack it as a real dev, better be sure your portfolio at least looks pretty since that's all you are good for
Me: Maybe I will, but maybe I'll try to learn some backend too. Maybe it isn't for me but it won't hurt to try to become more well rounded

Negative thought: Everyone else is understanding this material faster than you
Me: So what, I am enjoying what I'm learning and I can't stay bad at this forever. Who cares about everyone else

Maybe it sounds kind of hokey, but as I told my therapist about how I interrupt my thoughts she pointed out that I'm essentially doing cognitive behavior therapy within my inner dialogue. Hokey or not, it helped! It helped with the rejection, the negative thoughts, and the hits to my self-esteem.

I would love to give you a recommendation to deal with the last point I outlined, cliqueyness, but for that one all I can tell you is that there are not some special social rules that mean the behavior is OK in tech where elsewhere in life we recognize that it's toxic. Cliqueyness is toxic everywhere. Early in, I realized I was never going to spend time in spaces with that vibe and I've been happier for it. Outside of occasional forays onto stack overflow, I rarely tangle with that side of the tech world.

In closing

I wanted to write this to say, I truly get it: there are a confluence of factors that make it truly a marathon of emotional regulation to learn to code, on top of all the hard learning and the non-emotional challenges!

I told you already, I was NOT a fluid coder. I felt like the ugly duckling of code for a good year or more. The one thing I did have going for me though were techniques to regulate my emotions and discipline my disappointments enough to continue on.

I was telling someone the other day that I feel like my first year of coding was more challenging than all the other years combined and god if that isn't the truth. I never wanted to forget that feeling and it has made me passionate about mentoring and being part of a community that could make newer developers feel more supported than I did. Thanks for reading, and if you have any of your own tricks for staying grounded while learning to code, drop them in the comments!

Diving into ETL and CQRS — developing a secret message encoder with Serialized

JTK — Mon, 04 Apr 2022 22:14:01 +0000

Today we are going to embark on Part I of building a “secret” message board. In the course of creating this app — which is built just for fun — we are going to touch on the concepts of ETL systems, CQRS, and also explore Golang, to which I am still a newcomer! With that said, let’s start with some background on those topics and get to coding later.

ETL (or Extract-Transform-Load)

One of my favorite jobs I ever had was working on an ETL system. Before that point I had heard the term thrown around a lot, but the concept never stuck in my brain past “shuffling data around” — and to be honest, that is the gist!

If it helps contextualize the use case where ETL systems are relevant, I’ll give you the tl;dr for project that ran one where I was Dev Lead. The company was paying something like $1,000,000 per year for some expensive software, so we were building out a replacement for it. But until my project was completely functional, we piped data from the old system into it and slowly migrated users over. The need to keep our new in-house learning software in lock step with the third-party system was the reason we needed to extract, transform, and load data. Every minute, hour, or day various ETL jobs were running to capture the data, put it into the format our systems wanted, and then let it work seamlessly in our app.

Today’s example is going to start off with a rough idea of these concepts to begin with, and then in later posts we will be exploring ETL functionality further.

One interesting note: As you can imagine, people have all kinds of data transformation needs! That is why there are ETL variants like ELT (extract, load, transfer) and ETLT (extract, transform, load, transform). In this article, we are going to focus on ETL, but we can look at some other data pipeline processes later.

CQRS (or Command and Query Responsibility Segregation)

For anyone who has worked with a heftier traditional relational database, eventually you run into problems related to the fact that not all data is of the same nature. Some data is requested millions of times in a fraction of a second but rarely changed. Other data is write-heavy, frequently needing to be accessed and altered. As a result, usage patterns across tables can vary hugely, both in the number, and type of requests your database must handle.

💡 Managing those different usage patterns in a database with a single set of tools and rules is a big ask.

However, command and query responsibility segregation (CQRS from now on to save me some typing) says, “These are different animals! Let’s let an eagle be an eagle and a cheetah be a cheetah”.

In that spirit, I’ll be using an excellent tool in this post called Serialized, which offers a comprehensive data management approach to both event driven data and CQRS. You can read more about them here, or very easily get set up as a developer on their very handy free tier to play with all the cool functionality available.

Getting set up with Go

Last but not least, this will be my maiden voyage with Golang, or Go! Go was developed at Google 10+ years ago and has exploded in popularity due to its performance, concurrency model, and keen developer interest. I had vowed that next time I got to build an ETL system I would do so in Go, so today it’s on! If you are set up with Go already, feel free to skip ahead to the next subheading, but I’m going to first get all the gears turning on my machine.

Not being entirely sure this Mac was even set up with Go, I started with the basics before embarking on this voyage (for non-Mac users, various Go install options can be found here if you want to follow along, and package manager aside most of these instructions should be fine for *nix users).

brew install go 
mkdir go_serialized && cd go_serialized
touch main.go

In my main.go file, I added a simple test just to get myself running:

package main 

import "fmt"

func main() {
    fmt.Println("Running!")
}

And then I confirmed my Go setup was at least taken care of before moving on.

➜  go_serialized go build main.go
➜  go_serialized ./main
Running!
➜  go_serialized

Getting set up with Serialized

The sign up for a Serialized account is here, and I’ll tell you two things I liked about the signup/auth process already. I used their authenticate-by-Google option (one less password to remember). Later on as I began looking for what URL I’d need to feed this app I noticed another nice touch, which is that account differentiation for Serialized is apparently done all by secrets/payload. No special URL to remember.

If you’re new, go through the standard sign-up though. Make an account and a project if you are following along — for clarity my serialized project name is “go_serialized”.

And with that, we’re off!

Project Structure

What we are going to eventually build is going to do the following:

Extract a message from a local file
Transform the message by encoding it with a passphrase
Load it into Serialized
Query it

Being a Go newcomer and this being more of a proof-of-concept, I organized this project pretty simply. On completion, our structure will look like this:

├── api
│   └── api.go
├── etl
│   └── etl.go
├── go.mod
├── go.sum
├── inputs
│   └── message_one.json
├── main
├── main.go
└── utils
    └── errors.go
    └── passphrase.go

As you may expect:

The API code will wrap our network calls.
The ETL module load and transform our data.
The utils module contains logic to generate our passphrase and to give some generic error handling throughout our app.
main.go in the root directory ties all of our logic together.

Keeping secrets secret

Before we get too far, I’ll let you know that if you want to follow along you can head to this repo and pull down from the main branch to get our empty boilerplate structure. (A separate branch called walkthrough will create our commit by commit progress and ultimately have the final project, so I’m going to switch branches).

git checkout -b walkthrough

Switched to a new branch 'walkthrough'

The first thing we will want to do is build out the directories we mentioned earlier, and create an environment file. This will be where we store our credentials for Serialized.

touch .env
mkdir api etl inputs utils
ls -a

.         ..        .env      .git      README.md api       etl       inputs    utils

We don’t want our .env file to make it up anywhere public, because it will contain secrets soon. So let's quickly add a .gitignore file and tell it to ignore .env before anything else. We can then verify this worked with the cat command, which should just return the contents of that file.

touch .gitignore
echo ".env" >> .gitignore
cat .gitignore

.env

Let’s commit our .gitignore and [README.md](http://README.md) before we move on.

git add .
git commit -m "Create README, gitignore"

[walkthrough aa1f032] Create README, gitignore
 2 files changed, 4 insertions(+), 1 deletion(-)
 create mode 100644 .gitignore

Now, let’s navigate to serialized.io and get both our access key and secret access key (my info blanked out for security).

Add to your .env file the values we retrieved here.

// .env

ACCESS_KEY=xxxxxxxx
SECRET_ACCESS_KEY=xxxxxxxx

Awesome! Now we are set up to authenticate with Serialized.

Jumping in with Go

As we mentioned in the directory structure section, the core elements of this app are: the ETL, the API, and the main module tying it together. To begin a Go project, we’re going to run an initialization step for the project, and then re-run our main.go from a previous step to confirm we are set up.


go mod init github.com/jtkaufman737/go_serialized
go: creating new go.mod: module github.com/jtkaufman737/go_serialized
go: to add module requirements and sums:
    go mod tidy

main.go can just be a simple print test for now, since we’re just making sure it’s running.

// main.go 
package main 

import "fmt"

func main() {
    fmt.Println("Running!")
}

Running it once more before we begin, we see we are all set.

go run main.go
Running!

There’s one additional thing I recommend that made this project really nice to work on. For VS Code users, you may have already been or soon will be prompted to install a standard set of Go plugins for the editor. They included some really nice tooltips that helped me debug quite a bit as someone new to Go, but the nicest feature was that on save it will auto-import necessary packages for you and also automatically remove imports you aren’t using. I found this very handy, so if you have the opportunity, I say add the plugins.

Error Handling

The first thing I’m going to do on this project is actually not at all exciting, but is more of a quality-of-life thing. You know I’m talking about error-handling. As I read through various Go tutorials, I kept seeing the following pattern repeated over and over:

/* 
doing stuff in go having a great time
*/ 
if err != nil {
  fmt.Print(err.Error())
}

I’m new, so maybe I’ll hear that people like repeating themselves that way for some reason, but it annoyed me to have those three lines of code some 20x repeated in this project by the end. For this reason my first order of business was creating a little utility function to deal with this and at least cut our three repeated lines of code down to one. Go ahead and make a utils/errors.go file and we’ll add the following code.

// utils/errors.go

package utils

import "fmt"

// LogErrors takes variadic, or multiple arguments for errors and formats/prints them
func LogErrors(errors ...error) {
    // will check if we ever receive no arguments
    if len(errors) == 0 {
        return
    }

    for _, err := range errors {
        // if we receive an argument list, for each that is not nil (falsey) we will 
        // format and print them
        if err != nil {
            fmt.Print(err.Error())
        }
    }
}

For Golang newcomers, a couple of interesting tidbits here:

Go is one of a small number of languages I’ve worked with where nil is our garden-variety falsey return value.
We’ve seen a nice trick already which is that we can specify a varying number of function arguments with the use of ellipses .... You can read more about that here.
Like some other languages, we see a package declaration up top that allows this function to be accessed in the future as part of the utils package.
You may also notice the naming convention as our function is uppercase. This signifies it is public and can be imported by other files.

Information flow for our secret messages

For the next sections, we are going to get into some cryptography utilities that Go has. I will be the first to admit to you that I borrowed the public work of others to figure out how to encrypt and decrypt messages, which we will talk about more soon.

For this article though, I want to recap the application logical flow:

We read JSON in from a local file containing a “secret message” (extract)
We will encrypt that message using a random passphrase (transform)
We will then post a “new message” event to Serialized with the message, passphrase, and some metadata like an ID field.

In the future, the idea for this application is that when you post a new message, the response gives you your “secret code” back. You then choose who to share it with, and only they can access the message. The message recipient, armed with the secret code, will be able to get the message back in plain text. The concept came to me as a high-tech version of passing notes in class as a kid, through the lens of ETL systems. We’re not going to fully implement it now — this article is already going to be quite long — but that’s the big picture we’re working towards.

To start that process, let’s make a new file and set about reading it. Create inputs/message_one.json and add to it a body with some silly value. I went for test test test but be creative with it.

// inputs/message_one.json
{
    "body": "test test test"
}

E is for Extract

Let’s create a file called etl/etl.go where our extraction logic can live.

// etl/etl.go
package etl

import (
    "encoding/json"
    "io/ioutil"

    "github.com/jtkaufman737/go_serialized/utils"
)

func ReadData() map[string]string {
    /*
        going to create a default message structure
        as a map, which to me felt analgous to JSON-friendly structures
        in other languages. It is similar to a JSON object or Python dict.
        In the future it might be more semantic to do this
        with structs so I may revisit
    */
    message := make(map[string]string)

    // ioutil reads our JSON file into a variable and handles errors
    data, err := ioutil.ReadFile("./inputs/message_one.json")
    utils.LogErrors(err)

    // the json package translates this into the format we want
    json.Unmarshal([]byte(data), &message)
    return message
}

If you installed the VS Code plugins I mentioned earlier, you will notice that if you skipped the entire import section of the file above it will automagically generate ALL of the import statements for you, which is super nice.

I’ll next add to our main function in main.go to test out the file reading capabilities.

// main.go
package main

import (
    "fmt"

    "github.com/jtkaufman737/go_serialized/etl"
)

func main() {
    fmt.Println("Running!")

    // First we are going to read in a JSON message
    rawMessage := etl.ReadData()

    fmt.Println("Message from filereader is:")
    fmt.Println(rawMessage)
}

If we run another go run main.go we will see the following output:

go run main.go
Running!

Message from filereader is:
map[body:test test test]%

Perfect! We have confirmed our main function is running and capable of extracting our silly message from the JSON file.

T is for Transform

Our next task, transforming our message, is actually the most interesting and complicated part of the app, at least for now. To be honest with you, I thought about trying to dig into the nitty-gritty of the encryption, but I realized that I had nothing special to contribute to warrant making an original version of it. Plus, the number one rule of encryption is “Thou shalt not roll thine own encryption”, and there’s good reason for that. I’d likely just end up creating vulnerabilities.

I should note that Serialized, the platform we’ll be using to store this information in the Load step, actually has some neat options for real encrypted data (unlike our pet project here, where we aren’t too worried about security). You can read more about the options available for encrypted data on Serialized by checking out Serialized’s docs.

During this step, we will also add a utility to generate a random password to use as encryption salt. This will be what lets the message recipient eventually retrieve and decode the message. Since that part is simpler, let’s start there. Create a utils/passphrase.go file and add the following:

// utils/passphrase.go 
package utils

import (
    "math/rand"
)

var letters = []rune("abcdefghijklmnopqrstuvwxyz!@#$%^")

func MakePassphrase() []byte {
    /*
        Answer informed by this SO answer
        https://stackoverflow.com/questions/22892120/how-to-generate-a-random-string-of-a-fixed-length-in-go
    */

  // rune is a go data type that aliases int32, you can read more about it here:
  // https://zetcode.com/golang/rune/
    b := make([]rune, 32)

    for i := range b {
        b[i] = letters[rand.Intn(len(letters))]
    }

    return []byte(string(b))

For me, coming from higher-level languages, this looks awfully fancy to solve the problem of “generate a random of n length”, but it does the trick, so I’m happy. Let’s verify it works by adding a test to main.go.

// main.go

...
func main() {

  // ...previous code 

  // Custom passphrase to salt our encoded message with
  passphrase := utils.MakePassphrase()
    fmt.Println("Random passphrase is:")
    fmt.Println(string(passphrase))
}

Let’s re-run main.go and see our new result.

go run main.go
Running!

Message from filereader is:
map[body:test test test]
Random passphrase is:
bph@bgzmiegpcry!lvfcp!is^lpywxvy%

Perfect! Now into the actual transformation. In etl/etl.go under our ReadData function from earlier, we are going to add some encryption logic. It isn’t necessary for what we’re building to get every nuance of the below code, although it is an interesting area and you can read some more about encryption topics here if you wish. Ultimately, the important part is that we are using our randomly-generated passphrase in an encryption algorithm on the message text, and then we return both the result of the encryption and the passphrase.

// etl/etl.go
package etl 

import (
    "crypto/aes"
    "crypto/cipher"
    "crypto/rand"
    "encoding/json"
    "io"
    "io/ioutil"

    "github.com/jtkaufman737/go_serialized/utils"
)

// ... previous code 

func EncodeData(message string, salt []byte) map[string][]byte {
    // taken from https://tutorialedge.net/golang/go-encrypt-decrypt-aes-tutorial/
    text, key := []byte(message), salt

    c, err := aes.NewCipher(key)
    utils.LogErrors(err)

    // gcm or Galois/Counter Mode, is a mode of operation
    // for symmetric key cryptographic block ciphers
    // - https://en.wikipedia.org/wiki/Galois/Counter_Mode
    gcm, err := cipher.NewGCM(c)
    utils.LogErrors(err)

    // creates a new byte array the size of the nonce
    // which must be passed to Seal
    nonce := make([]byte, gcm.NonceSize())

    // populates our nonce with a cryptographically secure
    // random sequence
    _, err = io.ReadFull(rand.Reader, nonce)
    utils.LogErrors(err)

    // here we encrypt our text using the Seal function
    // Seal encrypts and authenticates plaintext, authenticates the
    // additional data and appends the result to dst, returning the updated
    // slice. The nonce must be NonceSize() bytes long and unique for all
    // time, for a given key.
    finalText := gcm.Seal(nonce, nonce, text, nil)

    return map[string][]byte{
        "passphrase": key,
        "message":    finalText,
    }
}

To test this encryption, let’s return to main.go and call our new function.

// main.go 
package main 

import (
    "fmt"

    "github.com/jtkaufman737/go_serialized/etl"
    "github.com/jtkaufman737/go_serialized/utils"
)

func main() {
    fmt.Println("Running!")

    // ... earlier code

    // Now we are going to encode it
    encryptedMessage := etl.EncodeData(rawMessage["body"], passphrase)
    fmt.Println("Encrypted message and passphrase are:")
    fmt.Println(encryptedMessage)

}

If we re-run main, we will now see our encoded message logged as well!

go run main.go
Running!
Message from filereader is:
map[body:test test test]
Random passphrase is:
bph@bgzmiegpcry!lvfcp!is^lpywxvy
Encrypted message and passphrase are:
bph@bgzmiegpcry!lvfcp!is^lpywxvy W?[??(/?"?S)c;????,?r?4$?4y??8o?w'

Well that looks like a lot of hot nonsense but signifies success!

Let’s now run a quick quality check to make sure we can demystify our data later on. In etl/etl.go, add the following below our other code:

// etl/etl.go 

func DecodeData(encryptedMessage map[string][]byte) string {
    // taken from https://tutorialedge.net/golang/go-encrypt-decrypt-aes-tutorial/

  key := encryptedMessage["passphrase"]
    cipherText := encryptedMessage["message"]

    c, err := aes.NewCipher(key)
    utils.LogErrors(err)

    gcm, err := cipher.NewGCM(c)
    utils.LogErrors(err)

    nonceSize := gcm.NonceSize()

    nonce, cipherText := cipherText[:nonceSize], cipherText[nonceSize:]
    plaintext, err := gcm.Open(nil, nonce, cipherText, nil)
    utils.LogErrors(err)

    return string(plaintext)
}

Add the following to main.go:

// main.go

func main() {
  // ..previous code

  // Lets check that we can decode it too
    decryptedMessage := etl.DecodeData(encryptedMessage)
    fmt.Println("Decrypted message is:")
    fmt.Println(decryptedMessage)
}

By re-running main.go, we confirm we’ll be able to decode this secret message when the time comes.

go run main.go
Running!
Message from filereader is:
map[body:test test test]
Random passphrase is:
bph@bgzmiegpcry!lvfcp!is^lpywxvy
Encrypted message and passphrase are:
bph@bgzmiegpcry!lvfcp!is^lpywxvy ?q?A_?L?w\??7<?ع4???SA??7X??O????
Decrypted message is:
test test test

L is for Load

The load step is going to connect us to Serialized and open the door for a lot of fun exploring this event sourcing platform and playing with all the functionality it offers later.

Quick heads up: In the next sections you may get a message about missing packages for our UUID package or others. You can run a quick go get plus the link to the package repo to fix this. For me, it happened on two packages, so I ran go get github.com/google/uuid and go get http://github.com/joho/godotenv. The errors will prompt you and supply the exact command.


// api/api.go

package api

import (
    "bytes"
    "encoding/json"
    "io/ioutil"
    "net/http"

    "github.com/joho/godotenv"
    "github.com/jtkaufman737/go_serialized/utils"
)

// for each event, a `data` field will contain custom data, 
// or here the payload we care about 
type PostData struct {
    EncryptedMessage string `json:"encryptedMessage"`
    Passphrase       string `json:"passphrase"`
}

// this is the larger `event` body with a subfield for data
// defined in the previous struct
type PostBody struct {
    EventID   string   `json:"eventID"`
    EventType string   `json:"eventType"`
    Data      PostData `json:"data"`
}

func Call(method string, url string, data PostBody) map[string]string {
    // Step 1: Loading environment variables including our secret
    err := godotenv.Load()
    utils.LogErrors(err)

    myEnv, err := godotenv.Read()
    utils.LogErrors(err)
    /*
         Step 2: We need to build out a JSON body adhering to the following format
        {
            "events": [
                {
                    "eventID": UUID,
                    "eventType": string,
                    "data": {
                        "encryptedMessage": string
                        "passphrase": string
                    }
                }
            ]
        }
    */

    // A bit MORE data wrangling: need a list of PostBody (events)
    type formattedData struct {
        Events []PostBody `json:"events"`
    }
    var events []PostBody
    events = append(events, data)

    finalData := &formattedData{Events: events}

    // then can finally put the whole thing in JSON format
    jsonData, err := json.Marshal(finalData)
    utils.LogErrors(err)

  // set up our network client & error handling
    client := &http.Client{}
    req, err := http.NewRequest(
        method,
        url,
        bytes.NewReader(jsonData),
    )
    utils.LogErrors(err)

  // add our headers so we can authenticate to serialize
    req.Header.Add("Accept", "application/json")
    req.Header.Add("Content-Type", "application/json")
    req.Header.Add("Serialized-Access-Key", myEnv["ACCESS_KEY"])
    req.Header.Add("Serialized-Secret-Access-Key", myEnv["SECRET_ACCESS_KEY"])

    resp, err := client.Do(req)
    utils.LogErrors(err)

    defer resp.Body.Close()

    bodyBytes, err := ioutil.ReadAll(resp.Body)
    utils.LogErrors(err)

    // get it back into a format we like
    var responseObject map[string]string
    json.Unmarshal(bodyBytes, &responseObject)

    // and return it
    return responseObject
}

There’s a lot going on here, but essentially our Call method is a generic wrapper that we can feed a URL, HTTP method, and relevant data to communicate with Serialized and any other sites we plug into. We then use the HTTP package to make our calls, and do some data-wrangling to get things back into a manageable format.

In our main.go file, we need to add a few new things:

we will use a UUID package to generate IDs for our new “new_message” event, which will be part of the payload and the URL we hit on the Serialized API
we will call our new API wrapper function to dispatch this logic
we’ll then do one last print to confirm Serialized got our info

// main.go

func main() {
  // ... previous code 

    // We are going to generate a unique identifier for our event object
    // and embed that into our URL
    newUUID := uuid.New().String()
    url := "https://api.serialized.io/aggregates/new_message/" + newUUID + "/events"

    messageData := api.PostData{
        EncryptedMessage: string(encryptedMessage["message"]),
        Passphrase:       string(encryptedMessage["passPhrase"]),
    }

    eventBody := api.PostBody{
        EventID:   newUUID,
        EventType: "new_message",
        Data:      messageData,
    }

    // Next we are going to post this to serialized
    responseObject := api.Call(
        "POST",
        url,
        eventBody,
    )

    fmt.Println("Response object from Serialized is:")
    fmt.Printf("%+v\n", responseObject)

}

And our final confirmation that the load step ran should look as follows:

go run main.go
Running!
Message from filereader is:
map[body:test test test]
Random passphrase is:
bph@bgzmiegpcry!lvfcp!is^lpywxvy
Encrypted message and passphrase are:
bph@bgzmiegpcry!lvfcp!is^lpywxvy ?f?̮?D/?I?'Ï?ܧ_9:O??yLC????c?
                                                             ?GBw5
Decrypted message is:
test test test
Response object from Serialized is:
map[aggregateVersion: result:SUCCESS taskId:51ccba05-33bf-4552-9539-9e9a547dadf9]

Confirmation of our work

Naturally we want to see a better signifier of what we did than this logging, so let’s head back over to Serialized’s dashboard. It’s the first thing you land on once you log in.

In my case I was testing this a few times so from our main dashboard we see a bevy of events.

By clicking on the last event id in the chronologically descending list, we can check on the payload of what we sent over and confirm our data arrived.

Conclusion

This was a fun one for me personally! Here’s a quick recap of some of the things I learned during the process:

It’s nice to feel like a beginner again every once in a while. I just got promoted to Team Lead at work so honestly it seemed about right that I take a step back and take a crack at Go, which I’ve never used seriously before. Discovering how to work with it and get the first part of this project done was incredibly fun.
Serialized was by far the easiest part of this work for me. We’re going to use it more when we continue this project in a future article, but it worked exactly how I expected it to going in.
I also learned that being away from strongly typed languages for so long has really done a number on me. Particularly, I hadn’t realized how much I leaned on the loosey-goosey nature of high-level languages to deal with things like varying fields and unstructured data with JSON. I still don’t feel like I’ve fully hit my stride in elegantly dealing with JSON inside Go, so I’m going to go chat with some folks I know who use it regularly and see what I really should have been doing with all this logic 😅
Despite the learning curve, I generally found Golang pleasant enough to use, and the docs were fairly good considering its relatively young age. I also was impressed by its relative brevity compared to Java, C#, or any of the other lower-level languages I’ve used in any significant way.

This article was intended to be the first part of a short series, so I’m going to leave it here and get started on building out the other half of this application for Part 2! I am looking forward to exploring both more Go and more of Serialized’s capabilities in the next post, where the latter will feature more prominently. Thanks for reading!

What everyone's getting wrong about bootcamps vs degrees

JTK — Sun, 03 Apr 2022 16:08:21 +0000

Considering how much I tweet, bootcamps and degrees are something I haven't spent extensive energy on. Compared to the numbers of tweets about animals, hiking, food, or just making terrible jokes I've been next to silent on this matter (comparatively).

But I did recently have this to say and I meant it.

// Detect dark theme var iframe = document.getElementById('tweet-1494010598023213057-620'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1494010598023213057&theme=dark" }

For the better part of a decade spent in tech, I've had to hear the endless choir of people pontificating about bootcamps and degrees and self-learning, each acting as if they were bringing down the word from the mountaintop. I've worked places where they assumed I had a degree and even repeatedly degraded people without them in front of me and I functioned as a “degreed engineer” with Schrodinger's credentials, you might say.

But now I am done, I have the degree, and I'm ready to talk.

Part I: No Degree Days

Why I don't/didn't have a degree

I'm writing this on February 21, 2022. Saying "I don't have a degree" will be accurate for about two more weeks, ish. I mean that's when my classes are done. I guess it gets conferred in May. But honestly, who gives a s@#$. The second that last assignment is turned in, in my mind it is done. (Updating this writing March 11th 2022 and I received two As, yay!)

For anyone wondering why I didn't manage to get a degree until now, there are many reasons. I didn't know for sure what I wanted to do. While in college, my dad got very sick and it became extremely difficult to manage his constant hospitalizations and keep up with the coursework. The teachers were not understanding. Also, I was poor and it was not clear that I would make enough money even with a degree to pay back student loans, and thus was hesitant to take them out (my sister had just gone through finishing a degree that she couldn't find work in, and her example was informative to me).

I'll pause here. For anyone whose thinking is so rigid that they don't understand that any one of those reasons by itself is good enough to not get a degree, you may as well stop reading.

It is a fool's errand to pressure people into getting degrees when they don't know what they want to do. It is harmful that we make students feel like they must go on when they are facing life and death family crises. It is irresponsible, honestly, it is like a pyramid scheme that we bully people into taking on debt to buy their way into a middle-class life when increasingly a degree does not guarantee it.

What it was like without a degree

As a developer, I have led a surprisingly charmed life. I made senior rapidly. Dev Lead early too. Money, equity, and what now comes easy once seemed like a distant fantasy. Early into my life, it at one point seemed impossible that I'd ever be able to get an office job at all or anything out of service/manual work.

It is not an exaggeration to say that doors were rudely slammed in my face, over and over and over again during that time of trying to find a first office job, over not having the degree. After the recession, there were enough job seekers that you could get rejected for $10/hr administrative jobs for not having a degree. (Nothing about that work requires one, for what it's worth).

There's a study I came across about how poverty lowers IQ, and I absolutely believe it. What I remember from this time were multiple prolonged cold wars waged between me and being poor and being an undesirable worker.

So bleak was my path in getting into any office role that I could leverage into a career that I found myself over and over again looking at online degree mills. When I say "degree mill" I do not mean some online college people turn their noses up at - I mean a literal degree mill where you mail them a check for a fake diploma. It was made so crystal clear to me that the degree was the problem, I really almost did it even though it would certainly have been grounds to be fired if I did and it came to light, it might have counted as actual fraud. Things looked that grim.

That's probably the most distinct memory I have of this time that I'll never forget: sitting in front of my computer with the degree mill website in front of me. I imagine if I could have looked at myself, I would have seen an eerie animation borne out of raw longing written all over my face. The lack of a degree was a black cloud trailing me, constantly.

That, and my credit score. Simultaneously I schemed and stewed and plotted as if in a long, arduous chess match. How do I get a job, any job where I'll build marketable skills? How do I budge my credit card up from 500-something? I can only imagine my IQ was lower. It felt like being haunted. I was always preoccupied.

Whatever forward momentum I now enjoy as someone with assets, the opposite was true as someone without them. To clean up my credit I paid additional fees for a debt consolidation service and high-cost, high-fee credit cards to begin to have a history. It was pay to play. But still, a far lower entry fee than what it would take to attend school.

I still feel strange sometimes saying that I was “poor” but I don't know what else to call it when they were at various points chasing me around trying to repossess my car, or that when living in Baltimore we kept the heat inside to 50 in the winter and used space heaters because me and my roommates were all broke. When I finally got a first office job, I volunteered at every opportunity to travel because I got a daily per-diem for it, but was constantly at risk of embarrassing myself on these trips with colleagues if we ever had to spend money because, with one wrong move, I would easily hit my credit limit or overdraw my bank account. I once volunteered for an extra two weeks away because I was too broke to pay rent and it gave me a pretext for the check to be late (ahh simpler times without Venmo). So, yeah. Plenty of people had it worse than me but I was pretty broke.

And yeah, it was miserably hard to live that way. When eventually I got a low-paying job but where I was taught transferable skills, there were no limits to my tenacity (for those of you who think people don't finish degrees out of "laziness"). I was a recruiter, and so had no reason to get tech certs or a security clearance but I saw a pathway. I was opportunistic to the extreme.

Anything I saw that would give me a foothold, give me legitimacy, give me something to propel me to a better life I grabbed on as hard as I could and didn't let go. I had no fixed plan of what I would be, I would have taken anything available to me that moved me forwards. The one exception to this remained school, infuriatingly. As a white-collar worker, I began to have tuition benefits at work, up to $5000 or so dollars which is the limit the company can use it as a tax write-off.

Here's the catch: you typically had to pay up-front and then be reimbursed. I never had the money. You also had to repay the money if you left, and I was constantly expecting the sky to fall and to be laid off. It's amazing when you don't have money how much you have to worry about it, constantly, to not be caught off guard by the next disaster. The tech certs, though, the company paid up front. So that's what I did.

Although this tenacity launched me out of being poor and into a stable enough career, the lack of degree continued to embarrass and dog me.

Fun things people have said to me about not having a degree

I would be remiss to not give my mom a shoutout here. I’m not in touch with her anymore but at the point of me struggling so hard to get into any office role, I was. At the point that I told her how frustrated I was by my job search and how little luck I was having, I once said something like “I'm so disappointed I didn't get that one, I know I would have been great at it,” and she replied something to the effect of “You aren't qualified for it, you are qualified to work at Target”. I'd be lying if I said that line won't be burned onto my brain forever, although it's pretty funny in hindsight. (Maybe I SHOULD work at Target, I shop there enough – could use the discount). But at the time, I knew it would do nothing to advance me towards something where I could use my brain and actually build a career.

Of course, there were other places where I got dinged over the lack of degree. In the screening call for what would wind up being my first programming job, they were so abjectly horrified when I answered that I didn't have a degree that I felt like an untouchable, like I might as well have had cartoon stink lines coming off of me. I finally just blurted out (having never done so before) that my dad had gotten very sick up until his death that year and I'd dropped out for family reasons but was trying to go back. After a stunned and uncomfortable silence, they apologized and moved on.

When hired, it was clear that because of my lack of degree they perceived me only suited to be sort of a code plumber, an unrefined hand who could do basic tasks (unlike the music major they also hired at the same time with no experience, which makes sense how?). My boss made his disdain for my unfinished degree known, he brought it up in our 1:1s frequently. And with great, great irony that job was in the Education unit of a research institute. So, I had the dubious honor of being the least educated person IN THE EDUCATION GROUP. Fun times.

It has actually been fun and funny at times to observe all the different reactions people have to the information. Stuff like, “But you are so smart!”, which.... thanks I guess?

My all-time funniest/favorite is when I have been in situations and realized that even with a degree there was a higher level of arbitrary gatekeeping that would have kept me out anyway. If my life had gone differently in a couple of spots I would probably be the proud owner of a Graphic Design degree from a state school. I've been in a small number of interviews where they led with “Well, we hire engineers from elite colleges but someone from other schools could start in tech support and potentially work their way up but it isn't a guarantee”. Cool!

At some point working in tech though, it stopped mattering. Actually, I guess I can pinpoint when: 2 years in. With two years of experience, it seems like no one gave a lick whether I went or not. Either that or people are exceptionally bad at reading. I put my field of study under a deliberately ambiguously titled Education/Training section on my resume. I never lied. I listed the field of study with no degree or graduation date. Having been burned over and over and over again for not having the degree, I suppose at that point I decided “if you don't care enough to ask, I don't care enough to tell you”.

It was always very hilariously clear when people assumed I had it and I didn't. A colleague once guffawed over me not knowing what Conway's Game of Life was and exclaimed “Did you even TAKE computer science!?”. In the moment I answered “Nope, they must have saved that for the fancy kids doing the Master's program” but internally I felt like shit. This colleague was a friend and I know only ribbed me that way because of the assumption that we were peers and it was in no way punching down. Little did they know, they had hit me where it hurt. Although having now finished a degree, I can tell you I STILL would never have learned about Conway's Game of Life! I have since of course looked it up. I do not feel as if it made me a better engineer to know it, lmao.

The other truly, truly funny one was a boss I had. I forget what level of education he had but a degree for sure, and he definitely assumed I had mine as well. He used to say benevolently rude things like, “You know JT, we could hire someone with occupational training and they could string code together but they wouldn't be able to write algorithms like you or (other teammate).”

Reader, I s@#$ you not, this man's code was THE WORST I'VE EVER SEEN. I WOULD GLADLY HAVE HIRED SOMEONE WHO “JUST STRUNG CODE TOGETHER” OVER HIM. He wrote API endpoints that were concatenated together in what sounds like a joke of a SQL-injection nightmare. He wrote all his joins in what would have been cartesian products/cross joins without the database engine cleaning up his misdeeds under the hood. He wrote a new-ass-application in Python 2 the year it was to be sunsetted when the maintainers had said they wouldn't even patch security bugs going forward, just because he didn't know about system python. He sure liked me though! He made me Tech Lead. I had an even more talented colleague at that job who switched teams to get away from this boss, and with great irony I later found out it was because the boss was giving him a hard time about having an unrelated Construction Management degree. (This dude was amazingly talented, and could code circles around the boss. Eeesh).

A lot to unpack there but I'm going to throw away the whole suitcase.

What someone without a degree would want you to know

The only thing I will ever say no one can accuse me of is a lack of work ethic or ambition. Whatever you want to say about me not having a degree, you could chalk it up to just about anything in the world but there is no credible case to be made that laziness or lack of tenacity were in play. I'm not unique.

I didn't write the first half of this post to be "woe is me", I'm obviously very lucky and have enjoyed a drastic change in circumstances. Honestly, it's been so drastic that for all I know I'm just part of a bubble about to pop and my fortunes will all reverse the next economic downturn. The reason I mention it is, that I was always the person capable of having the degree I'm about to earn. I mean heck, I used to work harder without it, I had to exercise such a high level of showmanship and boldness to compensate, I was probably a better worker.

So please take away from this that education != intelligence, education != work ethic, and in many cases, it is just a reflection of someone's luck. When you judge people harshly on it, that is in fact what you are judging and it is frankly crappy.

Part II: Bootcamps

What brought me to bootcamps

When in 2015 my father died, he had some miscellaneous life insurance policies that resulted in me inheriting something like $40,000. That might have paid for school. Unfortunately, the price I had paid to get to the fringes of the middle class was to work jobs with ridiculous hours. I felt extreme concern that I could keep it up if I added school.

At this point, my recruiting background had taken me into recruiting for technical roles in government contracting, most often for Network Engineers and Cybersecurity professionals. My claim to fame in recruiting was that I was very good at boolean searches, which was fortunate since government contracts often had roles with complex mixtures of security clearance and technical certification requirements.

There is an entire DoD regulatory proviso code for the Security+ exam by CompTIA (DoD 8750), so invested are government agencies in these various tech certs. Some jobs had requirements with a complex mix of certification combinations, where you would need one or more certifications from group A, one or more certifications from group B, and then a security clearance.

Point being, I come from a world where ongoing professional development was very respected and in-demand.

(One thing it's funny to note here. Probably based on what I said about being so broke, you would think I got into tech for financial security. Nope! I took a 16k pay cut for my first programming job. My dad's death was really hard on me and I felt like I just couldn't talk to people for a living anymore. It felt unbearable. At the end of my recruiting career, I made 70k which was a fine living in 2016 and if I had liked it, could have pursued management if all I wanted was more money. Along the way of trying to be a good recruiter and learn tech terms, I crossed over to the dark side. It was so much more fascinating than having the same intro candidate call dozens of times a week. Although there is nothing wrong with getting into tech for money, I actually have done it out of love from day one. And that's the only reason someone with so little natural talent at coding became so successful. I just happily plugged away at it in most of my free moments until I got suddenly shockingly good).

After my dad died and I debated using the inheritance for school vs getting more technical certifications, somehow strategically placed advertisements for bootcamps found their way to me. I immediately liked the idea. It sounded like a combination of instruction and apprenticeship, I would actually be BUILDING things not just memorizing and regurgitating things. I bought in completely.

It is worth pointing out that although I wasn't recruiting developers heavily, I was in the tech industry and had no idea that bootcamps were widely criticized and considered by many totally illegitimate. I want to pause on that.

If me, a white-collar worker actually already working with these types of roles and around tech people, wouldn't know the baggage that came with that path, how do you expect someone working at Mcdonald's and dreaming of a better life to know it? Part of the hate on bootcamps honestly in effect s@#$s on people who risked a lot and put their neck on the line trying to do the really hard work of building a career. So, consider that.

The mixed bag that is bootcamps

Here's where it gets dicey. I just finished telling you that you need to show people who attend bootcamps some respect. I can't and don't necessarily say the same for the people running them.

I attended General Assembly's first remote Web Development Immersive program in 2016 (since rebranded). I very clearly remember that the marketing was emphatic that the program could take you from zero to sixty, even with no experience. I even had some experience! I had been doing Coursera, CodeAcademy, and technical certs, I thought I'd be coming in from a good starting position.

I was very wrong.

About half the people in my bootcamp were technical. One did tech support, one had a CS degree and had freelanced, one was a graphic designer who did some code, on and on. By the end of the first unit, about half of the cohort's projects did not pass, a condition that could cause us to fail out of the program without a refund. As you may guess, the entire "almost failing half" were me and the other people not working in technical roles before joining the program. (I remember now and laugh that on the day we were supposed to present our projects I said something like “This is my project and spoiler alert it doesn’t work”).

With that large of a group failing, it was impossible to make us individually shoulder the blame - it was too blatant. The bootcamp split us up into groups, with mine to repeat our first unit. As time went on, myself and one other woman in the program realized how screwed up it was that we were missing instruction and content that we paid for (at the time, on MEAN stack, all the rage). We raised hell. We made them agree to extra workshops after and other conciliatory measures.

As a sidebar, I felt such stress and desperation that I had thrown all my eggs in this basket (quit my job, spent all that $) and then let down by the system I'd pinned all my hopes on, my brain got mean and focused and steely. After the first unit I moved, up-up-up towards the top of the pack in my group. I was stressed, I was angry, and I was scared but I channeled it and I liked code, even as bad as I was back then.

Since then, bootcamps have arguably only gotten worse. Parasitic income sharing agreements are a Faustian bargain. Bootcamps are functionally unregulated, and cherry-pick statistics to give the reporting body CIRR, or just stop doing so altogether. The inexperienced instructors and recycled recent graduate TAs don't know what they're doing. As the market is flooded with junior candidates many of whom are all trained in the same things, bootcamps have to pivot wildly and come up with new curriculum. I could go on and on and on, and often do.

Some of them have gotten better. I have seen B-Corps, free bootcamps, bootcamps to get former convicts into coding, schools free to disabled people. Some of the programs are now two years long. In response to critiques that bootcamp graduates don't know data structures well, some have added things like that. There are also orgs like Code The Dream that give me a lot of hope in the nonprofit model, they follow the bootcamp with an apprenticeship period building sites for nonprofits to give the students real experience.

Least favorite takes on this topic and rebuttals

“Bootcamp people are ok as programmers but will never be software engineers”

As someone who just finished four years of schooling largely debugging and learning alone, you can f@#$ right off with this. Nothing magical happened in my degree that transformed me. There was little new in my coursework and I could plow right through on code alone. Most of our books were open-source resources.

“People who join bootcamps need hand-holding and structure, but if you don't you can self-teach”

Yes, I hear this, not always this clear but I hear comments reeking of it often. You know why I wanted instructors? I had worked around software engineers and the field seemed hard. I wanted to do it right, I wanted to give it my all. I trusted that people in the field might have something to teach me. I trusted that learning these skills would be difficult and I wanted to give myself the best possible chance of success. Wanting instructors came from a place of humility. It was a lot of money and a lot of risk, but it was a representation of my commitment to choose active instruction – I wanted to do it right, I was giving this challenging material respect by wanting a guide.

What someone who attended a bootcamp would want you to know

It is possible to be critical of bootcamps but still respect the people who attended them and treat them as people who took a huge risk out of willingness that they could build a better life for themselves and their family. Doing a bootcamp is PAYING to work extremely hard. These people don't deserve your scorn or your in-group out-group blanket dismissal.

What does it say about you if you are someone that enjoys deriding people for trying to learn something? What did it ever do to hurt you? What makes you the authority on high to pass judgment on someone else just trying to be compensated for their labor and make a living? The more you unpack it, the more appalling it sounds.

But the number one, super-duper incredibly important thing I need you to remember is this. Yes, you, you who secretly has a low opinion of bootcamp people: there is also a lot of bias at play here, and I don't mean credentialism.

It is a known fact that bootcamp graduates are far more diverse across multiple categories than CS degree students. In-group/out-group bias is a real thing and no matter how nice of a person you are, the human brain was built to categorize and we all must contend with that. It is not a coincidence that this group of people who is maybe queerer, browner, and more female than you and all your CS cohort peers show up and the initial brain reaction is mistrust or disbelief. As one study of in-group/out-group bias puts it, “Experiments […] have been repeated time and time again, demonstrating that the favoritism people show for their own group doesn’t need to founded in anything particularly meaningful.” The same linked article goes on to say:

Another basic truth about people: we have a need to feel positive about ourselves, and we are frequently overly optimistic about how exceptional we are compared to other people. These processes of self-enhancement guide our categorizations of ourselves and others and lead us to rely on stereotypes that demean the out-group and favor our in-group. In short, because our identities are so heavily reliant on the groups we belong to, a simple way to enhance our image of ourselves is by giving a shiny veneer of goodness to our in-group—and doing the opposite for our out-group.

In this sense, I feel for people with skepticism about bootcamps. Your brain is really not set up to like this concept. It is invalidating to people to have spent ~2 years of a Bachelor's Degree on Computer Science concepts and have someone who spent a fraction of that be competing at your level.

I recently saw a tweet that was a pretty terrible take tbh but the comments were illuminating in one way.

Above ^ this the poster had a video of them saying “I took six years to get my degree, you went to a three month bootcamp. We are not the same”. I laughed a little at this because the “no one is better” at the end clearly is tacked on to avoid criticism. This person riffing on the “we are not the same” meme makes it clear they think one is better than the other. Some of the replies were:

And another, that most closely supports my point about in-group/out-group bias:

While I'm skeptical that anyone who knew Canva alone was off finding work as a designer, this person recognizes what's happening in their own cognition: they feel invalidated by people from other paths. It feels invalidating to the work they did and that's upsetting to them. I appreciate that they had the clarity to recognize it that way because at the end of the day I think that's part of what is going on with a lot of people.

One final thought to leave you with: if like the poster above you find yourself thinking “a few months can't compare with years”, I want to reframe that a bit for you. An average college student goes two semesters a year with a 12-credit workload. The average semester is 15 weeks, with 2.5 hours of classroom time per course. So, 10 hours a week over 15 weeks, x2, or 300 hours of instruction time per year. Using General Assembly, my own bootcamp as an example, the schedule was 10-6 with an hour break at lunch. (That is not including time alone coding afterwards for homework). So that's 7 hours a day for 12 weeks, although I'm going to subtract that last week because I think we spend it on the final project. Let’s call it 11 weeks x 35 hours = 385 hours of instruction. Draw from that what you will.

Part III: The Degree

What made me decide to finish the degree

This is a question I get asked A LOT by bootcamp people - “Why did you go back, did you feel like you needed it?” and it isn't that easy to answer. People in this field feel all the time like they are “imposters” regardless of background and experience, and I wasn't immune. Being a woman in a male-dominated field, it would be a lie to say that I never felt the need to be unimpeachable in my track record, credentials, and output. Having seen the fate of people without degrees in the last economic downturn, it also seemed like a way to protect myself. If the job market ever got bad again to where employers just needed to filter out some arbitrary number of people, degree requirements are a very easy way to do it.

I also just wanted to know. To know what it was exactly that I was missing by not going. What exact things I wasn't aware of that seemed to make this whole industry think people with my background were a joke (in the abstract anyway, individually people treated me well and I was obviously very successful even before finishing the degree). What exact topics a degree would cover that would take me from the fringe to being a Real Programmer :Tm:. You might say, I wanted the degree just so I could tell everyone to STFU up about this all. (That is literally what I tell people who want to talk about this topic, as I showed in the tweet included in Part I. There are a very very small number of topics where my mentality is “you can't tell me s@#$ about this”, but this is one of them. I have the complete perspective. At this point I have done a degree, done a bootcamp, and have professional certifications. I have tried it all).

But was it worth it?

The mixed bag that is degrees

Ok so. You've already heard that I was not super impressed by my bootcamp and it would probably be more satisfying to end this arc by me saying the degree was far superior.

Meh.

Some of the stuff was pretty cool. I think my favorite courses included one where I got to use JavaScript and Flask to make demos of different software vulnerabilities. I came up with behavior with VueJS to imitate a bot doing a brute force attack on a login page for instance. By far the HARDEST courses I took were actually during a brief time when I was doing a Network Engineering major and had to do coursework aligned with the CompTIA A+ and Network+ certifications. I don't know how hard the certifications were, but the classes relied on all these virtual labs and they were incredibly time consuming and finicky, if you did things correctly but in the wrong order you wouldn't get points, that kind of thing.

Most of the coding, I could just show up and code without much paying attention to anything else. Ironically, some of the things I was best at (SQL) were some of my worst grades, just because I would do things the “industry way” not the “textbook way” or use techniques that accomplished what was needed but weren't what the book was trying to highlight, etc. There were some more relevant things (like AWS) and some less relevant things, like assignments so old that they were dependent on Java tools that could no longer be installed on modern machines. At the end of it all, it was probably equal parts Python, Java, JavaScript (used in classes where I was allowed to pick), and SQL. Not a bad mix for employability.

I will say that neither the types of tasks we were assigned nor the way content was presented were particularly engaging. A lot of “build a desktop Java app to do payroll”, “build a Java app to look up semester grades”, “build a Java calculator”. As someone who didn't like Java, I definitely had the realization that if college HAD been my first introduction to code, I would have abandoned it. I would have switched majors I'm almost certain. If I thought all coding was was building dull apps with Java, where it takes a bazillion lines of code to do the simplest thing, I would have noped out hard.

The degree also pointedly side stepped a lot of extremely relevant industry concepts. There was one class where a classmate managed to bork his code irreparably and I was trying to help on the class forum. I was curious if anyone would tell this person about version control to save them from similar issues in the future. Knowing the professor would respond I said something like, “It would be great if there was a way to go back in time with code”, wondering if they would mention git at all. Nope!

We also got assignments where to follow the instructions the cyclomatic complexity of the end result was horrendous. To get an A, you had to follow the arbitrary instructions even if the resultant code was poor. We also spent an honestly, somewhat embarrassing amount of time on things that are decidedly not industry standard anymore, like drag and drop UI builders in Java, incredibly old versions of Oracle, etc. Some of the more relevant things we learned were approached in very weird ways, like having to use spool files (???) for SQL, and topics like types of software testing or code efficiency were entirely skipped while instead, we spent a mind-numbing amount of time on things like the Software Development Lifecycle.

Least favorite takes related to this topic and rebuttals

“Degrees are pointless”

Look, while ultimately mine may not be worth it you’ve seen from my experiences listed earlier how rudely the world may treat you for not having one. That’s not to mention that for groups that aren’t well-represented in tech there is often additional pressure to “prove” ourselves, and it is not always possible to be taken seriously without credentials. People might do it to advance themselves, just out of the curiosity to learn, or a multitude of other reasons. Same as we aren’t going to s@#$ on bootcampers for the act of learning, we are not going to s@#$ on people getting degrees. Why even would you? What is scorn worthy about someone wanting to work and learn?

“People who get degrees are fools, a bootcamp is enough”

Nothing is enough. Truly.

Our society has increasingly moved to requiring degrees even when they are only nominally of benefit for the role or even irrelevant. Automated hiring is filtering people out in bizarre ways like screening nurses who need data entry skills for computer programming.

And yet...

I don't know if I know ANYONE who walks onto their first programming job and doesn't feel kind of lost. In mine, I was able to immediately make contributions with bug fixes but even so was in way over my head on other things. Most schools still teach Java or C, most workplaces use a variety of things. If I extrapolate where I'd be starting from with the degree alone, my whole atrocious learning curve with git would have had to happen at work, I would have known little to nothing about CSS or HTML much less accessibility. It would still have been a steep, STEEP learning curve.

If someone feels like going to college because they want to feel prepared for the workplace, good for them! Just as it doesn't invalidate people from college for bootcampers to exist, it is pretty iffy to completely disregard any time spent learning period.

Also: this statement is clearly wrong, there is a reason people continue to find CS grads more prepared although it is not the reason people think and I'll elaborate in the next sections. Whenever I hear someone so self-satisfied with their conclusion, I can't help but think of the saying, trust those who seek answers not those who claim to have found them. Anyone too gung-ho with no nuance, I do not trust your conclusions.

What someone with a degree would want you to know

The reason I’m so sick of how people talk about this subject is the complete lack of nuance brought to it. And yes, you are hearing me correctly that my view that IMO does have sufficient nuance boils down to, people may be leaving degree programs slightly better prepared than bootcamps but there is also a lot of bias impacting how we evaluate people in these two groups, and the difference has absolutely nothing to do with degrees being superior or the people who get them being superior.

At the end of the day, despite everything I've said, me with a degree may still be a little better than the version of me existing in some alternate universe without it. Not because the degree itself was special, but simply through the act of having to engage with the material and the extra hours put in. That didn't need to occur in school and I don't know that school was even the right place for it. ALL of the more advanced things I have learned or done in my career have been learned on the job, period. But attending school of course didn't make me any worse.

As time has gone on, as I've worked with people with and without degrees, as I'm now on a team that is made up largely of career changers with degrees (but unrelated) who are among the smartest colleagues I've ever had, my views on this entire thing remain hard to put into words. While I DON'T think the act of attending college is magical, I am comfortable roughly saying that college does seem to generally be producing more prepared entry-level programmers than bootcamps are.

As time goes on, the more I see though, I am increasingly sure that it is not because these two forms of training are so fundamentally different or that one is inherently better. The longer I go on, I think the benefits of college are simply that it goes on longer. There is evidence that we retain more when concepts are explored slowly. Something called the spacing effect has studied that our brain holds on to more and benefits from superior long-term memory performance when the time spent learning is spread out. At the end of the day, I am of the opinion that that is the only difference that matters in this debate.

Part IV: Where does this leave us?

If everything sucks, what's the point then? What's the solution?

Well, okay, so I know I was critical of my bootcamp and now critical of my degree. That being said, I wouldn't change either experience. When prospective career changers ask me which they should do, or ask what language they should learn, or how they should spend their time, I often tell them something like “any time spent learning will be valuable”. Pick what interests you, or what you think is the direction you want to take your career. It doesn't matter. I have a suspicion if you pick something enjoyable to you it will enable you to spend more time learning and overall have the biggest ROI.

The real takeaway I would like people to have from this post is that bootcamp vs degree training is not a valid premise for evaluating programmers. Period.

If your brain has a sticky attachment to the idea of this hierarchy, ask yourself why. What benefit HAS ever really come from broadly profiling a huge group of people without any nuance? What benefit does it really give you?

An alternate framework

With that said, we reach a conundrum: if concrete credentials are not the way to evaluate programmers, how do we? We all know everyone hates whiteboarding and take-home projects. Although I will say, certainly me feeling stupid for an hour while live coding is less trouble than getting a whole ass degree.

At the end of the day if a company wants to have an entrance test and uses those in lieu of specific years of experience or degree requirements, I don't think that's the worst thing in the world. It at least would be something consistent. And while yes, it's a time commitment, again, less so of one than either a bootcamp or school to prove yourself.

But really, I don't even think that's a necessary alternative if degree requirements were scrapped.

One philosophy in hiring that I like is the competency-based approach, described here as being the solution for hiring challenges where people were coming in with the right backgrounds, degrees, etc, and still having issues on the job. This approach relies on job profiles and structured interviews, and as discussed in the article above has also been linked with facilitating more diverse hires. It is also an excellent way to account for the fact that not all years of experience are equivalent and some people progress faster than others to the same level of mastery. This article goes on to say that naturally there are numerous ways of doing this for software engineers, whether it be assessments, technical questions, whiteboarding, etc.

I think we'd be better off even with some combination of measures. Like a degree requirement OR you can do a take home OR you can whiteboard to demonstrate whatever level of mastery. Ultimately, my opinion is that you can assess an engineer without the whiteboarding but if it moves places away from arbitrary degree requirements then fine.

Also worth considering

It is also worth noting that both bootcamps and colleges have numerous other issues I didn't even get to cover here. My personal opinion is that the bootcamp model is not worth throwing away despite the issues with poorly run and disorganized ones, but it is screaming for legislation. Some of the things I've heard go on are borderline fraudulent, like ISA repayment being triggered by jobs that are barely tech-adjacent.

It is valid to have specific issues with the way bootcamps are run, but it is also valid to have extremely real concerns that stop you from wanting to go to college. When colleges shut down due to the COVID-19 crisis students were displaced from housing and meals that they were still on the hook for financially, and many are now suing. This excellent episode of Patriot Act addresses that increasingly schools cost is only half related to genuine fees, and schools are using atrociously paid adjunct professors to keep their own costs low while institution president salaries and investment trusts have exploded. The number of administrators at schools has increased in some cases 135% while adjunct pay has gone down 50% but executive level salaries are up 35%.

That's not to mention, that for cases like mine with a sick parent, it isn't like in return for all that money you are getting any kind of understanding. Even in the recent invasion of Ukraine by Russia, students requesting deadlines to account for the fact that they are literally in a war zone are being refused. I'm not even going to touch the ongoing embarrassment of sexual assault on college campuses and its abysmal handling. So with all that said: even if college might send you into the world SLIGHTLY more prepared for an entry level programming job, is it worth it? At the end of the day? For some people the answer is no, and for others it’s a moot point: they don’t have the money, they don’t have the time, and it was never in the cards for them.

In closing

Where does this leave us? The prospect of learning challenging material with no direction is somewhat bleak.

If I had it all to do again and had known about these options, I might have turned to something like the Open Source Software Degree or a new development, one I have my eye on as potentially the most promising revolution in software engineering education that I've heard of: the Free Code Camp accredited, free(!) computer science degree. I think if supplemented with learning groups like I've found on Slack communities, an option like this when combined with community could likely do as good of a job as the path I wound up taking.

If I can ask anything of anyone, after this absurdly long diatribe on this topic, it would be to realize that people's lives are messy and unpredictable. The same options are not realistically available to everyone. When compared side by side, the amount of instruction in a bootcamp vs a degree is not actually all that different, and there is nothing about degrees or the people who get them that is inherently superior. I truly believe that if CS grads are coming out slightly ahead, it is a function of the degree having a longer duration and nothing else.

Take a moment to examine why you might feel so strongly about someone else's educational background when it doesn't hurt you either way. And overall, realize that people's capabilities are not neatly assessable through the lens of bootcamp vs degree vs self-taught.

Negotiation Advice From a Former Recruiter

JTK — Fri, 11 Feb 2022 21:39:15 +0000

Something I enjoy - A LOT - is helping people negotiate, particularly for software engineering jobs, particularly if they are early-career or coming from a different background. To people in other fields, or new to this field, some of the stories I have from negotiating software engineering jobs sound made up. You got what!? They offer what kind of signing bonus!? They're paying what to mid-level people!??? I personally am of the belief that the boom times will not last forever, so I try to encourage people to seize the moment of the particularly competitive job market we find ourselves in.

As a disclaimer, I live in the US and that's also where I recruited except for one weird assignment in the Kingdom of Bahrain. Other countries are of course, different with different markets/salaries than what I may mention.

Part I: Get Your Mind Right

Your negotiation stance is a collection of beliefs, behaviors, and boundaries that starts far before you have received a job offer. Sometimes I see new people say "I'm open ended on salary for my first job, I really just need the experience".

They aren't wrong - a single year of experience in software engineering can be leveraged almost immediately into a 10, 20, 30k raise, perhaps more. But this approach is still slightly flawed.

It is good to have some flexibility if the ends justify the means. But I encourage you not to limit yourself here. The biggest reason for that is that you are negotiating not just for current you, but future you.

Scenario: you get offered a really great position at a very entry level salary. You join and it is everything you wanted, your skills grow phenomenally and you love the org and team. But within one year, that entry level salary adjusted 3% for cost of living is far beneath your skills. It is too irresistible to leave 20k plus on the table, so you leave after a year although you'd have liked to stay.

Negotiating aggressively is betting on yourself, and how much your new job will be a great fit for you for a long time. You owe it to yourself to set the tone with a salary you'd be happy with for a long while.

This is not limited to new people. I also see experienced people who have stayed with an employer say, the past five years vastly underestimate how much salaries have exploded over the past couple of years.

If you go into the job hunt with an undefined sense of what you should aim for, someone else will try to define it for you. What's worse, sometimes a sense of "sunk cost fallacy" will mean we go farther in interview processes than we should even though our spider-sense tells us this job may not be right. Defining your salary goal and your hard limits as part of your negotiation mindset up front can save you a lot of interviews, and who likes those!

Part II: Things to Figure Out Way Before the Offer

Average and stretch goal salary: anecdata, a lot of people I know job hunting in 2021 were starting from about 125 base, hoping for 150 and with a stretch goal of 180. The collective experience of people I know in that boat was that getting to 160 was surprisingly doable, and that everyone was giving out signing bonuses like candy. Some of these conversations happened in developer communities, and I have no doubt people on the sidelines at 125 that had been seeking 145 adjusted accordingly based on what they heard.

Benefit hard limits: I fell into this trap at one point. Entertained a role with under three weeks PTO. The farther I got into the process, the more I realized that amount of PTO worried me, especially in the time of COVID. It wound up being a dealbreaker. I honestly wasted everyone's time and went through a lot of stressful interviews I could have skipped by not knowing that limit sooner

Tech stack hard limits: Another one I speak from from experience. I would have not thought this mattered to me, but at a certain point I realized I didn't want to walk away from Python and start over with something else

Interview hard limits: Is this related to negotiation? Technically yes. It makes perfect sense to be willing to do a stupid amount of interviews to make Google money, but otherwise nope out.

Your real current comp value: Consider that imaginary tech workers Joe and Sarah both make 200k. Joe has three weeks vacation while Sarah has six. Sarah is actually in the grand scheme of things compensated annually ~$11,640 more than Joe for her time. Does your work have a tuition program? 401k match? Commuter benefits? If you need a nudge to ask for a big scary number, sit down and consider EVERYTHING you currently have.

Where the Job Lands: Before you even get to the negotiation stage, you want a reasonable assurance that the negotiation could result in an offer you'd take. So that means getting baseline information on comp, benefits, and other hard-line items like (for example) remote work if that's important to you.

Part III: Negotiation Time

Negotiation is not a one time event. If you think about it (not to get too meta) life is a series of negotiations. We are constantly negotiating the place we hold in the world and how people treat us.

People are trained to think that a job is something you compete for, that employers have the power in the equation. Or if people don't think that, they think that power in negotiation comes from competing offers only, and otherwise they don't have much leverage.

Not true!

I once heard it said that in romantic relationships, whoever is more willing to leave has more power. The same is true for negotiation. So your power in negotiation can come from:

Willingness to stay at your current job until the right thing comes along
Willingness to leave a current job and job-hunt full time if you are underpaid and they won't approve a raise
In demand skill set
Knowledge that the market is tight
Awareness that the role you are being offered has been unfilled for a while
Niche domain knowledge
Willingness to say no, no matter what

I encourage you to say no early and often. No to interview processes that are long and arduous if you don't know if the pay is worth it. No to recruiter conversations if they won't reveal the pay range or benefits. You are never obligated to finish an interview process just because you started it.

Talking Early Numbers

Companies are all over the map in terms of talking salary. I've seen recruiters send me the range in the very first cold email, I've had people who would have me pry the information out of their cold dead fingers only after I'd signed my life away. It can be hard to develop a consistent system of salary boundaries when recruiters are so all over the place. Below is my advice for some common pit-stops in the early negotiation process.

Applicant system where salary is a required field: Put 0 or n/a depending on what the form allows. Nice try recruiters!

First Call/Desired Salary: Much like jobseekers are trained not to give away numbers, some recruiters are too. Some will say they are flexible for the right person, or pay "market rate", or that the pay depends on where you fall in the company leveling guide, which they can only tell you after more interviews. In general it is worthwhile to play this game, using phrases like:

"I can be a little flexible for the right role, but could I ask the general budget?"
"I've honestly been seeing ranges all over the place in my current job search and it is going to depend a little bit on the total benefit and comp package. Can you tell me anything about that?"
"You know, I just started my job hunt and haven't really had a chance to crunch the numbers on the going rate for my experience. If you have an idea for this role though I can work on that and get back to you with something firm"

It is possible to be completely likeable while denying the recruiter the information that they want. Don't feel bad, they're used to it.

First Call/Current Salary: For starters, be aware if your state prevents this question being asked. But don't fret, even if you aren't in one of those states we aren't going to answer. My personal favorite is to say "I think that comp details are actually covered by my NDA" (Non-Disclosure Agreement). Totally out of your hands! Some people will share things like that they currently are underleveled and are changing roles to get a salary more in alignment with their expertise. To me, that's overkill, and shares that you may be in a vulnerable negotiating position as you are currently underpaid. My advice, is keep it simple, and make a third party (NDA) the bad guy in why you can't share the information. Others will advise you to answer "I'm not really comfortable discussing it". I like that answer less, because it's a job hunt - obviously money will come up. It seems disingenuous, and imo less like you are interested in cooperating with them. Maybe I'm splitting hairs, but that's my honest opinion.

Verbal Offer Stage

A lot of companies do this thing where they will want to get a baseline agreement on the numbers before drafting up an offer letter. As someone who has had to edit zillions of offer letters after negotiations changed the base pay, I sympathize with this impulse.

That being said, if you are wondering when to start negotiating hard, verbal offer is a great time! And also a dangerous one.

Dangerous, because sometimes the details of the verbal offer miss critical information, or in your excitement it is hard to keep the details you are being told straight. Two examples from my own life of verbal negotiation misfires:

I once thought an offer included stock grants instead of stock options - one would have been free stock, the other is just an agreement to let me buy stock
I once named my asking salary as 1-1-5 (verbally) and I think the recruiter misheard because the offer came back at 1-5-5 (not kidding!)

Takeaway: get the written offer as soon as you can. It is fine to say "So far this is sounding reasonable but I will want to see everything in writing". If there is one thing you read about negotiation (other than this post, of course) it should be this fabulous article. The author, Haseeb Qureshi, says something to the effect of "not being the decision maker". I love this idea! I am single but I always borrow the line "let me run it all by my family". The recruiter just doesn't know that means my pitbull.

Written offer & getting to the finish line

At this point, maybe you are sick of negotiation. You may have already negotiated several times just for this job! But hey, you're almost done.

You might wonder, "How long/hard can I negotiate at the offer stage? Aren't things pretty final?"

Well, let me tell you about a Project Manager candidate I was once responsible for recruiting. She managed to get something like 50k additional comp after negotiations were already down to the wire. (As the recruiter, it was SO STRESSFUL haha). So anything is possible.

That said, to paraphrase the fabulous article I mentioned earlier, companies are motivated to negotiate by believing that you are winnable. If you have too many "volleys" of back and forth, they may start to believe you are just leveraging them against another offer or not really interested in the role. If I had to give a completely random rule of thumb here, probably more than three volleys on the main stuff (base pay, stock, bonuses) may start to seem like you are not interested.

You can save yourself some rounds of back and forth by making the first volley a significant ask. There is something called the anchoring effect which describes the phenomena that humans tend to be unduly influenced by early information. Basically in this scenario, it means that the point of reference becomes your negotiation point instead of theirs. They may be inclined to try to "split the difference". This is the most common response I saw in recruiting to negotiation.

Examples:

We offer 85k, candidate asks for 100, we come back with 93
We offer 120k, candidates asks for 150, we come back with 135

And on and on. So a big bold number here sets the stage for what "meeting in the middle" looks like. If you want a paraphrased version of what I usually say here, following the Habeeb article about being winnable, it is something like this:

I'm incredibly excited about the idea of joining the team. I know that with my background in a and b, I can make an immediate impact on projects d, e and f. I'm confident that we can find something that works for everyone, but because I am looking at multiple offers I wanted to know if x is possible. If the pay could be x, I can sign on immediately.

Why those elements?

Communicate that I'm winnable
Remind them of the value they see in me
Take the adversarial element of negotiation away by reminding them you are motivated to find something mutually beneficial ("works for everyone")
Give them a specific end state ("could sign on immediately")

And boom! That's it. Cradle to grave, that's how I approach negotiation.

Gotchas and special cases

Q) What if the place has strict salary bands?
A) Some places really do, some don't. Still negotiate. Unfortunately, I have heard of numerous cases where people are told this to find out exceptions can be made but just weren't for them. Also, you can sometimes negotiate to be brought in at a higher title. I saw this happen all the time as a recruiter. No joke.

Q) What if the place says they don't negotiate?
A) Same answer as above, sometimes that is entirely made up. If they stick to it, it is your call whether it is worth proceeding.

Q) Can I negotiate other things than salary?
A) YES! You can also negotiate an earlier time to revisit your salary than annually if it is lower than you want. While you can negotiate things like conference budget, bonuses, etc there are certain things that are harder than others. Many companies I know do not vary on PTO. It is difficult to track if it is irregular between employees and could cause resentment if it came to light that someone got more.

Another one I would get asked a lot by veterans was if salary could be increased if they don't need healthcare. Usually, the answer is no since if someone changed their mind at any time, the workplace still would be on the hook to provide healthcare.

Aside from those two items, the sky is the limit though!

Q) Can my offer be revoked for negotiating?
A) Actually...yes. There is actually a lot of bad information out there on this. People say "Oh that never happens!" as if it is that simple.

It has happened - to me. After being a recruiter for years. So not someone approaching the negotiation in an insulting way, naturally. I know other people it has happened to. Most all of them were part underrepresented groups in tech.

The bottom line here is yes sometimes it happens although it is very rare, but it is not something that you should let stop you from negotiating. As upsetting as it is when it happens, it is a blessing - it is the reddest of red flags about that org. Run, do not walk away from that place.

Q) What if neither party budges in the game of don't-name-a-number chicken?
A) I tend to throw out a ridiculous number to get them to talk, bonus points for a bafflingly huge range. It usually gets a reaction out of them but also doesn't give much information away. Example: you expect the job to pay 60k, but the recruiter and you keep going back and forth in the won't-name-a-number dance. Eventually you give up and say "I'm seeing 80-120 for these types of roles". The recruiter will likely balk and blurt out something that tells you where they are out of surprise.

Conclusion

So that's it! That's just about everything I would want someone to know when approaching negotiation. If I can encourage you to do anything it is to make yourself comfortable with the process, whether using any of my tips helps you feel prepared or reading from other sources does. It doesn't have to be adversarial, it can be something really positive, and it certainly shouldn't be built up to be this magical thing only certain people can do. Go forth and negotiate!

And if I missed any Q & A topics you would like covered, feel free to drop them in the comments and I may add them to the post.

Vue vs. Svelte vs. React

JTK — Fri, 28 Jan 2022 21:25:31 +0000

For those of us who don’t get to work with frontend technologies, it can be hard to stay up to speed on the latest & greatest developments with JavaScript frameworks. For this reason, today we are going to explore two very popular frameworks, Vue and React, as well as a newer one I’ve heard great things about: Svelte.

In this post, I'm going to walk us through a side-by-side comparison of a Hello ~~World~~ Fauna app, built in each React, Vue, and Svelte. By running through the same core tasks and building the same end product, we will get to examine these tools in contrast.

In each, we’ll retrieve some data about Pokemon from Fauna and display it like this:

After building each, we’ll take a ten-thousand foot view of each framework’s developer experience and my observations on them as a Backend Engineer. Let’s get started!

Getting set up with Fauna

After playing with the Fauna GUI and CLI, I found the CLI a bit more to my taste, so that’s what I’ll be using here. You can get it on *nix machines with NodeJS/NPM installed by running npm i -g fauna-shell.

npm i -g fauna-shell

> faunadb@4.4.1 postinstall /usr/local/Cellar/node/17.3.0/lib/node_modules/fauna-shell/node_modules/faunadb
> node ./tools/printReleaseNotes

+ fauna-shell@0.13.0
added 325 packages from 242 contributors in 27.304s

If you are following along it is at this point that you'd want to head to Fauna's site and make an account. Quick heads up: if you’re going to use the CLI, don’t use an SSO option when you make your account. The CLI isn't set up to work with that authentication. I also built us some boilerplate code for a Vue project, so if you want to follow along clone the repo and start with origin/boilerplate.

Returning to the command line, let's get into our Vue project and create our database, first by logging in with the Fauna CLI. You’ll go through a process a bit like this:

➜ fauna cloud-login
? The endpoint alias prefix (to combine with a region): cloud
? The endpoint alias already exists. Overwrite? Yes
? How do you prefer to authenticate? Email and Password
? Email address: jtkaufman737@gmail.com
? Password: [hidden]
? Endpoints created. Would you like to set one of them as default? Keep 'cloud-us' endpoint as default
Endpoint 'cloud-us' set as default endpoint.

Then, let’s create our database and an API key. Keep your secret key private and be sure to mark it down somewhere as we will use it later.

➜  fauna create-database pokemon
creating database pokemon

  created database pokemon

  To start a shell with your new database, run:

  fauna shell pokemon

  Or, to create an application key for your database, run:

  fauna create-key pokemon

➜  fauna create-key pokemon
creating key for database 'pokemon' with role 'admin'

  created key for database 'pokemon' with role 'admin'.
  secret: xxxx-xxxxxx

  To access 'pokemon' with this key, create a client using
  the driver library for your language of choice using
  the above secret.

An important note before we move on

This is a very simple HelloWorld type exploration of three frameworks, and as such not how we would use any of these tools per se in the real world. We are going to let our JS code call the database directly, picking up credentials from local .env files.

As such, if you follow along you should not deploy this code, since the secret picked up will get built with the other static assets, making your secret visible if someone were to visit the web page.

If you do want to turn any of these demos into a larger project, tools like Netlify and Vercel have great options for how to deal with environment variables, or the database response could be returned through an API and server-side code.

Getting started with Vue

If you are familiar with modern single page apps, even if you haven’t worked with Vue, our project structure may seem somewhat familiar. You’ll notice that src/ contains the files we’d actively work on.

A typical Vue project can contain elements that have been removed from the boilerplate code we’ll use here, which was done to keep things extremely simple and make comparison between Vue, Svelte, and React easier and clearer.

With that in mind, our main active files are going to be App.vue and main.js. Let’s also add a data.json file to seed our database with some records.

// data.json 

{
    "id": 1,
    "name": "Bulbasaur",
    "imageUrl": "https://i.imgur.com/e7VtLbo.png"
}
{
    "id": 2,
    "name": "Pikachu",
    "imageUrl": "https://i.imgur.com/fmMERCo.png"
}
{
    "id": 3,
    "name": "Snorlax",
    "imageUrl": "https://i.imgur.com/TGf6qB8.png"
}
{
    "id": 4,
    "name": "Caterpie",
    "imageUrl": "https://i.imgur.com/A21Gpql.png"
}
{
    "id": 5,
    "name": "Jigglypuff",
    "imageUrl": "https://i.imgur.com/SU7yF1f.png"
}
{
    "id": 6,
    "name": "Abra",
    "imageUrl": "https://i.imgur.com/f59APqT.png"
}
{
    "id": 7,
    "name": "Weedle",
    "imageUrl": "https://i.imgur.com/XDeqSAB.png"
}
{
    "id": 8,
    "name": "Dratini",
    "imageUrl": "https://i.imgur.com/K9DxFvF.png"
}
{
    "id": 9,
    "name": "Charmander",
    "imageUrl": "https://i.imgur.com/KuZEzvo.png"
}

The Fauna CLI lets us easily import either JSON or CSV data to our new collection. Let’s import our new data.json:

➜  fauna import --path=./data.json --db=pokemon --collection=pokemon
Database 'pokemon' connection established
Start importing from ./data.json
Average record size is 113 bytes. Imports running in 10 parallel requests
9 documents imported from ./data.json to pokemon
 ›   Success: Import from ./data.json to pokemon completed

And we can confirm our records have made it up to the database by visiting our Fauna dashboard and drilling down into the right collection:

First, let’s connect our Vue app to the database using Fauna’s JavaScript driver. We’ll take the API secret that we wrote down from earlier and add it to a .env file - for Vue apps, anything prefixed with VUE_APP gets seamlessly picked up by the application so our .env file will look like this:

// .env 
**VUE_APP_FAUNADB_SECRET=xxxx**

We’ll then move over to main.js and run through our main logic. Here are the general steps:

Import Fauna
Use a constructor function to create a new database instance
Bind that instance to the application

// main.js 

import Vue from 'vue'
import App from './App.vue'
import faunadb from 'faunadb'

// This constructor creates a new database instance, and supplying the secret
// authenticates us 
const db = new faunadb.Client({
    secret: process.env.VUE_APP_FAUNADB_SECRET, 
    domain: 'db.us.fauna.com',
})

Vue.config.productionTip = false
// binding $db and $q means our database instance and query commands 
// are easily accessible from here on out anywhere in our Vue code 
Vue.prototype.$db = db
Vue.prototype.$q = faunadb.query

new Vue({
  render: function (h) { return h(App) }
}).$mount('#app')

Here’s the fun part: Vue’s basic unit of functionality is a Single File Component - split into a <template> tag for our markup, a <script> tag containing our JavaScript functions and logic, and optionally a <style> tag for our CSS.

Let’s start with our <script>. We are going to deal with a few handy tools Vue gives us:

data is an object for storing easily accessible values local to our component. We will start it with an empty array and fill it with Pokemon from Fauna.
methods is an object whose properties can be functions for any behavior we want - here, we will call our database and get our records.
lifecycle methods are special events you can attach to behavior in Vue. I have arbitrarily chosen the moment our component is mounted to trigger the method call to Fauna. There are more lifecycle events than that, but let’s keep it simple today.

// App.vue 

<script>
export default {
  // data = our local component data, will be filled with  
  // pokemon later 
  data() {
    return {
      pokemon: []
    }
  },
  methods: {
    // our fetchPokemon method calls fauna, then updates  
    // data.pokemon 
    async fetchPokemon() {
      const q = this.$q 
      const response = await this.$db.query(
        q.Map(
          q.Paginate(q.Documents(q.Collection("pokemon"))),
          q.Lambda(item => q.Get(item))
        )
      ) 
      this.pokemon = response.data.map(item => {
        return item.data
      })
    }
  },
  // this lifecycle method is what kicks off the 
  // fetchPokemon() function running
  mounted() {
    this.fetchPokemon()
  }
}
</script>

My favorite part of Vue is the concise shorthand it offers to dynamically generate HTML based on component data. I’m going to make use of it here by leveraging a Vue construct called v-for to iterate over our array of Pokemon. It’ll look something like this:

// App.vue 

<template>
  <div id="app">
    <div class="home">
      <h3>Pokemon</h3>
      <section class="grid">
         <!-- 
             v-for iterates over pokemon array, :key loads the 
             javascript object value id, :src loads the
             Pokemon image url from each
             array item, so on and so forth 
          --> 
        <div class="card" v-for="p in pokemon" :key="p.id">
          <div class="header"><h6>{{ p.name }}</h6></div>
          <img 
             :src="p.imageUrl" 
             :alt="p.name"  
             class="pokemon"
           />
        </div>
      </section>
    </div>
  </div>
</template>

<script>
...

See the colon before a few of those HTML attributes? That lets Vue know that the attribute values aren’t strings, they’re dynamic JavaScript values based on the p variable we defined in the v-for shorthand. We also get to use those JavaScript values inside double brackets to interpolate into the content of the HTML tags — here that’s used to display the Pokemon names.

Last but not least, I hacked together some CSS that will be used (with minor edits) throughout each demo app. It’s not absolutely necessary, but we want these components to look good, so here it is:

// App.vue 
...

</script>

<style lang="scss">
#app {
  font-family: 'Inter', sans-serif;
  -webkit-font-smoothing: antialiased;
  -moz-osx-font-smoothing: grayscale;
  text-align: center;
  color: #2c3e50;
  margin-top: 60px;
}
h1,h2,h3,h4,h5,h6 {
  font-family: acumin-pro,sans-serif;
  font-weight: 500;
  font-style: normal;
}
h3 {
  font-size:2em;
}
h6 {
  font-size:1.25em;
}
div.card {
  border-radius:30px;
  border:.25px lightgray solid;
  margin:5px;
}
div.header {
  width:100%;
  color:white;
  background: linear-gradient(90deg,#3F00A5 0%,#813EEF 100%);
  display:flex;
  justify-content:space-around;
  align-items:center;
  border-top-left-radius:30px;
  border-top-right-radius:30px;
  height:60px;
}
a {
  color:#3F00A5;
}
a:hover {
  color:#813EEF;
}
div.home {
  width:60%;
  margin:auto;
}
section.grid {
  display:grid;
  grid-template-columns:33.33% 33.33% 33.33%;
  grid-template-rows:auto;
  margin:auto;
  padding:5% 20% 5% 20%;
}
img.pokemon {
  height:100px;
  width:100px;
  padding:25px 10px;
}
</style>

With all that in place, let’s return to the command line and kick off a local build:

➜  npm run serve

> vue-faunadb@0.1.0 serve /Users/jkaufman/Code/vue-faunadb
> vue-cli-service serveDONE  Compiled successfully in 97ms                                                                                            11:30:06 AM

  App running at:
  - Local:   http://localhost:8082/ 
  - Network: http://192.168.58.105:8082/

  Note that the development build is not optimized.
  To create a production build, run yarn build.

Voila! if you return to localhost:8080 you will be able to see our Vue app displaying the Pokemon gallery shown earlier in all its glory. With that out of the way, let’s do the same with some other tools!

Recreating the app in React

React is the most popular web framework out there (or so it has seemed to me for a while), so it’s much more likely you’re familiar with how it works. Let’s try to reconstruct this application and see the differences between Vue and React. If you’re following along, pop over to this repo and grab the latest from origin/boilerplate.

For those familiar with React, you will again notice that I’ve stripped out a lot - some of the files are outside the scope of this article and would just cloud the fundamental comparison of these frameworks.

We’ll run through the same logic, although this time in index.js:

Import Fauna
Use a constructor function to create a new database instance
Bind that instance to the application

// index.js 
import React from 'react';
import ReactDOM from 'react-dom';
import App from './App';
import faunadb from 'faunadb';

// create database connection/instance through constructor 
const db = new faunadb.Client({
  secret: process.env.REACT_APP_FAUNADB_SECRET, 
  domain: 'db.us.fauna.com',
});

ReactDOM.render(
  <React.StrictMode>
    <App 
        // For react, we're going to bind the database instance 
        // and query functionality as props 
      db={db}
      q={faunadb.query} 
    />
  </React.StrictMode>,
  document.getElementById('root')
);

In our new React app, let’s go to .env file and add our Fauna secret — this time using a React specific prefix:

// .env 
REACT_APP_FAUNADB_SECRET=xxxxx

Our last piece of business here will be some work in App.js. We will need to:

Create methods to call Fauna and access our list of Pokemon.
Iterate over the data and display it through dynamically generated markup.
We will use useEffect and useState instead of lifecycle methods to trigger our API call. This is because React doesn’t have lifecycle methods, which is something I miss coming from Vue — more on that later.

// App.js

import './App.css'
import { useEffect, useState } from 'react'

function App(props) {
  const [pokemon, setPokemon] = useState([])
  const q = props.q

  // use effect means this will fire on render
  useEffect(() => {
    // this function uses our props for the database and Fauna query capabilities
    async function fetchPokemon() {
      const response = await props.db.query(
        q.Map(
          q.Paginate(q.Documents(q.Collection("pokemon"))),
          q.Lambda(item => q.Get(item))
        )
      )

      let updatedPokemon = response.data.map(item => {
          return item.data
      })

      setPokemon(updatedPokemon)
    }

    fetchPokemon()
  }, [])


  return (
    <div className="App">
      <div className="home">
        <h3>Pokemon</h3>
        <section className="grid">
          {
            // without v-for like in Vue, we instead use vanilla JS 
            // to get the iteration over our data 
            pokemon.map(p => {
              return (
                // all dynamic JS values in React are expressed as {x}
                <div className="card" key={p.id}>
                  <div class="header"><h6>{p.name}</h6></div>
                  <img src={p.imageUrl} alt={p.name} className="pokemon"/>
                </div>
              )
            })
          }
        </section>
      </div>
    </div>
  );
}

export default App;

Let it rip with an npm run start, and you should see a perfect replica of the app we created with Vue running on localhost:3000. Again, a pretty seamless experience to get a simple application prototype running.

Last but not least, we are going to do the same again, with one final tool.

Recreating the app in Svelte

I was honestly very excited for this one because I have been primarily a backend developer for quite a while, and as a result, I’ve had fewer opportunities than I’d like to play with the newest JavaScript stuff. Svelte has sounded interesting for a long time and I was happy to finally get to give it a whirl.

As before, grab the boilerplate code and checkout to origin/boilerplate if you’re following along. You know the drill at this point: step one is sticking Fauna in our entry point JS file (which is main.js here). Here’s how we tackle it in Svelte:

// main.js

import App from './App.svelte';
import faunadb from 'faunadb';

// create db instance through constructor, secret for authentication
const db = new faunadb.Client({
    secret: process.env.SVELTE_APP_FAUNADB_SECRET, 
    domain: 'db.us.fauna.com',
});

const app = new App({
    target: document.body,
    // to make the db and query functionality available widely, we are 
    // going to pass them as props in the main application instance
    props: {
        db: db,
        q: faunadb.query
    }
});

export default app;

You’ll want to remember to make an equivalent .env file here too of course.

// .env 

SVELTE_APP_FAUNADB_SECRET=xxxx

The main App.svelte file reminds me a lot of Vue, sectioned out for different functional areas by default. Take a look:

// App.svelte 
<script></script>

<main id="app"></main>

<style></style>

Here’s where it gets interesting. Svelte, like Vue, supports special shorthand iteration operations in its markup. For Svelte, these are denoted both with characters and keywords — for instance, {#each /} will allow us to iterate over an array. We also get lifecycle methods back, and can tie our API call to component mounting. The expression of {#each /} to me was particularly fascinating. It reminds me — visually speaking — more of templating methods in Rails or Django than the equivalent version of this functionality in React or Vue. There’s nothing wrong with that; it feels natural to me.

In App.svelte, next we’ll register the “on mount” behavior, a method containing our database call, and the iteration and display of the results in markup.

// App.svelte 

<script>
    import { onMount } from "svelte"; // Yay lifecycle methods! 

    let pokemon = [];
    // the double $$ is also visually interesting, maybe not what I'd expect
    // to signify accessing props, but nothing wrong with it either 
    const db = $$props.db; 
    const q = $$props.q;

    // method to grab our pokemon records 
    onMount(async () => {
        const response = await db.query(
            q.Map(
                q.Paginate(q.Documents(q.Collection("pokemon"))),
                q.Lambda(item => q.Get(item))
            )
        ) 
        pokemon = response.data.map(item => {
            return item.data
        })
    })
</script>

<main id="app">
    <div class="home">
        <h3>Pokemon</h3>
        <section class="grid">
         <!-- here starts our interesting loop block syntax --> 
           {#each pokemon as p}
              <div class="card">
              <!-- Like React, dynamic values grabbed and expressed with {x} --> 
                  <div class="header"><h6>{p.name}</h6></div>
                  <img src={p.imageUrl} alt={p.name} class="pokemon"/>
              </div>
           {/each}
        </section>
    </div>
</main>

<style>
...

At this point, we can put the pedal to the medal with a npm run dev. If you look at localhost:5000 you should again see a perfect replica of our Pokemon gallery, as pictured in the intro.

Comparison of these tools

As someone who leans towards the backend but thoroughly understands frontend concepts, I think I have more of an objective wide-lens view of these JavaScript tools, which led to some interesting observations:

All of these tools are simple enough for a backend developer like myself to jump in without much trouble. For example, I recall finding the class-based syntax React had at one point confusing to wade through. Being able to return to React and use functional components was great
Svelte made a strong first impression on me as a long time Vue user. I liked it but could also imagine someone coming from React liking it. Having never touched it before, getting the markup and methods working took pretty much zero time for a rookie.
The only area in which Svelte felt weak was in build configuration. Unlike Vue and React which by default utilize Webpack to build, bundle, and minimize code, Svelte uses another tool: Rollup. Although I saved you from this experience in the tutorial, dealing with Rollup presented multiple hiccups that reminded me of the pre-Webpack-3 days of working with Single Page Apps, when extensive Webpack configuration was sometimes required. For anyone interested you can read more about these two tools here
On a related note, I did feel like the amount of time it took to figure out the standard method of passing Svelte environment variables felt surprisingly long. I would consider that an area to be improved upon - if a .env file with SVELTE_APP_VAR was plug and play (as it is for React or Vue) I would have been a much happier developer
Both Svelte and React are by default stricter — I forgot that it is normal for your JavaScript build to scream at you about unused CSS classes. This is probably a positive, especially if you’re going to be building something for production in the long run
Plenty of people prefer React because of fewer framework-isms, like Vue’s v-for or Svelte’s {#each /each} for dynamically generated markup. For me, they are intuitive enough that I like having the option, but I can see how they’d be confusing for many
I also like Vue and Svelte’s component lifecycle methods. I find their names (mounted, onMounted) more intuitive than useEffect in React. If I were new to JavaScript, I might expect something called useEffect to be related to DOM behavior or something else
If I had to make a decision for a personal project after this experience, I’d rank my favorites as Svelte first, then Vue, and lastly React. Svelte just gave me a warm and fuzzy feeling that puts it over the top for me, especially with all its potential

The elephant in the room, of course, is that Svelte, uniquely, does not utilize a virtual DOM. In their own words, Svelte cites the rationale for this as a way to get away from the overhead required to sync and compare the actual and virtual DOM. As someone with a utilitarian need for JS frameworks, I didn’t mind the virtual DOM when working with it, nor could I particularly think of ways I missed it playing with Svelte. Truthfully, that conversation doesn’t seem relevant until we’re building much larger apps with much more at stake if our performance starts to dip. I do find Svelte’s argument compelling, though, so I’ll definitely be following the large scale adoption of Svelte.

Wrapping Up

I enjoyed building all three demo apps. React once felt very intimidating to me, but the move away from Redux and class based components has made it feel more intuitive.

Vue is the only one of these technologies I have worked with significantly, ever since 2018. I continue to like it as a “mellow” tool, and it is a recurring comment I hear from other backend folks that Vue feels approachable to us.

Svelte lived up to the hype, at least in this very minimal example! Although I have been a Vue person for years, I would strongly consider using Svelte instead on my next personal project based on my initial positive experience and wanting to learn more.

Let's talk about Docker

JTK — Sat, 15 Jan 2022 19:14:20 +0000

This post is equal parts actually about Docker, and the hilarity of being a woman on the internet in tech.

Who knew anyone read my tweets

It is funny to me that I would get any kind of attention on Twitter, I don't post for engagement, I barely write about tech, and I'm otherwise barely on social media at all. I post on FB every six or so months so my out of state relatives know I'm alive and that's about it.

So I was certainly not expecting an offhand tweet complaining about Docker to get any kind of attention.

// Detect dark theme var iframe = document.getElementById('tweet-1481652902913458184-963'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1481652902913458184&theme=dark" }

What are my qualifications to hate Docker

I never knew that in order to have opinions about Docker I would need to announce my pedigree, but apparently I do.

To my utter astonishment, someone on the Docker team found their way to my thread to call me a junior without the background to appreciate the problem set Docker addresses. And then asked to be unsubscribed like a heeeelllllla boomer. That part made me laaaaaugh.

// Detect dark theme var iframe = document.getElementById('tweet-1481893730684526592-557'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1481893730684526592&theme=dark" }

Anyway here are my qualifications: Have been in this field for many years, made senior in two and a half, have worked on giant gnarly legacy code bases and alternately built multiple products out of nothing to be hugely successful.

I am a code mentor and have volunteered teaching code, I am (as of recently) a paid guest writer for a database company, and managed to get straight As in senior year of a Software Engineering & Security degree while also working full time. My GPA was something like 3.8 and my major GPA was 3.9. Actually, I've been in school at least half time since 2019 so I have experienced a meteoric rise professionally with much less time on hand than equally qualified peers.

In one past role, I was so brutally effective that I received an off-cycle promotion from Software Engineer, not to Senior Software Engineer but directly to Dev Lead. Within less than a year of working there.

So...yeah. I am both educated and experienced, and as entitled to my opinion on Docker as anyone. (Although hot take, anyone can hate Docker for any time and for any reason, who cares lol).

Most notably, I actually had to spend somewhat extensive time using a dockerized development environment when at a past job, we had dependencies with no good MacPorts option that did not run on Mac period. In that role, I was also responsible for software that dealt with tracing vulnerabilities through containerized environments which introduces a special, interesting set of challenges.

What I actually think of Docker

Surprise surprise, twitter is not a tool of nuance. There were more to my comments than "Docker is terrible", naturally.

I made the mistake of wondering why this hit such a nerve with people and googled latest Docker tweets period, thinking surely I wasn't the only one with these issues. (Bad idea). Aaaaaaand that's when I found the subtweets. They centered on a few themes:

Everyone in the discussion were junior engineers

No one was providing any concrete reasons for disliking it (spoiler, I did and will again)

If you aren't using Docker in x way/on y OS its your fault it is miserable (ok lol)

People goading me "what's your alternative then"

The internet is a wild place lol. Imagine me tweeting "I don't like broccoli" and a bunch of nerds demanding I explain why or can't make that comment. Or saying I can only dislike broccoli if I mention that cauliflower is a valid alternative. Or say "it is your fault you don't like broccoli you should have been cooking it with this technique, DUH!"

But as explained before, I AM ACTUALLY PERFECTLY QUALIFIED TO HAVE BOTH THE OPINION, AND A RATIONALE BEHIND IT. The internet neckbeards I'm sure will be so relieved to hear it (not).

For starters, junior developers finding Docker un-intuitive and with iffy docs is also my problem, since what junior developers struggle with rolls downhill to me as their senior
I have consistently found the cli poor, things I need to do regularly are not prefab commands or require large numbers of fiddly flags, I need to wind up making aliases for tasks I find common enough that they warrant a built-in command and it seems like something so easy to fix if you care about dx at all
Resource consumption and old volumes hanging around remains annoying
I find the docs very sparse, my particular criticism with them could probably be addressed by a 1-2 liner in various sections just showing one vanilla example of cli options. My experience in the docs is being redirected around like a pinball and eventually finding what I need outside of the official docs. If the rest of the internet is better than you are at explaining your product, imo that's an issue
In light of I think, two Docker team members finding their way into my thread and being obnoxious, I now have a personal animus that I didn't even have originally so thanks guys lmao

I also have a criticism much more meta than Docker, and hate this opinion all you want but oh well:

Software is bad. Computers were a mistake, (To quote an old coworker). The less software my app depends on, the happier I am - and that includes Docker.

Particularly for my local dev setup, I tend to find that Docker wastes much more of my time than it saves. Tending to this tool and its various needs takes time I don't have. I'm busy, I'm sure if I had time to plumb the depths of the Docker documentation I could come up with a super cool setup that addresses every little thing about it that I don't like. For me personally, I have NEVER liked the "f@#$ you figure it out or you don't deserve the tool" mindset.

Give me things that work seamlessly. I have s@#$ to do.

I don't want to toil away customizing a linux box, give me an already usable computer. If you like that, great, but not everyone does. With more hours in the day I'm sure I could sit down with Docker and get into the weeds but I don't have time and for a tool so popular I don't feel like I should need to. Containerization is an area I have experience but not one I wish to specialize in, I'm hardly the best person around to come up with a complex Docker setup and feeling like I have to know it to do what used to be just running a flask app on localhost seems extremely overwrought.

Other criticisms I'll address

If you like Docker, from the bottom of my heart I think that's awesome. My least favorite use case (local dev) emerged in the comments that equally frustrated and delighted people. I think its great that we all are so different and have different likes and I'm happy Docker works for some people
Things were bad before Docker - perhaps they were, some of it I wound up liking better for whatever reason. I've worked with Vagrant, VirtualBox, deploying to vms up on GCP, and whatever else. Maybe my preference just settled on those. I certainly miss local dev without Docker to mess with. It doesn't need to be how everyone feels
Docker has its place - I agree with you! Yeah, Twitter is not great for nuance but sometimes I have really enjoyed Docker being in the mix
Come up with something better then - well that's ridiculous. Returning to broccoli, clearly I can't invent a better vegetable. It is beyond absurd to imagine I can only have an opinion on Docker if I have a competing container solution's software ready to go. News flash, there are plenty of things made by people smarter than you or I that we still won't like for whatever reason. Also, the hilarious detachment from reality here - I have a mortgage and am in school full time? I don't have time to invent a better container solution if no one is paying me to do so, some of us have day jobs babe

The hilarious thing about this experience

No one expects the internet to be nice to them, let's be real. But in the negative commentary I will say, themes certainly emerged.

// Detect dark theme var iframe = document.getElementById('tweet-1482076709927141380-894'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1482076709927141380&theme=dark" }

I'd love to never have to talk about this again, but it was hard not to see a sexist tilt to the replies. For starters, seemed supremely random that so many people wanted to discredit me by calling me a junior and implying that that could completely explain me not liking Docker.

It is not difficult to find the podcasts I've appeared on, my long held Senior or higher title has remained in my bio for years. Disagree with me all you want but you do not have a leg to stand on if you want to come at my seniority. Which is fortunate, because honestly as a newer and less secure dev maybe a zillion dudes telling me I have no idea what I'm doing would have bothered me.

That critique is even sillier if you unpack it: if I were junior would I not have permission to dislike technology? The software I admire makes complex things simple, so the highest compliment I could give Docker would be that its a good tool for junior devs. Some of y'all really have it twisted on the topic of juniors not understanding software and how that reflects on the software.

Lastly I'll say, a lot of bias is sub-optimal associations. People associate women with less expertise in tech, and these replies showed it. I think sometimes non-URM people think it is a security blanket, like somehow URM feel better if we can blame things on bias issues. I will assure you it is not the case, it just means for me that people are coming down hard on me regardless of the validity of my opinions and even if I respond in-depth, they are going to discount my expertise and conclusions. Makes it feel very futile to put effort into defending my views.

Something about my personality, I have really always hated status symbols or things like pulling rank. I would normally never say "I'm a senior AF dev here's my credentials", I find it honestly very tacky and no one should feel like I'm cooler than them if they are less experienced. The fact that I have to trot out this litany of things to say I can have an opinion is honestly bizarre, but here we are.

It is also not at all fun or a trump card to have to bring this up, frankly it sucks, but I feel like the kid in The Emperors New Clothes who speaks an unpopular truth. I am successful enough that I can take the flak for raising this issue so if something hits me as sexist you bet your ass I'm going to talk about it.

Takeaways and positives

Despite some annoying replies (admittedly tame for the internet at the end of the day) I actually got some cool takeaways from this experience.

I had three people reach out to me about container tooling they want me to try, I'm going to set up three meetings next week for things I'd previously never heard of - considering my work uses Docker if any of them seem good I'd be thrilled to introduce them to our tech stack if they make Docker less of a headache.

A ton of people also responded in earnest with cases they love Docker for, which made me happy! Trust me, no one is happier than I am that fewer people feel how I do on Docker.

A lot of people just seemed relieved that anyone else struggled with this supremely popular tool that they hadn't gelled with.

And finally, there were some good-faith replies by people who really like Docker that had suggestions on things to make the experience better, or with alternatives - many I'd never heard of and sound extremely interesting and I'm really stoked to get into further.

At the end of the day, willing to write it off as a funny fluke and move along. But did teach me a valuable lesson that sometimes when you yell into the void, the void yells back.