DEV Community

Cover image for Technical Interview Performance by Editor/OS/Language

Technical Interview Performance by Editor/OS/Language

compumike on March 12, 2018

Originally posted on the Triplebyte blog. My co-worker Daniel is really into Emacs. It's his primary editor, his grocery list (via org-mode), and ...
Collapse
 
sam_ferree profile image
Sam Ferree
Collapse
 
pat_metzdorf profile image
Patrick Metzdorf

This whole thing does seem a bit disconnected, but that may be because of the type of jobs and the technologies their recruits were applying to work with.

I have no idea what Triplebyte is or does, but if they do less conventional stuff or search for different kinds of developers compared to whoever answers SO surveys, would explain why these statistics look so different to what StackOverflow seems to find in their annual assessments.

Collapse
 
compumike profile image
compumike

Our data comes from software engineers applying to work at a few hundred top tech companies. See triplebyte.com/blog/we-ve-raised-a... for more background -- we've helped software engineers get more than $50 million in job offers in just the past 6 months.

Instead of relying on self-reporting from a low-response-rate survey, we record what editor and programming language all engineers actually use during a two-hour technical interview with us. (You can use any programming language you like!) We don't actually use these signals to decide who we work with. Instead, we have a structured rubric and are looking for things like coding productivity, coding professionalism, and debugging skills. I've personally interviewed a lot of engineers applying to Triplebyte, and ultimately we're looking to predict whether this person has what it takes to perform well as a software engineer at one of the companies we work with. And more importantly, we use structured data from the interview to match you to companies, so we can say, "Jane is a really speedy and natural Python developer who also speaks very well about database design," from which we can match you to companies we know are looking for those strengths.

You can see triplebyte.com/interview_guide for more information about the interview in which we collect this data.

Collapse
 
compumike profile image
compumike

It's there, shown as Visual Studio!

Collapse
 
sam_ferree profile image
Sam Ferree

Are Visual Studio and VS Code lumped together? Or is the label just incorrect?

Thread Thread
 
compumike profile image
compumike

Lumped together!

Collapse
 
cathodion profile image
Dustin King

If the interviewer knows the candidate uses a given editor/OS/language when making the determination, I'd interpret these statistics as suggesting that your interviewers may be using these as proxies for the competence of the candidates. That might be a hard bias to correct for without hiring Java developers to interview Java developers, Windows users to interview Windows users, etc.

Collapse
 
apotheon profile image
Chad Perrin • Edited

(In the following, I use "external" to mean "interviews conducted by an agency that performs screening interviews on behalf of employers", and "internal" to mean "interviews conducted by the employers themselves, 'internally'". I think other terms I use should be pretty obvious, or at least I hope so, in context.)

Assuming no evaluation bias (a big assumption; we all have biases), I have a few wildly speculative ideas to offer for things to consider when trying to impose some kind of interpretation on these numbers:

  • Sublime is probably popular because it's free-or-cheap, and because it was the early free-or-cheap entrant in its specific editor-style market (including TextMate, Atom, and VS Code). Even on MacOS (TextMate territory) it's unsurprising, because many devs on MacOS are migrants from Linux-land.

  • Vim is probably popular in the interviews because it likely dominates among Linux users, and some migrants to MacOS carried that with them.

  • Atom users are probably mostly ex-Sublime users, and many of them seem to have trouble with it due to negative transference of knowledge (call it an "uncanny valley" effect, if you like) given the similarities with Sublime coupled with some differences that subtly trip up people used to Sublime.

  • Vim users are clearly well-suited to picking up skills involving systematic linguistic traits, given the fact the vi editing paradigm is basically a text processing language with a relatively simple grammar.

  • C++ is probably a very difficult language to use to perform the same kinds of tasks suitable to quick/easy completion in an interview environment. For exactly that same reason, if I was interviewing for a C programming job (because I'd rather have a C job than a C++ job), and was given the choice of any language at all to use in a skills evaluation for the job, I'd use Ruby. Once people who prefer C++ in the external skills interview actually manage to pass it, though, you've probably selected for the very best of them, so those candidates are probably much more likely to have a good chance of getting the job.

  • Ruby is a very productive language, and easily solves many problems more difficult in other languages, and it's thus possible that a lot of candidates do well in an initial skills test screen that are then judged not suitable to the job. A very smart bootcamper could lack the specific qualities to get past the internal interviews, for instance, even if that person excels in the external interview due to a solid grasp of the basic language and simple skills-test problem solving, while using a language that lends itself well to fast completion of tasks.

  • Emacs seems to be rapidly losing popularity for software devs overall, and I suspect that leaves those most "niche" in personal manner, preferences, and so on, which might put off interviewers at the target employer. They are also probably more GNU-favoring overall, given the pedigree of, and community around, Emacs.

  • Linux users are likely to be either fairly technically competent in a wide range of skill areas, or fairly overconfident in such skills, so you likely get a balance of the two resulting in a close-to-average overall success level for the group as a whole in the technical screen. Once they get there, those that are most skilled may also be the most "niche" in preferences, skill distribution, and so on, thus making them appear less desirable at the internal interview. This bodes well for me in terms of competence, or ill in terms of overconfidence, for a technical screening, but doubly ill for me at the internal interview stage; I actually prefer OpenBSD for my daily driver, and I have many eclectic technical skills (I like to explore interesting, quirky skill areas at times), most of which are more likely to make me less enticing to employers if they have any effect on the evaluation at all, thanks to the common human fear of weirdness.

  • I could be full of it. This is all just wild speculation, after all.

Collapse
 
elmuerte profile image
Michiel Hendriks

So your process is biased towards Vim and Sublime?

Collapse
 
compumike profile image
compumike

Correlation != bias :)

Also, Sublime was quite popular with engineers applying to Triplebyte, but Sublime users passed our interviews at a below-average rate. See "Triplebyte Technical Interview Pass Rate by Editor."

Collapse
 
manueltorrez profile image
Manuel Torrez

I think developers tend to use macOS not only by the OS itself (and Unix-like) but the hardware quality