How I hire Anyor developers to not regret

I did quite a lot of technical interviews and I wanted to keep some guidelines to my future myself and improve it over time. The most of the interviews I did was for backend positions (Kotlin, Java, Python, Node.js) but I also did some frontend interviews (TypeScript, React.js).

I believe that no interview gives 100% proof that the candidate is a good fit. The point of my interviews was to know if the candidate has really basic understanding of the technology used and fits into the team personally. But that don't mean to refuse the candidate with different thinking. The diversity is also important in the team. For anything more deep the transition period is needed unfortunately.

I start with some not so interesting preparation notes here but you can jump directly to the technical section.

Preparation

1. Check the CV profile if the candidate sent one.
2. Check the GitHub code or example code sent by the candidate. If there is something interesting (positive or negative) make a note so you can talk about it during the interview.
3. Check LinkedIn, Facebook, Twitter, StackOverflow profiles. I usually google the candidate to get as many info as I can.
4. Tell the candidate the interview usually takes from 1 to 1.5 hours and he will be writing basic the Python/Java/Kotlin/JS and SQL in an online editor. It's good to set expectations.
5. Ask the candidate if he can bring his own notebook for it. It's better because he knows the keyboard and settings there so the code writing will be smoother. Writing on the paper or the whiteboard is not good because it is usually not readable and it takes much longer time.
   1. If he brings it, then prepare WiFi password for him to connect to the internet during the interview.
   2. If the candidate does not bring his own notebook you need to prepare the interview notebook. Always check if it works and does not need updates. Is there EN/DE/... keyboard installed?. Is the battery charged or is there an electric socket in the room? Are WiFi and the browser working? You will need https://codeshare.io/ or http://collabedit.com/ document prepared there (syntax highlighting for the language). You can copy and run the code in https://repl.it/languages. For FE the good tool is CodeSandbox
6. Prepare anything so you can write notes there about the interview.
7. The good thing is to set some "secret" signal with other people if the candidate will be really bad and you will want to finish the interview as soon as possible. In that case you simply stop asking more questions, move to the outro and I often just ask him if he has any questions and let him go.
8. Prepare a glass of clean water for the candidate and ask him if he wants coffee/tea/... :)

Introduction + questions

Interview starts with small talk about the candidate and the company. I usually ask some questions below and tell the candidate about the company/architecture/tech. stack. You don't need to ask all the questions. Just those which make sense of course. And you can ask more questions specific for the candidate.

1. Introduction of the people in the room and the team.
2. We have a few questions for you to know you better but feel free to interrupt us and ask your own questions anytime. Or you can start asking if you have something on your mind already.
3. Did you check something about the company? Maybe our websites? - if yes ask the candidate to tell you what he knows so you can only clarify where he is not completely right. If not just tell him something interesting and not too long :)
4. What are you working on now?. - If the candidate is a student he can tell you about school and the projects there.
5. Why are you considering to leave your current company? What do you like/dislike on your current job and/or the project(s)? If the candidate hates the company it's a bad sign. There will always be something what could be better but the candidate should be able to list also good things. The worst of all is when the candidate tells you how all the people there are assholes and he is the only one who saves the whole company every time. It's probably the opposite. I would probably ask for specific example of the problems or the situations when the candidate saved the company. And more than one specific example. Also, it's good to explicitly ask for good things.
6. What's the most/least exciting thing you worked on? - I usually ask if there were some big issues (least exciting thing :) he caused in the past and what would he change on the issue to make it properly next time?
7. What do you expect from work here? What makes it an interesting company for you? - You probably want to know how he knows about the company, what he thinks is better here than in other companies, ...
8. Are you looking for a long term cooperation (years) or the short term (less than a year)? How long do you think you are going to stay here if the job will be OK for you? - You probably want long term candidates. If the candidate tells you he plans to travel around the world in 6 months you should really consider if it's OK for you to spend 6 months with teaching him and then he leaves.
9. Where do you learn new things about the technology? Blogs, books, twitter, podcasts, RSS, Reddit, StackOverflow, side-projects? - What book are you reading right now or you read the last time? What is your favorite blog/podcast. Simply ask about specifics of the channel he mentions. This is because everybody says they learn and read books. But you want to see which one. Even non-technical. Maybe even if you don't hire him at the end he can recommend you something inetresting :)
10. Is there anything what would annoy you in the office? Dogs, open-space, air conditioning, noise, ... Is there anything you need to have in the office?
11. Was there any disagreement about what technology should be used, or about the process in the company you've been involved? How was it resolved? - This is something I don't ask often, but if I feel the candidate is more conflicting type or he mentions something from the past I want to know something about these things. He should not be stubborn, but assertiveness is good.

Introduce the company tech. stack

After all the questions it's good to tell the candidate again something about the company and more in-depth technologies used.

1. The languages and the main frameworks used (Kotlin, Python, Spring, Flask, Celery, React.js, Redux, TypeScript, ...). It's good to mention also versions if that matters.
2. GitHub/GitLab/BitBucket/Space
3. CI - GitHub, GitLab, Travis, Jenkins, ....
4. Slack, M$ Teams, Space, ...
5. Trello, Jira, Product board, Redmine, Asana, ...
6. ELK, Graylog, Grafana
7. Intellij IDEs/VS Code

Ask if the candidate knows the tools. If no, it does not matter, but if yes, ask about how he used it in the work. The big warning sign is if the candidate knows everything. It's so called superman (with small s) and you should ask him about really specific things like:

1. In what project did you used the technology? What was the use-case?
2. Why did you choose this technology and not any other similar?
3. What were the problems you had with the technology?
4. What do you like about the technology?
5. When would you use it again and when not?

If he knows the answers on all of them then it's probably Superman (with capital S :) but really make sure. If you hire a superman it's really painful to fire him because they know everything in theory but nothing in practice. Of course every new person should be hired only if you are at least 90% sure you want to work with the candidate, but the candidate who knows every technology is very suspicious.

Hard core skills

This is a part where I really want to see what the candidate knows and how confident he is with his decisions and knowledge. Sometimes (OK, almost every time) I play dumb here and pretend I don't know about the answer the candidate is asking. Instead, suggest to the candidate to Google it - it's part of our job anyway. If the candidate is really confident and I would like to stress him a little I would ask him if he can find the bug in the code even if the code is absolutely OK. I ask questions like: "Are you sure? Any flaw?" To see how he reacts. But I tell him at the end that the code is really OK and it is nice job :)

If you spot any minor issue, like typo in the variable name, just tell the candidate. Don't stress him/her with silly things that IDE can warn him about. This is not a school. But don't answer him a questions like "Is this OK? Is it like that?" If (s)he is not sure about what he wrote he can Google it. When the candidate is writing the code you see how it goes immediately. My strategy is to make the requirements harder and harder until I hit the spot when the candidate does not know. This spot is different for everybody. Simply stop asking when you feel you know where is the upper limit of the candidate's skills. Bellow are the questions I ask along with my favorite exercise tasks.

Python/Java/Kotlin/TypeScript

1. Java/Kotlin: What is the (functional) interface and why does it exist? What's difference between interface and abstract class? - If the candidate does not what the interface is, I would not waste the time and don't hire the candidate. The good place for the secret signal and finish the interview as soon as possible. Otherwise keep going to check the candidate's seniority :)
2. What is the decorator (Python/JS) / annotation (JVM)? Can you name any? Did you write your own already? - This question is about to find out if the candidate knows some more advanced features of the language, but not too advanced. I can decide within one minute based on answers if I can give him harder coding exercise or not. Also if it makes sense to ask about generics during the coding.
   • Java/Kotlin - @Override, @SuppressWarnings, @Deprecated in Kotlin, @JsonProperty from Jackson
   • Python - @property, @classmethod, @app.route from Flask
   • Javascript - @connect from Redux, @Component from Angular, @withRouter from React router
3. When would you use thread and when the process to scale the application and why?
4. (Python) Do you know GIL? Can you tell us about it?
5. Do you know any library, standard or 3rd-party, to scale your application?
6. Do you have experience with tests? Which ones (unit, integration, E2E)? What is your favorite framework?

Code

1. Write function find_max({1, 2, 3, 4, 5}) (or find_min) that accepts an array of numbers and returns maximum (or minimum) from them.
2. Can you find any issues in your implementation? - Common bugs here are:
   1. People don't consider empty array can come - results in the Out-of-range errors.
   2. Sometimes the candidate does not know what to do when the array is empty - raising an exception or returning null is OK IMHO. NaN is kind of half-OK because it does not work with other objects (read later). I try to ask for something else. But returning some special string or INT_MAX/INT_MIN/0 or another weird things are no-go. Any number can be the valid result so we can't use it like special value for empty array.
   3. People don't consider negative/real numbers - these are also numbers and the function should work with them.
   4. People try to find some MIN_INT/MIN_FLOAT or something to initialize the maximum value - the 1st item from the array should picked instead because it can be Decimal number or complex number or we could change the function to accept also DateTime and there is no minimal/maximal value.
3. Can you write some basic tests for your code? What values makes sense to test? How many tests are enough?
4. If the basic code is OK and has no issues I change requirements a little. For example, what if I wanted to call the function like find_max(1, 2, 3, 4, 5) what do I need to change? You can use Google if don't know. The right term to search is "variable arguments".
   • Java private static int findMax(int... items)
   • Kotlin fun findMax(vararg items: Int): Int
   • Python def find_max(*items)
   • TypeScript function findMax(...items: number[])
5. What if I would like to use the function for non-number objects? Can I do it? How? I want to hear about generics, comparators in Java or magic methods (for operator overloading) in Python/Kotlin/TypeScript.
   • Java private static <T> T findMax(T... items, Comparator<T> comparator) and then if (comparator.compare(item, maximum) > 0) { maximum = item; }
   • Kotlin fun <T> findMax(items: Array<T>, isFirstGreater: (a: T, b: T) -> Int): T and then if (isFirstGreater(item, maximum)) { maximum = item }
   • Python def find_max(items, is_first_greater) and then if is_first_greater(item, maximum): maximum = item

/** Kotlin */
fun main(args: Array<String>) {
    assert(findMax(arrayOf(1, 2, 3, 4, 5), { a: Int, b: Int -> a > b }) == 5)
}

fun <T> findMax(items: Array<T>, isFirstGreater: (a: T, b: T) -> Boolean): T {
    if (items.isEmpty()) {
        throw IllegalArgumentException("Empty array is not allowed")
    }

    var maximum = items[0]
    for (item in items) {
        if (isFirstGreater(item, maximum)) {
            maximum = item
        }
    }

    return maximum
}

class Main {
  /** Java */
  public static void main(String[] args) {
    Integer[] numbers = {1, 2, 3, 4, 5};
    assert(findMax(numbers, (a, b) -> a - b) == 5);
  }

  private static <T> T findMax(T[] items, java.util.Comparator<T> comparator) {
    if (items.length == 0) {
        throw new IllegalArgumentException("Empty array is not allowed");
    }

    var maximum = items[0];
    for (final var item : items) {
        if (comparator.compare(item, maximum) > 0) {
            maximum = item;
        }
    }

    return maximum;
  }
}

import pytest

def find_max(*items, is_first_greater=lambda a, b: a > b):
    if not items:
        raise ValueError("Empty array is not allowed")

    maximum, *items = items
    for item in items:
        if is_first_greater(item, maximum):
            maximum = item

    return maximum

assert find_max(1) == 1
assert find_max(1, 2, 3, 4, 5) == 5
assert find_max(-1, -2, -3, -4, -5) == -1
assert find_max(42, 42, 42, 42, 42) == 42
with pytest.raises(ValueError):
    find_max()

1. If I feel the developer is more experienced or he claim he is senior in CV, or has "excellent" Python/Kotlin skills I have more advanced task prepared - length of the longest continuous non-descending subsequence (sub-array). I ask him if he knows what it is, if not he can google it or you can tell him. The steps are the same as for the previous example. I can add generics or comparator. Also the common bugs are usually similar. But people also forget about previous = item often :)

def longest_increasing_subsequence(items, is_first_increasing=lambda a, b: a >= b):
    if not items:
        return 0  # longest sequence for empty list is zero

    length = 1
    max_length = 1
    previous, *items = items
    # for previous, item in zip(items, items[1:]):
    for item in items:
        if is_first_increasing(item, previous):
            length += 1
        else:
            max_length = max(max_length, length)  # can be if length > max_length: max_length = length
            length = 1

        previous = item

    return max(max_length, length)

assert longest_increasing_subsequence([]) == 0
assert longest_increasing_subsequence([0]) == 1
assert longest_increasing_subsequence([1, 2, 3, 4, 5]) == 5
assert longest_increasing_subsequence([-1, -2, -3, -4, -5]) == 1
assert longest_increasing_subsequence([-5, -4, -3, -2, -1]) == 5
assert longest_increasing_subsequence([42, 42, 42, 42]) == 4
assert longest_increasing_subsequence([1, 2, 1, 0]) == 2
assert longest_increasing_subsequence([1, 2, 1, 2, 3]) == 3

1. Can the candidate design some relational DB? Postgres, MySQL? The candidate doesn't need to know the DDL for the tables from memory. Can just write what are the tables, columns and types - pseudo-code. You have system with users and departments. User has name and the salary and department has only name. User can be in no department or in more than one.

CREATE TABLE "user" (
    id BIGINT PRIMARY KEY,
    name VARCHAR NOT NULL,
    salary NUMERIC NOT NULL
);

CREATE TABLE "department" (
    id INT PRIMARY KEY,
    name VARCHAR NOT NULL
);

CREATE TABLE "user_in_department" (
    id_user BIGINT REFERENCES "user" ("id"),
    id_department INT REFERENCES "department" ("id")
);

1. Can you find all users with salary bigger than 100k? SELECT * FROM "user" WHERE "salary" > 100000
2. Can you find all users with name starting with "M"? SELECT * FROM "user" WHERE "name" LIKE 'M%' I meant any "m" ;) ILIKE or "name" LIKE 'M%' OR "name LIKE 'm%'. There is also function to search substring in PG position or the candidate can use regular expressions.
3. Can you find all names of the user who are not in any department? SELECT "user"."name" FROM "user" LEFT JOIN "user_in_department" ON "id_user" = "id" WHERE "id_user" IS NULL
4. Can you find users who are in "IT" department? SELECT "user"."name" FROM "user" JOIN "user_in_department" ON "id_user" = "user"."id" JOIN "department" ON "id_department" = "department"."id" WHERE "department"."name" = 'IT'
5. Can you list the average salary in every department? Including average salary of the people without assigned department? SELECT "deaprtment"."name", AVG("salary") FROM "user" LEFT JOIN "user_in_department" ON "id_user" = "user"."id" LEFT JOIN "department" ON "id_department" = "department"."id" GROUP BY "department"."name"
6. Can you sort departments by the count of the users? The biggest departments first? If there is the same amount of the users can then alphabetically according to department name? SELECT "deaprtment"."name" FROM "user" JOIN "user_in_department" ON "id_user" = "user"."id" JOIN "department" ON "id_department" = "department"."id" GROUP BY "department"."id" ORDER BY COUNT("user"."id") DESC, "department"."name"

JavaScript (frontend)

1. What kind of object inheritance is used in JS and how it works?
2. What is "pure" function? Why we should use it?
3. What is "lambda" function? How is it written? What are (dis)advantages?

Code

1. Write function sum(a, b) that returns a + b if 2 arguments are given and another binded function if only one is given so I can call it either as sum(2, 3) === 5 or as sum(2)(3) === 5. What if we would like to support impossible number of parameters like sum(2, 3, 4)(4, 5)(1, 1, 1, 1)(3, 7)() === 32.
2. What happens when you execute the following code? How would you fix the code if we would like to alert all values in i?

for (var i = 0; i < 3; i++) {
    setTimeout(function () { alert(i) }, i * 1000)
}

1. Find even minimum in the array of the numbers. Can you write it in the functional way? (if the candidate does not know how, you can suggest him to use Array.filter, Array.map, Array.reduce).
2. If the candidate is really good and confident, try to ask what will be stored in numbers in this code const numbers = ["10", "10", "10"].map(parseInt)? The correct answer is [10, NaN, 2] because parseInt(number, radix) accepts also radix from array index so the call will be ["10", "10", "10"].map((value, index) => parseInt(value, index) and radix = 1 does not exist and 10 in radix = 2 is 2.

Outro (practicalities)

This is the section I skip if I know right away we don't want the candidate because we all gave each other the secret signal.

1. When can you start working with us?
2. What contract do you prefer?
3. What is your expected salary? In gross (or net if he's student - students don't pay taxes if they meet certain conditions :).
4. List the company benefits. It's questionable if you want to list the things everyone in the industry has like flexi-time, home-office. It's good to mention some activities people in the company do like football, board games, movie nights.

More sources:

• https://github.blog/2022-03-31-how-github-does-take-home-technical-interviews/

Comments

[Shai Almog]

I like most of what you said but I'm not a big fan of live coding. I feel the environment can be a bit stressful and might not bring the best in a person. Some people (such as myself) thrive under this type of stress, but others don't. I prefer evaluating the debugging capabilities and code reading skills instead of code writing skills. I think the latter are far more important.

I wrote some of my thoughts on the matter here.

[Mišo]

Thanks for the inputs and the link. While I agree some people can not bring the best during the interview I think the reading code is much simpler that actual writing. You see how fluent the candidate is in the language when writing and you can distinguish superman who can talk about everything and can't write a line of code. That's why the exercises are so simple and not some bubble/quick/... sort algorithms or similar stuff. In the end, you don't pay the programmer only for reading and debugging but also for writing the code.

I believe any programmer should be able to write for-loop and if-condition wrapped in a function. No matter how stressful the interview is for him/her :) And if it is too easy for the candidate I can always harden the requirements ;) During the interview it's more a pair programming than live coding and it can lower the stress significantly.