Passionate generalist conquering the web one project at a time. Whether authoring libraries for node, JS, PHP, or Rust, I am always on the lookout for better solutions to common problems.
Location
USA
Work
Lead Developer & Co-founder at corpscrypt, CTO at REtech
Finally have some time to get to this. I apologise for the delay. Let me start by saying that this approach seems very time intensive for both sides and therefore probably not realistic for many companies.
But I have a lot of questions:
Regarding 1.
When you say references in the context of internships, what do you mean? I assume interns usually have little to no previous experience?
Regarding 2.
Is this interview usually in person or over the phone? And you mention that the prospect can learn about the company, but what do you want to learn about the candidate in this interview?
About 3.
This sounds very interesting. Here are the things that come to mind: how do you ensure the result is based on the candidate's work alone? I tutor on wyzant on the side and the amount of students that basically want to pay someone to do their homework is relatively high. And then there is the majority of students who are eager to learn while we work through the assignments, but probably would have some trouble reproducing the complete assignment on their own. So I am torn. On one side, the ability to "google for" help is an important skill we usually don't test for, but effectively will show the long term abilities required to keep up with emerging technologies. On the other side, I would want to know if a candidate can gather information or work through a process completely on their own. While you do mention that in 5., I assume that you will find yourself having invested quite some time for a "poser" sometimes.
About 4.
I respect that dedication. And maybe on a side note for readers: your GitHub account is always more important than the portfolio you polished to hell. Nothing gives me a better understanding of your abilities and practices then your code.
I usually do this in the resume screening phase, but frankly I have never hired interns so I am not sure what I would get out of it in this scenario. Additionally, I work in a field where proprietary work is common. So more often enough people (including myself) cannot share the work they are most proud of.
So I guess this approach is overall fair, but frankly sounds like the opposite issue to what I am describing in the post (and explains your reaction even better): You invest a lot of time with a relatively high risk. In this case it's just you who spends an unreasonable amount of time without knowing "what's in it for you"
I might lack some important details, but overall, I feel like this approach is not scalable. It might work well for you and your candidates/target group, but I am sceptical of this approach being something that can
a) provide companies with an accurate assessment of a candidate's skills while
b) give the candidate a fair chance to show abilities and
c) design the process so both sides have a reasonable amount of time and risk investment
You invest a lot of time with a relatively high risk. In this case it's just you who spends an unreasonable amount of time without knowing "what's in it for you"
Um, ha, that's assuming I go through all these steps with each person regardless. They don't necessarily make it all the way.
When you say references in the context of internships, what do you mean? I assume interns usually have little to no previous experience?
That's correct. I want to know about their work ethic and their character. Often, references are current/past professors or teachers, or supervisors at prior jobs (usually not tech jobs), and the like. Not having references is not a deal breaker, but it certainly helps.
Is this interview usually in person or over the phone? And you mention that the prospect can learn about the company, but what do you want to learn about the candidate in this interview?
We're 100% remote, so this is over video chat. I'm finding out about their career goals, what they want out of the internship, their work/study habits, and generally whether our program will be a good fit for them.
How do you ensure the result is based on the candidate's work alone?
Well aware of this problem. The obvious first step is to check for copy/pasted code from online, but that doesn't guarantee they didn't outsource it. We discuss it and have the candidate work on it during the final interview, and that always routs out the rest.
I will have to answer the rest later, but understand: I've been putting this into practice for years, with a high hiring accuracy.
Passionate generalist conquering the web one project at a time. Whether authoring libraries for node, JS, PHP, or Rust, I am always on the lookout for better solutions to common problems.
Location
USA
Work
Lead Developer & Co-founder at corpscrypt, CTO at REtech
Sorry to switch gears here, but this little sentence opened up a world for me. Are you saying that you are applying long-term metrics to measure "hiring success"? If so, how does that look like? How do you evaluate e.g. that you made the right choice? And after which timeframe do you come to such conclusions?
You must remember, this is an internship, a training program. Whether someone thrives in and successfully completes the program, or washes out (variety of reasons), is a significant indicator. I'm also the internship supervisor, so I'm the one responsible for mentoring, tailoring the program to individual needs, answering questions, and helping the intern succeed.
It is worth noting that, of our 12 program graduates, 9 are in mid-level coding jobs, 1 is in the army, and 1 in graduate school. Only 1 is not in the industry because of the geographic area he wants to stay in.
The percentage of how many hirees actually graudate from the program has gone up significantly as we've refined our hiring process (although our hiring rate really hasn't dropped). I've even checked my assumptions inversely: we've hired some candidates despite some red flags from the hiring process, and they've always wound up washing out due to serious problems.
You also need to know that we have a very diverse team from many backgrounds, cultures, and fields of interest. I encourage interns to be involved in project management and standards development, to share their opinions openly, and to challenge even my ideas! So, their success really is dependent on forming the core soft skills and work habits necessary to thrive in software as a whole.
Regarding the rest of your original response:
Your GitHub account is always more important than the portfolio you polished to hell. Nothing gives me a better understanding of your abilities and practices then your code.
I agree! I allow applicants to use their GitHub as their "portfolio" if they wish; and many do.
There are times when applicants have no portfolio, and I account for that as well.
You invest a lot of time with a relatively high risk. In this case it's just you who spends an unreasonable amount of time without knowing "what's in it for you"
Oh, I know plenty of "what's in it for" MousePaw Media. We have a number of projects going, and I also consider where an intern will best fit. There are times there's a mismatch between goals and interests that becomes quite apparent during the initial interview, wherein the applicant won't continue. We're not right for everybody - we do have actual projects that have to get done too.
We're not an easy internship program to get into. Two of the local universities advised on our program, and one of the Associate Professors (whose job history includes many years at Microsoft) called our program one of the most robust he's seen. He refers students to us directly.
I might lack some important details, but overall, I feel like this approach is not scalable.
So far, it's scaled well. 30+ interested parties, 14 pending applications (as of writing), and it's working well for a fairly reasonable effort from me.
It might work well for you and your candidates/target group, but I am sceptical of this approach being something that can
a) provide companies with an accurate assessment of a candidate's skills...
Again, note the outcomes.
b) give the candidate a fair chance to show abilities and
We've been able to spot, not just present ability, but raw and undeveloped talent beyond that. Remember, those same applicants are now in mid-level full-time software development positions at other firms, and very much enjoying their work from all reports.
c) design the process so both sides have a reasonable amount of time and risk investment
I deliberately take a bit more 'time risk' on, so as to provide constructive (and legally appropriate) feedback to applicants. Often, this is the first tech position these applicants have applied for. We're forgiving of many reasonable faux-pas, and help applicants understand how to improve before they interview elsewhere.
All of that combined is a major reason why the Career Services departments at two of the universities we work with use us as an example for other internship programs.
Passionate generalist conquering the web one project at a time. Whether authoring libraries for node, JS, PHP, or Rust, I am always on the lookout for better solutions to common problems.
Location
USA
Work
Lead Developer & Co-founder at corpscrypt, CTO at REtech
Just to clarify: by "scalable" I meant "applicable to a wider range of companies". While a lot of what you write sounds great, it is a very specific and unusual scenario. In most settings, interns are the exception, not the rule.
So while I believe you that this might be the ideal approach for your organization, I remain sceptical of how transferable such an approach is.
And I don't say that to criticize, I say that as I am searching for solutions that can be applied in the broader context.
Unfortunately, the classic career "You come as a junior and stay for the senior positions" seems to be outdated. Most devs make their move after about 3-5 years.
Certainly, there is some blaming on the employer side to do, but there are many other reasons as well. This poses two issues: it's hard to have employees stay with you. And it is costly to fill positions. Especially in the higher roles, it sometimes takes up to a year to have the full output, as complex onboarding and in-depth understanding of code-base and processes are necessary. This makes "wrong decisions" often not immediately apparent.
It sounds to me (and forgive me with not being familiar with MousePawMedia), that you kind of place yourself between the industry and universities in order to buffer this risk. So I assume both the academic institutions as well as the private sector have an interest in "funneling resources" through you in order to bridge the gap.
And if I am right with this assumption, it makes sense that your approach is vastly different.
Unfortunately, the classic career "You come as a junior and stay for the senior positions" seems to be outdated.
Agreed, and my approach has nothing to do with it. They leave us by design and move into a mid-level position elsewhere.
This poses two issues: it's hard to have employees stay with you.
It's a year long program. Our goal is for them to move on to paid positions at other firms; it's actually how we're structured.
And it is costly to fill positions. Especially in the higher roles, it sometimes takes up to a year to have the full output, as complex onboarding and in-depth understanding of code-base and processes are necessary. This makes "wrong decisions" often not immediately apparent.
The program is a year, but a number of interns have stayed on longer to keep working on the projects. So, I actually have observed the full output and the in-depth understanding. (Don't get hung up on "intern" — this program goes significantly deeper than standard internships.)
I think you're confusing my motivation and evidence of success. Just because I am hiring interns does not mean my approach is only applicable interns. I account for the lack of experience by deliberately going a lot easier on candidates than I (or anyone) would be. There are other companies that successfully use similar practices to what I describe.
You do have to be careful of confirmation bias. I'm reading "I don't think coding challenges work, therefore the fact they work for you must be an unusual circumstance."
As to myself, I can only speak to my own experience, and I can share that of the hiring managers (etc) I know...although you know neither of us could confirm or deny it second-hand.
So I assume both the academic institutions as well as the private sector have an interest in "funneling resources" through you in order to bridge the gap.
The private sector hardly knows I exist. I run the program for the sake of the students, to help them gain the all-important soft skills necessary to thrive in the coding field. The Career Services know this, which is why they work with me.
And if I am right with this assumption, it makes sense that your approach is vastly different.
Except it isn't all that different...based on talking to hiring managers, former interns who have gone on to other jobs, career services faculty, and my own experiences being a candidate. I've observed four types of companies:
Passionate generalist conquering the web one project at a time. Whether authoring libraries for node, JS, PHP, or Rust, I am always on the lookout for better solutions to common problems.
Location
USA
Work
Lead Developer & Co-founder at corpscrypt, CTO at REtech
I'm reading "I don't think coding challenges work, therefore the fact they work for you must be an unusual circumstance."
No, apparently I haven't put that in the right context: I didn't refer to coding challenges when mentioning scepticism. I also don't think coding challenges aren't working in general, it's the code golfing challenges I have issues with.
The private sector hardly knows I exist. I run the program for the sake of the students,...
Well, that might be true, but I am looking at what you are saying from a different perspective. What I am hearing is: "students that ran through our program end up being very employable and capable". So from the perspective of companies, this translates to: "students vetted and 'formed' by MousePawMedia are low-risk, capable employees".
So maybe it makes sense for you to form relationships with the industry as well. You more or less offer a service that circumvents the very problem I am reporting here. And that service has a value you might be able to capitalize on without selling your ideals and while still fulfilling your main goals.
I still can't believe we're so unusual that our hiring practices couldn't possibly work elsewhere. I'm able to determine the skill level of applicants with the coding challenge, and that has proved accurate. It could be made more restrictive, or more tied to a particular language or domain, without taking additional time from either party.
In any case, 20 minutes of me (the hiring manager) reviewing the coding challenge gives me a fairly accurate snapshot of the applicant's present skill, and that always either saves me a final interview (2 hours saved) or advises the final interview, making it far more reliably insightful.
So from the perspective of companies, this translates to: "students vetted and 'formed' by MousePawMedia are low-risk, capable employees".
Ah, yes, that is indeed what I'm hoping our company represents to employers, especially in terms of the letters of recommendation we provide to program graduates. It's a win-win for our internship graduates and their future empoyers.
So maybe it makes sense for you to form relationships with the industry as well. You more or less offer a service that circumvents the very problem I am reporting here. And that service has a value you might be able to capitalize on without selling your ideals and while still fulfilling your main goals.
Ahh, I understand you now! Thank you for that. I would need like to form stronger connections between MousePaw Media and the industry.
That said, I'd still understand the need for vetting, such as how we do it, even for our own internship graduates applying elsewhere. Someone may be fluent in (say) C++ and algorithms, but that doesn't mean they're good at (say) API testing in Python. In my mind, half the beauty of a coding challenge such as I describe is "are you a good fit for this particular role?"
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Finally have some time to get to this. I apologise for the delay. Let me start by saying that this approach seems very time intensive for both sides and therefore probably not realistic for many companies.
But I have a lot of questions:
Regarding 1.
When you say references in the context of internships, what do you mean? I assume interns usually have little to no previous experience?
Regarding 2.
Is this interview usually in person or over the phone? And you mention that the prospect can learn about the company, but what do you want to learn about the candidate in this interview?
About 3.
This sounds very interesting. Here are the things that come to mind: how do you ensure the result is based on the candidate's work alone? I tutor on wyzant on the side and the amount of students that basically want to pay someone to do their homework is relatively high. And then there is the majority of students who are eager to learn while we work through the assignments, but probably would have some trouble reproducing the complete assignment on their own. So I am torn. On one side, the ability to "google for" help is an important skill we usually don't test for, but effectively will show the long term abilities required to keep up with emerging technologies. On the other side, I would want to know if a candidate can gather information or work through a process completely on their own. While you do mention that in 5., I assume that you will find yourself having invested quite some time for a "poser" sometimes.
About 4.
I respect that dedication. And maybe on a side note for readers: your GitHub account is always more important than the portfolio you polished to hell. Nothing gives me a better understanding of your abilities and practices then your code.
I usually do this in the resume screening phase, but frankly I have never hired interns so I am not sure what I would get out of it in this scenario. Additionally, I work in a field where proprietary work is common. So more often enough people (including myself) cannot share the work they are most proud of.
So I guess this approach is overall fair, but frankly sounds like the opposite issue to what I am describing in the post (and explains your reaction even better): You invest a lot of time with a relatively high risk. In this case it's just you who spends an unreasonable amount of time without knowing "what's in it for you"
I might lack some important details, but overall, I feel like this approach is not scalable. It might work well for you and your candidates/target group, but I am sceptical of this approach being something that can
a) provide companies with an accurate assessment of a candidate's skills while
b) give the candidate a fair chance to show abilities and
c) design the process so both sides have a reasonable amount of time and risk investment
Um, ha, that's assuming I go through all these steps with each person regardless. They don't necessarily make it all the way.
That's correct. I want to know about their work ethic and their character. Often, references are current/past professors or teachers, or supervisors at prior jobs (usually not tech jobs), and the like. Not having references is not a deal breaker, but it certainly helps.
We're 100% remote, so this is over video chat. I'm finding out about their career goals, what they want out of the internship, their work/study habits, and generally whether our program will be a good fit for them.
Well aware of this problem. The obvious first step is to check for copy/pasted code from online, but that doesn't guarantee they didn't outsource it. We discuss it and have the candidate work on it during the final interview, and that always routs out the rest.
I will have to answer the rest later, but understand: I've been putting this into practice for years, with a high hiring accuracy.
Sorry to switch gears here, but this little sentence opened up a world for me. Are you saying that you are applying long-term metrics to measure "hiring success"? If so, how does that look like? How do you evaluate e.g. that you made the right choice? And after which timeframe do you come to such conclusions?
You must remember, this is an internship, a training program. Whether someone thrives in and successfully completes the program, or washes out (variety of reasons), is a significant indicator. I'm also the internship supervisor, so I'm the one responsible for mentoring, tailoring the program to individual needs, answering questions, and helping the intern succeed.
It is worth noting that, of our 12 program graduates, 9 are in mid-level coding jobs, 1 is in the army, and 1 in graduate school. Only 1 is not in the industry because of the geographic area he wants to stay in.
The percentage of how many hirees actually graudate from the program has gone up significantly as we've refined our hiring process (although our hiring rate really hasn't dropped). I've even checked my assumptions inversely: we've hired some candidates despite some red flags from the hiring process, and they've always wound up washing out due to serious problems.
You also need to know that we have a very diverse team from many backgrounds, cultures, and fields of interest. I encourage interns to be involved in project management and standards development, to share their opinions openly, and to challenge even my ideas! So, their success really is dependent on forming the core soft skills and work habits necessary to thrive in software as a whole.
Regarding the rest of your original response:
I agree! I allow applicants to use their GitHub as their "portfolio" if they wish; and many do.
There are times when applicants have no portfolio, and I account for that as well.
Oh, I know plenty of "what's in it for" MousePaw Media. We have a number of projects going, and I also consider where an intern will best fit. There are times there's a mismatch between goals and interests that becomes quite apparent during the initial interview, wherein the applicant won't continue. We're not right for everybody - we do have actual projects that have to get done too.
We're not an easy internship program to get into. Two of the local universities advised on our program, and one of the Associate Professors (whose job history includes many years at Microsoft) called our program one of the most robust he's seen. He refers students to us directly.
So far, it's scaled well. 30+ interested parties, 14 pending applications (as of writing), and it's working well for a fairly reasonable effort from me.
Again, note the outcomes.
We've been able to spot, not just present ability, but raw and undeveloped talent beyond that. Remember, those same applicants are now in mid-level full-time software development positions at other firms, and very much enjoying their work from all reports.
I deliberately take a bit more 'time risk' on, so as to provide constructive (and legally appropriate) feedback to applicants. Often, this is the first tech position these applicants have applied for. We're forgiving of many reasonable faux-pas, and help applicants understand how to improve before they interview elsewhere.
All of that combined is a major reason why the Career Services departments at two of the universities we work with use us as an example for other internship programs.
Just to clarify: by "scalable" I meant "applicable to a wider range of companies". While a lot of what you write sounds great, it is a very specific and unusual scenario. In most settings, interns are the exception, not the rule.
So while I believe you that this might be the ideal approach for your organization, I remain sceptical of how transferable such an approach is.
And I don't say that to criticize, I say that as I am searching for solutions that can be applied in the broader context.
Unfortunately, the classic career "You come as a junior and stay for the senior positions" seems to be outdated. Most devs make their move after about 3-5 years.
Certainly, there is some blaming on the employer side to do, but there are many other reasons as well. This poses two issues: it's hard to have employees stay with you. And it is costly to fill positions. Especially in the higher roles, it sometimes takes up to a year to have the full output, as complex onboarding and in-depth understanding of code-base and processes are necessary. This makes "wrong decisions" often not immediately apparent.
It sounds to me (and forgive me with not being familiar with MousePawMedia), that you kind of place yourself between the industry and universities in order to buffer this risk. So I assume both the academic institutions as well as the private sector have an interest in "funneling resources" through you in order to bridge the gap.
And if I am right with this assumption, it makes sense that your approach is vastly different.
Agreed, and my approach has nothing to do with it. They leave us by design and move into a mid-level position elsewhere.
It's a year long program. Our goal is for them to move on to paid positions at other firms; it's actually how we're structured.
The program is a year, but a number of interns have stayed on longer to keep working on the projects. So, I actually have observed the full output and the in-depth understanding. (Don't get hung up on "intern" — this program goes significantly deeper than standard internships.)
I think you're confusing my motivation and evidence of success. Just because I am hiring interns does not mean my approach is only applicable interns. I account for the lack of experience by deliberately going a lot easier on candidates than I (or anyone) would be. There are other companies that successfully use similar practices to what I describe.
You do have to be careful of confirmation bias. I'm reading "I don't think coding challenges work, therefore the fact they work for you must be an unusual circumstance."
As to myself, I can only speak to my own experience, and I can share that of the hiring managers (etc) I know...although you know neither of us could confirm or deny it second-hand.
The private sector hardly knows I exist. I run the program for the sake of the students, to help them gain the all-important soft skills necessary to thrive in the coding field. The Career Services know this, which is why they work with me.
Except it isn't all that different...based on talking to hiring managers, former interns who have gone on to other jobs, career services faculty, and my own experiences being a candidate. I've observed four types of companies:
No screening.
Traditional whiteboarding.
Leetcoding screening. (The topic)
What we do.
No, apparently I haven't put that in the right context: I didn't refer to coding challenges when mentioning scepticism. I also don't think coding challenges aren't working in general, it's the code golfing challenges I have issues with.
Well, that might be true, but I am looking at what you are saying from a different perspective. What I am hearing is: "students that ran through our program end up being very employable and capable". So from the perspective of companies, this translates to: "students vetted and 'formed' by MousePawMedia are low-risk, capable employees".
So maybe it makes sense for you to form relationships with the industry as well. You more or less offer a service that circumvents the very problem I am reporting here. And that service has a value you might be able to capitalize on without selling your ideals and while still fulfilling your main goals.
Ah, I understand now.
I still can't believe we're so unusual that our hiring practices couldn't possibly work elsewhere. I'm able to determine the skill level of applicants with the coding challenge, and that has proved accurate. It could be made more restrictive, or more tied to a particular language or domain, without taking additional time from either party.
In any case, 20 minutes of me (the hiring manager) reviewing the coding challenge gives me a fairly accurate snapshot of the applicant's present skill, and that always either saves me a final interview (2 hours saved) or advises the final interview, making it far more reliably insightful.
Ah, yes, that is indeed what I'm hoping our company represents to employers, especially in terms of the letters of recommendation we provide to program graduates. It's a win-win for our internship graduates and their future empoyers.
Ahh, I understand you now! Thank you for that. I would need like to form stronger connections between MousePaw Media and the industry.
That said, I'd still understand the need for vetting, such as how we do it, even for our own internship graduates applying elsewhere. Someone may be fluent in (say) C++ and algorithms, but that doesn't mean they're good at (say) API testing in Python. In my mind, half the beauty of a coding challenge such as I describe is "are you a good fit for this particular role?"