I think the core issue is that hiring a bad programmer is so costly, and there are so many bad programmers in the hiring pool that companies tune their filters to avoid the worst, while thinking they are aiming for the best.

After a few years of doing it, I think those fears are silly and it’s all a lot of overblown scare tactics to pressure us into cargo-cult adoption of what Google has done, usually through consulting firms that advise on the recruiting process or through new businesses like HackerRank, etc., that try to commoditize this fictional hiring fear.

What has worked for me is trusting recommendations from my existing team, employees in a company, or my extended network. I’ve also selectively reached out to specific users of GitHub, Stack Overflow, Kaggle, and other community sites. I will also travel to university recruiting fairs, professional conference recruiting fairs, and sometimes local meetups or technology groups.

I avoid opening up a job ad to the whole internet and accepting an undifferentiated stream of resumes. I avoid it not because it means I have to weed out “bad coders” or some nonsense — in many cases, most applicants are fully skilled enough for the job, and it’s just a big lie in our industry when people whine about inability to pass FizzBuzz. It keeps the incentives aligned towards hiring hyper-competitive 22-year-olds who just spent 10 months on leetcode and end up working for a lot less money, while nobody thinks about the impact on culture, long-term design or other expertise, diversity, etc. They just want the 22-year-old leetcode jockeys to arrange them as a doll collection in open-plan seating for VCs and investors to walk by.

Instead I an internet stream of resumes because it’s not cost effective to put myself in a position where I or especially engineers on my team have to scour huge piles of resumes.

I can’t stress this enough. You should not open yourself up to a firehose of resumes. And adding extra filters, like coldly making candidates complete a code test before they even speak to a human about the nature of the job is just a symptom of the contortions you have to do if you open yourself up to a firehose.

Focus on improving the useful signal of the source of resumes. Keep diversifying and improving this, and take on small batches of higher signal resumes.

If I could summarize what has worked efficiently for me:

- keep job ads short and technically focused on the intended job duties

- don’t open yourself to a firehose of resumes

- use technically probing conversation, recursively digging into really technical details of the candidates past experiences or studies

- code tests, if used at all, should be used lightly, involve no hazing-style algorithm trivia, and candidates should have options regarding their comfort and preferences (e.g. doing it on-site vs take-home, using their editor & programming environment of choice, generous time limit, and absolutelt nothing resembling HackerRank/coderpad/whiteboard hazing nonsense).

- Keep initial interviews short and informative.

- Use the max over a candidate’s performance in early rounds, and min in later rounds, explained below:

- In early rounds, always have more than one interviewer to get multiple impressions, and use the best candidate impression to decide to continue them to the next round.

- in late rounds, have many interviewers and use something closer to the candidate’s worst session to decide on an offer, but perhaps with some careful discussion to mitigate the chance it was unrepresentative.

- And of course always give the candidate a chance to ask questions and see the facilities. I would expect a candidate to absolutely turn me down if they aren’t allowed to ask questions, and why shouldn’t they? Especially if my company comes across like we care more about cramming in one more trivia question than about the candidate’s curiosity and preferences, frankly then we don’t deserve to hire a good candidate.

There really is such a woeful lack of Peopleware-style common sense in modern tech hiring; a lack of basic humanity or empathy to such a degree that it actually hurts business.

Kudos to the author for being so forthcoming about his experience. Maybe if enough programmers start making a big stink about this problem, something will actually get done about it.

The process Amazon and many other tech companies use is fucking terrible. Algorithm tests and surprise CS 101 questions do not identify good employees. They're biased towards recent grads, do not address most real world situations, can be gamed through studying, and do not identify people who can actually think. You should not be testing for people who interview well, you should be testing for people who will make good employees.

You need to test real world scenarios. If they're going to be re-implementing bubble sort and doing binary math then fine, use HackerRank. Otherwise you need to have candidates work on a project based on what they'll actually be doing. Will they be working on APIs? Have them build or integrate with an API. Mapping in an iOS app? Have them do that.

Do not drop big surprises on candidates. Respect them and they will respect you. Tell them what to expect up front. The first email we send includes an outline of our entire hiring process with a list of each step. I've lost track of how many people have thanked me for this. Going through an interview blind is extremely stressful and increases the likelihood of losing good candidates.

My goal is to set people up for success. If their skills match what we're looking for I want them to succeed. I don't want them to fail because they are bad negotiators, didn't have time to study CS questions, or don't fit the typical stereotypes of a programmer.

In my career I've dealt with quite a few people like this, both as a hiring manager and as a candidate. As a hiring manager you get the opportunity to ask them some detailed questions about how they filter candidates, and I even asked one, once, to send me all of the resumes they got for the position, after she'd applied her filter. I went thru the entire stack of resumes myself, and filtered them by looking at them briefly, and then compared the ones I pulled out to hers. There was zero correlation. None of the ones I pulled out she did. All of the ones she pulled out -- and was busy giving me the hard sell on-- were ones I'd excluded. In fact my belief that the pickings couldn't be so slim was the reason I asked her to send me all of them, though I gave her some other excuse at the time. I also pulled twice as many candidates as she did, and was willing to spend up to an hour of my own time on the phone with them.

All of the people in your position think they know what they are doing and think they are good at being able to tell good candidates from bad ones. I understand why this might be, as this seems to be what they think is their job. "Anybody can exclude the janitors-- but I add value by being able to tell good programmers from bad ones!"

If you'd like to be a better recruiter, here's some changes to make:

1. Stop reviewing people's code. I've participated in many code reviews. I've often seen stuff that looked stupid, and asked the programmer about it, and then found out that they were expressing a level of understanding of the problem that was stronger than mine. Makes sense given that it wasn't my code I was reviewing, I was just reviewing, not writing it. That you think you can review someones contributions to open source in isolation is a big red flag. Just stop. Even if you're a great programmer, you can't do it fairly. In fact, if you were good enough to do it, you wouldn't want to do it because you'd know how easily it is to get wrong.

2. Just because someone doesn't list python on their resume, does not mean they wouldn't be a great python programmer. I've been using python off and on since the mid 1990s. I use it for utilities, tools and other support code, not for major projects, and it turns out (I just checked) I never actually put it on my resume. By your statements above I would have a "strong career but zero commercial or practical experience with python".

3. Just because someone has never programmed in python before, does not mean can't be the best candidate for the job. Programming is the talent. Python is a language. If you would say I'm unqualified to drive a ford because I've been driving toyotas, you'd be laughed at, wouldn't you? This is what you're doing. Every language has its strengths and weaknesses, sure. Unless you're hiring someone for a 3-5 week job, the quality of python code produced in the 6th week will have everything to do with the strength of the programmer, and nothing to do with the amount of python they've used in the past.

4. I once was interviewed by a recruiter and it became immediately obvious he was something special. I could tell he wasn't technical, so after he said he'd be setting up an interview with the hiring manager, I asked him how he'd come to the decision. He gave me a solution that you, and every non-technical person can apply, which is really quite brilliant: He said (paraphrasing) "I look for three things: ABC: Attitude, Bandwidth and Confidence. If they have a positive attitude, can explain things well to show they have a bright mind, and have some confidence, then I ask them a follow up question. I ask them to tell me about some technical thing they did that they're proud of. If my eyes glaze over, I know they are qualified to pass on." He didn't have to understand what the thing was, but he was uncanny in finding great candidates... in large part because his reputation caused them to seek him out. Maybe someone could bullshit him with a bunch of technical jargon that he didn't understand, that's fine, that's not his job. The idea of him judging my code is absurd, and would be to him too.

Please take this in the spirit it is intended, which is constructively.

More importantly, you need to look beyond yourself. I'm an old developer and I don't like to be grilled on algorithms either. But my understanding is that it works very well and that no one's ever come up with anything that's better in terms of effectiveness at scale. Giving random hiring managers discretion to hire who they want simply doesn't work if you're a desirable place to work and need some minimal level of competence across the board. What do you think would work any better? This focus on how it feels from your perspective is fine if you all you're trying to say is that it sucks for you personally but ultimately if you're trying to prove that this is a bad process for those who employ it, you need to make a case that there are better, more efficient processes. Every process will lose some qualified people - so that you may have been qualified, but were rejected isn't meaningful evidence against the process.

Btw, appeal to special credentials as a way to criticize standard procedures is an implicit argument that hiring managers should be given discretion to bypass standard procedures to hire special candidates. I've never really seen this go well, at a level below, say, industry pioneers. I do distinctly remember the one time I did that as a manager to hire someone with a rather long series of accomplishments who refused to take coding tests - I had to fire him rather quickly. The interesting part, other than the complete absence of any sort of actual productivity, is that he couldn't work with anyone, was right about everything and his long list of accomplishments was constantly used as a reason why he didn't have to do anything he didn't want to and any rule he didn't want to follow was stupid.

So I don't think standard algorithm tests are that great as a filter, but at the very least, the willingness to go through it communicates some level of willingness to do things you don't like or aren't best at. A huge part of what makes engineers successful isn't the ability to do things they are good at, but the willingness to do things they are not good at.

The best you can do is sort out the people you absolutely cannot use or don't want. The rest? The one's that you reject because of not matching some part of the interview process but largely are a good fit? They're probably fine, they're probably better than you think they are. In fact, statistically speaking it's probably better to have gambled on these people than the people you've hired and then had to fire.

We don't have the industry standards, the experience, the data, or the worker pool to fuss around with these ridiculous interview processes some companies employ. In fact these interview processes actively filter for people who are only willing to go through with the process, not necessarily the best talent.

Candidates shouldn't be groveling on their knees, and any hiring manager who expects that should be avoided at all cost, because they're selecting on willingness to be abused. But it shouldn't be the other way around either; hiring an asshole is not good for the company.

As I said, this is just not how things are done at modern tech companies and there are good reasons - hiring managers generally don't have the authority to arbitrarily bypass standard processes because hiring managers are inherently biased towards hiring someone good enough for the time being, even if it lowers the bar. So the goal isn't to hire somebody that the hiring manager feels is good enough, but to objectively compare thousands of candidates against one another in a systematic way that keeps the bar sufficiently high enough. You should understand at least what the problem is before you can criticize the solution.

Also, generally speaking, if you pay attention to research into hiring, the two things that really stand out are that 1) general abilities are more important than specific skills and 2) standardized processes outperform ad hoc evaluation. If you're asking to be evaluated on ad hoc credentials that cannot be objectively compared to others (most people's best work cannot be shared) - you're asking people to forego processes. And that is simply not known to work very well, mainly because it's not evidence-based. Your unique set of accomplishments that cannot be compared is not known to correlate with any measure of performance because there's no way to say, people who tend to have accomplishments like yours tend to perform at a certain level.

I also don't know what you mean by "top-shelf" corporation but I'm not aware of any top software company that doesn't basically do this - your claim to have been a hiring manager at a top-shelf corporation isn't consistent with your lack of understanding of the realities faced by top-shelf corporations where there are far more candidates than qualified due to paying well above market and the need to ensure that the bar is kept consistently high, regardless of the motivations and qualities of individual hiring managers and interviewers (unless your top-shelf company isn't actually a software company, in which case it's likely not top-shelf at software).

When you never did any hiring, you assume you're mainly looking for good candidates. When you actually have the experience, you know that at least 30% of applicants can't even code the most basic thing.

So I was wondering how you, hiring more than 100 developers, weeded out those candidates who "do the talk but can't walk".

Another thing: how many of those 100 developers you hired were bad hires?

We used to send out open-ended programming assessments as part of our hiring process. It's really hard to keep track of everything without an ATS and good internal processes and even with those it's hard to predict how many candidates will drop out at this stage so there's a strong incentive to invite too many and risk getting swamped than vice versa.

Not proud of it but I've definitely kept candidates waiting for several weeks due to being unable to keep up with the rate of submissions.

I think there's a non zero chance this guy is assuming too much malicious intent and could get a positive response still

That's not to say don't make sure people are smart. And the idea of proving coding skills is good too. But the manner has to be consistent with respecting the time and skills of an expert.

On another hand, think about it from a company perspective: maybe 100 people apply for an advertised role. 25 might be excluded based on CV. Remaining 75 are phone screened and given coding tests online. 35 might be excluded based on that. 40 invited to proceed to on site to interview. 5 out of 40 candidates might demonstrate reasonable ability during on site interviews in terms of actually being able to write code, and demonstrate some problem solving ability, and some familiarity with common data structures, and not have obvious red flags about how they relate to people. Two of these 5 candidates who gave a strong performance drop out due to competing offers, one runs into visa issues, remaining two offers are made and accepted.

Some candidates will apply for roles with no relevant experience, some candidates will demonstrate no ability to program in their preferred programming language during on site interviews after doing well during an earlier remote coding test (e.g. they got a friend to sit the test for them), some candidates have 10 years commercial experience but cannot assess when to use an array versus a hash map.

This might boil down to something like 150 - 200 hours of human effort, much of this engineering effort required to assess engineering ability, to identify a single candidate who is a strong fit and goes on to accept an offer.

From the company's perspective, the company has an imperfect hiring process. Sometimes candidates have a rough day, freeze under stress and don't perform well during the interview- even if they could be a good fit for the job. Sometimes the hiring process emphasises measuring things that aren't necessarily the most important things for the role. But it costs the company a lot more money to hire the wrong person versus missing hiring a person who's a great fit, so it's probably the right tradeoff to bias the hiring process to reject more often than accept, and risk rejecting a number of candidates who could have turned out great.