The problem is that this doesn't scale with talent and/or experience: I certainly wouldn't be better at coding interviews than I was out of college. People who aren't great at hiring take an approach that's valid at the low level and extrapolate it well beyond the bounds where it makes sense.

On the other hand, this only works if the pedigree is recognizable and reliable. I can be pretty confident that I don't need to test basic coding literacy of someone who was an engineer at Google for four years, but I've interviewed people with ten years experience and great references who were clearly dumb as a brick during the interview. In one case, we hired the guy over my objection and spent the next year fighting the fires he'd set whenever he touched anything (the founder eventually found some relatively unimportant, isolated niche for him to keep busy in).

It's a really hard problem, and most complaints I've come across are generalizing from their skewed view of what the candidate pool is like (usually assuming that most candidates are roughly as talented as they are, when the reality is that many candidate pools consist of some of the dumbest people you're likely to ever encounter)

[1] ie, practically everyone

Whatever happened to getting to know a candidates work? How about look into the work that a person has done and take the time to understand where a person's skills are. The problem with tech hiring is we have people trying to cut corners. So-called 'non-technical' recruiters doing interviews with a checklist, companies that treat people like hoop-jumping monkeys, and generally f*king idiots that won't do their job (they get paid for it, why again? They're not actually doing their job.)

Hiring is not a complex problem. The problem is literally incompetent people doing hiring.

The best you can do is sort out the people you absolutely cannot use or don't want. The rest? The one's that you reject because of not matching some part of the interview process but largely are a good fit? They're probably fine, they're probably better than you think they are. In fact, statistically speaking it's probably better to have gambled on these people than the people you've hired and then had to fire.

We don't have the industry standards, the experience, the data, or the worker pool to fuss around with these ridiculous interview processes some companies employ. In fact these interview processes actively filter for people who are only willing to go through with the process, not necessarily the best talent.

Because of that I no longer bother looking at Who Is Hiring or Work At A Startup.

From the perspective of more junior developers this must look like a game with checkboxes and scorecards. That’s exactly how third party recruiters view it. So, I completely understand use of AI boilerplate. Recruiters are the only way most of us are getting past resume submission now and they will game the hell out of your resume to get their commission.

Look, hiring is broken. It’s not clear what most employers are actually looking for if it’s not addressed in the job title. I just won employment with this big rocket company and it comes down to Java, test automation, cucumber. The interview was 30 minutes and I was selected because of my experience knowing I have never written Java. A tech stack never came up, but the requirements and expectations are clear on all sides.

In order to fix hiring employers must be willing to hire competent people to solve problems, not replaceable cogs to fill a seat. Most developers cannot communicate in writing, so make that part of a job filter that a candidate must write some essay in real time and must demonstrate some super trivial code solution (not leet code nonsense) in real time. Require they have code online somewhere if they pass all the filters. Garbage in, garbage out.

I also suggest reading about shovel vs spoon. The requirements for hiring in software are so failingly low that it resembles job placement. If you want candidates to take this seriously then don’t cast a wide net.

My problems:

1. Makes demands of the candidate before selling the role (in fact, explicitly!)

I recommend you don't do this. Instead, make a point of going out of your way to sell your company and your role to your candidates before you do anything to screen them.

This serves three purposes: (1) sure, it makes it marginally more likely that you'll keep high-value candidates engaged who'd otherwise fall out of your pipeline at/after "stage 1"; (2) more importantly, it helps disarm the interview process, both by establishing that you, the employer are going to put the effort and initiative into keeping the process running and establishing that tone for the whole process, and, even more importantly, by making the candidate actually want the job so they'll perform better during the process; and, (3) it's both the right thing to do and a noticeable difference from most firms' terrible hiring processes, which is good for marketing.

2. Is extremely subjective

Right out of the gate you're asking the candidate questions you can't possibly be benchmarking. "Code you're proud of"? What if the code the candidate is most proud of is something simple that been super useful on lots of projects? What if the candidate cherry-picked it from their code to demonstrate the most complicated stuff they've worked with? How will you compare it to every other code sample you get across the lifetime of your company? You're doing that, right? Keeping track of how well your hiring process predicts performance?

2a. Culture fit

Beware culture fit questions. You state it outright later in your process when you say (paraphrased) "you can train technology but personality mismatches are impossible". That's a recipe for hiring the same 10 dudes who like roughly the same X360 games. I know that sounds hyperbolic, but look around you at other companies and tell me that isn't a real concern.

Two other problems with culture fit questions: (a) they mask irrational responses from interviewers, which happen all the time. Some of your best performers do badly on interviews, because they just don't interview well. Some of those people (ironically, it seems to be those people in particular) will be hard on candidates who have the same problem. Why are you setting the expectation with your team that they should ding candidates for intangible, irrational, or subjective concerns?

In neither (2) or (2a) am I arguing that you should only be hiring based on measured defect density or ability to correctly estimate how much time feature X would take† or typing speed. If there are specific personality traits you need to select for, that's fine: just select for them deliberately, and train your team how to select for them, and track how you're doing.

3. The tryout project

This is going to make me sound like a jerk for at least 2 reasons I can immediately think of, but, to sacrifice (a lot of) tact for (a little) clarity:

I think less of teams that propose short contracting projects to candidates to try them out, because I would never agree to interview on those terms.

Here are the problems I have with this approach: (a) it invites/provokes an argument during your hiring process about what a good daily rate is, (b) it asks the candidate to negotiate an hourly rate in a situation where their leverage is egregiously compromised, which is unethical, (c) job searches are full-time jobs by themselves, and your best candidates are almost invariably already employed, so how could it be reasonable to demand they take a temp job with you as part of that search, (d) it requires you to demand the candidate produce billable work and assign the resulting IP before the candidate is sure they're going to accept the offer they might get from you, (e) the tryout project is deceptively low-information.

On that last point: I guess I've hired a person or two in my career that ended up not working out almost immediately. But usually, the dealbreaker problems I've had with coworkers or employees took months to surface. Lots of people can be productive for a short, high-stakes sprint. But virtually no dev team is hiring for that; they're hiring for the ability to be reliably productive, to exercise good judgement in design and estimation, and be compatible with the decision making process the team uses.

Yes, the tryout project proves the candidate can produce code you might deploy on your website. I hope I'm not the first to tell you that that is a very low bar; lots of high school students can do the same.

Please please please take this comment in the spirit I intended it. I expect lots of smart people will disagree with some or all of it, and would be happy to learn from them. I'm pretty convinced though that the conventional interview system that some of the smartest teams use is fatally flawed in a bunch of ways, and this hiring process is an opportunity for me to start talking about why.

%. One last point because I am on a tear about this lately

I would like to start establishing a meme: Job Interviews Are Hostile For Candidates.

Normal people don't enjoy interviewing for jobs. Normal people find the experience off-putting. They're nervous. They're dealing with a procession of people each of whose stated position is to judge them. Normal people don't like being judged by strangers. They sit there and read tea leaves from behavioral cues about whether they're coming across well. In my experience, this happens even in phone interviews, which (can we also establish this meme?) are the worst.

I am not saying don't interview people. I'm just saying when you do it, assume all the signals and readings you're getting from your candidate are janky as hell, because they are. Lots of strong performers can't do their best thinking and judging when they're conflicted and anxious. Compensate.

Whoah, long comment.

† BTW: my favorite dev interview question

†† There is a better way to write this paragraph but I'm pretty depleted right now so please excuse the clunky, combatitive tone.

We used to send out open-ended programming assessments as part of our hiring process. It's really hard to keep track of everything without an ATS and good internal processes and even with those it's hard to predict how many candidates will drop out at this stage so there's a strong incentive to invite too many and risk getting swamped than vice versa.

Not proud of it but I've definitely kept candidates waiting for several weeks due to being unable to keep up with the rate of submissions.

I think there's a non zero chance this guy is assuming too much malicious intent and could get a positive response still

Why go through all the levels of indirection of a LinkedIn search and then sift through hearsay? Why assume that the one or two people you talk to actually know what they're talking about? You haven't even vetted the candidate, so you're going to put blind faith in the two strangers you find via LinkedIn? What if they are not very good, or have poor judgement, or what if they hate your candidate because he was really good and they weren't (or for any other irrational reason)?

I'm not trying to be rude, and maybe I'm missing something brilliant here ... but I just don't see any "secret sauce" here.

Basically need a Codality for normal companies with reasonable mixture of real world practical problems, and reasonable algorithm questions. Then hopefully the copy cat culture can pick up on that and we’ll have an organic fix to the shitty programming interview puzzle garbage filtering out decent developers who are just fine for most companies.

If all they do is read the resume, then we know, based on other people's comments here, that it's clearly not good enough of a method to find the best talent. So, if this is the case, why do we keep relying on recruiters to find us talent?

Instead of throwing tricky algorithm questions at a candidates, I scour their detailed employment records for the most relevant experience for the first project. In other words, I'm looking for relevant experience rather than top-of-the-head algorithmic brilliance. In the interview, I pose our project problem, and the candidate who gives me the most impressive proposal to get that done gets a chance to solve it.

I hire from an international pool, often on Upwork, so I can start developers on a project basis.

If the developer does a great job, we hire him/her for another project, and so on. At some point, this becomes a full-time relationship, with stock options and other perks.

Using this approach, we value experience over "raw intelligence," per se, and we end up with a team of self-directed developers who are fabulous at delivering great finished products.

It's amazing how well this has worked out for us. I think there's an arbitrage opportunity to avoid coding tests and hire on this basis.

I was tricked into a code golf challenge which was a hiring test. The result was good I was surprised that slacking off could get me a nice discussion, and the company got to pitch their positions. I've never done a hacker rank thingy for hiring and probably never will, I loathed them in school. They do not tickle my mind nor can I scratch my itches with them.

I know this sentiment is shared by many high value employers I've had the pleasure to work with.

1. Weed out the truly incompetent. A very simple programming exercise is sufficient. Keep in mind that the goal of the exercise is not to identify a great programmer, but rather to identify those who can't program at all.

2. Set expectations. During the interviewing process, try your best to work out what you would expect from the individual if hired. Would you expect them to churn out a ton of quality code? Do you expect them to be great at determining business needs and writing less code, but code with a high impact? Is this going to be your go to guy for technology X? Do you expect your junior person to learn Y and Z so they can start contributing after N months?

3. Communicate your expectations clearly. Make sure the candidate knows what's expected of him or her. Towards the end of the interview discuss how they feel about those expectations. Ask how they plan on going about meeting them.

4. After hiring: If you get it right great, if not, let the person go sooner than later. You do yourself and the individual a great disservice if you try to turn them into something they are not. Don't try to turn a lump of coal into a diamond. It's a trap. It doesn't work. They will be unhappy and resentful if you try. Nobody likes to be fired (or do the firing) but you have to tell yourself it's best that they get back out there and find the job that they can be happy and successful at.

We recently did a bunch of hiring. Our coding excersize was practical and involved working through some existing code that had a bug and extending it. They had to be able to state the problem back to us clearly. It was take home solutions submitted to git. It was simple to weed through candidates looking at the code for 30 secs. We looked for things like clean code and well thought out solutions. We didn't do the silly BigO optimization stuff that every company is obsessed with. We've been very happy with our hires.

As I said, this is just not how things are done at modern tech companies and there are good reasons - hiring managers generally don't have the authority to arbitrarily bypass standard processes because hiring managers are inherently biased towards hiring someone good enough for the time being, even if it lowers the bar. So the goal isn't to hire somebody that the hiring manager feels is good enough, but to objectively compare thousands of candidates against one another in a systematic way that keeps the bar sufficiently high enough. You should understand at least what the problem is before you can criticize the solution.

Also, generally speaking, if you pay attention to research into hiring, the two things that really stand out are that 1) general abilities are more important than specific skills and 2) standardized processes outperform ad hoc evaluation. If you're asking to be evaluated on ad hoc credentials that cannot be objectively compared to others (most people's best work cannot be shared) - you're asking people to forego processes. And that is simply not known to work very well, mainly because it's not evidence-based. Your unique set of accomplishments that cannot be compared is not known to correlate with any measure of performance because there's no way to say, people who tend to have accomplishments like yours tend to perform at a certain level.

I also don't know what you mean by "top-shelf" corporation but I'm not aware of any top software company that doesn't basically do this - your claim to have been a hiring manager at a top-shelf corporation isn't consistent with your lack of understanding of the realities faced by top-shelf corporations where there are far more candidates than qualified due to paying well above market and the need to ensure that the bar is kept consistently high, regardless of the motivations and qualities of individual hiring managers and interviewers (unless your top-shelf company isn't actually a software company, in which case it's likely not top-shelf at software).

There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire. If there were, everyone would be using it.

Instead, third-party HR monoliths have moved in and snatched up (and more or less created) the 'market' for screening and filtering applicants. Larger companies sigh, throw up their hands, and say 'well how else can we deal with hundreds of applicants?'.

As if that 'funnel' of applicants contains what you think you want. As if it's just an exercise in reductionism, each applicant a data point to evaluate.

Given this, why not hire lightly and fire lightly?