The technical interviewing scheme is not great, but I haven't seen another system that works.
> I cannot help but think that these big tech companies (FAANG, et. al) are missing out on diversifying and increasing their engineering expertise by passing over developers like you.
I think this is certainly true.
> I often think what would Google/Facebook would be like if they hired in some experienced engineers that may not be able to whiteboard a BFS tree or can tell you Djikstra's algorithm, but have proven business track records of getting projects done, on budget, and on time.
Well... how do we find these people? By looking at their resumes where they claim this? By contacting references who will attest to it? By trusting the intuition of subjective evaluators of the candidates?
Practically speaking, FAANG companies do hire such individuals, they just do it through acqui-hires. If a person works at a company that is good enough to be worth acquiring, then we have a good signal that they are effective employees even absent a direct evaluation of their technical abilities.
"Technical interviews are feared and hated in the industry, and it turns out that these interview techniques may also be hurting the industry’s ability to find and hire skilled software engineers."
It doesn't hurt "the industry", there's a competitive edge to hiring otherwise superior engineers that are passed up by poor hiring practices at certain companies.
Not everybody needs to work at FAANG, or worse, companies that act like they're FAANG. Less money, less bullshit.
> "but most of our interviewers are senior engineers at FAANG"
Wait, so the result is that "interviewers from FAANG companies rate highly interviewees from FAANG (and FAANG-adjacent) companies"?
Or maybe more causally: "those who have already passed FAANG-style interviews are more likely to pass interviews conducted by FAANG people"?
I appreciate the mission here - but if the idea here is to give people a fair shot even if they don't have the pedigree of FAANG, building FAANG interview styles into the system seems counter to your stated goals. If anything these results are concerning - you can interpret the findings in (at least) two ways:
- these companies hire or produce superior engineers, the results you got are indicative of a broader higher caliber of engineer in those companies.
- the interviewing exercise is optimizing for "people who can pass/have already passed FAANG-style interviews", which rolls in all of the myriad biases of FAANG hiring and perpetuates them.
HN loves to hate FAANG interviews. There's always a monthly passion post about how flawed they are.
I don't think it's unfair to say that Google and FB are considered to have strong engineering talent. So the interview process has to have _some_ hand in that success, even if it's also turning away some would-be strong performers.
I'm not saying the interview process shouldn't be criticized or that it's beyond improvement. And, yeah, it's a little silly to go memorize a bunch of algorithms so you can recite them on the stop. Furthermore, so many interview questions are posted on leetcode... and there's so many more flaws to be pointed out, but in spite of that, they still manage to have a high engineering bar.
But if you only read HN you'd think it's a total failure.
1. He noted that this wasn't in an engineering field (which I would presume to include programming :) ).
2. A technical interview is hardly a panacea against bad hiring decisions. It's just the least crappy way to hire people that anyone's found (that I know of).
(About me: I was an early engineer at one of the fastest growing SaaS company on the planet. I had the privilege of contributing to our interview process that took us from 5 to 250 engineers. I'm now trying it from the candidate side: http://InterviewKickstart.com)
tl;dr
You have three choices:
1. Apply with a strong referral and/or
2. Apply to exactly the same kind of jobs/projects that you have had work experience in, and/or
3. Apply to a company that's hiring only one or two people a quarter (slow, careful)
A bit more
Everyone hates the tech interview process. Everyone. Including people on the other side.
Let's understand the process, by taking the most typical case, where a candidate with experience in X type of projects, is applying for a job with Y type of projects.
e.g. If you have worked on say Payment Systems' projects for last 5 years and you are now applying for Deployment Systems' role, then as an interviewer, how do I evaluate you? I don't know enough about Payment Systems to judge your work of 5 years in 45 minutes. And I can't presume you know enough about Deployment Systems that I can ask you questions built on my knowledge of several years.
I hence have the following options:
1. I ask someone about you. That's possible, but not systematic, because I need to find a person that I trust very well. It's much better if that person refers you. Hence your option #1.
2. I can hope that you and I have some project in common in the same domain. That's getting much harder these days, as programming is getting very vast. Hence your second option. Then I can evaluate your work and decide one way or another, quickly.
3. I can look at your work of a different domain and give it a proper evaluation. That's takes time, often hours and days. I can't spend that much time, when my goal is to hire 20 engineers a quarter (and hence interview 5x more). Not to mention that evaluation is also subjective, error-prone and contentious. Everyone has different ideas on design and code-quality. Add that to the fact that I need your work evaluated by everyone in the team, not just me. It just becomes untenable. Hence your third option (apply to a place that's hiring slowly).
Being human, hence, I have to fall back to the path of least resistance. That path is to find the lowest common denominator of our knowledge, which can be evaluated in a short amount of time. Yes, it's not perfect and it's still a bit subjective, but there aren't that many ways of writing quicksort <or insert your favorite question>, you know.
I hence ask you that question, begrudgingly, knowing fully well, that I'm myself going to be subjected to the same treatment when I go out interview. But at least now I know why, and I look for the best I can in the current process.
I think a more apt title is "technical interviews suck", and speaking personally, I have entirely given up on them. (I would say that I gave up on them after two decades, but the truth is that I gave up on them a decade ago -- and I really tried hard in that first decade to develop the perfect technical interview.)
My belief has become that the only way to hire the traits that I'm looking for (high technical ability, sufficient education, predisposition to rigor, and -- most importantly -- indefatigable persistence) is by judging a candidate on their work, not their performance in an interview. (After all, software engineering isn't done via pop quiz -- and it's not even a timed test.)
The problem then becomes: how do you judge the works of someone you've never worked with before? Three ways that have worked for me:
(1) Rolodex. This is an easy way to hire reliably: someone on the team has worked with the person in the past, and vouches for them as one of the tribe. Assuming that you trust the person vouching for them, the interview consists of you selling them, not them selling you. This method has high fidelity (though is still fallible), but also suffers from obvious limitations in terms of scale and breadth.
(2) Known curricula. There are some schools where I know (or someone on the team knows) the computer science curriculum very well, and can assess a student simply by the courses that they have taken (or, better, TA'd), the labs they have worked in, or the professors that they have worked for. The fidelity of this method will depend on the school, how well one knows the curriculum, etc. -- and it has all of the strengths and weaknesses of hiring university grads. (It also suffers from serious scale problems and runs the risk of creating monoculture.)
(3) Open source. If you lead open source projects that attract community contributors, you may find it surprisingly easy to coax the best of those contributors into working for you full-time. While I know that this isn't a fit for every company, it has become my preferred way to hire: you are hiring someone who can obviously do the work (because, um, they're already doing it) and who has demonstrated interested in the problem space (especially if their contributions have come during their free time). Importantly, you are also not limiting yourself to a particular geographic area or school or company history or whatever; the people I have hired via open source are people I never would have found otherwise. And, it must be said, they have proven (without exception, actually) to be great hires. There are obvious problems with this as well in terms of scale and (especially) predictability, but I view open source as the farm system for software engineering: it's a way to find and develop talent at lowest opportunity cost.
Edit: I forgot a fourth method that has actually worked for me.
(4) Homework. When interviewing someone who I don't know and who is otherwise a stranger, I will ask them some exceedingly basic questions to assess technical viability, and then proceed to discuss the problems that we're solving and why I personally find them exciting. If they are sufficiently interested at the end of this conversation (which is really more me selling them than them selling me), I will assign homework -- which is generally to take some fixed amount of time (I have usually said no more than eight total hours) to build something fun in one of the technologies that we have built or otherwise use. (When node.js was new, this was to build something in node.js -- but more recently, it's been to build something fun with Manta.[1]) If a candidate comes back with completed homework, you each know something about the other: they were sufficiently interested in you to actually play around (and they like playing around to begin with -- which is essential for us) and you now have some works by which to judge them.
> A HR professional can identify the candidates who don't come close to meeting the requirements of the job and filter them out.
In my experience this is mostly not true. Most HR people don't know tech so what the HR screen devolves into is cramming your CV with acronyms so you make it pass the HR grep filter.
> You will want to interview a decent number of folks for every position.
If you're bringing in more than five people for onsite interviews then (IMHO) you're doing it wrong. You're wasting your time and theirs and it's clear to me that you don't know what you're looking for.
At least Fred advocates phone/video screens (of 15 minutes) although he suggests 6-12 onsite candidates.
> Many employees don't know how to interview and you should teach them the basics ...
This is true but I would go further: most people shouldn't interview. It's arguably a talent not a skill. At Google, it's viewed as every engineer's responsibility to do interviews and I disagree with this. Many of the stories of bad interview experiences can be attributed--at least in part--to people who just don't have the interest or talent in interviewing.
Anecdote: in one quarter I once took 10 people to lunch as part of their interview slate. In those lunches I just talked about the company, answered their questions, etc but didn't get into any technical discussions.
Of those 10 I predicted 8 wouldn't get offers, 1 would and 1 I was borderline. Actual results: 1 offer, 9 no offers putting me at 9-10 out of 10.
> If you connect to the candidate on LinkedIn, you can quickly figure out who you know that knows them.
What an incestuous world the NY/SF tech scene must be if this is true. Well it might be true if you're a veteran VC. I can't see it being true for anyone else.
Otherwise this is all straightforward good advice although it skips the most important step: how to determine if someone is a good fit and competent but that's a topic we'll probably argue about in perpetuity.
Well, it's just your wishful thinking that it's a such kind of interview, not reality, and any personal attacks on me won't help you to prove your point. I have software engineering management experience in multinational companies and I have hired other managers: there are much more effective ways to find a person with good soft skills than such remote screening with a purely technical checklist. This way it's simply too costly: first, you need really smart recruiter with good soft skills himself, so he will expose the candidate's weaknesses and strengths. Then, there should be very well designed checklist that will allow to derive candidate's mentality from answers on purely technical questions. That's almost impossible, I'd say.
I agree with all you said. However, 99% of major tech companies employ exactly the same process of hiring software engineers. I've been on hundreds of interview loops and unfortunately just one bad interview in a loop of 5-6 could make the candidate look like a "bad" fit and thus receive no offer. That particular candidate might be in fact great fit, but the way he was tested could not reveal that.
I don't know the solution to this problem. Eliminating tests whatsoever and just talking with the candidate about his experience and probing his knowledge on different topics is not efficient either. There are a lot of talking heads around who when given a simple task fail miserably.
We as a tech community need to come up with better ways to assess other people's competency while also making sure those people fit within our company's culture, work efficiently with others and after all create value. This is a hard problem. So, we try to simplify the problem by imposing the "proven" way of finding such people -- give them arbitrary tests and hope they pass them.
For starters, I find it odd that you claim that the recruiter gave you feedback, and on the technical skills of all things. If you reached the last round of interviews of a FAANG, that round does not focus on technical aspects and it's more focused on soft skills and personality traits that emerge during the interview. In fact, I know for a fact that they don't look at the solution at all, and look at the process that the candidate follows such as how he takes suggestions and how he attacks problems or how he adapts to changing requirements. Sure, there are technical questions, but they barely fill 30% of the interview time, and even then their main goal is to gather info on the candidate's personality traits.
With that in mind, it's easy to believe you bombed your interview if you are the type of person who opens a discussion on a public forum attacking your interviewer with accusations of being too incompetent to evaluate you and too immature and having no clue about how great you are and how awesome you did. Just from this post alone I was readily convinced I would hate to work with someone like you, and apparently so did your interviewers.
The technical interview is overwhelmingly the primary screening method of choice at the world's most competent software development companies. The team that builds the match engine for the electronic exchange, the team that builds the boot ROM for your smart phone, the team that builds the Google crawler and indexing, the teams building kernel SAN storage code --- these teams are hired by technical interviews.
The "audition project" is a trend story with, from what I can see, very little empirical evidence to back it up. When the most notable example of a company routinely employing audition is Treehouse --- all due respect to Treehouse, but still --- that's a red flag.
The bigger red flag is that the paid audition project method has obvious flaws. The top 20% of software developers are almost uniformly employed. Prospective new employers for these people court them actively; in fact, the problem of companies luring top developers out of other companies is so challenging that large SFBA tech companies have entered into illegal compacts to stifle the practice. Companies are so hard-up for talent that they'll "buy" startups just to get access to their teams.
In this environment, why would a top developer, who has their choice among tens of different high-paying interesting jobs, moonlight for a prospective new employer just so they can make sure the relationship's going to work?
Most technical interviews are terribly flawed. They aren't standardized and they aren't rigorous so you can't compare candidates on any apples-apples basis and you can't correlate them to job performance to make them more predictive. Most of the developers tasked with conducting them suck at interviewing; many use interviews as a sort of hazing ritual, and most use them as opportunity to project their own subjective views about how software should be built onto candidates. Many technical interviews are trivia games punctuated by awkward attempts at working through code on a whiteboard or a piece of paper in a high-pressure environment.
The solution to this problem is to improve technical interviews. It's not to pretend that the whole market for devs has suddenly embraced temp- to- perm hiring. It hasn't. It's getting harder to find good developers, not easier, and the notion that companies have the luxury of inflicting "audition projects" on candidates is counter to reality.
In the last 10 years this is how I got my last 3 jobs: single stage 30-45 minute technical conversation with a lead engineer. I had other multi-stage interviews that went nowhere. Those were not FAANG companies obviously, but FAANG are overrated anyway. I'd rather be a big fish in a small pond.
Bottom line is - if a company actually wants to hire, they don't waste time. I've met engineers from other companies at meetups who told me they have rejected every single candidate because they didn't want to hire, management was pushing them to. Others said they have deliberately chosen "Elite Language X" to weed out "bad developers" and so on.
The tech interview system employed by FAANG is an example of a weak correlation style interview. Actually the structure of the interview is mostly fine, but the criteria being judged is usually off.
1) Questions should be structured such that they're modeled in a real world context, and somewhat close to the nature of the type of problems your company/division solves.
2) For most companies, coding ability and quality is more important than CS theory strength. Success on a coding problem for these companies should be judged by pace of coding and quality of solution, rather than time complexity of the result. Run time complexity of an algorithm is almost 100% orthogonal to ability to write high quality code quickly, yet this is where 99% of the interview focus is for most companies.
That being said, if you're hiring somebody to design a database storage system, sure, theory is more important in that context. But 99% of jobs are not that.
Can't tell you how many people I've seen join FAANG that I've worked with who are actually quite poor performers in a real world context. It's very easy to grind leetcode and game the system as its structured.
Its true too that at scale the correlation is probably good enough to end up with a decent workforce though. But also very easy to tweak judging criteria to be more highly correlated to real world success. I've hired close to 100 engineers, and its immediately obvious to me who will perform well by how they carry themselves in the interview. I pretty much don't even take into consideration whether they reach an optimal runtime solution. One of the best guys I hired couldn't even implement a tree traversal/DFS/BFS in the interview
"They eat large sprawling problems for breakfast, but they balk at 45-min algorithm challenges."
Care to guess what we're looking for in screens and interviews?
Some of our best performers were poor interviewers. Likewise, we've had interview aces not pan out. Rather than expecting the world to become interview clones (and make our hiring decisions even more difficult), we're learning how to be better interpreters of people to get to the answer of our real question -- is this person an engineer we'd like to have on our team?
It's still a work-in-progress and we're nowhere near perfect, but we're simply not going to outsource our decision-making to the status quo of technical interviewing in 2016.
The software industry's greatest blog post sin: "Hiring in tech is broken"
This topic is trite. Furthermore no one seems to be able to offer up actual tangible suggestions as to how to fix this, or some objective 'better way'.
Disclaimer I work at a FAANG etc company:
Also this has been distinctly "not" my experience at many FAANG type companies. Usually these interviews, even the one's I haven't done well in, have been highly conversational, and all contain a fairly in depth 'soft skills' interview as well.
Yes, your technical knowledge has to be good, but a good interview will keep pressing at the edge of your knowledge (not to make you 'fail'), but to see how you handle the unknown.
In fact for one of the companies I ended up working at I was called in for a second soft skills interview just to dig even deeper on the dimension the author claims is 'never tested for'.
Much like these authors I don't have a perfect solution to this, however I will say one of the most rewarding interviews I've ever done (at a well funded startup where I did NOT get the offer was comprised of):
- 1 Algo question (standard fare)
- 1 Architecture question (standard fare)
- 1 'find a bug in this mock part of the codebase / pair program with me' question. Implementation didn't necessarily have to be perfect, mostly was interested in diagnosing the actual problem and talking about 'how' we might solve it (very fun)
- 1 Product Sense (!) question with actual PMs (very fun)
You may not have a better way, but one has existed for decades, maybe longer. Combine a personal interview with a company that's worth working for and you'll have no problem finding qualified candidates.
This means one phone screen of about 30-60 minutes to both introduce the candidate and decide if they are worth a follow-up with the team followed by a one hour interview with the whole engineering team more focused on technical capabilities directly related to the job. Finally, just to be sure, a one hour take-home assignment that is simple just to make sure they are not bullshitting. This last one only goes to the top few candidates of which one will be chosen (assuming they can code the simple assignment). People have been hiring this way (ok minus the coding assignment) for decades, maybe even a century or more. And you know what? It works.
Now on to the second requirement: having a company that's worth working for. That means a company that at least partially values its employees. The best way to do this if you don't have FAANG money? Remote work. Ideally for everyone. That guarantees your pool of hiring (even if limited to certain locations for artificial reasons) is much higher and the demand for the job will be huge. Other things that make a company worth working for: good salary, yearly raises, performance bonuses, 40 or less working hours a week, flexible schedule, decent managers who understand engineering, interesting tech, etc. Note that none of those is out of reach of any company, even the smallest small-business.
So yeah, let's stop with this bullshit that we don't know or can't think of a better way to hire. That's idiotic and frankly, closed-minded. Let's stop pretending like there aren't a ton of qualified candidates just waiting to be hired when there are so many. Let's give people a chance for once to prove that they can do the job. Not everyone will work out, but you know what, in the US with at-will employment, that is not a big deal. Doesn't work out, fire them. Easy fucking peasy. If you can't do that, you shouldn't be leading a team/company.
My company currently works this way and we have had NO problem hiring. Whenever we want a great candidate, we go from resumes to hired in less than a month. There is nothing special about us. We don't pay top dollar. We don't even have a lot of the perks I listed above, but we have enough of them that candidates love us and we have no problem attracting great talent (the top one being remote work, of course). If a company isn't willing to make itself desirable, I have no pity for them. It's like never showering or grooming and expecting to find a hot girlfriend. Highly unlikely.
> They also assume collaborative interviewer, which has never happened in my experience.
As a FAANG interviewer myself, this is very sad to me. System design interviews are inherently collaborative and they can provide a pretty good signal on "do I want to work with this person". I thoroughly enjoy conducting system design interviews -- It's always a joy to talk through the problem with the candidate and sometimes I even get to learn something new myself!
I used to be a recruiter. Roughly 1 in every 25 developer I worked with told me they wouldn't do silly arbitrary technical interviews and that if the company wanted them to they would decline the interview.
I respected that decision but it none of those engineers every got hired by companies I was working with. They all had jobs already so clearly that was fine with them but it seems like there needs to be a better alternative.
One developer would ask what the role entailed, then find a relevant job/project that showed he had done similar work. That was helpful but obviously only works if you have a longer career with many projects to pull from.
> I cannot help but think that these big tech companies (FAANG, et. al) are missing out on diversifying and increasing their engineering expertise by passing over developers like you.
I think this is certainly true.
> I often think what would Google/Facebook would be like if they hired in some experienced engineers that may not be able to whiteboard a BFS tree or can tell you Djikstra's algorithm, but have proven business track records of getting projects done, on budget, and on time.
Well... how do we find these people? By looking at their resumes where they claim this? By contacting references who will attest to it? By trusting the intuition of subjective evaluators of the candidates?
Practically speaking, FAANG companies do hire such individuals, they just do it through acqui-hires. If a person works at a company that is good enough to be worth acquiring, then we have a good signal that they are effective employees even absent a direct evaluation of their technical abilities.
reply