The real truth is: hiring is a crapshoot. For any position and any field. Software engineering is no different.
There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire. If there were, everyone would be using it.
Instead, third-party HR monoliths have moved in and snatched up (and more or less created) the 'market' for screening and filtering applicants. Larger companies sigh, throw up their hands, and say 'well how else can we deal with hundreds of applicants?'.
As if that 'funnel' of applicants contains what you think you want. As if it's just an exercise in reductionism, each applicant a data point to evaluate.
Given this, why not hire lightly and fire lightly?
Right. It logically follows that random applying would be a good strategy: selecting randomly from top-paying jobs, you would reply to job offers for e.g. software engineer, brain surgeon, architect or hedge fund manager, to increase the chances of being randomly hired.
That is effectively how I got my last 2 jobs in software engineering. Picking a few keywords and spamming out your resume seems the only way to match the equivalent efforts from recruiters.
>That is effectively how I got my last 2 jobs in software engineering
Are you saying that you had no qualifications at all before? Because that's what they're saying, just applying for any high-paying job, even if you don't have the requirements.
I mean my anecdata to serve as a realistic example of the hyperbolic concept of just wildly applying anywhere at random.
In both cases, I was switching industries (from ASIC design to IoT cloud software to data analytics). So, I arguably had some qualifications and arguably none at all.
>It's not totally obvious to me that random hiring would be worse
Does this have some implied conditions I'm missing? Because to me it seems absolutely obvious that filtering for people who have experience/a degree will vastly improve your outcome over true random hiring.
A lot of things seem obvious but aren't actually true. Plenty of people with degree and experience coasted on nothing, or have personalities so abrasive as to make working with them all but impossible. Disappointingly I have poor experience with degrees actually instilling any sense of diligence, or even research capabilities, thus making them questionable even when recruiting for fields such as R&D, where it should be truly an obvious choice. At least a retrained nurse has some basic curiosity and a critical eye, though it's a bit of a shame, what with the nursing staff shortage.
And, also personally, I've had great experience with people who migrated from absolutely unrelated fields (e.g. several people who retrained - within same company - from support into software development). Similarly I had great experience with fresh interns turning out to be brilliant theory people.
I propose the Monte Carlo hiring process: heads means the applicant is hired without question, otherwise they get to go through the regular interview routine.
I like it. Exploration vs exploitation is used in a lot of optimization algorithms (and in reinforcement learning) meaning that there's a low probability that the agent just decides to make a random choice instead of making a greedy choice.
Why do you think that? Isn't it rather sexist/racist to argue this?
The candidate stream is itself not "diverse" (assuming you are using the normal meaning of the word of lots of women+racial minorities). Simply randomly making offers wouldn't alter the outcomes because those people aren't being de-selected by skills testing to begin with - unless you believe the 'diverse' candidates are less skilled programmers.
Isn't it? While machine intervention might be able to weed out the most obvious low-quality candidates, I don't think there's any strong evidence that most hiring processes actually select the strongest hires - just that they, out of the set of candidates that progress to the point of human intervention (i.e. interview), select a decent one.
The problem is that interviews don't scale well, and some kind of automatic culling of the field of candidates is necessary. Engineers, managers, etc all want to feel that whatever machine solution (keyword searching in resumes, applying AI to recorded video statements, HackerRanks) they select is better than random - but there's no incentive for them to check that that's true.
Obviously randomness + interviews is better than pure randomness, but ultimately hiring processes are still pretty random. If you need to weed out most of your candidates, you can't do much better than throwing most of their applications away - and companies end up doing just that, only they cargo cult an "automated process" that claims to do better.
I would say that a random selection process would be worse than most hiring funnels that I've seen. This is doubly true considering the abject terrible quality of the lowest decile of candidate in software engineering. (The "spray and pray" resume machine-gunners would be way over-selected by any hiring process that relies on randomness when the rest of the industry is trying to select by interviewing.)
Has random hiring often been tried? I've never heard of anyone doing it. Not sure random hiring would be optimal, but it would be interesting if there were some data, doubtful it exists, though.
It has the same amount of evidence as all these "hiring is broken [for the most profitable and transformative industry in all of human history]" posts.
If hiring in tech is broken, then all of hiring is broken, whether it's fast food cashiers or aerospace engineers. Can you seriously claim that any of those have better hiring practices? From personal experience, I can say they are far worse.
That's not the ideal response if you actually wanted to defend your claim about the internal data and special competitive advantage rather than just abandoning the claim.
Seriously, I would say that the competitive advantage of a company comes primarily from its product and the vision and competence of the company leadership. You can't just hire your way to success without that foundation. Bad leadership can destroy the greatest employees.
The company founders almost always come together in a very informal manner, and often the early employees do too. And that's the basis of the company. It has nothing whatsoever to do with some special snowflake data-driven formal hiring process.
Indeed, I would argue that once a company becomes very large and successful, they become bloated and bad at pretty much everything, including hiring. At that point they have to get better by acquiring other companies, not by hiring through their standard bureaucratic processes. The competitive advantage of giant corporations is the ability to swallow other companies whole.
Google basically admitted that GPA and where you went to school don't matter and do not correlate with job performance. And yet here we are still optimizing on the Ivy League. It'd be a safe wager that algorithm whiteboard interviews don't either except that they screen out folks who don't pass the interview. So because everyone has that baseline, that baseline is by definition meaningless. The other side of the coin is that performance management is also just as bad.
Work results are only sensible measure for the work performance. Easy hire and fire is one solution. Also a short trial period at the start of the work agreement should show if results happen or not (fg. 4-6 months).
I don't know if it was ever made public, I think I heard this story via backchannels. But Google did a research project at some point to discover why GPA/interview performance in general did not seem to correlate with post-hiring job performance. The project was never allowed to reach full completion because the direction their data and investigation was going looked like this:
"Our hiring process does not correlate with post-hiring job performance because we have a large and measurable bias in interviews in favour of Ivy League candidates and women."
In other words, GPA didn't correlate with job performance for Google because it was a confounding variable: they were selecting the sample for top tier universities (which select on GPA) at hiring time, but not when it came to promo reviews (which had a different process and anyway, less rhetorical ideology involved than hiring, at least at that time).
Because hiring has costs, this requires substantial risk tolerance, and is more suitable to a bigger corporation that can bear them. It will also give you a reputation — though some companies seem to manage quite well even with an iffy reputation in this area (e.g. Netflix). You need a certain amount of prestige to pull that off, I think.
Construction and trades takes that approach. It’s a mixed bag. I would say implicit job insecurity removes a lot of potentially good employees from the applicant pool. People who value stability and support families tend to be reliable, and some of them won’t work for someone who has a reputation for firing everyone as soon as things get slow or if they have an off day.
I think my industry (civil engineers) is a bit easier to hire for because credentialing weeds out people who are completely out of their depth, and lying about qualifications can get you in real trouble. Managers can assume you’re basically competent and can focus on hiring people with decent personality and a good reputation. Some of my software friends use brain-teasers to screen candidates, and it comes off as a bit silly to me.
Funny you should mention that. At my college (Cal Poly) I had fraternity brothers who were Civil Engineering majors. They would look up to me because I had a natural ease with computers. But in the end, I watched them sweat their way through classes that would have wasted me.
Many engineering fields are ones were being studious and knowledgeable can give you a solid career. Software development has a lot more upside potential for the flexibly creative and the lucky.
Which is another way of saying that the "advanced" subjects (vector calculus, multivariable calculus, fourier transforms) really aren't that bad if you're prepared to put the work in.
I think my industry (civil engineers) is a bit easier to hire for because credentialing weeds out people who are completely out of their depth, and lying about qualifications can get you in real trouble. Managers can assume you’re basically competent and can focus on hiring people with decent personality and a good reputation. Some of my software friends use brain-teasers to screen candidates, and it comes off as a bit silly to me.
Pretty much the case for every non-Software Engineering discipline.
But then again, people in the tech field will vehemently defend the system - because it's one of the very, very few fields where you don't need a college degree, can go on and "grind leetcode" for months, and land a six-figure job.
And it is because of that, that tech companies are making their recruitment process bordering the absurd. They're simply deadly afraid to vet frauds, that have "gamed" the system.
So in short - tech companies would rather let 10 decent candidates slip by, if it means they can block 1 fraud.
I've been to many engineering interviews, outside tech, and it is not even remotely the same process.
That fear may be true for candidates without experience and/or university degree. But why is it still present for candidates with lots of experience and education? In other words, if other technical/engineering disciplines can manage without this absurd hiring process, then why can't IT field (leaving aside the candidates without experience/higher education)?
Because every engineer has some outlier horror story in the lines of "We hired this dev/engineer with legit CV, but he turned out to be useless. Couldn't code a line of code"
Universities are terrible at teaching programming/software engineering. Many also suck at teaching computer science. There's no incentives in academia to be good at teaching, and many lecturers only care about their research.
There are arguments here and on the blog post that the industry doesn't have credentials. We do have credentials, it's the CS degree, but universities suck at issuing them so we don't rely on them much. Sometimes not at all.
The people saying, hey, why can't we be like civil engineering are thus perhaps not sounding as re-assuring as they hope: why would universities be more trustworthy at teaching civil engineering than other kinds of engineering? The problems are incentive related. Ultimately the software industry is highly egalitarian and a bad university experience can't hurt your career permanently, like it could in most other fields.
When I say “engineer” in disciplines other than software, I mean somebody with a professional license. You can only get that by passing exams and having a combination of experience, education and recommendations to pass a state board. It’s much harder and riskier to fake that than to make up experience on a resume or make a blog to appear like a subject matter expert. Unless you know the candidate personally, it can take a lot of effort to uncover a carefully crafted lie. Plus, recognizing quality work in software isn’t even a solved problem, to my understanding.
Other engineering disciplines hire engineers who don’t have a PE all the time. There are more engineering disciplines where taking the PE exam is the minority than the other way around.
Of course they hire non-PEs. You can’t get a license without years (at least 4) of experience first. Most consultancies I know of would not hire anyone who couldn’t pass the FE, though.
> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire. If there were, everyone would be using it.
I can say with confidence that certain resume features are incredibly strong predictors of interview performance (at least in the related measures, in the field and/or team I'm working in). I'm not HR, but your statement seems to also deny the possibility of this empirical observation.
> Given this, why not hire lightly and fire lightly?
Because it would be a complete dick move to have someone relocate (possibly involving change of country) and then fire them 2 weeks later? That's not even looking at the formal, bureaucratic or training overhead, or any of the other factors in this industry that make hiring and firing a bit more complicated than handing someone (and later taking away) a hardhat and a shovel.
Disclaimer: This is for a quite particular set of positions, and it's not in the US (so I have no reference on US GPAs). The EU supply/demand situation is also different from the US, so your FAANG mileage may vary.
Here are a few (at a level of detail that I'm comfortable sharing):
- Outstanding (in either direction) performance in certain subjects in school is a very strong predictor.
- Cover letters are mostly noise, but incoherent, overbearing or careless ones don't go very far. Lack of curiosity and interest is also a negative predictor (but these are of course easily feigned).
Let me be clear though that we consider applications on more than these factors. These are just predictors, and sometimes they are wrong. I have interviewed actuaries that couldn't calculate dice roll probabilities, C++ programmers that were confused by the class keyword and C# programmers that hadn't heard of virtual.
You are correct to observe that what I listed is primarily relevant to people who have less experience. While prevalent, that's not exclusively the demographic of our applications, and of course the weight of these factors is reduced for someone who has been in the workforce for longer. But they remain good predictors nonetheless.
No offense, but that is a very silly thing to consider "foundational" in 2020. The heavy, all encompassing use of inheritance patterns these days is much less prevalent than 10 years ago, even in the C# community.
Extremely silly, I got a C# job recently after I went about 7 years without touching C#. I forgot virtual existed until my first day when I had to look it up. I would have failed the "what's virtual?" question, yet I am very good at my job (just got a big raise).
Certifications are a strong negative predictor. Someone who turns up with a lot of corporate certifications is unlikely to do well when asked difficult questions.
People who have only ever worked in low grade or low tier banks tend to do poorly. There are a few exceptions. GS has good technologists. Most big investment banks have a few programmers (older ones usually) who have solid experience - but who are unlikely to put any creative effort in and may have a "here's why it can't be done" attitude. May not matter for some roles.
People who list a very large number of short roles tend to do poorly.
People who give a list of projects in their resume experience for which they only assert they were part of the group that delivered it, without concrete claims of tech leadership, often do poorly - generally it indicates a "hiding amongst the crowd" problem in which someone is a poor performer and attempts to hide lack of any actual achievement by reference to group projects.
People who list obscure programming languages as a skill do relatively well, people who list obscure products as a skill do relatively poorly.
In light of this particular team having been very successful at recruiting (i.e. extending offers based on interviews) and retaining people that succeed at the challenges they face for the better part of a decade: Yes.
But again, I'm not claiming this to be universal. Just that it's possible.
> it would be a complete dick move to have someone relocate (possibly involving change of country) and then fire them 2 weeks later
This. I hold a slightly lower bar (meaning more willing to take a chance) on someone who is unemployed than if they're working now. Worst case if I take someone unemployed, hire them "light", and they don't work out is they're back where they already were, whereas if someone quits their job to come work here and gets fired, they're worse off than they were before.
I’m not sure I understand. Someone is unemployed, applies, interviews just a whisker under our normal bar, and if I offer them a job, I’m the bad guy in this story?
This isn't true, depending on your definition of large. For example, Stripe is known to have an engineering interviewing process that diverges from the typical leetcode-style questions.
I think hiring is "doing just fine" and there's room for order-of-mag improvements on precision/recall.
Just because it's widely used doesn't mean it's good.
Source: they let me interview people at one of those fancy FAANGMO places (and I don't actually know what I'm doing). Interview success comes down to how lucky you get in the loop imo.
> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire
This is a manifestly absurd claim. It is possible to improve the outcome of a hire, and almost everyone is using those kinds of processes. The firm I work for does an excellent job filtering people for skill and compatibility. The trick is that if you want to capture the top e.g. 10% of applicants, you're going to have to pay them several times more than the median applicant. Most companies are too cheap to attract distinguished talent.
> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire.
I don't have stats but anecdotally, hiring by personal reference seems by far the most effective way to hire.
> Given this, why not hire lightly and fire lightly?
Because it's a huge financial and organisational burden on the hirer and an even huger (sometimes existential if you live in the U.S. due to health insurance but that's a whole nother rant) burden on the hiree.
It shouldn't be. From my experience, the financial cost that comes from increasing the number of people you fire after a few weeks is less than what companies spend furiously guarding against making a bad hire.
To add to this, as a senior employee, bringing someone junior up to speed to the point where they can make actual contributions is a big consumer of time typically. Less so for employees who come in at medium rank but even then there is a hand holding period as the new person gets used to the company culture.
In a sufficiently large company you could have a specialized onboarding team, but you’d need to be careful that the specialized team doesn’t get isolated from the technical culture and needs of the rest of the company
> There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire.
Daniel Kahneman analyzed a bunch of data that lead him to concluded that the typical interview process did nothing to help select the best candidate. There's a chapter about it in Thinking Fast And Slow [1] and the advice he gives is summarized in this article [2]. I remember thinking after reading this book that it was just a matter of time until everyone everywhere would be denouncing interviews but here we are - old habits die hard.
What are you talking about? The article is nearly the exact opposite of what you claim. It says:
> A vast amount of research offers a promise: you are much more likely to find the best candidate if you use this procedure than if you do what people normally do in such situations, which is to go into the interview unprepared and to make choices by an overall intuitive judgment such as "I looked into his eyes and liked what I saw."
the typical interview is a series of unstructured intuitive judgement calls - Kahneman suggests something more like a survey with a strong emphasis on its calibration and isolating/controlling biases. Thinking Fast And Slow goes more in depth on how terrible typical interviews are - the article merely summarizes the advice he provided for what to do instead.
I think a lot of us suspect that just choosing applicants randomly would probably have a very similar outcome to what we do now, but nobody wants to be the first to try it.
The above statement is really just validation that the current process sucks so much for everyone that it's conceivable to replace it with random choice. However, it's not actually true (for reasons explained in the blog) - as a hiring manager, I am confident you couldn't ship "Hello world" by hiring randomly.
id also say it works out okay. its just long and tedious and it feels more painful than it should - but really its not all that bad in general.
I see very toxic people being rejected that think its because of their gender/color/whatever <= lots of complains but the process work
I get rejected sometimes because of misalignement from me, or from them <= feels bad man.. but the process works
I've never been hired to a place where I found that this was all a terrible idea, in 25 years. Theres been places better than others, theres been adjustments after hiring.
I've seen places hire people using simplified processes and the candidates were unsuccessful and unhappy after being hired <= process fail
> Given this, why not hire lightly and fire lightly?
Because any company that adopts such an approach will never be able to recruit anybody who wants to work in a stable environment with a consistent set of smart people who aren't in constant fear that they will be fired the next day.
Yeah, exactly... depends a lot on who is doing the firing and why too.
I suppose if ALL companies did that then there might be some sort of fluid equilibrium state where most companies had some mix of mostly competent engineers who chose to work there for positive individual choice/reasons. But in a more winner-take-all world where the "best" need to hire the "best", because that means second tier companies get the rejects and therefore will fail to compete and the chosen few get success and riches? Set the bar incredibly high and forget fairness. (also an extreme for purpose of conversation, just on the other end of the spectrum)
When you know you have a job for the next n years at least, you also tend to put more effort into efficient processes. When you have to wonder every other month if you will still be there a lot will be just quick and dirty ad hoc solutions.
It is still not easy to find good people, but the truth is: HR are not the best people to decide who is. If you have good people already and they don't try to build an empire, they will select good people. The danger there is only that they will select people that are too much like themselves or they will select the worse candidate because they fear about their own standing etc.
You should fire lightly early, and fire hard if late.
Where I live you have a month when starting were you can be fired with 2 weeks let-go period, after which it takes 3 months or more.
I think you should have an intensive monitoring/evaluation in the first month, and fire lightly if you think there is a mismatch. After you have determined an employee fits you, if they stop fitting you try to figure out why, help them to fit again (are they the kind who gets bored, needs new challenges?, do they have problems at home? If you found someone who fits I think it is more efficient to make them fit again then try to find someone else who fits). I believe this is the way to go even in areas where you do not have the rules about the dismissal period that you have here.
More accurately: such an environment requires paying a premium to compensate for the decreased job security. (As with any market, employees expect to be compensated for their higher risk with higher returns.)
Netflix, for example, and Amazon (to some extent) have gotten away with this model but the vast majority of companies are simply unwilling to pay top of market.
Additionally, in many markets there's no such thing as 'firing lightly'. Here in Scandinavia firing people requires a significant amount of work on the part of the manager and HR.
In Germany new employees usually have a probation period of 3 months, during which companies can pretty much fire them at will if they don't perform as expected.
I think that's almost always enough time to assess whether someone is a good fit for long term employment.
>The real truth is: hiring is a crapshoot. For any position and any field. Software engineering is no different
I tried building spaceship and failed. Fiend of mine tried building a spaceship and failed too. However, I know of people who built a spaceship. So is building a spaceship a crapshot?
Reality is: 95% of HRs are incompetent. That's where randomness comes from. Take a random man from street and assign him for HR role after a brief training -- you'll achieve a similar performance to those 95% HRs.
Everybody's talking about hiring the engineer, but so fewer people talk about finding a good HR that would help you to solve a "simple" problem: how to find a good engineer with small experience and small salary that would not leave your company for the next few years and would become after several months just as good as 70-100$/hour engineer hired from top software company, while later most likely still gonna need some time to adapt to your product to become efficient.
There is no meaningful data that any hiring process--good or bad--improves the outcome of a hire. If there were, everyone would be using it.
Instead, third-party HR monoliths have moved in and snatched up (and more or less created) the 'market' for screening and filtering applicants. Larger companies sigh, throw up their hands, and say 'well how else can we deal with hundreds of applicants?'.
As if that 'funnel' of applicants contains what you think you want. As if it's just an exercise in reductionism, each applicant a data point to evaluate.
Given this, why not hire lightly and fire lightly?
reply