If you wanted ideal data, you'd want an employer to hire large numbers of people for a single role, hire them as randomly as possible, and be willing to train them. You could then see what sort of candidate was successful there.
There are a handful of militaries that effectively do that, but it's pretty much impossible for a normal employer.
Great point. The word "science" appears in the name of the field that I studied, yet I haven't seen any evidence that our current hiring processes in this field are better than random (i.e., the null hypothesis).
If I made a software company who openly declared that their hiring process was purely random, would it work as a company? Would it be legal? For that matter, is a standard tech interview legally valid[1]? What if there is no known valid method?
Classifying humans into hire or no-hire is literally the point of every recruiting process. I’ve heard that random selection is as good as various interviewing techniques, but I have to admit I’m skeptical. Though I do agree that doing better than drawing lots should definitely be table stakes for any hiring or promotion process.
I'd bet dollars to doughnuts the people doing this are collecting zero data to analyze to see if this new process improves or hurts the number of good candidates they actually hire.
The irony of it is putting processes like this is actually turning away good talent, but they wouldn't know, because they aren't doing any analysis; they just read an article somewhere.
There are enough lemons on the job market that this barely even bears examination. Anyone who's ever looked at a pile of incoming resumes can see at a glance that selecting at random from them is going to get a viable employee with vanishingly small probability. Anyone's who's tried to interview from even a selected pool of those can assume that the actual pool of candidates is overall even worse than it looks on paper.
Actual hiring results look a lot in abstract like dartboarding after trying to filter out the obviously unfit applicants, and while people can reasonably disagree on how hard to filter, not at all seems like a good way to put yourself out of business.
That's true of any hiring system though: how do you gather data on the job performance of the people who don't pass a resume screen? A coffee date? A phone screen? Missing counterfactuals everywhere.
To run a full analysis, you need to hire people randomly - both people who fail and pass your hiring process - and assess their subsequent performance. This never happens in real life.
However, you can still run an informative statistical analysis based on the variability in interview scores and performance scores. For example, the people who scored 5/5 on the interview should perform better than the ones who scored 4/5.
Randomness is a factor whenever you have social interaction; as it turns out, people are subjective to the core. Trying to gauge someone's ability to deliver results is educated guesswork at best. You can only be so objective about it; there's going to be some measure of variance.
Google errs on the side of rejection because one bad hire has a much bigger impact than one good hire.
Not even Naive Bayes would be needed if recruiters would use a "more suited technique" to predict exactly who is going to be interested in a job and only bother emailing those.
Isn't it? While machine intervention might be able to weed out the most obvious low-quality candidates, I don't think there's any strong evidence that most hiring processes actually select the strongest hires - just that they, out of the set of candidates that progress to the point of human intervention (i.e. interview), select a decent one.
The problem is that interviews don't scale well, and some kind of automatic culling of the field of candidates is necessary. Engineers, managers, etc all want to feel that whatever machine solution (keyword searching in resumes, applying AI to recorded video statements, HackerRanks) they select is better than random - but there's no incentive for them to check that that's true.
Obviously randomness + interviews is better than pure randomness, but ultimately hiring processes are still pretty random. If you need to weed out most of your candidates, you can't do much better than throwing most of their applications away - and companies end up doing just that, only they cargo cult an "automated process" that claims to do better.
> You're assuming that it was based on whether or not the candidate was hire. Nobody with an iota of experience in machine learning would do something like that. (For obvious reasons: you can't tell from your data whether people you did not hire were truly bad.)
It's a fine strategy if all you're trying to do is cost-cut and replace the people that currently make these decisions (without changing the decisions).
I agree that most people with ML experience would want to do better, and could think of ways to do so with the right data, but if all the data that's available is "resume + hire/no-hire", then this might be the best they could do (or at least the limit of their assignment).
And yet unstructured interviewing basically works and the world keeps spinning. If 'random' truly was a better result (as the article suggests), then in a hiring round for a programmer last year, we might have discarded our candidate that won and is awesome for the one that couldn't conceptually 'get' FizzBuzz.
Or put another way: if 'random' was better than 'unstructured', you'd never have a round of hiring where there were no unsuitable candidates - one will have been chosen.
I would say that a random selection process would be worse than most hiring funnels that I've seen. This is doubly true considering the abject terrible quality of the lowest decile of candidate in software engineering. (The "spray and pray" resume machine-gunners would be way over-selected by any hiring process that relies on randomness when the rest of the industry is trying to select by interviewing.)
You have everyone's interview scores, performance feedback and promotion histories in a database at a company with tens of thousands of employees. You also have the interview scores for everyone who failed the interview process. Put a statistician on that for a day and you will get a lot of significant data about your hiring pipeline.
It is not hard to do, the data just isn't public and such data will never become public. Therefore public researchers will always lag behind private ones, since the private ones have access to the interesting data.
Edit: Also it is not a theory, I have seen internal studies on this myself.
So, that's a sample size of 1. Not quite what we would call scientifically rigorous ;). I'm not saying it's not a useful question. I'm just saying, you jump to a whole lot of conclusions that are clearly not based on evidence in your hiring process. That's fine, just be aware that your hiring process is not doing what you're thinking or asserting it's doing.
Sure but you’re bound to say that ... Find me a hiring manager than publicly says anything else. That’s kind of part of the point.
I roughly recall google touting ~50% hiring success rate and that’s with a 4 interviewer, 10-15 man interview process, and undoubtedly a massive false negative rejection rate.
I think some undesirable attributes are observable in fairly short screeners ; sure screen away . But I bet you that you’d do at least as well if not better by just selecting at random past that point.
These were their conclusions:
1. The ability to hire well is random. This is referring to individuals, not the system as a whole.
2. Forget brain-teasers. Focus on behavioral questions in interviews, rather than hypotheticals
3. Consistency matters for leaders
4. Grades don’t predict anything about who is going to be a successful employee. School grades, that is.
So, stop making stuff up from behind your throwaway account.
reply