One possible explanation for this is that talented though start-up engineers may be, they are in many cases over-worked and under pressure to put out revenue-generating product.
As basic and important as salting + hashing passwords is, it is the sort of task that ends up on the technical debt list or scrum back-log, never to be looked at until it becomes a problem.
I'd have to disagree with the passwords thing. This is a design decision from the beginning, not a "I'll get around to it" type of thing. It's just about the same amount of work to implement salted and hashed passwords as it is not to. And it's much easier to just do it right the first time than trying to go back and fix it later. I can't imagine how a skilled web developer would miss something that glaringly obvious.
An experienced web developer, no. A skilled-but-never-built-anything-but-a-toy developer? I'm not so sure. Best practice is rarely sexy. Agile development methodologies over-emphasise YAGNI and do-the-simplest-thing (to the point that I want to scream -THAT-WORKS every time I hear that phrase being parroted by junior developers).
Sure, once they've been on the wrong end of a high-profile identifying information leak, they'll never forget to do it right. But the best of the best? They're building air traffic control systems, or embedded missile guidance, or airbag controllers, or F1 ECUs, or million-simultaneous-call telecoms payment processors, or any one of a thousand other systems where you don't get a second chance to get it right.
Total newbies, agreed. I'm thinking a progression that goes high school [aptitude] -> college / work exp. [skills] -> employments [experience] - ie. I'm hoping that many developers at least go to college, or built things by themselves, or had some kind of work experience before they get their first "real" job, on the back of any natural aptitude.
1.- How do those companies get the best of the best?
2.- Are you sure it's talent and not process? Where you can't fail, you usually have even the smallest change specified in thousands of pages and reviewed by many people.
1. In the UK, via the "milk round" of good universities, personal referrals, industrial sponsorship... Also, some universities won't take undergraduates straight from school, which forces a "year out" in industry which plays into recruitment. It was kind of fun watching the big players fighting over those new grads with 1st class degrees.
2. I think, at it's best, the process is the safety net. At it's worse, it's a substitute for talented people - but I don't think that's sustainable for long. Perhaps it's just a different talent, for working in an extremely tightly controlled way without going crazy.
It would be unfortunate if the best technical talent went to "web 2.0 companies". I'd hope the best are working on our operating systems, our programming languages and their APIs and virtual machines, our database servers, and other technology that "web 2.0 companies" and untold others rely on.
+1, you are right. That's the right perspective on the issue. But there's some hype around these companies that is said to attract more good developers than normally go to companies of that sort.
Do Google and Amazon count as web 2.0? Maybe they're 1.0. Anyway, what I think of as the three biggest "internet companies", Google, Facebook, and Amazon, have all decided it's worth their while to get into this kind of systems work. E.g. HipHop (Facebook), Cassandra (Facebook), BigTable (Google), Android (Google), Go (Google), AWS in all its myriad manifestations (Amazon). Maybe when you do anything on a large enough scale, this stuff becomes important.
Re Salesforce: we were evaluating them, and I discovered you have to pay extra if you want non-company customers. I realized it wasn't a ploy for extra money, they just hadn't used the Party model, and thus had to do extra work to have individual and company customers. We didn't go with them.
I have a feeling the best technical talent still goes to big blue for the "impress mom" rep and R&D, or NSA for the spook factor
My take is Google have acted as a talent black hole. They have absorbed such a massive proportion of the top talent over the last few years it's starting to cause a serious problem to their competition, such as the iOS Maps business.
Apple still has a reputation/image plenty good enough to attract good talent. I assume they just don't invest as much. Hardware and marketing is a much bigger part of their business.
The big differences I've seen between Google and iOS Maps are in the data-cleanup aspect, not the technical aspect. That doesn't take top tech talent, it just takes a lot of man hours (or truly spectacular tech talent to come up with algorithms that don't need the cleanup, but Google isn't there yet themselves).
His whole article is premised on a false dichotomy:
> So is it really the case that these silicon valley/web 2.0 companies have the best developers, or they are just regular companies that have average developers doing stupid things?
Third option: silicon valley has (many) of the best developers, but great developers do stupid things al the time.
Doing The Right Thing (tm) is always easy when looking from the outside in. In my experience, good developers write shitty code all the time, because the world is a messy place and quality comes at a price.
As an anecdote, I once worked on an application where we had to store passwords in plain text, because the password should also be used in a piece of hardware and to provision that hardware, we needed to send the password to said hardware. There's just no good and easy way to solve that and when you're constrained by real world limitations, sometimes you just have to accept a suboptimal solution.
You could at least encrypt the passwords on disk an decrypt them when needed. this is not ideal because you still need to keep the key around, but it is better than plaintext.
That's what we ended up doing. As you say, not ideal and given that we were in a hurry, it didn't get priority for a long time (Since it was mostly futile anyway).
Anyway - Just an anecdote to show that things aren't always that simple.
The coders who join these companies are not in it to do the right things (cf. the popular things), to make sound decisions (even if they are based on "old" thinking, like the notion of privacy, and appear to be boring). These developers are in it for the perceived prestige of working for a high profile company, for the nonserious working environment (if they are young), and often simply for the money. These jobs often pay better than, e.g., working on projects where there is great concern with doing things the right way. But the high-profile "Web 2.0" company does not necessarily produce better code.
Fourth option: Silicon Valley companies are more likely to do thing that have a broad impact.
I've worked with genius level people in environments varying from Welfare departments to IBM to industrial companies. Their genius is a big deal to steel production or food stamp printing, which are important things, but not sexy & important.
I think the biggest factor that might come into play though is that even top developers write bad code sometimes (weird behaviors that they think might work out but prove to be hard to maintain) especially when working on things they've had no experience in the past writing. (The first facebook api was probably not written by somebody who's done a lot of work with APIs). Or some other legacy issues caused the API to be written in non-optimal way.
The whole structure behind a lot of companies that he criticizes is that there's a small team building big stuff. In that environment, developers need to take on multiple roles and work on projects unlike anything they've worked on in the past. So I think you're absolutely correct, the first facebook api was not written by someone who had done a lot of api design. And I'd bet a lot of these other cases are similar. The best devs will still struggle a bit with projects that are unlike anything they've ever done before.
I'd also say that having the best technical talent is not a prerequisite for building the coolest or most popular stuff. The author assumes it is, which is a major flaw. There are a ton of other factors that leave you with a successful product.
That being said, leaving passwords in plain text is just plain stupid.
Probably the truth is that most Web 2.0 companies have a mixed bag but by the nature of their business and the startup ecosystem every web 2.0 company has to hype their business and consequently hype the talent they have.
Just about every startup will tell you they have A-players, rockstars, etc etc - its called marketing.
What would your job ad say if it was designed to solicit applications from 20 to 22 year old males willing to work long hours for peanuts?
"Come be a frankly mediocre programmer writing pedestrian code to solve unimportant problems"?
A job, like any other product, has to be sold. Beer ads don't say "makes you stupid and fat; occasionally ruins lives; it's going to kill some of you; no, really, you will not have beautiful women drape themselves over you, that virtually never to men sitting at home playing Xbox and drinking beer does not make it substantially more likely." For a reason. Similarly, there's the intimation of a cultural/status/etc component to the product offering for working as an engineer at one of these places. (n.b. Not unique to working at them, incidentally. i-bankers, Japanese salarymen, nurses, and PhD candidates in English all get a sales pitch -- in no case is it totally 100% representative of the actual job because, hey, sales pitch.)
Most of my (ex israeli military) and extremely talented people didn't even look at web startups but went to work for companies "doing real products" where "there are complex challenged".
They view 99% of web startups as glorified TODO apps with the biggest challenge being playing around doing A/B on front page headlines. Hardly an engineering challenge.
They want to build stuff like Terabite-size de-duplicated storage infrastructure, or embedded devices.
And they are the majority of the brilliant people I know who all started as "programmers" (java/c/c#/etc)
Comments like this perpetuate the idea that it's somehow immoral to want to earn money. We spent all this time and energy learning a commercially valuable skill, which is going to make a pile of money for someone else. It would be foolish to undervalue that skill at the endpoint of the developer.
It's good to distinguish between people who want to make a decent amount of money in order to live comfortably from those that want to make as much as possible in order to compete with their peers and perhaps make up for some self-esteem or other issues. I think OP was referring to the latter while it sounds like you're referring to the former.
Seems pretty silly to think that anyone who wants to make more than enough money to live comfortably wants to do so only to compete with their peers or make up for self-esteem issues.
"Golddiggers" go work as quants on Wall St. People that want to work on startups generally dream of the big hit, but also want to be fully engaged with their job and create vital code. At least that was my motivation.
Well I think many of these startup companies are looking for "rockstar", "ninja" code warriors..whatever that means. I think they are trying to attract the most promising talent in the next generation of up and coming programmers. They try to search these candidates out by making a github profile (or some other open source contribution) mandatory.
I think experience does matter though. Even a talented young programmer can get into big trouble by trying to do cute and clever things. They may not have the experience to know that generic database design is generally a really horrible idea, etc.
I'm afraid to mention(considering many might feel offended) that IMO most "web 2.0 startups" are full of ill-competent self-claimed "software developers" who have 3 months of experience in PHP and JS and then build "cool apps", who then are considered "technically talented" just because their "cool app" has had some exposure on different media which has brought in some users.
There's much more to actual technical talent when it comes to building software than what you get from year or two of experience, let alone 3 months. Though, in this era when you can actually build usable software products in weeks, if not days(as there have been many interesting "weekend projects" mentioned here in HN) this doesn't matter as much than it did in the past. For example game development has never been as easy it is today, there are thousands if not millions of young people creating games for mobile devices. Some of them succeed in making money but that is most certainly not due to technical talent, as many may easily think.
Back in 90's kids(under 20 yo) literally wrote their own 3D renderers in software because there was no hardware acceleration available. Now 20 years later, how many teens would be able to write their own 3D rendering engine even when using graphics APIs, let alone when doing it all in software? Although the amount of programmers has gone up, I think amount of competence has gone down simply because technical competence is far less needed these days.
These days there's far less need for technical talent when it comes to "building web apps" than what one would've needed say 10 years ago to create a simple client-side desktop program. Yet, now more than ever there's talk about technical talent. Strange.
These days people are experimenting with compilers and parsers.. . If you want to reinvent the APIs of the JVM or the DOM be my guest. My college computer science teacher said it in the best, "software is about standing on the shoulders of giants."
Its not so much about brains but technique, the technique of the software community as a whole is increasing, we are way better in making software that REACH than the 90s, I am 20 btw
The ability to reach is a problematic one; consider e.g. Google's Play Store or Apple's App Store in which you see literally thousands of new products appearing daily. Technical excellence is secondary and often neglected(see e.g. WhatsApp's security issues!) which results in poor quality software and poor progress.
Simply the barrier of entry is much lower, which in turn reduces the programming talent. I'd be interested to see what the quality of software would be when average 65 yo granny would be reading API documentation of some "web app" because it's just so easy and it just works. :)
I think the difference between "average programmer" 20 years ago and now is the fact that back 20 years ago you had nothing fancy. A text-based editor, command line and a compiler. You were perhaps able to set graphics mode with some obscure commands and get some pixels on the screen. And that was just so fascinating. You really had to me one-in-a-hundred kind of person to be proficient with that stuff. If you put an average programmer of 2012 to such situation, I really bet a huge percentage will just drop the hobby and do something else. Though, on the other hand, things were so simple back then. What does it take these days to have access to the screen pixels as an array? Libraries, frameworks, documentation, APIs and whatnot. Back then you just moved a value to CPU register and called a certain interrupt and voila, 320x200x256 graphics mode with framebuffer starting from a well-known address. Then it's just a matter of writing to memory addresses, very simple. Couldn't be any simpler really!
Talent takes many forms. For web 2.0 companies engineering matters, but hustle product vision and teamwork come together in a unique way. Talent is not measured on a single axis.
Most startup engineers couldn't build design and build something like hadoop, but I doubt many database authors would perform well at a startup.
In addition, quick and dirty hacks can be a virtue in a startup (Say leading up to an important product demo). Some engineers have a good sense of what it takes to meet a tough deadline. A language designer does not face that challenge.
Who ever thought that "web 2.0" (whatever that means) companies have the best talent? They are mostly started by kids fresh out of college and have no real world experience at all.
What they DO have is a lot of energy and creativity. The talented developers usually come in to those companies a little down the road when things start to take off, or are there in the rare case of a veteran founding team.
Security and clean code are not sexy parts of the job, so a lot of startups make them secondary concerns. Young programmers don't understand technical debt because they simply haven't dealt with it, so things start off with a poor structure and need to be refactored down the road.
This a question I think about often. Like everyday.
Here's a few tenative conclusions:
1. The best coders are not necessarily interested in the same things as people who start "Web 2.0" companies.
2. Many of these coders who never get involved with "Web 2.0" and probably never will. (The good news, if you could call it that, is you can still benefit from their skills. They often like to share code freely. They are generous.)
To find the evidence for these conclusion you need to venture off the beaten path. To be receptive to this evidence you have to take respite the herd mentality and buzzconcepts like "Web 2.0".
3. This is the one that's continually obvious and the cause for a blog post on the topic: There are some very sub-par coders heavily involved with things like "Web 2.0". And they are very vocal. Much more so than excellent coders who not involved in things like "Web 2.0".
I'm always open to evidence that counters these conclusions.
But the more bad coding I see the more skeptical I get. And it just keeps coming. The bar is set very low. Who cares if some Web 2.0 site is a horrible privacy risk if they can still manage to IPO for a few billion dollars? The best coders care very little, it seems. They are more interested writing smart code than in a multi-billion dollar IPO.
As basic and important as salting + hashing passwords is, it is the sort of task that ends up on the technical debt list or scrum back-log, never to be looked at until it becomes a problem.
reply