Civil engineering had thousands of years of development when Galloping Gertie happened. Why would their methodology work in other, different engineering fields when it fairly routinely doesn't work in its own? I mean, if you look at things like project failure percentage (and cost!) over time spent developing the field, you probably have software winning.
Software is developing in to an engineering profession at an astonishingly fast pace. It's just currently at the point where it's differentiating between tradesmen and engineers, and without that clear distinction, it's hard to compare apples-to-apples between fields.
> And the handyman and the engineer have very little in common.
Why not? You're lampooning software development for the fact that a lot of not-quite-professionals develop mixed results because they churn a lot of product onto the market. The damage done to house integrity all over the world by questionable repairmen estimating the engineering impact of a change they make is comparable.
My experience with high-assurance software is that it's similarly well constructed and engineered to high-assurance civil engineering, eg, major bridges. Failures happen, but are sporadic rather than regular.
Software is rather good actually. Only a small percent of any of it is faulty. It could be improved but it is largely not considered worth it.
Civil engineering is not as perfect as often supposed. There are plenty of defects from barely noticeable to catastrophic. The software world simply knows nothing about other engineering. There is an easy way to see: just check the references in software engineering books and papers and count how many point to other such disciplines: practically none.
Neither is civil engineering as predictable (or controllable) as supposed. The majority of time, cost, and risk is borne by construction, so this overshadows the part most equivalent to software -- the engineering design proper. But this stilll has the same inherent difficulties as any design work.
Saying software makes other engineering look trivial shows a lack of knowledge. If you learn a little of civil engineering you will find it is just as complex as software.
Software is not all entirely new: look at all the reused components, libraries, etc. That is substantial, perhaps even the majority proportion in some sense.
There is probably more complexity of interrelations of parts in software, since physical designs are constrained to a more 'regular' physical substrate. But I don't think it is too bad, because the design activity is intrinsically isolating to a large degree.
Software has almost none of the physical constraints in other engineering, and it is so changeable at any stage. It really has distinct good fortune and advantages. And it is everywhere, it runs the world. It is a great success. Yet it is seen as failing. We don't have a mature, perceptive, understanding attitude to it.
It is difficult to compare software development with civil engineering as they operate on different levels.
Software development is better compared to building houses. No one expects single story house to support 4 extra levels or for internal walls to be resistant to hammer blows.
Imagine what civil engineering would produce if you were building a bridge from a to b and then 3/4 way through you were required to add two extra exits c and d that were miles apart from a and b and another 4 lanes and extra level for train traffic...all in the same time frame and for no extra cost.
Civil engineers would not be able to do it and no-one would even expect them to but this is regular occurrence for software developers and most of them actually manage to deliver.
Sure there are bugs but bridges need constant maintenance too, some quite substantial and that is expected.
For some reason, lots of non-software engineers expect software to be perfect when their own products are not, not even close.
Civil engineers are self-confident and don't go comparing themselves to all the other engineering disciplines trying to convince themselves they are real engineers. Software engineering is prone to a significant minority of people who are not so self-confident.
Personally, I am. In my opinion, whenever someone tries to drag in a methodology from other engineering disciplines they do so with a blinding ignorance of "cost" side of the cost/benefit ratio (and often non-trivial ignorance on the "benefit" side too), and the real differences between software and physical things. If car engineers could smash a car into a wall, observe something they didn't like, tweak something and 30 seconds be smashing another car into a wall effectively for free, their design methodology would be different too. (And if you feel inclined to get into a really detailed argument about how it would affect them because you think I'm assuming something about exactly how it would be different, you're missing my point, which is simply that it would indeed be different. Especially towards the beginning phases of the design.)
As someone who learnt to code as a kid, then studied and worked in civil/structural engineering, then ended up later on back in software, I do find comparisons (somehow always involving bridge analogies) between software engineering and 'traditional' engineering to always be off somehow.
Most traditional engineering projects are not magical bastions of rigor and certainty. It is really only in very large budget or critical projects that a lot of analysis or rigour comes into play to create that certainty.
At least in the civil/structural world you'd be surprised how much comes down to just gut feel by experienced engineers who then throw down some very quick calculations (budgets don't allow for much more than that) to back up their choices. Most numbers are looked up in tables. They then hopefully get run past another experienced engineers gut feel for validation then get signed off. And most of the time the public agency will just accept that uncritically - as they no longer have the resources to double check designs themselves.
The main difference with software as I see it, is that software is binary and the real world is analogue. Civil/structural engineering standards have factors of safety for materials and loading codes etc to cope with inconsistencies or minor mistakes/oversights or unforeseen circumstances etc. Even most failures are not catastrophic and can be detected and fixed before a catastrophic failure eg things can yield or crack rather than snap.
Software being binary though means for a certain set of circumstances it either works or it doesn't. You don't have the luxury of factors of safety and just overspeccing components to be sure.
That seems like a really unfair comparison. I'd argue that the software industry has evolved magnitudes more in the last 20 years than what you call civil engineering has evolved in the last 100. Sure, the latter has existed much much longer, but it's by no means evolving very quickly. Software development has traded safety for rapid evolution in areas where possible. This could be argued for or against, but I'd rather have a multiple generations more mature software industry at a point I'd need to develop "safe" software, than a safely evolved industry several generations behind.
Software is completely different from your typical other engineering fields. You just can't apply the same methodology there. In other fields such as building bridges you are quite often taking what has already been proven to work well and building it, while in software if you start to repeat yourself you are doing things wrong.
Why do people have a penchant for insisting on comparing software engineering (or programming or whatever) to civil engineering, specifically? Isn't there a whole world between the structural engineering of a skyscraper and the engineering of a new todo-app?
Are really all other engineers doing work as delicate and safety-critical as brain surgery? I'd imagine (not that I know, really) that a lot of risk-averseness in more concrete engineering fields simply has to do with the bottom line: if a design has already been put into production, what if there is a fault in that design and subsequently all the final products of that batch? With software, you can just nag the end users to update.
Building software is often compared to building houses. If civil engineers can design and create a very complex bridge or dam or whatever, and it keeps working for 150 years without breaking. Why can't us software engineers make software that will work without bugs for more than 10 minutes?
Problem is, the two aren't really comparable.
A while ago someone wrote an article that we are Software Gardeners not engineers[1]. And I think that comparison is much more apt.
Software is maleable, it can change, the layers of complexity build up so high it's impossible for any one person, or team of persons, to fully understand. Even the mathematical theory of software isn't fully understood yet, and yet we are piling those little lacks of understanding on every layer of the architecture ... they add up and things become monstrous.
BUT! This is a good thing. Software isn't a physical object. I want to be able to change the specs half way through a project rather than having to wait another 50 years to design a better bridge. This makes progress quicker.
As always, when progress is quick (and easy), it is also messy.
For example: it took builders thousands of years to invent The Arch and trully revolutionize the industry. It took software only a couple of decades to go from machine code to python.
Oh and never forget, the Turing Machine was "invented" in 1948. Our lives revolve around an industry the theoretic principles of which were first defined less than 70 years ago.
Well, the classical engineering domains are all pretty different anyway (electrical, structural, civil, etc). I remember my grandfather saying he thought it was ludicrous that they were all covered by a single qualification.
Building software with an engineering approach is a real thing. Not everyone subscribes to it, but doesn't mean it isn't a real thing.
For the record, I do a lot of gardening too, and building software has fuck all to do with that.
I also don't get how everyone who argues against software-as-engineering talk like buildings and bridges never fall over. They do.
Actually I rather enjoyed the article, especially the opening bit about the pursuit of popularity destroying blog value, but you can't just pretend that software engineering is just simply a form of engineering still in its dark ages (though it's obviously cruder than traditional engineering disciplines). Computer science is an extension of pure math going back thousands of years, of which civil engineering is another only slightly older offshoot. The problem spaces are simply not comparable.
In civil engineering you are dealing the same constraints: physics, which can be refined to perfection because they will be used billions upon billions of times throughout history and will never change. In civil engineering the stakes are high, people will lose lives. Finally, the parameters for success are very easy to measure in advance... it's easy to calculate how much traffic a bridge will carry, and you can even render exactly what it will look like for aesthetic evaluation before construction begins at all.
Contrast with software where the problem space is infinitely larger (ie. the entire realm of abstract thought vs physical space, which also makes it less intuitive by people btw). The costs of failure and repair are most often trivial (especially for web software). And the suitability of software is often not known at all until it is delivered (in this regard architecture has more in common with software engineering). You can't just hand-wave these things away by stating that software engineering is an immature discipline. Even with the exact same number of man hours behind software engineering that you have behind civil engineering, there would still be a such a tiny dent in the problem space as to leave software engineering looking primitive.
It's not that no one takes software engineering seriously either, take military or NASA software. Those guys have the most advanced software engineering principles on the planet, but even there, it's specifically suited to their use case. I'm sure there's a lot we could all learn from them, but if you want Microsoft Word to have that quality are you prepared to pay $500,000 a head to run it?
I agree that Joel totally threw the baby out with the bathwater in the duct-tape programmer defense, but let's not disparage the state of software engineering. It's certainly much less disciplined than other forms of engineering, but that's simply a cost-benefit tradeoff that seems reasonably close to optimal.
No, precisely the opposite should be said of engineers. Engineering is about reusing well-known techniques to achieve repeatable, predictable results whose timelines can be estimated. Engineering is not art. When civil engineers build new buildings or bridges (and I'm talking about your common building or bridge, not some iconic project), do they try to innovate and undertake to do things in some new-fangled way every time? No, because this would be extremely costly and dangerous and would render useless much of the past work that's been done testing materials and loads and so forth. When a method is known of constructing a bridge or an overpass that is both safe and economical, a good engineer should just keep doing that, with perhaps some adaption as appropriate on a case-by-case basis.
If software developers would stick with tried-and-true tools instead of inventing new frameworks to solve the same old problems and immediately obsoleting the "old" way of doing things each time, we'd have much less legacy code and likely would be much better at estimating software development tasks, and wouldn't have to solve the same problems over and over again.
"Software engineering estimates and plans often fail to live up to the reality that follows. It seems to be the only engineering discipline in which this is regularly the case."
Is it though? I live in Boston, home of the notorious* Big Dig[1]. While particularly egregious, it's far from the only large-scale civil engineering project that's gone off the rails. In fact, I'd argue that until fairly recently, many more public works projects shared the "surprise factor" of software projects. I'd recommend Caro's "The Power Broker"[2] for a fascinating history of NY-area public works (among other things - great book all around), including how much of that process was about adapting the plan to new things the builders were learning along the way ("oh, turns out that soil is completely different than we planned...")
That's not to say that there aren't particular features that make software engineering its own special snowflake - as there are meaningful differences between how civil, structural, mechanical, etc. engineers operate. But spend some time in another engineering organization and you'll find it's different, but not as different as you think it is.
(And FWIW, even civil engineers sometimes follow "agile" concepts - a company I once worked for was contracted to design a highway, and even after the construction started, engineers were "embedded" with the builders to make on-the-fly adjustments based on the environmental factors they discovered throughout the process... I wish I could find their project write-up, but it was a while ago and the company has long since been gobbled up by a bigger company).
* As a (subjective) kicker, I'd add that the Big Dig, over-time and over-budget as it was, was ultimately quite worth it... much like many software projects!
You are comparing apples and oranges here. Bridge building should be compared to aerospace software engineering, not to the average CRUD app.
Apart from that, I think you are giving too much credit to construction engineering outside of the highly regulated sectors. They mess up just as often, and expensively so. But stakeholders in construction are slighly more aware that their requirements are going to be set in stone, pun fully intended.
What really makes a difference is the malleability of software. This is an invitation for feature creep and last-minute changes. Also, the impact of overengineered, unmaintainable software is not quite in-ya-face as for physical assets. Therefore, in software engineerin project managers need to be religious about these matters. And software engineers must get better at communicating about these concerns.
The fact that the feedback loop of a software engineering process is much tighter and cheaper than that of civil engineering does not mean that it is not engineering - that's a fallacy.
Perhaps it's less applicable to civil engineers, because of the scale and safety requirements of their profession, but let's look at electrical engineering. An electrical engineer will build prototypes; they will build models; they will use computer simulations to predict how systems will behave; they will build testing and QA infrastructure. It's definitely harder to iterate—once you've built a million widgets, shifting circuits around is going to be expensive—but it's not qualitatively different to developing a software product.
As in any engineering discipline, software engineers must optimise for a system's required qualities. In some cases, reliability might not be that high up the list. In others—let's say medical systems, or those used in space travel—software engineers have rigorous systems in place to ensure quality, and in many cases substantial up-front planning and certification will be involved.
Ultimately this seems to say that software engineering is different from (specifically) civil engineering. I think that's broadly rather true – the nature of civil is such that generally there are stringent safety requirements, and indeed massive costs involved in making subsequent changes. The majority of software does not have the same requirements, and so it's no surprise that the process is somewhat different. And I agree that not all development is engineering, of course, but to dismiss the entire field is a bit of a shallow argument.
Precisely. I would have been less charitable in my response. Most software development is plumbing. To say we're solving new problems all the time and then to set that in some unclear opposition to what structural engineers do... I mean, just like software, each SE's project is different and yet similar across cases. That's uninformative, to say that least. So if you want to contrast them, it's best to actually know what you're talking. Also, I always hated occupational narcissism. It's a sign of narrow-mindedness.
(Besides, the question about whether software development is engineering or craft is very much real. John Allen, John McCarthy, Bertrand Meyer and Peter Denning have all written interesting things about this subject.)
Thanks. You are right, the comparison is off probably on both sides.
As with critical civil/structural engineering projects, there are software development projects built with correctness in mind. I've worked on a few of those, mostly in aerospace and VLSI simulation in the late 70s til the early 90s, and the level of engineering was very high.
Fast forward two decades to the world of web development and corporate IT and, wow, the lack of rigour is simply abysmal. I can imagine the same been true in the housing/building sector.
I am sure other fields have a better development story these days, but isn't it a matter of perceived risk and relative effort?
I.e. we rarely hear about plane crashes caused by software, or power plants accidentally blowing up due to an integer underflow.
Civil engineers make solid bridges, but most software is not the golden gate, it's a shed thrown together by some guy who's not an engineer nor a professional mason.
I think domains are too different for this analogy to apply.
When building bridge civil engineer knows exatly the environment wher it will stand and pretty much knows all the likely variations of it.
Software engineer? Not even close. I'd argue, that sofware solutions are _more_ robust given all the uncertainty.
It can be, as in the world of formal proofs and languages like Ada which facilitate them. It just typically isn't, as such formal procedures are far more costly and slower than than just dealing with bugs as they happen. For critical software systems like say avionics where a failure case is extremely costly (in both money and casualties), there do exist certification programs and rigorous methods as you describe for structural engineering.
It could be argued that avionics behaves more as engineering than as programming. Then we have a self-fulfilling definition: if we define fields with rigorous procedures as engineering, and fields without as software development, then of course software development doesn't have formalized methodology.
You're also comparing the largest scope of structural engineering - a massive bridge - with the entire spectrum of software. Engineering does happen on a smaller informalized scale too, like a homeowner building a shed or a Boy Scout troop building a rope obstacle course. That's still engineering, applying skills in carpentry or knot-tying to build something.
Software is developing in to an engineering profession at an astonishingly fast pace. It's just currently at the point where it's differentiating between tradesmen and engineers, and without that clear distinction, it's hard to compare apples-to-apples between fields.
> And the handyman and the engineer have very little in common.
Why not? You're lampooning software development for the fact that a lot of not-quite-professionals develop mixed results because they churn a lot of product onto the market. The damage done to house integrity all over the world by questionable repairmen estimating the engineering impact of a change they make is comparable.
My experience with high-assurance software is that it's similarly well constructed and engineered to high-assurance civil engineering, eg, major bridges. Failures happen, but are sporadic rather than regular.
reply