"So here is the bottom line. If you really need your code to run as fast as possible, and you are planning to spend half your development time doing profiling and optimisation, then go ahead and write it in C because no other language (except possibly Assembler) will give you the level of control you need. But if you aren't planning to do a significant amount of micro-optimisation then you don't need C. Haskell will give you the same performance, or better, and cut your development and maintenance costs by 75 to 90 percent as well."
Note the 'same performance or better' in there.
Maybe you missed that bit in the original article?
This wasn't a large effort by any stretch of the imagination and a factor of 5 difference compared to the Haskell code isn't even in the same ballpark as "the same performance", and about a factor 10 difference with the C code listed in the original article. You'll notice no micro optimizations were done in the re-write, just some global re-arranging of the way data flows through the program.
The rest is in response to the misleading title, that Haskell is 'faster' than C, faster to program, faster to debug, easier to maintain and so on while making claims about performance that are not born out by real world comparisons.
Speed was the wrong thing to compare on here, and should not have been part of the original article without a serious effort to become equally proficient in both languages.
> This article is in response to an earlier one comparing Haskell and C, which made the claim that Haskell beats out C when it comes to speed.
Perhaps my reading comprehension of the original post is different from Jacques' (or I'm just wrong), but I don't think that the original article made such a claim. Here's the TL;DR of the original article:
> TL;DR: Conventional wisdom is wrong. Nothing can beat highly micro-optimised C, but real everyday C code is not like that, and is often several times slower than the micro-optimised version would be. Meanwhile the high level of Haskell means that the compiler has lots of scope for doing micro-optimisation of its own. As a result it is quite common for everyday Haskell code to run faster than everyday C. Not always, of course, but enough to make the speed difference moot unless you actually plan on doing lots of micro-optimisation.
From this, I understood that in a larger program, most programmers wouldn't be doing the kind of micro-optimizations that they do for the Benchmarks Game. I figure that most code would be written following this pattern:
* Write code to follow the spec (usually without thinking too much about performance)
* Run the code, and evaluate its performance
* If the performance is good enough, you're done
* If the performance needs to be better, run a profiler
* Find the biggest bottleneck and optimize it
* Re-run the code and re-evaluate its performance
The original article took a micro-benchmark (a mistake in my opinion, because it's easy to micro-optimize every single detail of that code) and showed that in the case of Haskell, the first version was too slow, but that with the help of a profiler, finding the cause was easy, and the fix was trivial, while in C the profiler didn't show any problem with the code of the user, so it must be a problem in the standard library's code, and to fix it required changing the design of the program and making it more different than the spec. And I felt this was the author's real point; that to get the maximum performance from a C program, you'll need to code it not like a human would imagine it, but like a computer would, and it makes the code harder to maintain later on.
"Third, a comparison on speed between Haskell and C is almost meaningless, speed is not Haskells strongest point and it would be folly to bench it against C for that particular reason."
I disagree wholeheartedly. If I am choosing Haskell over C, I am giving up some (possibility of) performance. The question of how much is an important piece of information, entirely relevant to that decision.
As I observed in a comment on that other post, the actual performance of both the Haskell and the C depends on the amount of effort expended to make them faster (first in general, and then possibly on a particular architecture). At the limit, the C beats the Haskell by some margin - the size of that margin is informative; but that's also not the whole story - what the margin is earlier also matters, and for a particular project, it might matter more.
This is not to say that the particular benchmarks in the earlier article were good choices - I don't have a clear position on that.
To be honest, your comment strikes me as far more "religious" than the article, which (since you obviously didn't RTFA) documents optimizing a particular bit of haskell code to be on par with an equivalent implementation in C. The article is not saying that haskell (the language) is faster than C.
>From this, I understood that in a larger program, most programmers wouldn't be doing the kind of micro-optimizations that they do for the Benchmarks Game.
The TL;DR; is wrong too. It doesn't require micro-optimizations for a large real world C program to be faster than an equivalent Haskell program.
All those abstractions and laziness in Haskell add up, and you get lower performance overall compared to ANY decent C implementation of a real world program, not just "highly optimized" ones.
I'm surprised by how dramatic the difference is between the speed of C and Haskell. One of my professors (at The University of Glasgow, so appropriately a Haskell fan) once claimed that it had "c-like performance".
I suppose that's the point of this paper though, that "c-like performance" is a terribly vague term, meaningless without knowledge of the specific comparisons being made.
I think it's worth pointing out though that idiomatic C is probably going to be more consistently performant. It seems common to run in to situations in Haskell where one change can cause 10X speedup, but I don't see that nearly as often with C code. I don't have a lot of evidence on hand to support this, just what I've observed personally. This seen fair? Relevant?
That's actually something I've been saying for quite awhile when people bring up microbenchmarks complaining about how a language like Haskell is slow.
Like, yes, no question, if you try and implement a tightloop in Haskell vs C, C is going to come out a lot faster.
However, no one writes Haskell like that! People rely pretty heavily on standard libraries (or even third party libraries), and on the optimizations afforded to them by the "rich" language of Haskell.
C might be capable of being faster, but that doesn't imply that a C program will inherently be faster than a similar Haskell program, especially if the programs are relatively large.
* The vast majority of code doesn't need to be fast, and it's trivial to write code that's within a factor of 5 of optimized C. I can write Haskell at least 10x faster than I can write optimized C for the same problem.
* If you want code that's within a factor of 2 of C then you'll need to understand quite a bit about the internal details of the compiler, but your code won't get too ugly. I'd say it's about the same amount of work in both C and Haskell to get this level of performance.
* If you want code that's just as fast as optimized C then you'll need great understanding of both the compiler and the underlying hardware. At this point, you can expect your code will get hideously ugly. Also your code will break a lot when GHC versions change. I can write optimized C about 10x faster than I can write optimized Haskell.
Of course every problem is different, and this is only a rough average of my experience.
If Haskell is easier to optimize than C, then it could easily be that there's some amount of programmer effort, for which expending that much effort in Haskell yields a faster program than expending that much effort in C. If that amount of effort is in the range of effort most people are able to expend on a class of projects, then Haskell is faster than C for those projects. It may even be that those are most projects.
It is, of course, not the case that Haskell is faster than C with arbitrary effort expended tuning to the specific hardware - no one is claiming that.
It's not neccesarily which language is faster, but the which algorithm is faster.
He said naively written C, which mean the algorithm may be entirely different and run in O(n2) and much slower than a Haskell version which use a different algorithm and run in O(nlogn).
I agree - if you're comparing runtime implementations.
I think this is actually a useful comparison, though. I would argue it's a lot easier to become a proficient, performance-conscious programmer in python, java, or even Haskell, than C. And you're more likely to shoot yourself in the foot with C.
From the point of view of someone who hasn't learned either language (maybe a scientist or engineer looking to do some simulation work), the message here is, "with the same time and effort, not only is Haskell as fast as C in many cases, but in some cases it will actually be faster than the C code that you, a beginner, can write."
Some people like bickering. But I think the point of the article is that C is, by nature of the language, easier to optimize than Haskell, since it is lower level.
>Conventional wisdom says that no programming language is faster than C, and all higher level languages (such as Haskell) are doomed to be much slower because of their distance from the real machine.
No: conventional wisdom just says that no higher level programming language is consistently faster than C AND/OR better to reason about with regards to memory consumption and runtime behaviour.
(Conventional wisdom also adds fortran, forth, Ada and C++ in the same "speedy" category).
Conventional wisdom adds that micro-benchmarks of some BS outlier examples (Java where the JIT can take advantage of some known condition to do something clever, etc) do not matter in real life programs, which are far more complex.
Conventional wisdom also adds that C to be speedier it doesn't even have to be highly optimized or carefully crafted by some C programming wizard or anything. Merely avoiding gross mistakes (like using an algorithm of the wrong complexity for the job) will do.
Conventional wisdom concludes that writing in your high level language in an unconventional (non idiomatic) way to get to "as fast a C" speeds is bullshit too, because it doesn't represent idiomatic (and far more common) high level language use.
Although the results are impressive, the title is a bit optimistic. Haskell beat GCC slightly in the first benchmark. Haskell beat C in the second benchmark because the C code is very poorly written. They did not compare with the obvious C implementation. Secondly, GCC isn't the best C compiler, even though the paper claims that it is the best compiler that "we could find". They should have used ICC. C++ is also 4x faster than Haskell in the second benchmark, giving further indication that the C is slow just because it is poorly written. So it's hard to justify such a title. The work itself is very good however.
Performance of a tiny code sample is not why people select one language over another. If that was the case, no one would be using anything besides assembly or C. Even in other languages, you'll use an abstraction heavy implementation for the benefit of long-term maintenance and code evolution and then performance optimise the bits that can pack a punch. Performance optimisation in any language looks ugly be it C or Haskell.
I don't think it was an assumption; I think Steve was speaking from experience. As someone working extensively in C and doing a fair bit of Haskell, I'm finding my C to be more verbose and less safe. The C runs faster, in my application; sometimes I write it faster and other times slower, depending on a bunch of factors.
That depends on a couple of things. First it assumes as fast as C is the goal, for my project the goal was faster startup than Perl and more safety. Also I remember hearing the requirements of the benchmarks had to be changed to prevent Haskell form lazily evaluating it's way to first place. I know in my case lazy evaluation is part of one of my optimizations.
This is an objection that people seem to raise disproportionately often - particularly when they have no performance tests and never profile their code. The one time I've seen a real-world head-to-head comparison between C and Haskell, the Haskell version performed 5x better than the C version. In fact I've literally never known a real-world Haskell program to have serious performance problems (I've known some that needed a small amount of time to be spent on profiling).
I realise this is much less satisfactory than a convincing theoretical solution; all I can say is that it just doesn't come up in practice.
The title is a bit click-baity, but the core point is sound: C is not fast, it's optimizable[1].
If you just write readable, friendly C code it won't be all that much faster than normal code in a high-level language like Haskell—it might even be slower. I know, I've done that myself. But you never see that in benchmarks, do you? That's not what benchmarks are about.
Here's another illustration: we can compile high-level languages like Haskell and Scheme to C. Does this make them "as fast as C"? Yes—they're literally running as C. But also no. Hand-optimized C code is going to beat those compilers any day. It's not even close.
C is not magical performance sauce you can sprinkle over your code to make it fast. It's just a language that doesn't have much mandatory overhead (no GC, minimal runtime... etc) and gives you access to certain low-level knobs that you can twiddle to optimize your code. I mean, that's important and useful, but unless you're going to apply that level of effort to your whole codebase, most of your C code won't be all that fast. Some applications need this level of optimization; most don't.
"So here is the bottom line. If you really need your code to run as fast as possible, and you are planning to spend half your development time doing profiling and optimisation, then go ahead and write it in C because no other language (except possibly Assembler) will give you the level of control you need. But if you aren't planning to do a significant amount of micro-optimisation then you don't need C. Haskell will give you the same performance, or better, and cut your development and maintenance costs by 75 to 90 percent as well."
Note the 'same performance or better' in there.
Maybe you missed that bit in the original article?
This wasn't a large effort by any stretch of the imagination and a factor of 5 difference compared to the Haskell code isn't even in the same ballpark as "the same performance", and about a factor 10 difference with the C code listed in the original article. You'll notice no micro optimizations were done in the re-write, just some global re-arranging of the way data flows through the program.
The rest is in response to the misleading title, that Haskell is 'faster' than C, faster to program, faster to debug, easier to maintain and so on while making claims about performance that are not born out by real world comparisons.
Speed was the wrong thing to compare on here, and should not have been part of the original article without a serious effort to become equally proficient in both languages.
reply