Callback-based code will always be more complex than synchronous code (and I believe that's an objective observation), but please let's not go there.
My remark was against their "Keep your code small and light". Making your code lighter doesn't always make it faster. If you can get your code to do less (like, skipping memory-based sessions), that will make it faster. If all you're doing is avoiding complexity, there's no guarantee it will be perform better.
The proper way is to benchmark and profile. You can refactor your code based on developer happiness, but only try increase its performance based on objective data.
EDIT: I should probably clarify this (although it seems silly to have to do this on HN): I consider callback-based code yet another tool for solving problems, not something you preach for/against. There is a time and place for callback-based code, just as there is a time and place where it's not a good idea.
First: code that performs simple sequential steps should look simple. With callbacks, it always ends up looking complicated.
Second: refactorability in callback oriented code is a lot worse than linear code. Even refactoring code with a single callback can be annoying. Async code with callbacks that looks and feels like imperative synchronous code is an enormous gain.
> I consider callback-based code yet another tool for solving problems, not something you preach for/against. There is a time and place for callback-based code, just as there is a time and place where it's not a good idea.
Node doesn't really give you much of a choice. If something is going to take awhile then you will be using callbacks, it's that simple.
Callbacks are a very limited way to do asynchronous programming. However they are a good way to create interfaces that let you call methods and insert your own functionality in the middle.
So yes. Better async is good. But don't take away callbacks. They have their uses.
> Callback-heavy programming is what you get when your language doesn't support better ways of controlling context.
UI programming is callback heavy, but I doubt that any language will support "better" ways for that.
So basically UI programmers are used to callbacks, others not that much, and now that event-loop servers are becoming popular, we see confusion from people not familiar with events.
Server-side people just need to figure out how to use callbacks properly and everything is gonna be fine.
All people need to understand is to nest callbacks properly, nested callbacks are the things that confuse most people.
> For Python, you can get the benefits of asynchronous IO without callback hell if you use the Stackless fork.
You can't just abstract away every programing feature that seems hard and expect everything to be fine, it just doesn't work that way, Java developers are still suffering because Java tries real hard to abstract memory management.
"Callbacks seem simpler to you not because they are simpler (try explaining them to someone just learning the language, and you'll see what I mean), but because you got used to them."
No, they're simpler in the literal sense: they introduce no new concepts into the language or runtime semantics. (The dynamic behavior is still complex, of course.)
"Even so, error handling and explicit thread synchronization make maintaining callback-ridden code painful. I think setting `Busy` to `false` in `finally` block is a great example (in the blog post). You just can't do that with nested callbacks—they are not that expressive."
Right -- nested callbacks aren't the answer, either. In JavaScript (where most of my non-C experience comes from), a good solution is a control flow function:
busy = true;
series([
function (callback) {
// step 1, invoke callback();
},
function (callback) {
// step 2, invoke callback();
},
function (callback) {
// step 3, invoke callback();
}
],
function (err) {
// finally goes here
busy = false;
if (err)
// ...
});
This construct is clear and requires no extension to the language or runtime.
This is fundamentally a matter of opinion based on differing values. I just want to point out that there's a tradeoff to expanding the language and to dispel the myth that callbacks necessarily trade off readability when control flow gets complex.
> The big reason callback-based systems are hard (despite the fact that built-in flow control constructs need to be re-implemented) is that functions are no longer composable
Absolutely, I've had it happen in "big" (client-side) JS projects.
Only way I've found so far to handle this is to make anything which might ever have any reason to become asynchronous (so anything but helper functions) take callbacks or return deferred objects. Always.
But then an other issue arises: for various reasons, callbacks-based code which works synchronously may fail asynchronously, and the other way around. And then it starts getting real fun as you still have to go through all your (supposedly) carefully constructed callbacks-based code to find out what reason it would have to fail (alternatively, you create both sync and async tests for all pieces of callbacks-based code)
I've heard this before. "Oh, callbacks are fine for little things like this hello world demo, but for Big Programs, you need (threads/coroutines/etc.) because they're Serious Business."
That might be true. Maybe for Big Programs, something else is better. But I think the problem is that setting out to build a Big Program for Serious Business is Doing It Wrong.
The best frameworks (or, at least, my favorite frameworks) are collections of small tools with consistent interfaces and interchangeable parts.
If you can manage callbacks for a 100-200 LOC program, then you can build a Big Program out of such modules with a little bit of forethought. Instead of setting out to write a Big Program, why not try to figure out how to express your Big Problem in terms of a bunch of Little Problems, and then come up with a way to have Little Programs assemble.
Then, assemble the Little Programs to solve the Little Problems. There is no solveable Big Problem which cannot be broken down into some finite set of Little Problems.
Callbacks make it natural to solve problems in this manner.
I actively develop a several thousand line NodeJS project. It's quite nice, actually.
> On the contrary, it gives your code clarity, consistency and predictability.
Well, when a product I worked on gradually migrated from event loops with callbacks to threads, clarity, consistency, and predictability made big improvements. Some callback usage would have been better with C++11 or C++14, but it's still way worse than what you can do simply by having variables go in and out of scope on a stack.
Callback-heavy programming is what you get when your language doesn't support better ways of controlling context. It's analogous to manual memory management: it's work that's better to outsource to a compiler or runtime, except in specialized circumstances.
For Python, you can get the benefits of asynchronous IO without callback hell if you use the Stackless fork.
First, the analysis relies on every step from request to response being evented, which in turn relies on the existence of event-based libraries for everything you want to do.
Also, most languages don't have a good way of dealing with callbacks. They tend to make the code verbose and difficult to reason about. This is especially true for non-trivial applications, where there are more than two or three callbacks chained together.
However, you can counter-argue that the additional complexity isn't endemic to the programming model, only certain languages. Also, in languages which support multiple threads (read: not Javascript), a hybrid approach that uses events where possible, and threads where necessary, may retain many of the benefits of the event-based approach.
Everything has overhead and the only way to control it is to have options that fit your proposed work load and then optimize based on empirical evidence. If however your only option is the callback, then you have no way to work around its overhead.
Additionally, callbacks have the same amount of overhead but it's not constant because you have to create a side-channel for the state management. That means, instead of a simpler stack for keeping the state, you have to have a periodic stack + a structure or object for all the state even when the callback isn't active.
This kind of misses the point - sure it's still simple for a four line function, but it's whether callback oriented code scales for anything other than the most simple of projects.
I agree. Sometimes inline asynchronous callbacks work, just like inline code blocks for if statements or for loops. You just need to train your eye to read them as if they were inline code blocks for an if/then/else block or a for loop.
But sometimes when you get many levels deep in if statements or if a synchronous function starts to reach the hundreds of lines it makes sense to break it up into multiple functions that each have a sensible semantic meaning and which fit on a screen or so. This makes the synchronous code easier to read.
The same goes for asynchronous callback functions. The callback hell that I see most often happens when people have hundreds of lines of inception style anonymous callback functions inside of anonymous callback functions. In this case, just as with the synchronous function that got excessively heavy it makes sense to break things up into multiple functions.
It's all about finding the right balance, and when you do the results are very readable and easy to understand whether you are writing synchronous or asynchronous code.
I agree that it is a bit overzealous to say that every function should return its data using callbacks, but sometimes it makes sense even for synchronous functions. By making a function which doesn't accept a callback, you are creating an API that can never be made asynchronous, which could cause you more work in the long run.
For example, consider a function with a synchronous implementation. It gets the data from a place that is quick and easy to access, so making it async is not necessary. But later on, that data gets moved to an external database. Now, in the best case, every place where that function is used must be updated to be made asynchronous. In the worst case, every caller of the function, as well as every caller of those functions, and so on, must be updated. And quite often, converting synchronous code to async is no small feat.
Oh come on. You've set up several straw man arguments here that don't apply to the real world situations that give rise to callback hell, like fetching multiple rounds of data and aggregating many parallel callbacks at each step, or callbacks that are difficult to trace the origin of. Sure you can write good code with callbacks -- you can also write good code with goto and callcc -- but building a better abstraction makes code easier to write and maintain.
Agreed, it was a HUGE improvement over callback soup.
The only cleaner methods I’m familiar with are CSP (as implemented in clojure and go, and to some extent in JavaScript with libraries like RX and Bacon). Basically you’re treating steams of events as first-class values. That’s a rather functional/mathy way to treat concurrency/parallelism but I’ve found it illuminating.
My remark was against their "Keep your code small and light". Making your code lighter doesn't always make it faster. If you can get your code to do less (like, skipping memory-based sessions), that will make it faster. If all you're doing is avoiding complexity, there's no guarantee it will be perform better.
The proper way is to benchmark and profile. You can refactor your code based on developer happiness, but only try increase its performance based on objective data.
EDIT: I should probably clarify this (although it seems silly to have to do this on HN): I consider callback-based code yet another tool for solving problems, not something you preach for/against. There is a time and place for callback-based code, just as there is a time and place where it's not a good idea.
reply