If anything, the amount that async/await spirals outward to other areas in Rust demonstrates that it doesn't seem to be a well contained feature.

In addition, async/await executors in Rust quite often fail on corner cases. If I want to only wait on a socket, stuff works great because that's what everybody wants.

If, however, I want to do something like wait on sockets and wait on a character device file descriptor ... that tends to break in fairly mysterious ways.

https://play.rust-lang.org/?version=stable&mode=debug&editio...

I think this restriction is because they are trying to figure out how to handle dynamic vs static generics. Presumably they will add support for this in the future.

The compiler also recommends https://docs.rs/async-trait which has a pretty clean solution.

As a new Rust user myself, this kind of snobbery is still off-putting.

> A mature and serious language such as Rust

About a language as new as Rust you can be sure that they're getting their grinding stone and axe ready for some serious use.

I agree that languages with invisible program effects are harmful. The author said it another way as "Inconvenient knowledge is better than convenient ignorance".

I've been bitten many times when using languages which have weak types and perform implicit type conversion. I much prefer to know exactly what type of variable I'm using and perform explicit type conversion instead of having the language "take care of it" for me.

It has been the standard in the web for time immemorial (blue for links, a shade akin to purple for visited links).

What would be the "optimal" choice?

    a, a:visited {
        color: #692fac;
    }

Microsoft's Koka is an example of a language that further empraces effect types and makes them easier to use: https://koka-lang.github.io/koka/doc/index.html

[1] https://github.com/unisonweb/unison

The thing is: having less generic abstractions (and fewer colors) makes certain things (like debugging tooling and assumptions when reading code) much much nicer.

[1] http://madoko.org/reference.html

[2] https://theory.stanford.edu/~nikolaj/programmingz3.html

Regardless, this blogpost is about Rust which has yet to gain either feature, for well-known reasons (for one thing, monadic bind notation interacts badly w/ the multiple closure types in Rust). Talking about Haskell monads just does not seem all that relevant in this context.

(On the other hand Lawvere theories may be useful for modelling effects in type theories. That may be so (I don't know) but is an orthogonal issue.)

https://github.com/jameshaydon/lawvere

Monads can be used to implement "effect types" and define the semantics of them. I believe this is how Koka works.

> Talking about Haskell monads just does not seem all that relevant in this context.

I respectfully disagree. Async/Await first appeared in F# as a Monad (Computational Workflow). Monads also do not necessarily have to involve closures at runtime, they can be useful just to define semantics and build type-checkers.

For example, isn't it true that you can't easily compose a function f :: a -> IO b with a function g :: b -> [c] (assuming you don't use unsafePerformIO, of course)?

Of course, there will be ways to do it (just as you can wrap a sync function in an async function, or .Wait() on an async function result to get a sync function), and Haskell's very high level of abstraction will make that wrapping easier than in a language like C#.

   h :: a -> IO [c]
   h = liftM g . f

Maybe because "composing functions" is the whole point of category theory, monads and Haskell?

But still, since it's all about composition, the simplicity is a direct consequence IMHO.

Still, there is a slight bit more to the story if you actually had to pass a [c] to another function ( you would have to liftM that function as well, I believe, and so on for every function in a chain).

In Haskell, you can’t automatically lift something into a Monad because this would require a type of ad-hoc polymorphism that is impossible with a coherent Hindley-Milner type of type inference system.

However, that exact type inference system is what lets you just write “liftM”, and it automatically does the Right Thing based on the type of monad expected at the call site.

I think ultimately—Haskell is in a sense an experiment to see if these alternative tradeoffs are viable, and it’s a massive success in that regard.

(Also: you wouldn’t need to liftM every function in a chain. Just liftM the chain at the end.)

This easiness comes from the existence of fmap, which is defined for any functor (monads are all functors). To instance it and get haskell's lovely syntax, any given "container" has to define its own fmap implementation. So it is a feature of monads. If you implement a monad in another language, you're still going to be defining fmap (or an equivalent), which lets you use this.

But this is fundamentally a very simple thing. If you use an async framework, you have to think inside that. It's just as fundamental as CPU architecture, or operating system. If you have an Intel CPU you need a corresponding motherboard. If you have Windows you need programs that use the Win32 API (or use the necessary translation layer, like WSL).

Similarly, if you use - let's say - libuv in C, you can't just expect to call into the kernel and then merrily compose that with whatever libuv is doing.

This "coloring" is a suddenly overhyped aspect of programming (software engineering).

And of course fundamentally both are wrappable from the other, but it's good practice to don't mix sync/async libs and frameworks and components. (Or do it the right way, with separate threadpools, queues, pipes, or whatnot.)

Quasar Fibers, which eliminated function coloring for Java async versus blocking IO, I wound up doing the equivalent of that in the Java ecosystem, and it works in production extremely well.

Defeating the coloring problem makes gluing all sorts of really valuable stuff together much easier. In a world where you can’t just rewrite from scratch the things you’re gluing.

Does function coloring solve this? I don't think it's the case. Python, JS, etc. don't see this issue much because the interpreters tend to be single threaded or have GILs. C#, for instance, would see all the same issues a Java or Go program might with respect to variables changing over time.

Contrived example:

    void ChargeAccount(Account account, int cents) {
        int balance = account.balance;
        int newBalance = balance - cents;

        Logger.Write("changing account from {balance} to {newBalance}");

        account.balance = newBalance;
    }

If it's explicitly async with async-await keywords etc, then the race is easily noticeable and the programmer will know they have to use a mutex or atomics or whatever.

If there is any other code that touches account.balance from any other thread of execution (OS thread, coroutine, etc), then this code is already not safe.

If there isn't any other thread of execution, then this is safe regardless of whether the call to Logger.Write is sync or async.

For example, the following program would be safe regardless of whether Logger.Write is sync or async:

  int main() {
    InitLogger();
    ChargeAccount(acc, 10); 
    // the program will only ever get here after
    // account.balance = newBalance has finished executing
    ChargeAccount(acc, 10);
  }

  int main() {
    InitLogger();
    launchInNewThread(() => ChargeAccount(acc, 10));
    launchInNewThread(() => ChargeAccount(acc, 10));
  }

In fact, the first program if written with something like Loom would be perfectly equivalent to the following C#:

  async void ChargeAccount(Account account, int cents) {
    int balance = account.balance;
    int newBalance = balance - cents;

    await Logger.Write("changing account from {balance} to {newBalance}");

    account.balance = newBalance;
  }
  public static void Main(string[] args) {
    InitLogger();
    await ChargeAccount(account, 5);
    await ChargeAccount(account, 5);
  }

You seem to assume it is equivalent to something like this, which would indeed be unsafe, but is not the case:

  public static void Main(string[] args) {
    InitLogger();
    var t1 = ChargeAccount(account, 5);
    var t2 = ChargeAccount(account, 5);
    await Tasks.AwaitAll(t1, t2);
  }

async/await and friends (generators, callback-passing, etc.) all provide some clues that are sometimes useful for solving concurrency problems, but they aren’t able to make unsafe programs safe.

Exactly what I said.

>but they aren’t able to make unsafe programs safe.

Exactly not what I said either.

My point is about concurrency, not parallelism. I didn't mention anything about threads, let alone that there's more than one thread involved or that threads are being spawned.

Take a simple node.js program or a Rust program that has a single-threaded tokio executor, or a C# WinForm program that reuses the UI thread for running Tasks, and everything I wrote applies. A single Account value used to be fine to be shared between coroutines because only one invocation of ChargeAccount would be active at a time, but the "transparent async" breaks that assumption.

Explicit async-await makes it more obvious that multiple invocations of ChargeAccount might interleave unless proven otherwise, and the programmer has a reason to confirm that it is not the case, or use locking / atomics as necessary.

>You seem to assume it is equivalent to something like this, which would indeed be unsafe, but is not the case:

That `Task.WaitAll` program is indeed what I'm talking about, though the part of you thinking that I was talking about it being equivalent to a multi-threaded program is bogus.

That's not true at all. Here is a simple NodeJS implementation that is completely unsafe despite having non-transparent async:

  function delay(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
  }
  async function LoggerWrite(s) {
    await delay(Math.random() * 100);
    console.log(s);
  }
  async function ChargeAccount(account, cents) {
    let balance = account.balance;
    let newBalance = balance - cents;
    await LoggerWrite("changing account from "+ balance + " to " + newBalance);
    account.balance = newBalance;
  }    
  async function Main() {
    let account = {balance: 0};
    let t1 = ChargeAccount(account, 1);
    let t2 = ChargeAccount(account, 20);
    let t3 = ChargeAccount(account, 18);
    let t4 = ChargeAccount(account, 12);
    await Promise.all([t1, t2, t3, t4]);
    console.log(account);
  }
  Main() 
  //will randomly end up with -1 or -12 or -18 or -20, instead of the expected -51

  func LoggerWrite(s string) {
    <-time.After(100) //this means "block for 100 ms"
    fmt.Println(s)
  }
  func ChargeAccount(account *account, cents int) {
    balance := account.balance
    newBalance := balance - cents
    LoggerWrite(fmt.Sprintf("changing account from %d to %d", balance, newBalance))
    account.balance = newBalance
  }

  func main() {
    account := account{balance: 0}
    ChargeAccount(&account, 1)
    ChargeAccount(&account, 20)
    ChargeAccount(&account, 18)
    ChargeAccount(&account, 12)
    fmt.Printf("%+v\n", account)
  }
  //will ultimately print -51

NodeJS: https://replit.com/talk/share/Async-race-condition/138001

Go: https://play.golang.org/p/ypKfQFUF5M2

Edit: > Explicit async-await makes it more obvious that multiple invocations of ChargeAccount might interleave unless proven otherwise, and the programmer has a reason to confirm that it is not the case, or use locking / atomics as necessary.

As shown by the example above, I think almost the opposite is true: async/await take code that looks linear and single-threaded and make it run concurrently. It's easy to accidentally end up with code like the JS one. Even worse, it's easy to accidentally call async code without await-ing it at all, and have unexpected behavior.

By contrast, transparent async + cheap coroutines means that you can make all of your APIs blocking, and rely on the user explicitly spawning a coroutine if they want any kind of parallel/concurrent behavior, at the call site. In the Go code, I could have called `go ChargeAccount(...)` if I wanted the ChargeAccount calls to happen in parallel - I would have explicitly created parallelism, and I would have known to be careful of sharing the same pointer with the coroutines running concurrently.

What does that have to do with my comment?

I said (paraphrased) "Transparent async is bad because sync code can end up being wrong when async-ness is silently introduced into it". You keep responding with (paraphrased) "Explicit async code can also be bad." Do you understand how this does nothing to negate what I said?

(This discussion is frustrating, and the walls of irrelevant text you keep posting aren't helping.)

I'd say that monads surface a problem that was already there, and give you a vocabulary to talk about it and potentially work with it. Composing effectful functions is still surprising even if you don't keep track of those effects in your type system (for example, combining async functions with mutable variables, or either with an Amb-style backtracking/nondeterminism function, or any of those with error propagation).

But yeah, monads are an incomplete solution to the problem of effects, and when writing composable libraries you often want to abstract a bit further. In the case of Haskell-style languages we have MTL-style typeclasses, or Free coproducts, or various more experimental approaches.

    -- suppose we have some x :: a
    fmap g (f x) :: IO [c]

It's true that "monad's don't compose" in the sense that you need to manually write monad transformers to correctly combine monadic effects, whereas functors compose automatically. However, that's not the same thing as saying it's difficult to use functions which happen to return different monadic results together, if the types match.

    (>>=)  :: m a -> (  a -> m b) -> m b
    return ::   a                 -> m a

That gives you `composed_function x = (f x) >>= (return . g)`, which a "do" block makes potentially clearer :`do {y <- f x; return (g y)}`

Anyways the idea is that each sort of "container" defines how to apply functions to it.

Second, the whole function coloring thing wasn't a complaint about limits on function composition. If it were, we might have to raise a complaint that arity is a kind of color. It was a complaint that functions of one color can't be called from inside functions of another color. And you don't really have an equivalent problem with using one monad inside the implementation of another monad.

Hence the semantics are the same - just without any compiler feature like async/await.

IOW, Haskell does have coloring, and its coloring does manifest through the IO monad, but that does not mean that all monads have color. Sort of like how you can't do, "Socrates is a man, Socrates is dead, therefore all men are dead."

But in general you can't do that with monads, in particular not if you just have "any" monad at hand but you don't know which one it is. And IO is one example, parsers would be another one.

Monad is a typeclass (analogous to an interface in OOP or traits in Rust). IO is a type. To make some type a member of monad (or any other typeclass), you just need to write the required functions in an instance declaration.

You can make a type a member of as many typeclasses as you want. List is a member of Foldable, Monad, etc. Calling that a special isn’t quite right as it is very common for a type to be a member of many typeclasses.

You’re entirely correct in saying the Monad typeclass offers no way to extract values. But there actually is an extract function called unsafePerformIO with type “IO a -> a”. The name says it all: use it and you’ll lose all of the safety normally afforded by keeping data derived from effects inside the IO type.

For example, if you have, say,

`getIntFromSomewhere :: IO Int`

and you want to apply a pure function of type `Int -> Int` to it. Supppose the pure function adds one. You might initially think that that's all there is to it, you don't want IO involved in what happens next. But of course, the only reason you would ever do some pure computation is to use the result to have some effect on the world, eg printing it to a console. So actually in a sense, you do want to stay in IO.

    myIOAction :: IO ()
    myIOAction = do
      myInt <- getIntFromSomewhere
      print (addOne myInt)

Monads aren't functions[0]; in the analogy, monads are colors.

> [F]unctions of one color can't be called from inside functions of another color. And you don't really have an equivalent problem with using one monad inside the implementation of another monad.

If you have a bunch of `a -> M b` and one `a -> N b`, it's much more straightforward to interleave calls to any group of the former than to incorporate one of the latter. You encounter this when writing a new `a -> M b`, much like you might encounter the corresponding issue when writing a new blue function.

I should point out that there is usually good reason for this in the implementation, and often good reason for it in the domain. When there is good reason in the domain, that's great - our type system is surfacing the places we need to make decisions, rather than letting us mash things together in ways that will cause problems. When there is only good reason in the implementation, or no good reason at all, it's unfortunate, but certainly something that can be worked around.

Unlike the original setting, Haskell provides the ability to write functions polymorphic in "color" ("polychromatic"?), but for the (unfortunately necessary?) distinction between `a -> b` and `a -> Identity b`, and a lot of colors can be easily removed or transformed.

[0]: Some types which are monads are functions but that's not the sense in which we're coloring functions.

Strong agree.

This list could also be extended endlessly, and with continuous dimensions: how much L1i/L2i/L1d/L2d/L3 cache does this consume? How many instructions will it generate? For AMD64, ARM? How well can it be reordered wrt this other function? Is it superscalar? It becomes dimensionally intractable, because these properties do interact with each other.

IMHO, all side-effects considerations should be (easily) surfaced as hints to the programmer, but not as constraints. At least that's what a language/runtime should aim for.

For example: Rust's asyncs can be called synchronously. That's fair enough (and ok), but the programmer should be made aware by the compiler that it will block the thread.

To me, the best thing to aim for is a language that gets out of the way (correctness in changing requirements context is difficult enough already), but informs you on what you are trading off.

Say your wrote a sequential part. A hypothetical 'parallelization opportunities' tab of your IDE should ask you the question: 'Do your business requirements allow that this part be made parallel?'.

The reason this is being considered for removal is that even though the mechanism is very flexible and powerful in theory, it is too elaborate to be used correctly by most programmers in practice.

This is by design to allow for hot code reloading. This is a very sound design choice because true concurrency is achieved through the creation of other processes (agents, genservers, tasks, ...).

You have nice things like Dynamic Supervisors to monitor those processes. And it works very well.

Try to do the same with uncancellable promises in Javascript/Typescript, or with channels in Go, while it's still possible, you'll find your code growing in complexity way quicker.

The actor model embedded in the language, like Erlang/Elixir, brings so much to the table, it's unfortunate that the author dismiss it without a second thought.

async/await being implemented and popularized as it is today everywhere might just be our generations null-mistake since they're so error prone, they're great to show off parallel IO performance increases for classically synchronous languages but the programmer-burden of explicit await's (luckily IDE's/compilers can catch them) with "weird" side-effects was a big mistake.

The author of the article seems to be firmly in a specific category of developers so the dismissal seems quite in-character.

I wasn't sure so I did not want to claim it, so thanks :)

> async/await being implemented and popularized as it is today everywhere might just be our generations null-mistake

Agreed, especially since each language implements it differently.

For example, Javascript's async/await is just syntactic sugar over promises and aren't truly "colored" functions.

The "all things are (posslby) async" is a language design choice similar to "all things are (possibly) effectful" for a procedural language. If you keep your language semantics in the less generalized domain (e.g. functional vs. effectful), you have to invent syntax to talk about the other semantics easily.

So you come up with monad systems in functional languages to talk about effects, and make some nice syntax for that. And you have your effectful functions and effectless functions segregated by type signatures that flow all the way up to the top.

And you come up with async/await systems in traditional sync-semantics languages, and provide syntax to help with that - whether Javascript or Rust. In both Rust and JS, async functions are distinguished purely in the type-system, with some helpful syntax around it. In both, they desugar to `func(...): Promise<T>` or `func(...) -> impl Future<T>`. Those types too, flow "all the way to the top", imposing a colouring on code.

Erlang solved this problem by adopting the general semantics for erlang to achieve erlang's goals, and it seems that the "everything is async" semantics works there (I agree that it's a model that deserves deeper exploration).

Languages like Rust can't solve this problem the way erlang solves it, because it takes away from other fundamental aspects of the language. It would necessitate the inclusion of a very specific and very heavy (relative to Rust's expectations) runtime, along with lots of changes to the language that would diminish it along other dimensions.

My point about the design being mistake is the way you need to be explicit in invoking with await and the often harmful side-effect of forgetting the await expression (that in practice becomes a code-path fork that can become tricky to debug if the awaited function is a void function with no unexpected values coming out of the missed await expression).

For JavaScript the dynamic type-system really leaves no choice since the caller has no information about the behaviour of the callee, so await for these dynamic languages is kinda unavoidable (unless the entire language is green-threaded thatis).

For Rust and C# (and probably others) the compiler knows about this in advance so the "await" expression just adds an extra layer of work for the developer (that is also error prone).

There is little reason that for example the C# compiler couldn't have auto-inserted the await for all Task<> returning functions inside every (async) marked function and instead had a (direct) operator for the FEW cases where a developer really wants to handle the Task<> object manually instead of just awaiting it (C# is quite pragmatic in many other cases so the little loss of direct developer control would have been far outweighted by better symmetry and less ceremony).

Practical? Definitely, the Zig language seems to even handle this entirely under the hood by it's compile-time functionality. So for a developer the red-blue distinction is mostly something handler by the compiler despite not affecting the runtime more than some extra code i guess?

The challenges and confusions you mention are pretty fresh in my mind.

Maybe switching the default semantic behaviour over to recognize the async type signatures and implementing this sort of default syntax transform is the better way to go.

I wonder if it might cause problems with generic code - where a return type for some otherwise non-async function happens to "become a future" due to type-specialization, turning the function async? But maybe that's not an issue because the call-specialization behaviour just transitively applies to all the callers up the chain, same as any other specialization.

Beyond that, I think this whole conversation has a hard time moving forward before we get a good industry understanding of high-level concurrency structures in practice. So far the major one people fixate on is "actors", which usually reduces to "stream processor".. which is a nice construct but by itself not enough to build complete programs out of. You need an entire structural framework to talk about what produces the things the stream processes. Sometimes that'll be other streams, sometimes that'll be static, sometimes that'll be some collection of random event sources, some of which are direct and some of which are indirect. Then you need ways of talking about how those producers relate to each other, and _some_ way of identifying and preserving global / regional / local invariants.

Error handling is a nightmare. I haven't yet found any good vocabulary to talk about shutdown order for tasks and error handling in, how to express it clearly in the program intent, and how to make sure that intent doesn't rot over time. It's all ad-hoc promises and hooks and callbacks wiring up _this_ event to _that_ thing.. accompanied by small or large comments along with the hope that your fellow developer is cleverer than you and can actually figure out the intent from just the text of the code.

After we get a few good examples of those sorts of things, it'll be easier to go back and discuss syntax and semantics, and we'll have a reference set of designs/patterns to check against and make sure we can express cleanly.

This statement confuses me. What if I never spawn a process? What if the process I spawn makes function calls into other purely functional modules?

Therefore, every function is interruptible (implicit async), and calls to another module function introduce the breakpoint (implicit await).

You always have at least one process, the one running your code. Even if you don't spawn any more processes.

And the BEAM still needs to handle signals (like messages, traps, crash, ...) that could be received from another node at any point in time, therefore it needs to be able to interrupt your code, which will be, by design, interruptible.

The very nature of Erlang/Elixir is asynchronous by design, even if you don't really see it in the syntax (as it should be, because your concern is the algorithm/data exchanges, not the underlying execution model).

Most other languages (static or not) are equally explicit (returning a `Promise<A>` and not an `IO a` but it's very similar. Most differences are because call-by-need vs. call-by-value language semantics (Haskell being non-strict and the I/O not coming out of "thin air" but coming from somewhere - but that's kind of a different point than function color type)

For perfect ideal beings, yes. Humans are a bit less straightforward, even for the particular case of red-blue functions.

Can't disagree with this statement more. I've worked with a coloured language for a very long time (C#) and also recently worked a lot with Go and both have huge advantages and disadvantages. Go is not conveniently ignorant. The green thread model which Go uses is actually a very elegant solution where one can write simple code without leaking abstractions down the stack as it happens to be with red and blue functions. It's not ignorance. It's focus! Only the function which deals with async code has to know and think about it. Calling code doesn't need to adapt itself to the inner workings of another function which it has no control of. I quite like it and don't see any particular problem or traps with it. Quite the opposite, as a very experienced C# developer I see a lot of less experienced developers do very fatal mistakes with async/await as it's so easy to write code which can end up in a deadlock or exhaust resources and run into weird issues which only a person with a lot of experience would be able to spot and fix. I think Go's model is much more tolerant and forgiving to beginner mistakes which means that writing fatal code tends to be a lot harder and almost impossible if not done intentionally.

While Go's model is certainly a valid point in the design space and keeps things nice and simple, there are of course going to be many trade-offs. For example, low-level control is very much lost. There are no opportunities for custom schedulers, optimising how non-blocking calls are made, or easily dealing with blocking syscalls (or foreign libraries that call them). It would not be the right approach for Rust.

We are?

I feel old reading these kind of blog posts that assume that we all know what the latest unicorn has been spewing out of its butt. It's like I'm missing out on the greatest debate. On dung.

I'm sure it's valuable debate for those in the know. But this is just how it translates to some of us.

Async/await has been in the discussion for most popular languages (C# and JS first for the masses, Python, Rust, Java, and others) for a decade now.

And the colored functions analogy was a very popular post (and was subsequently referenced and discussed in lots of others).

Besides, HN is not some website about just knowing the core concepts used in the trade. There's Hacker Noon and others for that. HN has always been all about these kind of intra-group discussions.

(edit) Really, everyone should probably start by reading the 1978 papers "on the duality of operating system structures" and "communicating sequential processes", which, when combined with the appropriate background in the continuation passing style compiler transform (which I am pretty sure was also developed around the same time) could have hopefully prevented Ryan Dahl from accidentally causing the callback-hell "dark age" of software engineering from ever happening in the first place ;P.

https://cs.oberlin.edu/~ctaylor/classes/341F2012/lauer78.pdf

https://www.cs.cmu.edu/~crary/819-f09/Hoare78.pdf

For completeness and curiosity maybe. Otherwise one doesn't have to read the "actually-seminal papers" if they already know the concepts from the 40 to 20+ years that followed.

Do physicists need to read the original Einstein or Maxwell if they had read tomes of subsequent course and academic books on the subject, plus modern papers for the later developments?

In any case, I'm pretty sure the author of that post [1] had read at least the CSP papers -- he works on the Dart language team, and has written Game Programming Patterns and Crafting Interpreters, both quite popular books, which have been discussed (as in first page) more than 3-4 times in HN in the past years.

[1] https://journal.stuffwithstuff.com/2015/02/01/what-color-is-....

I guess maybe the issue in your comment is that my complex sentence with "is more than a bit frustrating" was mis-parsed, as you left off the direct referent of it and then seem to be arguing back at me with the same point I was trying to make with some emphatic irony (I was really hoping to provide three papers all from 1978, but there wasn't a perfect one on continuations)... to re-architect the center a bit:

> ...the idea that I must have read this one random recent blog post [--when I can't imagine this person has read the specific literature I think everyone should have read, as there was an infinite amount of it and a lot of it is equivalent to the rest: the only thing that makes this blog post exciting is almost certainly some kind of recency bias--] is more than a bit frustrating.

The point the original post is making is that in languages that support asynchrony using callbacks, promises or async/await, they've split the language in two: the "synchronous" and "asynchronous" parts, and reconciling them is a pain. The only languages which avoids this split comfortably are Go and a few other more niche ones (notably BEAM languages like Erlang and Elixir).

This post makes the counter-argument that it's actually a good thing that functions are "colored" differently, as synchronous and asynchronous functions have different execution models, and that should be reflected in the language.

[0]: https://journal.stuffwithstuff.com/2015/02/01/what-color-is-...

I was just put off from doing so because of the author's assumption that everyone in tech is on the same page when this isn't necessarily the case. Many or even most folks "in tech" give little thought to PL design, at least not in the way that would have them read that article, though influential it might have been to some folks in tech that are into these discussions 6 years ago.

This way, others in a similar situation to you get to benefit from GP's explanation. Moreover, GP got to interact with others on this concept.

I'd also wager it took less effort for GP to write this, and then for you to read it, than it would have taken you to do all of it yourself.

Point being, your question led to a very useful answer, and you should be proud, not sorry.

I think the post was great for making people think about this in an abstract way, but I don't think it was meant to reach beyond that mental exercise.

At the time I wrote the original article there were a lot of people really excited about the performance of async IO in Node (and JS in general) and were implicitly accepting the async programming style that went along with it. They liked the former so much that it muddled their perspective on the latter. Sort of like how when you go to a shitty diner after an amazing concert and recall the meal as amazing when really it's just because it got wrapped up in the emotional experience of the entire evening.

I had many many discussions of people where I tried to talk about how the async programming model is really cumbersome and their response was "but it makes the app go faster!" We were talking at totally different levels.

It can be really hard to get people to have a clear perspective on something they have an emotional attachment to. By making up a fake language and a made-up concept of function color, it let me show them "hey here's this thing that kind of sucks" without triggering their "but I love async IO in Node!" response.

I had no idea the article would catch on like it did. Something I find interesting about it in retrospect is that because it is kind of strange and abstract, readers take very different things away from it while assuming their interpretation is exactly the one I had in mind. Some folks (like the author here) think it's about effects and static types (even though there is nothing about types at all in the article). Some think it's a love letter to Go (a language I'm not particularly enthused about in general).

Really, I just wanted to talk about the unpleasant user experience of a certain programming style without getting hung up on what underlying behavior the style is attached to.

Also, I was one of the readers that managed to interpret your original basically the way you meant it. (One thing that helped is that I had previously read your criticisms of Go.)

I am curious though: what do you think about structured concurrency? It was your post that made me search for a better solution than async/await, and I am of the opinion that structured concurrency might be better. Am I wrong?

I could, but I'm never sure how much work I want to invest in maintaining and updating old content. I try to focus most of my (limited) time on making new stuff and, even though I'm generally a perfectionist, I think it's generally best to just keep moving forward.

> I am curious though: what do you think about structured concurrency?

I'm not familiar with it, sorry. My overall impression is that there is no silver bullet when it comes to concurrency. Humans aren't very good at thinking concurrently, so there's always some level of cognitive inpedence.

  @sync:  #00f;
  @async: #f00;

This function is a here is a rough function, this function there is smooth. This function yonder is damp, that one is dry. Careful, you are dealing with a sour function!

Wait, let me open up my blog

but Java still have checked exceptions which is also a form of coloring.

If you take a look to the history of Java, checked exceptions was there since the beginning but were weaken several times, first by the introduction of generics+inference (a type parameter can not represent an union of exceptions) then by the introduction of lambdas, the type of a lambda (the function interfaces in Java lingo) used by the standard lib, do not declare checked exceptions.

It's so bad that IOException, the king of checked exception, has two unchecked dual defined by default, UncheckedIOException and IOError that are raised randomly by the new APIs like java.nio.file.Files that do not want to deal with the complexity of checked exceptions.

Java has a checked exception problem but it does not want to recognize it.

BTW, there is another coloring in Java which is primitive vs object, but at least this one is recognized as an issue (see OpenJDK project Valhalla).

"Project Loom intends to eliminate the frustrating tradeoff between efficiently running concurrent programs and efficiently writing, maintaining and observing them." [0]

http://cr.openjdk.java.net/~rpressler/loom/loom/sol1_part1.h...

But it already works well, you can try it on the project’s branch. But there is no official timeline, it will be ready when it’s ready.

Don't know about go, but in elixir "We have to use Task.await", is like, you can use it if you want exactly those semantics, just because it has await and task in its name doesn't mean you have to - although simple in usage it's quite a higher level abstraction inside the BEAM - the base unit is the "process", and you can make any sort of asynchronous or synchronous flow on top of that.

But even with Tasks, they can be called inside other processes and then instead of having to wait for all tasks to finish they'll be delivered one by one to that process mail box and you can decide at all points of execution what to do. Of course if you're running your code like a sequential block... You'll have to wait until they sequentially finish?

The whole async/await thing is just a convenience feature that looks great but distracts you from thinking about the true concurrency of your program.

This is a massive over-generalisation. In particular, you're making a huge assumption that looping is possible, leaping from there to having loops be desirable, then pole-vaulting to claim that anything not architected entirely around looping is "not well-designed".

Based on this generalisation, I'm going to guess your experience is limited to languages which (a) have some sort of looping construct built-in (e.g. a `for` or `while` keyword), (b) do not eliminate tail-calls and (c) either don't support, or discourage (perhaps until recently) higher-order programming. This would include languages like C/C++/Go, Python/PHP/Perl, Java/C#, etc.

In contrast, I consider looping to be a problematic, error-prone construct in general (for reasons outlined at http://lambda-the-ultimate.org/node/4683#comment-74302 ). Recursion is much safer, in languages which eliminate tail-calls.

I also consider "main loops" to be an anti-pattern (for reasons outlined at http://disq.us/p/kk1da6 ). They break all sorts of modularity/encapsulation; force bizzare control flow patterns (e.g. lots of highly non-local 'returns', AKA unlabelled GOTOs); force us to make arbitrary (yet highly impactful) decisions between 'carrying on' or 'trampolining off main' (these decisions are often hard to change after-the-fact, regardless of their suitability); require lots of boilerplate to marshal and manage intermediate values which, by their nature, must conflate all sorts temporary values from the entire codebase into amalgamated blobs; and so on.

Rather than shoehorning important logic and decisions into a global, manually-kludged tail-call trampoline (AKA a 'main loop'); I find it far easier to write, reason-about and debug simple code which performs a 'step' then tail-calls to the next.

As far as concurrency goes, I would claim that "well-designed programs" with control following data flow, it's usually pretty trivial to spin off parts as Futures/etc. as needed. A type system will tell us where to place the necessary calls to "map", etc. as needed.

A "main loop" doesn't necessarily imply an actual loop. A game written in Haskell or Scheme probably has a main loop implemented via recursion, but it's still called a main loop. "Main tail-calling function" sounds somewhat ugly.

>Far better to write functions which perform a 'step' then tail-call the next function.

This sounds like a recipe for spaghetti. The "main loop" approach is much more conducive to the clean approach of "functional core, imperative shell" (https://www.destroyallsoftware.com/screencasts/catalog/funct...), where IO, asynchrony and the like are cleanly separated out from business logic.

If such a 'main loop' is a single function which (a) dispatches to one of a bunch of 'handlers' then (b) calls itself recursively, I would still call it a 'main-loop anti-pattern'.

More formally, it's likely to be in "defunctionalised" form https://blog.sigplan.org/2019/12/30/defunctionalization-ever...

The problem with such 'main loops', whether actual loops or self-recursive functions, is that control flow has to keep doubling-back to this 'central dispatcher'. In the case of actual 'main loops' this is often an unavoidable consequence of lacking tail-call elimination. In languages with tail-call elimination, we're just over-complicating the problem.

> This sounds like a recipe for spaghetti.

The control flow for a 'main loop' program goes like this:

    main function     fooHandler     barHandler
        |
        +------call-------+
                          |
        +-----return------+
        |
        +------call---------------------+
                                        |
        +-----return--------------------+
        |
       ...

    main function     fooHandler      main function    barHandler    main function
        |
        +----call--------+
                         |
                         +-----call----------+
                                             |
                                             +------call----+
                                                            |
                                                            +----call----+
                                                                         |
                                                                        ...

> The "main loop" approach is much more conducive to the clean approach of "functional core, imperative shell"

I don't see why. The "functional core, imperative shell" approach is mostly about writing as much as possible using pure functions, then 'lifting' them to the required effectful context (via map, join/>>=, etc.); in contrast to the common approach of writing everything within the required context to begin with (e.g. making everything "IO" just because we need one 'printLn' somewhere).

That doesn't require manually writing a global "main loop". For example I like to use polysemy, where all the business logic can remain pure, and the `main` function just declares how to 'lift' the application to `IO`. Under the hood it uses a free monad, which dispatches to 'handlers'; but (a) that's all automatically managed, and (b) it recurses through calls spread across the application, rather than returning to a self-calling "main function".

That's backwards IMO. The first example is structured, hierarchical, organised. The second example is spaghetti because there's no difference between returning and calling, so all you see is a bunch of functions connected to a bunch of other functions.

> Hence we can avoid special-cases and handler-specific checks migrating their way into the 'main function'; even if we want such things, we don't need to pollute the 'main function' itself (or defunctionalised intermediate data) with such cruft;

It's the opposite of cruft; the whole point of defunctionalising is that you replace invisible control flow with a reified datastructure that you can actually inspect and understand. You can properly separate the concerns of "figure out what state we're in" and "actually execute the things that need to happen now" rather than sprinkling in special-case tests at random points in your execution.

It embraces a defect introduced for BC reasons as if it's sound engineering. It really isn't.

If you start over, there's no reason any blocking API should really BLOCK your program. Your functions can look completely synchronous, but yield to the event loop when they invoke a blocking API.

Basically like Erlang works.

Eventually it will be the case, but these kind of "love the flaw" attitudes are slowing us down.

This require either runtime-based or operating system support. If need be, you can always rely on the OS scheduler to be your "async blocking API".

Well, the original red-blue function article did say "just use threads", didn't it?

That implies a level of sophistication and abstraction that simply doesn't exist today, and probably won't for a while.

For this to work you either need an ultra-sophisticated compiler what will transparently auto-async your code at compile time (as realistic as a unicorn at this point) or a heavy runtime which will have performance implications.

So in the meanwhile I agree with TFA that coloured functions is probably the pragmatic choice if you want pay-for-what-you-use async right now.

Personally my main issue with async is that it seems to me that a generation of devs who grew up on JavaScript see it as the Alpha and Omega, the one and only way of handling any sort of parallel or event-driven system, not considering other approaches such as threads and/or polling. This sometimes results in spaghetti code that I personally find over-engineered.

To give you an example I hacked on rbw a week ago, a Rust command line client for Bitwarden. It's a good program, but it's completely built with async code and I genuinely don't get it given that it's almost entirely synchronous in practice (you query an entry, you enter your passphrase, you get a result). As a result I found it a lot harder to follow the flow of the program and understand how it all fit together. But hey, I eventually got it and managed to fix the issue I was debugging, so it's not the end of the world.

>For this to work you either need an ultra-sophisticated compiler what will transparently auto-async your code at compile time (as realistic as a unicorn at this point) or a heavy runtime which will have performance implications.

I may be misunderstanding this point, but if I'm not, it seems like gevent managed to do this for Python a decade ago and it still works well today. I still use gevent for new code because it gracefully sidesteps this function-coloring problem. (I know people don't normally associate "monkey-patch" with "graceful", but they somehow made it work. I guess this falls under the "heavy runtime" approach, but it doesn't feel heavy at all.)

True, but is this not also the case when using Python's native async/await functionality?

The difference is that async/await doesn't even work with previous pure-Python code which performs any I/O, either. There are thousands of ordinary third-party libraries which basically have to be tossed out the window and rewritten from scratch, despite none of them using any C extensions.

gevent works well with almost all of those old libraries.

Yes indeed.

> The difference is that async/await doesn't even work with previous pure-Python code which performs any I/O, either. There are thousands of ordinary third-party libraries which basically have to be tossed out the window and rewritten from scratch, despite none of them using any C extensions.

Hmm, yeah. I sometimes wonder about a language style where functions are monadic by default, and I guess being able to monkeypatch the internals is a way to achieve the same thing - rather than introducing a syntax for a "programmable semicolon", we can just monkeypatch the actual semicolon. Introducing effectfulness into code that wasn't written to accommodate it worries me, but I guess gevent usually works in practice. I guess a language like Frank is a more principled way of doing that kind of thing.

And the external API they provide is really clean; you generally don't have to think about any of the internals. You just use everything exactly like you already would and pepper in straightforward abstractions like gevent.Pool and gevent.spawn. Internally, it might be the antithesis of Pythonic, but the external interface and developer experience is very Pythonic.

you don't [1]

[1] https://www.gevent.org/api/gevent.html#gevent.sleep

> That implies a level of sophistication and abstraction that simply doesn't exist today, and probably won't for a while.

This sounds like a description of UNIX and was true from the very first versions: a blocking call would trap the system, select a process task that can continue and jump to that instead. At the language level, I'm pretty sure we had this in the 1980s in Forth, and I don't think it was considered new even then (although I don't have a reference handy); libdill does this for C programs.

Or am I misunderstanding what you're saying?

OS-level async is what I accuse some devs to underuse, preferring language-level async constructs. I've read a lot of frankly FUD and cargo culting about how inefficient things like pthreads are. Let's not even talk about pre-fork parallelism which is probably the simplest way to use multiple cores when dealing with independent tasks but hardly anybody seems to use these days. I wonder if most devs even realize that it's an option.

In my experience coroutines were never really popular in the C world simply because without GC or at least a borrow checker it's just very hard to write correct code. Avoiding memory violations in C is tricky, avoiding memory violation in async C is asking a lot.

Rust does a lot better here thanks to borrow checking but it still leads to, IMO, very complicated code.

Speaking from experience: they don't. I recently picked up a Linux kernel development book [1] and after four or five chapters I was able to eliminate several bottlenecks and eliminate latency edge cases in personal projects with a greater understanding of the scheduler algorithm and process priority/affinity/yields.

I think the problem is that the vast majority of developers work on top of cross-platform runtimes that hide any detail that can't be easily implemented for all operating systems. It's been more than a decade since I've used Unix sockets directly in any language and I don't how to set SO_REUSEPORT in Rust. I'd probably have to implement it using `TCPListener::as_raw_fd()` and `unsafe libc::setsockopt`? It's just easier stay sane within the bumpers.

[1] https://www.amazon.com/Linux-Kernel-Development-Robert-Love/...

> you need an ultra-sophisticated compiler (as realistic as a unicorn at this point)

Isn't what Zig does, described in the post linked by a few other comments [1], basically this?

[1] https://kristoff.it/blog/zig-colorblind-async-await/

Depends on how much of it you want to solve automatically. The Zig compiler does essentially auto-async your functions. So I think you can write a whole lot of synchronous code, and then just have a single "async" at the top to have it yield to the event loop whenever it hits blocking IO.

So I think you can solve 90% of that problem without being ultra-sophisticated.

Automatic yield points and a proxy to syscalls are used for that.

It's also still plenty fast (as opposed to Erlang, which was mentioned in other comments).

Therefore, you would need to deal with far more race conditions.

Even in single-threaded languages, your entire state may change during a single asynchronous function call. By differentiating between synchronous code (i.e. code that needs to be run in a single slot) and asynchronous functions, you can immediately see where you need to account for unexpected state changes.

(Yes, it's more complicated in multithreaded languages, but the main idea still holds because they usually all sync back to a single thread)

Like the article points out, then you would need green threads to do asynchronous work simultaneously. And if you are already using threads then there is no need for "async". The point of "async" is to give you the ability to efficiently manage the event loop WITHOUT threads.

> Is it unnecessary performance optimization to parallelize multiple adjacent asynchronous functions?

The important word in your question is "async". Take that out and the answer is "no" - it's not unnecessary to parallelise. But the example serves well to illustrate at least one aspect of why red/green is wrong in my opinion: who decides what should be able to run concurrently?

Let's modify the example slightly, and assume we have a function `size(url) -> Int` that returns the size in bytes of a given URL. It's making network calls, so it'll have latency. Should the function be async?

As the developer of that function, I can't reasonably make that choice beyond "probably". But it comes with a cost. The caller can't now just call it: they have to await, and perhaps unbox the answer. If they're only calling it once then you've just added unnecessary complexity to their code.

The _callee_ is the one who's appropriately placed to decide. Using the example, we're calling it 3 times. So, as the developer writing the _calling_ function, I can decide that it's appropriate to paralellise.

To use GP's example, Erlang gets that the right way round. Functions are functions are functions. If I want to call a function synchronously, I just call it. If I want to parallelise, then I simply spawn the function instead.

> or should we just hope that the runtime/compiler gets smart enough to know what is safe to parallelize?

That would be a step on further still. It's definitely doable - compilers already do a buch of pretty sophisticated dataflow analysis.

But even today, the red/green async model is a very limited thing. With significant constraints. Other than perhaps rust - with its focus on zero-cost abstractions - it feels at best an interim step that asks programmers to compensate for a lack of good fine-grained concurrency support in language runtimes.

Cilk requires annotating async functions calls at the call point (with the spawn keyword to make them async), but not the funciton themselves, so there is no red/blue coloring.

Synchronization is explicit via the sync keyword, but in principle I think a compiler could insert the sync point on first use of the value.

I don't think a cilk style runtime would be heavier than an executor behind async/await.

> The _callee_ is the one who's appropriately placed to decide. Using the example, we're calling it 3 times. So, as the developer writing the _calling_ function, I can decide that it's appropriate to paralellise.

The right answer is polymorphism, and it's easy in a language with HKT (I do it in Scala all the time). If you make every function implicitly async (or possibly-async at the runtime's discretion) then you lose the ability to know where the yield points are, to get higher performance from blocking where appropriate, but more than that you just don't get consistent performance behaviour.

   Functions are functions are functions. If I want to call a function synchronously, I just call it. If I want to parallelise, then I simply spawn the function instead.

Why not put the async idea on the caller side alone, as suggested and as seen in go?

fetch(url) // synchronous

go fetch(url) // asynchronous

if the compiled code of synchronous and async functions really has to look different for fundamental reasons, the compiler can still silently emit both versions (and gc unused variants)

Someone please help me understand, or rid the world of this async-await mess.

> the compiler can still silently emit both versions

I think that is zig's strategy under the hood (not 100% sure)

1. The function does not have a suspend point, so it is emitted as a normal function, and it just gets called like a normal function at the async callsite. Await is a no-op.

or...

2. It has a suspend point. It is emitted with the async calling convention, and any normal callsites to it also become suspend points.

`go fetch(url) // asynchronous`

This is not something you can readily abstract over. That depends on whether language is compiled vs interpreted, has a runtime or not, is scheduled or not, stack model, threading model, ownership model, and so on. `go myfunc()` implies something to dispatch, execute and reap that function and its metadata, and there's no one-size-fits-all solution.

You can design a language that has this feature, but it's not without tradeoffs, and it's certainly not something you could just introduce to a language unless it already lends itself to it.

Cilk is designed for parallelism more than for concurrency, but with a runtime augmented idle cpu detection, it would work perfectly for async io.

Thread-based code (CSP) is, after all, the most well studied model of parallel computation, and for a reason. Async code which achieves real parallelism may get much harder to interpret, especially if it's non-trivial (e.g. requestA.then(requestB).then(requestC); in parallel, requestD.then(requestE); and mix this up with mild processing between A and B and C, and between D and E - you can easily get to race conditions in this type of code even in single-threaded runtimes like NodeJS).

Huh? Traditional threading is not CSP; threads are generally defined as sharing memory. Async is much closer to the CSP model.

The main difference is that in CSP (and thread-based models) the thread of execution is explicitly represented as an entity in the code. In async models, the threads of execution are mostly abstracted away and implemented by a runtime scheduler, while communication is done purely by function return values.

Of course, you can typically implement any of these models in terms of each other (e.g. you can easily implement CSP channels in a language which offers mutexes and shared data; or you can implement them with async/await primitives). But the base support in the language and the patterns you'll see emerge in idiomatic code will be different between goroutine/CSP, shared-memory parallelism, and async/await.

  Threads + some kind of channel (a la Erlang processes or Go's goroutines) are pretty much direct programmatic representations of CSP.

afaiu erlang is based on non-sequencial processes more like the actor model, there is no "communicating sequencial" process at the base of it (though you can implement csp in actor languages) afaik

here you can see joe armstrong and carl hewitt fundamentally disagree with sir tony haore (csp) and sequential channels as a fundamental construct [1] https://www.youtube.com/watch?v=37wFVVVZlVU&t=720s

again, apologies if im arguing something that wasnt implied, it was just something i noticed ^^

This approach tends to become a lot less simple if you have any sort of significant resource usage to manage: you’ll be fighting all of that magic when you have non-trivial memory usage, need to avoid limits on file handles or connections, etc. not to mention the runtime support for things like debugging.

Any time you see a long-running widespread problem and think there’s an easy fix, ask whether you’re really that much smarter than an entire field or whether it might be harder than you think.

   db = connect_to_db()
   result = run_query(db, …)
   result2 = run_query(db, …)
   result3 = run_query(db, …)
   return build_response(result, result2, result3)

The same problem arises for memory usage: if you allocate memory it's going to stay allocated until the last async response arrives and if that's non-trivial it means that the async work might not be worth the hassle because your concurrency is actually RAM-limited below the point where you see any benefit over synchronous execution.

One common challenge here is that it's usually quite easy to miss this being a problem no matter what style you use if you don't carefully test with real data volumes or under perfect conditions. For example, if my code looks like this:

   result_1 = run_query(db)
   result_2 = call_other_service()
   return build_result(result1, result_2)

There are, of course, ways to guard against this but then it's a lot harder to have a language make synchronous-looking code run asynchronously and leaving out async/await isn't saving you much if you have to sprinkle synchronization primitives all over the place. That doesn't mean you don't do it, of course, but it means there isn't a shortcut.

https://github.com/rust-lang/rfcs/blob/0806be4f282144cfcd55b...

Haskell does that, and it is great. But there is no chance at all that I would want anything like that in Rust. It would be completely out of place, and break the entire concurrency model.

That would make the language unfit for any task I use it for today.

As if you matter? The conclusion could be stripped, and the article would still make its point.

I interpreted the points in the article as fairly generic. The idea of "Inconvenient knowledge is better than convenient ignorance" goes well beyond Rust.

The author of the article does imply that Rust does a lot of things right, but I'm curious how one could say "Hey, I like this language it does a lot of things well and fixes problems I see in other languages" without sounding like a snob.

Just not writing “Feel free to create and use languages with invisible program effects or asynchrony, and I'll feel free to not pay any attention when you do.” would do the trick in this case.

Or, generally, show your reasoning, show your arguments, acknowledge any tradeoffs, contextualize against other languages as needed. And perhaps do not jump into every programming language discussion with “this wouldn't be an issue with Rust” unprompted, as that's generally not constructive, and does not acknowledge that there is more to software engineering than just what programming language you pick.

I'd say maybe conclusion was the point. The author clearly said others who wrote Rust articles were not forceful enough that Rust color functions are actually good thing. So they needed to make position clear.

That way it doesn't sound like the author's opinion is what matters.

and with this being true, the article's argument does not work, there is no types-mark-io happening there.

the argument applies better to Haskell, where you cannot do any IO without the types saying so.

interestingly, it also applies somewhat to Javascript, where because of backward-compat and history, it's not possible to do sync IO (mostly :) welcome to javascript), so if you do not see an "async" annotation, you can be mostly sure that this is pure cpu-work.

Rust allows to simply spawn threads, right? So a function not being marked 'async' doesn't necessarily give any guarantees about potential side effects.

Spending most of my time writing Elm, having side-effects present in the type system is great. But colored functions in Rust (as well as most other async/await langs) only creates a devide between functions that provide a specific kind of asynchrony, and those which do not.

So async/await improves the syntax of writing a specific kind of async code, at the cost of composability (the red/blue divide). It does not, however, make any guarantees about the asynchronous nature of your code.

I may be wrong but most of the C calls in Python do not have callbacks and you can pin the coroutine to the kernel thread if there is a callback.

https://kristoff.it/blog/zig-colorblind-async-await/

I highly recommend watching Andrew Kelley's video (linked in that article) on this topic:

https://youtu.be/zeLToGnjIUM

Sadly I think that their disregard for safety (the manual states that dangling pointers is the developers problem) kinda makes it a no-starter for many people. I personally consider possible information breach far worse than a crash or runtime stall.

Your opinion seems to be that any future systems language that doesn't implement a heavy Rust-style borrow checker and explicit safe/unsafe modes "disregards safety"?

Zig does have a lot of checks, such as bounds checking[1]. There are also test modes that help catch errors. I don't know what you're referring to about "information breach".

> The manual states that dangling pointers is the developers problem...

In a systems language where you can cast an int to a pointer:

    const ptr = @intToPtr(*i32, 0xdeadbee0);

Zig made the design choice to not split the world into safe vs unsafe. It seems a bit unwarranted to say that because they didn't make the same design choices Rust did that they have a "disregard for safety".

[1] https://ziglang.org/documentation/master/#Index-out-of-Bound...

As for the exact methods I'm not really partial to anything (Be it borrow-checker,gc,compiler-time-analysis,etc) but we've seen time and time out of bounds(handled by Zig atleast), UAF and other issues be exploited so having "safe" defaults for the majority of code isn't something I think we should skip on, especially if it's a "systems language" since the exploitation surface will end up everywhere.

The Rust borrow checker can probably feel cludgy at times, and in many ways it's a result of creating a scheme that is verifiasble. Since Zig already has comptime I guess having expand compile time capabilties to analyze for common UAF conditions,etc shouldn't be unfeasible?

I wouldn't call it a disregard. Zig's approach is to give you easy and automatic tools to detect and handle most of those memory safety problems in its language-first-class test system, which you can think of as a carrot to get you to write tests.

That's not the case for Zig. There's no special decorator, no block_on/await distinction, and regular calls will return the expected value by implicitly awaiting (in case evented mode is enabled). A callsite may decide not to implicitly await the result by using the async keyword, which then requires an explicit await on the returned promise.

edit: fix caller/callee typo

> there’s plenty of applications that need evented I/O to behave correctly. Switching to blocking mode, without introducing any change, might cause them to deadlock

    a = x + z;
    b = y + x;

With async functions, the CPU isn't actually doing anything. It's just waiting for something to happen, like a network response from another machine. It's almost always faster to do them in parallel if possible.

Don't take the example too literally, some function calls can be here.

Running computations in parallel is often valuable. Or run computations in parallel with waiting for external resource - why does not the code in the article compute something while waiting for a, b and c?

Anyways, if async functions are so good, why not have all functions async?

The article says this a kind of "documentation" that tells you what functions can wait for some external data and what functions are "pure computation". If it was so, it would be OK. Such a documentation could be computed automatically based on the called function implementations and developer is hinted: "these two functions you call are both async, consider waiting for both in parallel". In reality, the async / await implementations prevent the non-async functions from becoming async without the calning code change. This restriction is just a limitation of how async / await is implemented, not something useful.

As other commenter says, the article "embraces a defect introduced for BC reasons as if it's sound engineering. It really isn't."

When my code is called by a 3rd party library, I can not change my code to async. That's the most unpleasant property of today's async / await. What yesterday was quick computation tomorrow can become a network call. For example, I may want the bodies of rarely used functions to only load when called first time (https://github.com/avodonosov/pocl).

The article suggests we have to decide upfront, at the top-level of the application / call stack, which parts can be implemented with as waiting blocks and which should never wait for anything external. This is not practical.

> It's almost always faster to do them in parallel if possible.

Yes, if possible. And it is determined, in particular, by dependencies between the operations. If the first call is writing to DB and the second call is a query that reads the same DB cell, they can't be parallelized. The same is with in memory computations - if next steps do not depend on previous steps they can be parallelized. Hopefully, compilers of the future will be able to automatically parallelize computations.

They aren't. They're only good when the primary thing that the function is doing is waiting.

> When my code is called by a 3rd party library, I can not change my code to async. That's the most unpleasant property of today's async / await. What yesterday was quick computation tomorrow can become a network call.

You can still do that. Network calls don't have to be async. Blocking network calls still exist, just as they did before async was created.

> Hopefully, compilers of the future will be able to automatically parallelize computations.

Ah, yes, the famous sufficiently smart computer that can determine if two operations happening on the other side of the network depend on each other or not. Personally, I'm not holding my breath.

Browsers have already deprecated sync XMLHttpRequest.

Needing to add async to a function is problematic in JS, but it's far less problematic than the alternative which is completely rewriting your function to be event-driven. Blocking network calls aren't a possibility in JS due to the choice to make the entire runtime single-threaded.

In my opinion the argument doesn't really apply to async await when you get down to the details, but I need to think about it more to be able to articulate why I think so with precision.

[1] https://kristoff.it/blog/zig-colorblind-async-await/

However, and maybe this is what you're alluding to with your mention of allocators, it seems that Zig has "colored" functions in its own way, with regard to allocation! If a certain function allocates, then to use it the caller will need to pass in an allocator, and that caller will need to have been given an allocator, etc. In other words, explicit allocation "infects" the codebase, too, right?

I can't help but see the parallels there. I think Zig's stance on explicit allocation is one of its strengths, but it does make me wonder if some compiler trickery can make it colorblind as well.

- declare a global allocator for your program and call it implicitly everywhere. This is very accepted by the community as bad practice for libraries but if you are writing end product who cares. In the end in most cases you will have to bootstrap your allocator against a global allocator anyways, and usually that in some form or another std.heap.page_allocator.

- bind an allocator into your struct when you build it. This makes the allocator "implicit" instead of explicit.

The important criteria about red/blue functions that a lot of people forget about is that one of the seams between the two colors is missing. As far as I can tell, for all of "red"/"blue" ish things in zig, (async, allocators, errors) there are clear and documented ways of annealing those seams in your code.

Then "Rust async is colored, and that’s not a big deal" https://morestina.net/blog/1686/rust-async-is-colored

And now it's a good thing.

Honestly, this sounds like somebody struck Achilles' heel of Rust and it hurts. They're neatly covering 3 stages of grief over these 3 articles, and I wouldn't be surprised if we could uncover the other 2 in the blogosphere somewhere.

A good example would be lifetimes. Once you introduce lifetime parameters into a data type, it starts to introduce ballooning complexity into everything that touches it.

I've run into cases with Rust where I refactored a significant portion of a program to avoid nested references and explicit lifetimes in a way I haven't had to in other languages

a) Accept the performance hit that comes with GC or defensive copying.

b) Convince yourself that developers can keep all those potentially dangling references in their heads at all times.

c) Use something like Rust lifetimes.

My preference would be (a) whenever possible, (c) when absolutely necessary, and never (b)