Rant over. `for await` and `async for` to me seem like a fairly natural extension once you understand async. I think the problem comes in where once you run into async the first time you are forced to fully grasp it (and potentially rewrite a ton of your code) to move forward which is really counter to the rest of the Python learning experience.

[1] http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...

With Async, people are still figuring things out.

I've encountered a bunch of code bases that are polluted with async when a better design could have removed it completely.

That is, there's the way to run one statement, then the next in the synchronously-colored portion of the language:

    oneStatement()
    thenTheNext()

    await oneStatement().andThen(theNext)

This does not, on its own, mean "async" is bad. It is a complicated decision with many pieces. But this particular aspect of it means that there will always be a barrier to a programmer encountering this for the first time, simply by virtue of adding a new dimension they have to constantly account for to constructs that they're still getting a handle on.

(Having languages that don't color this code different has its own tradeoffs, and its own challenges. One of which is precisely because there isn't the separate color in the language itself, it's much easier to blunder into problems without even realizing it! People posting "why is this code broken?" to /r/golang, getting the answer "it's a race condition", and then the poster replies back with basically "what's a race condition?" is at least an every-couple-of-months sort of thing. I don't think there's an easy way to introduce concurrency to new programmers, honestly.)

Golang gets it right with goroutines. Languages should have extremely light and performant concurrency that efficiently map to OS threads. And the developer should only care about synchronization, not personally accounting for every possible context switch.

Python, as a language, has a lot of reliance on global context and a lot of dynamic functionality. Imports happen at runtime, not compile time. You can modify the attributes of a class. You can reassign a function. And so forth. You can do nonsense like this, where a isn't even defined until the second-to-last line:

    $ python3
    >>> def f():
    ...  yield 5
    ...  yield a
    ... 
    >>> x = f()
    >>> next(x)
    5
    >>> a = 10
    >>> next(x)
    10

https://glyph.twistedmatrix.com/2014/02/unyielding.html (which predates Python having proper async-await support by a few years, and references the "Tulip" project that became asyncio) makes a good argument that "Don't worry about it" doesn't work and "Make the programmer explicitly acknowledge concurrency" is fine in practice.

Anecdotally, my (limited) experience writing concurrent Python code in Trio has required no learning of the intricacies of coroutines and the implementation and also relatively easy-to-follow code, but yes, with explicit references to concurrency. Also, anecdotally, since my performance problem is just "I would like to parallelize multiple HTTP requests" and not "I am an OS scheduler and need to optimize every cycle" (in which case I wouldn't be writing Python), I'm happier writing code that accounts for context switches but in turn never has to care about locking than trying to properly manage locks and lock ordering and think about types of locks and all that.

Yes you can do very magical things with Python, and no they're not very safe, but at least that level of magic requires some level of skill, and the assumption that with that skill comes a responsibility for putting up warning signs and providing guidance. But most of the time, people should not be doing magic, and if they are, it should be hidden so completely, that it appears no magic is actually happening.

Anecdotally, my experience with working with engineers who are trying to understand new async code, has been dismal. I used to think "oh, these engineers just don't want to put in the work to understand what is happening in the codebase." But it's actually hard to explain what is happening to other engineers who have not been "initiated" in async. So they end up writing broken solutions and then feeling frustrated, because there's a resistance to the nature of the abstraction. Now you could argue that they should just hunker down and learn more, but personally I think if new abstractions do not come with a natural intuition, then they are not good abstractions.

There are plenty of languages with both async model and multicore runtime support. For example Kotlin on JVM. You still need to synchronize on shared resources (or use other patterns like message passing).

Async does not magically save you from synchronization. It's just a quirk of node.js implementation.

In Python land explicit is better than implicit as this is cooperative multitasking after all.

It's also worth reading through https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

Context switches remain expensive.

I'm kind of on the fence about coroutines. They can sometimes simplify writing performant IO-dependent code, or can make it harder to debug. It feels that they've not reached their "final form", nor has the right set of structured programming norms developed around them yet.

Then people start trying to apply coroutines to a bunch of arbitrarily complicated and unbounded business logic things and everything blows up.

https://github.com/Qbix/Platform/blob/01604218d06ed158c921ab...

(There are many other cool things in that js file btw)

This is already possible with:

  await Promise.all(arr.map(async item => {
    await item()
  })

Is anyone working on a proposal for this?

  async for (const item of iterable)
    await item()

  for (const item of iterable)
    async do {
      await item()
    }

  Promise.all(arr.map(async item => {
    await item()
  })

    for await (bar of foo) { }

    await Promise.all(
      arr.map(item => item())
    );

The only case where you would need to await inside the mapper function is when you would further process some asynchronously returned value. This seems like it would add bloat and do unnecessary sync processing inside the async function. It is highly advisable to separate the concerns using function composition, e.g.

    await Promise.all(
      arr.map(async item => handleItem(
        await item()
      ))
    );

One might be tempted to use the thenable interface (don’t!):

    await Promise.all(
      arr.map(item => item()
          .then(handleItem)
      )
    );

No.

    async item => {await item()} // Promise<void>
    item => item() // Promise<ReturnValue<typeof item>>

> The only case where you would need to await inside the mapper function is when you would further process some asynchronously returned value

Which is what map is for. Also, once again, that was just how the async loop can be implemented. If anything you're saying that it's "a misuse" of map and thus we need an "async for"

    >  arr.map(async item => handleItem(
    >    await item()
    >  ))

> await arr.map()

That's actually a good QoL improvement, but it doesn't address either of the points I'm talking about (only works on arrays + it requires a function)

> but it actually only waits for the promises returned from item() and not the .then() part

Wrong. The mapper function returns a Promise that resolves with the return value of handleItem. `await a.then(b).then(c)` awaits the 3 promises in row.

To achieve that race condition, you should not return nor await the return value of `then` inside the map method:

    await Promise.all(
      arr.map(item => {
        const p = item();
        p.then(handleItem); // "Race condition"
        return p;
      })
    );

> One might be tempted to use the thenable interface (don’t!):

The code that followed was fine and it had no race condition.

> Seems like it would work, but it actually only waits for the promises returned from item() and not the .then() part!

This is the wrong part. The blockness is irrelevant.

My belief is that async is going to complicate things. I do not foresee myself using it. This is just not a problem point I’ve wanted to solve enough that I’ve wanted the two color problem. And I find coroutines quickly difficult to follow.

I’ve been learning Elixir these last few months. When I need to solve the kinds of problems async supposedly address, I find the basic fundamental nature of Erlang/Elixir so much better suited to this.