Terrible idea. Not only is it completely counter to the python zen koan of "explicit is better than implicit", you really don't want implicit concurrency without some sort of zero-cost abstraction and really strong guarantees against side effects.
> Rationale: Python is designed to be easy and simple. Concurrency in general is not that.
Python is way too dynamic for that. Furthermore, you have a syscall at the start of the loop, and one syscall per loop. Even if you did this in Rust, it probably would not give you the true results you wanted, if that file tree were manipulated in any sort of way during the loop, was a network FS, or any of a number of other assumptions were violated.
That's exactly why automatic concurrency is a bad idea. Now you have no idea if your code is executing in "simple mode" or "complex mode".
[Foreword: I agree that it would make no sense for any language to automatically parallelize loops, especially over filesystem mutations. However I am very happy manually doing it with one function so it's explicit when it happens and maintainers know to take it into it account.]
> Even if you did this in Rust, it probably would not give you the true results you wanted, if that file tree were manipulated in any sort of way during the loop, was a network FS, or any of a number of other assumptions were violated.
That's factually true but not really relevant.
Introducing your own parallelism barely adds to this problem. Your program cannot assume it is not parallel with another program affecting the same file tree in arbitrary ways.
The parallelism you add to your own code (say, with Rayon) doesn't change the set of operations that can occur to those files. If you design your own operations to be non-overlapping then parallelism is no concern, and if you didn't then that's a bug and you might get non-deterministic results even without parallelism.
For what it's worth, even for writing that kind of code, Rust's standard library has taken the recent wave of TOCTOU vulnerabilities much more seriously than contemporaries like C++ implementations. Good writeup here: https://www.reddit.com/r/cpp/comments/151cnlc/a_safety_cultu...
This of course only makes safer operations possible in more cases. Code still has to do what it can with the constraint that filesystem contents may change in arbitrary ways at arbitrary times. Adding your own parallelism does not add significantly to that. And if one is to add parallelism, it may as well be with something as high-level as Rayon and with all of the usual data race protection Rust provides, so that you do get to focus on filesystem concerns and not also on thread-safety in general.
> Rationale: Python is designed to be easy and simple. Concurrency in general is not that.
Python is way too dynamic for that. Furthermore, you have a syscall at the start of the loop, and one syscall per loop. Even if you did this in Rust, it probably would not give you the true results you wanted, if that file tree were manipulated in any sort of way during the loop, was a network FS, or any of a number of other assumptions were violated.
That's exactly why automatic concurrency is a bad idea. Now you have no idea if your code is executing in "simple mode" or "complex mode".
reply