Hacker Read top | best | new | newcomments | leaders | about | bookmarklet login

Did you use WAL?

But in general, funelling writes into one thread is the way to get around it



sort by: page size:

that's exactly the case; make sure only one thread writes, but how you feed data to that thread is up to you.

Ummm, if it's going to be hit by more than one thread, you should lock it. I don't see how this is GCC's fault, wait till you learn about write barriers.

This can't be right. As far as I can tell, WAL allows concurrent READS and WRITE, not concurrent WRITES. Am I doing this wrong all these years?

That's to make it thread safe without the GIL.

If you only care about single thread there's all kinds of stuff you can do.


That's not too bad assuming that only that thread is affected.

I can't think of another way to do it that wouldn't effectively reduce to this anyway.


WAL allows one writer concurrent with any number of readers. It does not allow two concurrent writers.

It's actually really easy. Traditional threading has a very obvious design pattern, where if someone isn't doing this you can be sure there's an issue.

TL;DR, if you don't have a "getter/setter" you're doing it wrong and it's a sign of bad code.

Firstly, make sure that there's only ONE place to write. Write being changing the variable in anyway(Obviously make sure you have a lock or other primitive here). If there's multiple places, that's a big smell for a race condition, and a massive turd for a shitty design pattern.

Secondly, just like the write, all reads should be done from a single place.

And then for the advance developer, you also may need to make sure that the cache is synchronized. Pretty much all libraries will do this for you, but if you ever need to dive into internals that alot of "guides" or papers DON'T cover, you need Fences or whatever your architecture provides. I have this page bookmarked for when I need a quick refresher https://stackoverflow.com/questions/27627969/why-is-or-isnt-...


But you don't need to write every time, only on occasion, so you can actually use a read write lock and in the nominal case many threads can read just fine.

That said, it's probably still better to avoid this unless it's absolutely necessary to modify the underlying structure sometimes, I recently had to do this for an LRU cache.


You are misunderstanding me. When I said no thread ever writes to it, I truly mean no one. There’s not a possibility of “someone writing to it.”

And I think you also have a very narrow view of multi-threading. Shared memory, mutable-everything isn’t the only way of doing multi-threading (or more precisely concurrency). This very article is trying to introduce to you a new way of doing concurrency that doesn’t involve mutable variables shared between threads.


> Even if you're appending lines to a text-file, if you do it from multiple concurrent threads, you're going to potentially corrupt individual lines.

Not true; the write(2) syscall is atomic. You just have put the the entire line in the buffer you pass to it.


That's nonsense. I can't think of any realistic examples in which it would be useful for multiple threads to write to the same StringBuffer.

You might as well sprinke random and pointless synchronized blocks around your code. The magical synchronization fairy will give you a false sense of correctness.


But that's nothing to do with thread saftey - even with a single thread if the char[] changes then stuff blows up right?

Copy on write is for fork. Threads share memory, so you specifically don't want to CoW.

> Reads and writes do not always happen in the order that you have written them in your code, and this can lead to very confusing problems. In many multi-threaded algorithms, a thread writes some data and then writes to a flag that tells other threads that the data is ready. This is known as a write-release. If the writes are reordered, other threads may see that the flag is set before they can see the written data.

> Reordering of reads and writes can be done both by the compiler and by the processor. Compilers and processors have done this reordering for years, but on single-processor machines it was less of an issue.

https://learn.microsoft.com/en-us/windows/win32/dxtecharts/l...


I guess if you don't see the interleaving in log(write(process(read(source)))) our mental models are too different to merit further discussion of this aspect.

And of course thread based code can do other things as well during blocking IO - on other threads.


You can have multiple threads generating code if necessary, you just need to ensure that each has its own pages to write to. Once the machine code is written the page can be flipped to rx and it's safe to share across multiple threads.

Is this not resolved with WAL mode? You can't have concurrent writes, but they can go into a "queue" so as long as you have fast writes and suitable timeouts they'll still be committed.

So whilst they're not technically concurrent, as far as a human using your app will be concerned it will feel concurrent because they were able to write at the same time as someone else, just at an imperceivable fraction of a second slower.


Yeah, I've found the simplest way to have multiple writers / readers is a mutex.

I was suprised to read that they actually consider removing the sync/atomic packages. It's crucial in my opinion, and would complicate a lot of code if had to be written with channels.


> Processes would just place messages into their circular buffers and update an index variable in the buffer header.

Did you implement a write barrier on this to ensure the compiler or CPU doesn’t run these two writes out of order?

next

Legal | privacy