> As node.js is not multi-threaded, we spin up 4 instances of node.js per server, 1 instance per CPU core. Thus, we cache in-memory 4 times per server.
And why not use a shared memory server?
> Operations started adding rules with 100,000s of domains, which caused a single set of rules to be about 10mb large ... If we weren’t using node.js, we could cut this bandwidth by 4 as there would only be one connection to the Redis cluster retrieving rule sets.
Maybe a 10mb json string isn't the best design decision.....
Or you know, you could have one node process connect to the Redis server, and have the local processes read from a shared memory server.. Or you could not store your rules as a 10mb friggin JSON string..
> When rule sets were 10mb JSON strings, each node.js process would need to JSON.parse() the string every 30 seconds. We found that this actually blocked the event loop quite drastically
Well then do it in another thread and save it to shared memory. Maybe, just maybe, JSON strings aren't the tool for the job here.
While I don't know node.js very well and can't speak to the "shared memory server", I do think it is valid to point out that this sort of redesign to fit the platform is frustrating. I've hit similar situations in Python where an async i/o framework is used and at some point there becomes a CPU bound task. The application was redesigned and new microservices were started in order to cope and the architecture became convoluted. This sort of question always seems to always show up in Python where limitations of the GIL influences the design of the software, the libraries used and deployment. It seems node.js, not surprisingly, has the same problems.
As the above comment clearly points out, there are workarounds to make things work. Still, when I think about the amount of time and effort that goes into debugging systems when they hit limits like the author mentioned, I imagine most of the the benefits gained by using a language like Python or JavaScript go away.
It's not a Node.js limitation either. The design was not thought out from the beginning.
It has happened to me before. I design something, then I run it in multiple CPUS and it doesn't make sense anymore...
Or realize that if try to scale the current design doesn't hold.
Node.js running in a single thread is a good thing if you know how to work with it. Or if you really need multiple threads, use another PL.
Or you could write it in C++ and bind it to Node.js. And run it as a background process. Or spin up a worker? there seems to be multiple solutions here...
The thread thing in nodejs seems very misunderstood. Only the javascript runs in one thread. I think its libuv that uses 4 threads to do most of the work in node.
And why not use a shared memory server?
> Operations started adding rules with 100,000s of domains, which caused a single set of rules to be about 10mb large ... If we weren’t using node.js, we could cut this bandwidth by 4 as there would only be one connection to the Redis cluster retrieving rule sets.
Maybe a 10mb json string isn't the best design decision.....
Or you know, you could have one node process connect to the Redis server, and have the local processes read from a shared memory server.. Or you could not store your rules as a 10mb friggin JSON string..
> When rule sets were 10mb JSON strings, each node.js process would need to JSON.parse() the string every 30 seconds. We found that this actually blocked the event loop quite drastically
Well then do it in another thread and save it to shared memory. Maybe, just maybe, JSON strings aren't the tool for the job here.
reply