I'd be interested in seeing how difficult it would be to modify this to use multiple communication servers (redis, cassandra, etc) so that it can be scaled across multiple instances.
Would also be interested in seeing how many connections it could handle doing say 5-10 messages per second.
I think that the main point of OP is that it's possible to serve production load using just one server.
I did not looked into source code yet but I suppose that OP if not implemented already than there should ideas for implementation.
In addition: from my POV implementation of scaling for such service should be trivial:
- sharding of data between instances by a criteria (e.g. regional) or by hash
- configure network routing
This is not a single instance solution. We can run as many instances as we want across multiple servers. It is currently a single-Redis instance solution, however we have some intelligent sharding planned.
I agree with you about Redis clustering, and would extend it by saying I don't think it'd be helpful for this problem regardless. Intelligent sharding with some gossip for slave promotion is where we'll have to go.
The tradeoffs are quite seductive, in my experience most companies do not operate their software on a scale where they really have to scale out, but they run multiple instances simply to increase availability.
But then those are probably the same kind of companies that will probably not have the required network hardware, and won't for many years to come.
I'm thinking about stuff like Redis or distributed caches like Hazelcast, maybe even Zookeeper and friends. Any ideas where this could shine?
Cloud vendors could manage the hardware part, but is this model attractive to them, essentially being constrained to single-node performance?
Pretty much any modern runtime (Java/Go/Node w/ native bindings) can handle that many connections per machine. You probably want to horizontally scale it with kafka or similar, but anyway, a single machine will work to start.
I'm working on a function as a service platform that will eventually proxy to a cluster of worker machines. Currently, I'm not scaling in the most literal tech sense; it runs on a single server until I validate that people will actually use it.
No, you could have multiple servers talking to each other via something like ZooKeeper. But individually, each node/server would use multithreading to keep track of its state.
I configured and launched a machine with redis and node in less than 5 minutes. Very cool.
How will you isolate instances from each other? My instance appears to have 24 GB of RAM and 12 cores, and it looks like I can use all of it in my instance.
This means then that if you run multiple application servers, which most do, that you’ll need to implement a data distribution mechanism of some sort.
I must admit, with limitations like this I’m struggling to figure out the use cases for this.
Edit: so I guess this is easier using the change subscriptions you mention in other comments. That does mean many subscribers, but hopefully that’s minimal load. This has the trade-off that it’s now eventually consistent, but I suppose that’s not a problem for many high read applications.
I’m still feeling like this could be solved in a simpler way with just simple data structures and a pub sub mechanism. Now I think of it, we do a similar thing with Redis for one service, and a custom Python server/pipeline in another, but we’ve never felt the need for this sort of thing.
Do you have more details about specific applications/use cases, and why this is better than alternatives?
Without having looked into it too deeply I feel that they are somewhat “cheating” by using a superserver to launch a new process for each connection, thus letting the OS handle the dynamic allocation needed for each connection.
Still pretty impressive project. Would be fun to take a deeper look at it at some point.
I run it on a single server. That said, I imagine in a distributed environment (where you have a couple of machines, not tens, hundreds or more) you can do something like this on each machine you have (probably not in parrallel though, if something happens you don't want to mess up all the servers?).
If someone has a link to a good article / book / video about best practices with this, please post it?
There's a lot of talk about how someone did massively scallable systems or deployments (and it's really interesting, lots of cool engineering problems & solutions there), but a real world "here's what we figured works really well for us for a smaller-scale distributed operation" would be really useful, too.
Depending on how you have it configured. Using a combination of worker pools, and taking advantage of the ecosystem config files. You can distribute work off to multiple threads.
In one of my use cases I have a Redis server between all as a POC to see the kind of redundancy that could be achieved. It's flexibility is its greatest feature imo.
I implemented a similar idea for a few companies. The response was always very positive.
In one company, it grew into a cluster of a few servers for higher capacity and availability. This load balancing and management code is a quite more complex than the original single-server snapshotting script. Unfortunately, it's not open source. For several years, the single server was enough. Go for it. :-)
Not on a single box, no - we run these machines in a cluster. Each chat machine scales up to a bit more than 80k simultaneous connections (each machine runs 8 processes, each of which can do about 10k simultaneous). There's some amount of IPC between all the chat processes in the cluster, but not too much, which means we can pretty much scale linearly just by adding hardware at this point.
That is really cool! We definitely see a lot of potential in this use-case, especially at scale. For example, you could use the Realm Object Server to synchronize the Realms used in the web server. If you need several servers, all the state coordination is then handled for you. Could be quite helpful, especially if the web servers are wrapped in containers.
In general I think you're right but I'm not sure why you would have 5,000 connections to each node. There isn't any reason I can think of to maintain more than 1 connection per client (application process) to each node. You don't gain anything by 'connection pooling' with Redis so it's not like each app server would want to keep a 100 connection pool around. I guess you could have an issue if you have 5000 application processes all wanting to connect to a single redis cluster, but I'd argue that you'd be well past the point of needing to break your system into services by then anyway.
Would also be interested in seeing how many connections it could handle doing say 5-10 messages per second.
reply