But if you start with 600 MB, you have quite a bit of headroom before you grow out of memory on a single server. Normal off the shelf servers fit 3-6 TB, which 5 orders of magniture up from 600 MB. Even AWS EC2 has 2 TB instances.
At this point you can get 24TB of RAM in an EC2 instance (along with 448 vCPUs, 100Gbps of network bandwidth and 38Gbps of EBS bandwidth). That won't scale forever, but Stack Overflow has been running on a single primary/standby setup with 1.5TB of RAM so that would be 16x Stack Overflow's RAM.
I think a lot of work goes into horizontal scaling which is necessary at a certain scale, but very few people actually get anywhere near that scale. It can be important to understand which things are needed at your scale and where you can simply buy some beefier hardware. I've been at places where people run a dozen sharded DB servers with each server having 16GB of RAM. Maybe that's resume-driven-development where someone wants to say they've done that.
Their FAQ has a section on how many instances are on each host, which is 40 for 0.5GB and 20 for 1GB etc (ie, 20GB of customer instances per host, regardless of instance size). Which would mean that when you go for twice as much ram, you also get twice as much of everything else. The front page lists disk space and bandwidth, and these also scale linearly with ram size.
In my opinion the correct answer is 255Gb. (i.e. AWS r3X8 High Memory instances ).
While one can purchase servers with larger memory most likely you will run into limitation on number of cores. Also note that there is at least some overhead in processing data, so you would need at least 2X the size of raw data.
Finally while its a good thing to tweet, joke about and make fun of buzzword while trying to appear smart. The reality is that purchasing such servers (> 255 Gb RAM) is costly process. Further you would ideally need two of them to remove single point of failure. it is likely that the job is batch and while it might take a terabyte of RAM you only need to run it once a week, in all these cases you are much better off relying on a distributed system where each node has very large memory, and the task can easily split. Just because you have cluster does not mean that each node has to be a small instance (4 processors ~16 Gb RAM).
1TB RAM Hertzner servers are available, so at least 8x more scaling before that's a problem.
2TB RAM is common in commodity servers, albeit expensive ones ($1000ish/month). Somewhere between 4TB to 20TB RAM is the pragmatic limit (where costs for vertical scaling start to get far worse)
You need as much space (as in, disk space) as there is content to server, plus a bit more. It's not the disk space you should worry about, though. (Even an entry level server at whatever hosting company will have plenty of space to host pretty much anything.)
Most servers support at least 128GB; it isn't even very expensive. And if you want to handle a million concurrent users you also need to consider CPU and latency, so for most real-world workloads the memory probably won't even be your bottleneck.
A lot of resources go to scaffolding for containers, runtimes, caching, etc. 4gb would make for a chugging experience if you run a Java spring boot backend for example. But an old school php + postgres stack would be fine, or modern dotnet, rust, etc. And honestly I'm not sure it matters since ram and compute is cheap for most small-medium sites (at worst maybe $30-50/mo vs $10-20/mo).
VPS services are pretty cheap these days. Hetzner, digital ocean, genesis, etc. Or you spend a couple grand to build a dedicated machine in raid6 and only pay utilities.
Yeah that's pretty wild. I managed to fit my entire serverless stack (roughly 1.2m checks per week) into 2x 256MB VMs and 1x 512MB VM, you could run a monster server on that.
Keep in mind that a single modern physical server that is decently configured (12-16 cores, 128GB of ram) is the processing equivalent of about 16 EC2 m1.large instances.
I think most people on here are coming from the perspective of startups, which scale out of a single server setup pretty quickly. At a bare minimum, most will have dedicated purpose-built servers like Redis or a DB, and often there's separate background workers, or a load balancer with a couple of web servers.
When your server requirements get into needing 5-6 servers (not at all atypical for a startup in their first year of being launched), running your own stuff becomes more of a challenge pretty quickly. Factor in 2-3x growth a year, and the challenges just mount.
That's not as bad as I was expecting. I was once working with a startups infrastructure (>100 servers) and it was near 20k/mo (mostly reserved instances)
That's bollocks. I've got a 4GB server which runs 5 production services, and (maximally) 20 test services, and only one of those ever goes over 40MB, and then drops below that again. The production services handle about 65k requests per day. Not much, but it shows that you can easily run go applications in less than 2GB.
While I fully understand why you run this thing with 300+ nodes as you do, I have to wonder, just for fun - could you actually fit this whole thing on a single large server? Looks like something with 16 TiB RAM and 2 PiB SSD storage is actually a server you could theoretically buy today?
reply