It's a neat technology and I'm very interested in using it. It's almost a perfect fit for my business, which deals with serving traffic during email spikes.
I think the questions are just around the economics to make sure you guys stick around and can continue to offer it. My initial thought was that being able to scale up and down so quickly would require you to have a lot of idle resources sitting around at any given time, both on individual hosts as well as across hosts.
I think this has potential to scale, but you'd probably need to throw in more servers if you plan to handle more than few hundreds of additions per second.
Would love to know how this affects my bottom line! if my servers are basically running idle, it would be great if they use as little resources (and money) as possible. Then at any moment they can scale up when needed!
What kind of services are you guys running that require you to scale up and down? Why not get one or two dedicated server(s) and run everything on them!? The post had no numbers, but I'm pretty sure you would come off even cheaper if you use dedicated servers, even managed.
> They actually DO need to be able to scale servers up and down based on demand-usage, time of day, growth patterns, etc.
Cost wise, your best scenario is usually going to be dedicated or colo + EC2 or similar for overflow / peak.
If you do just dedicated hosting you need to leave enough spare capacity that you feel comfortable handling the spikes for whatever the worst case provisioning time your host has.
It's still cheaper than EC2.
But if you do dedicated + ability to spin up EC2 to take peaks, you can go much closer to the wire with your dedicated hardware, and increase the cost gap to EC2 massively. You don't need as much spare capacity to handle peaks any more. You don't need as much spare capacity to handle failover.
It's rare for it to pay off to spin up EC2 instances to handle inter-day "normal" load changes, though - most sites don't have differences that are pronounced enough over short enough intervals for it to be worth it. If you do the dedicated + EC2 model, your EC2 instances needs to be up no more than 6-8 hours or so on average per day before it becomes cheaper to buy more dedicated capacity.
Oh, may be I misunderstand you. So far it was ~1500 users per day which is quite small amount considering I don't need to store any big amounts of data. I pay for server and domain myself, I can afford it and don't think it can be overloaded in the near feature.
Not sure who your question was directed to, but I've implemented a similar system in the past for a website with 10m+ users and 15+ web servers. It scaled just fine, we were sending close to 150K emails a day, among other background jobs for reporting, data cleanup, automated spam checks, etc
Much much larger, albeit distributed across multiple datacenters and completely isolated environments. The site I was giving as an example was going possibly serving around 10-20M/day, and it isn't an isolated case: I know plenty of companies that allocate such resources for something like that.
What type of overhead is associated with this? Are you experiencing an increase in server costs? What about the resources it takes to build those relationships, maintain the content, or manage the infrastructure?
I understand why would think so, because you don't know our setup. Just to give you an idea, we process 16PB of data every single month. This is an ingress traffic. If we would pay for this traffic going out (egress), we would end up paying over $1M just for the traffic itself. By keeping everything in a single spot it costs us literally 0.
That said, I've tried. I reached out to many hosters, like OVH and others. They just don't have the capacity we need. 20 servers will not make a change for us. We wanted to start with 500, but it would take them 9 months.
To address your question better, the plan has now turned into creating a subscription services which will allow to setup better services whilst keeping traffic standardised to a medium that allows us to grow the server farm as more people join. Development is underway to turn it into a fully scalable system for future growth, whilst continually adding features etc.
The growth was pretty steady, but I was always extracting the maximum from the hardware I had to save money, so for a long time I had intermittent issues. I also had my own server colo'd for a while, which I do not recommend for a one man shop. It nearly broke me a few times having a server go down and spending all day trying to fix it. I had a sever go down hard 2 days before I got married, but I had to let it go until 2 days after the wedding. I didn't think the site would recover from that.
The hardest part was to scale it economically. I have always been a big fan of SSD's, which are instrumental in scaling a site that doesn't have the resources to put data they need fast access to in RAM. For example, a site with more money might use redis to cache all the pics for each user (sorted, unique lists), plus all the tweets that point to a pic, but there is too much for me to do that in RAM without selling organs. Also, redis is not very memory efficient. But the advantage is that technologies like that are easy to use and easy to scale. If you have money.
The Twitter rule changes freak me out every day but I have not seen any specific moves that make me think they are after me. It is probably a matter of time, but then you need to think about what form that will take. Will they just cut off the Streaming API? That would be a big move. Will they remove pics from the streaming API? I doubt it. Will they make moves to single me out? That would be weird. Will they come after me with lawyers for caching tweets? I don't see that either. But, while I don't see anything specific, there are a lot of what ifs.
My advice is to use Twitter to get off the ground and then move away from Twitter altogether if at all humanly possible. That's the strategy they are forcing with their stupid rules, so maybe they will see the light at some point and rethink their strategy.
Startups? I work for a finance firm, and while we certainly have a need for large farms of servers to store data, my current team keep talking about web request latencies as an important infrastructure concern when the literal maximum number of users is in the tens of thousands.
Which is, you know, 1 maybe 2 machines with nginx plus 1 for redundancy. Our internal services are slow because people cohost them with batch jobs, and no other reason.
Interesting, wouldn't mind having a chat outside of HN if you're interested (see my profile for mail).
I've spent much of my career working on systems with active users from the hundreds to low thousands, but which process a huge number (50k/sec scale) jobs/tasks.
It's a totally different kettle of fish, and if I'm totally honest I'm shocked at how badly "web" scales and how common these naive and super inefficient implementations are (hint: my bare-metal server from 2005 was faster than expensive cloud VMs).
Recently I've worked on two high-usage systems (one of which was "handling" 30k requests/second for the first couple of week).
If you want to be able to instantly spin up servers collocated you just need to have spare capacity. There is nothing that restricts you from delaying expanding your capacity until you need it. That is what slows you down.
The $5 instance can easily handle 50k connections, depending on what you're doing. I scaled to 40k clients and 50 megabytes a second sustained on a $10 Linode (but that was nchan, not this). That was after my first 50 paying customers...
Well, if you can triple your performance, you will require 1/3 of the servers, and your clients will enjoy faster responses. I thought that was the idea, good enough is not good enough.
Anyways, congrats on your success. I would love to read more about the business side - finding clients, profits, expenses, etc.
Only if you extract a ton of value per user or can serve a huge number of users per machine. Otherwise, upping the number of servers by an order of magnitude will start requiring an operations team to maintain.
That was a pretty common truism 10 years ago, but machines haven't gotten faster at the same rate lately.
I think the questions are just around the economics to make sure you guys stick around and can continue to offer it. My initial thought was that being able to scale up and down so quickly would require you to have a lot of idle resources sitting around at any given time, both on individual hosts as well as across hosts.
reply