But for VM, do customers pay by CPU usage or runtime? I am sure the Netflix of this world optimise their CPU usage but I also suspect the majority of VMs are mostly idle or have little traffic as their task are either intermittent or are sized for peak usage.
Not at netflix but at scale any savings in CPU usage translate directly into spend savings. If you have 100 machines at peak and autoscaling rules in place, you can generally assume that a 1% saving in CPU usage translates into one less machine running. You never get "headroom", you just have less machines running at any given time.
Hey, just woke up. I don't see anything wrong here, so I think I will second the other commenter. I wonder if the VMs are over-subscribed? Is there any way to tell or would that be a business secret?
I also think we should consider how CPU usage is calculated as well. I don't know how it's done inside VMs, but I can easily imagine that there are several cases where it would include time where the CPU is just waiting for things like access to memory or even waiting on I/O as long as that happens through a shared memory ring buffer, via spinning or something to that effect. So, if that is how it works, then over-subscription of VMs could cause them to show higher CPU usage. But, I don't know, and I hope someone more familiar with the subject will join the discussion.
Virtualization is one popular way to drive up CPU utilization. The more diverse workloads run on a given server, the more even the CPU usage tends to get. This way, if you have 100 workloads that peak at 100% but average at 1%, your CPU usage will tend to be smooth at 100%, any overallocation will smooth out over time (a job that would take 1 second may take up to 10).
Yeah that stood out to me too, since when do people boast about how much resources they consume?
That's 2k customers (not concurrents) per core, which is a terribly low rate. Even if they've only been running 1 month that's 1200 seconds (20 minutes!) of CPU time per customer, and it gets rapidly worse the longer you assume they've existed...
It only requires CPU usage when it's streaming the contents of your screen. For Youtube/Netflix, all it does is tell the key the url of the stream and everything is done either on the key or the server.
It makes business sense: server cpu cycles you have to pay for, client ones you don't. (If the client is fast enough not to notice -- if not, of course, you could lose customers).
But why raise tickets though? Since they already measure CPU utilization, it's not hard to also lower priority or throttle CPU or move such VMs to a separate machine where they can't affect anyone but other heavy CPU users.
Same. Our compute hosts are generally not using 100% of their cores at all time.
There’s computes that are not full, computes that run rather diverse tenants, and even the fully utilized computes responsible for CPU optimized VMs have enough variance in their workload for frequency scaling to occur.
Sorry, I was sloppy. I meant that it generates CPU load somewhere (or I/O load, for those OSes where those two are cleanly separated).
If you measure the CPU load on half the CPUs that are involved in serving a user, you're not really measuring anything. Ditto if you're outsourcing (parts of) the backend and not measuring the outsourced service level.
reply