My company has just purchased 150-200 ksplice licenses, and it's really a fantastic technology. It loads up a kernel module that does all the runtime patching for you, but if you don't run it, it doesn't update.
The benefit I see here is that it provides you with a kernel that has all the latest security updates without having to recompile, upgrade, etc. When you reboot next, you have the same kernel you had last time you rebooted (unless you've done an upgrade in the meantime), so you know it's going to boot.
ksplice doesn't modify the kernel on-disk, only in memory. In the unlikely (so far for us) situation that one of those patches is incompatible, causes problems, crashes your system, etc. and you want to reboot to a known-good configuration, just log into their system and deactivate the server, and it won't be able to do any updates.
Your post seems to assume that ksplice changes your booting kernel, and thus who knows if it will boot next time. This doesn't happen. Unless you upgrade your kernel yourself (e.g. through yum or apt), you're always booting from the same (insecure) kernel, and the ksplice daemon reapplies the patches again after you boot to get you up-to-date.
It's really a marvellous technology, and it's worked very well for us so far.
This idea that OS (or application) startup provides a valuable upgrade sanity check under controlled conditions is an interesting point and one that hadn't occurred to me in my work with runtime upgrades.
However, Ksplice mainly supports security patches which tend to be localized and less risky than large semantic changes or feature additions. I suspect that such changes are extremely unlikely to produce an unbootable system.
The latest work has been adding live update, making it possible to upgrade
to a new version of the operating system WITHOUT a reboot and without running
processes even noticing. No other operating system can
do this.
Can anyone correct me if I'm wrong – but can't Linux do this with Ksplice, and the more recent live kernel patching by Red Hat?
Both have updates in which you do or don't need to restart to apply them. Both also have methods of hotpatching all the way to the kernel level depending how much it's worth it to someone as well.
What about when a kernel update is due? Is something fancy like live patches (via http://en.wikipedia.org/wiki/Ksplice or the like) happening or are you simply skipping security updates?
Full disk encryption or other situations where manual intervention is a hard requirement to get the system fully back online is one situation.
Any instance where you're running untrusted binaries on your server you'd want to update ASAP and not worry about getting pwned while waiting for scheduled downtime.
Before Oracle closed it to new customers, Ksplice was fairly cheap, ~$3-4/mo/server.
Using Ksplice means that you're letting a private company inject custom code directly into your running kernel. The paper explains that most Linux security updates are applied without modification, but some updates require Ksplice to write custom code for the patches to work (specifically if the update changes any data structures). My pessimistic side says that sooner or later this custom-written Ksplice patch will contain a bug that causes your kernel to crash - worst case would be if that crash happens after the kernel has been running for a while.
I'm sure the people at Ksplice are much smarter than I am, but every time a security update for the kernel is released they will be stressing to release a Ksplice update to be able to provide value for their customers (the whole point of a subscription is so that you get the security update faster than planning a reboot takes).
The way Ksplice works is that it, for each update, injects all changed functions into the kernel and replaces the old functions with a jump to the new version of the same function. Even though it might not cause what you would consider a production-threatening memory leak, I would still like to clear out the old code from memory every once in a while.
You only need to restart to install kernel updates that you need. While I'd normally just install all updates, most kernel bugs that I recall seeing the past few years are local exploits. If you're running few/no external services, you might not need to upgrade. And often bugs are in little-used subsystems/protocols -- often those will be off-by-default, or turned off by a diligent administrators (never run code you don't need).
It's rare to see a kernel-bug that can't be worked around in some way other than patching.
From what I have heard, ksplice is pretty popular with VPS (virtual private server) providers. When there is an exploitable bug, they have to announce the planned upgrade to their customers and wait for a few days until they can run the upgrade. Even worse, they basically announce that there is a hole in their system to the very people that already have elevated access to that system. (Even non-remotly-exploitable bugs are a problem here.) Ksplice allows for an upgrade as soon as the bug is found/fixed in source code without tipping off potential attackers that could exploit the bug.
services: they are specifically designed to be updated and restarted
kernel: https://en.wikipedia.org/wiki/Ksplice
reply