Hacker Read top | best | new | newcomments | leaders | about | bookmarklet login

software: can upgrade binaries and libraries without affecting already running programs

services: they are specifically designed to be updated and restarted

kernel: https://en.wikipedia.org/wiki/Ksplice



sort by: page size:

Oracle Ksplice[1]: patches the kernel with security updates without rebooting.

1. http://www.ksplice.com/


My company has just purchased 150-200 ksplice licenses, and it's really a fantastic technology. It loads up a kernel module that does all the runtime patching for you, but if you don't run it, it doesn't update.

The benefit I see here is that it provides you with a kernel that has all the latest security updates without having to recompile, upgrade, etc. When you reboot next, you have the same kernel you had last time you rebooted (unless you've done an upgrade in the meantime), so you know it's going to boot.

ksplice doesn't modify the kernel on-disk, only in memory. In the unlikely (so far for us) situation that one of those patches is incompatible, causes problems, crashes your system, etc. and you want to reboot to a known-good configuration, just log into their system and deactivate the server, and it won't be able to do any updates.

Your post seems to assume that ksplice changes your booting kernel, and thus who knows if it will boot next time. This doesn't happen. Unless you upgrade your kernel yourself (e.g. through yum or apt), you're always booting from the same (insecure) kernel, and the ksplice daemon reapplies the patches again after you boot to get you up-to-date.

It's really a marvellous technology, and it's worked very well for us so far.


This idea that OS (or application) startup provides a valuable upgrade sanity check under controlled conditions is an interesting point and one that hadn't occurred to me in my work with runtime upgrades.

However, Ksplice mainly supports security patches which tend to be localized and less risky than large semantic changes or feature additions. I suspect that such changes are extremely unlikely to produce an unbootable system.


> routine kernel updates

ksplice


  The latest work has been adding live update, making it possible to upgrade 
  to a new version of the operating system WITHOUT a reboot and without running
  processes even noticing. No other operating system can
  do this.
Can anyone correct me if I'm wrong – but can't Linux do this with Ksplice, and the more recent live kernel patching by Red Hat?

Right - POSIX is generally "advisory locking" rather than "mandatory locking," so an update process is free to overwrite any library.

Windows does not allow this, which is why we have Patch Tuesday outages.

Also, Oracle KSplice was the first free tool to apply updates to a running Linux kernel without rebooting.

KSplice is free on Ubuntu. I think other free services have become available since.

https://ksplice.oracle.com/try/desktop


Kernel updates probably need a restart (or use kexec), but most other updates only require a service to be restarted, right?

Both have updates in which you do or don't need to restart to apply them. Both also have methods of hotpatching all the way to the kernel level depending how much it's worth it to someone as well.

What about simple stuff, like updating the kernel for the latest security patch and needing a small restart?

LTS kinda garantees that the internal APIs will not change, allowing for upgrades without reboot using something like Ksplice.[1] (aka "Hot patching")

[1] https://en.wikipedia.org/wiki/Ksplice


What about when a kernel update is due? Is something fancy like live patches (via http://en.wikipedia.org/wiki/Ksplice or the like) happening or are you simply skipping security updates?

Full disk encryption or other situations where manual intervention is a hard requirement to get the system fully back online is one situation.

Any instance where you're running untrusted binaries on your server you'd want to update ASAP and not worry about getting pwned while waiting for scheduled downtime.

Before Oracle closed it to new customers, Ksplice was fairly cheap, ~$3-4/mo/server.


Using Ksplice means that you're letting a private company inject custom code directly into your running kernel. The paper explains that most Linux security updates are applied without modification, but some updates require Ksplice to write custom code for the patches to work (specifically if the update changes any data structures). My pessimistic side says that sooner or later this custom-written Ksplice patch will contain a bug that causes your kernel to crash - worst case would be if that crash happens after the kernel has been running for a while.

I'm sure the people at Ksplice are much smarter than I am, but every time a security update for the kernel is released they will be stressing to release a Ksplice update to be able to provide value for their customers (the whole point of a subscription is so that you get the security update faster than planning a reboot takes).

The way Ksplice works is that it, for each update, injects all changed functions into the kernel and replaces the old functions with a jump to the new version of the same function. Even though it might not cause what you would consider a production-threatening memory leak, I would still like to clear out the old code from memory every once in a while.


What is the difference between maintenance and regular updates? If I for software it should be complete, correct, and secure.

No new features? Fair enough.


New software is written with newer kernels in mind. To continue running newer software, you will need to update your kernels.

Additionally: you might update infrequently when it comes to feature releases, but you'll definitely need security patches.


> Anything that involves system services

This is not true. NixOS, as one example, is able to figure out which services (including system) need to be restarted.

> god forbid the kernel

This is not true either. Live kernel updates are possible (but are usually a paid addition, e.g. https://ubuntu.com/security/livepatch).


No security patches to the kernel. You can certainly upgrade user-space items (daemons and the like...)

You only need to restart to install kernel updates that you need. While I'd normally just install all updates, most kernel bugs that I recall seeing the past few years are local exploits. If you're running few/no external services, you might not need to upgrade. And often bugs are in little-used subsystems/protocols -- often those will be off-by-default, or turned off by a diligent administrators (never run code you don't need).

It's rare to see a kernel-bug that can't be worked around in some way other than patching.


From what I have heard, ksplice is pretty popular with VPS (virtual private server) providers. When there is an exploitable bug, they have to announce the planned upgrade to their customers and wait for a few days until they can run the upgrade. Even worse, they basically announce that there is a hole in their system to the very people that already have elevated access to that system. (Even non-remotly-exploitable bugs are a problem here.) Ksplice allows for an upgrade as soon as the bug is found/fixed in source code without tipping off potential attackers that could exploit the bug.
next

Legal | privacy