Friday, July 29, 2011

Ubuntu, you're fired.

Attempted to upgrade a workstation (dual-xeon, sata3 ssd for /; sata 3 hdd for /usr, /var, /opt, /usr/local; sata2 mirrored raid (via dmraid) for /home) from 10.04 to 11.04 via do-release-upgrade (in update-manager-core) last night. Why do that? Because Ubuntu only updates software packages for its newest release, so if you need, say, Python 2.7 in order to work on a project, you're SOL unless you upgrade.

Everything went well until the final reboot, which left the machine at:

error: the symbol 'grub_xputs' not found.

Verified that the grub software is all present:

grub rescue> ls (hd0,1)/boot
... lots of modules and such...

Everything seemed to be there, so attempted to boot manually, under the assumption it was all due to misconfiguration:

grub rescue> set root=(hd0,1)
grub rescue> set kernel=/boot/vmlinuz-2.6.32-33-generic
grub rescue> set initrd=/boot/initrd.img-2.6.32-33-generic
grub rescue> boot
error: no loaded kernel

Second attempt, loading the grub modules manually (per Ubuntu Grub2 Docs and this thread):

grub rescue> set prefix=(hd0,1)/boot/grub
grub rescue> insmod normal
error: the symbol 'grub_xputs' not found.
grub rescue> insmod linuxgrub
error: the symbol 'grub_xputs' not found.

For some reason, the symbols in the grub2 bootstrap code do not match the ones expected by the modules. One of them is the wrong version. Looks to be a reported bug.

The solution, per Ubuntu Grub2 Docs and this fix, is to boot a LiveCD (which must be the same version as the newly-installed-but-broken Ubuntu) and run grub-install. This turns out to be a huge delay, as the upgrade took place without a LiveCD, so an ISO must be downloaded. It is also a huge pain when one is out of blank CDs, and has to press a usbstick into service (accomplished per Ubuntu Install Docs).

It also doesn't work. The new error on reboot is

error: file not found
grub rescue> insmod linux
ELF header smaller than expected

It turns out that 10.04 upgrades to 10.10, NOT 11.04, so the LiveCD contained the wrong version of grub!

Fortunately, there is a guide to purging and reinstalling grub once booted from any LiveCD.

The steps are (details such as hdd devices and mount points are specific to this workstation):

bash$ sudo bash
bash# mount /dev/sdk1 /mnt
bash# mount /dev/sdf3 /usr
bash# mount /dev/sdf4 /var
bash# mount --bind /dev /mnt/dev
bash# mount --bind /dev/pts /mnt/dev/pts
bash# mount --bind /proc /mnt/proc
bash# mount --bind /sys /mnt/sys
bash# chroot /mnt
bash-chroot# apt-get update
bash-chroot# apt-get purge grub grub-common grub-pc
bash-chroot# apt-get install grub-common grub-pc
bash-chroot# #NOTE: Be sure to select /dev/sdk for Grub2 install location
bash-chroot# exit
bash# umount /mnt/dev/pts
bash# umount /mnt/dev 
bash# umount /dev/sys
bash# umount /mnt/proc
bash# umount /mnt/var
bash# umount /mnt/usr
bash# umount /mnt
bash# reboot

That does the trick.

But you know what would really be nice? A way to upgrade the OS, including the kernel and all packages, WITHOUT TOUCHING THE GODDAMN BOOT-LOADER. Seriously, it's an upgrade, the bootloader already works -- don't mess with it!


  1. It seems to me that during the do-release-upgrade, it is asked what to do with grub. Something like
    -keep actual
    -pick new
    -view diffs in grub.cfg
    -go bash

    Now maybe I am wrong

  2. That might be the case now -- it certainly wasn't the case when I did this upgrade.