Saturday, August 30, 2014

Zen and the art of system maintenance

I recently discovered that my linux box was woefully low on disk space. So much so that the boot partition was entirely consumed and so was the root partition. I had separated /boot, /home and / on to various partitions. While I reserved several GiB for /home, I reserved only and about 4.5G for / and about 200MiB for /boot.

And I'm still suffering because of it.

I was unable to update the system because it kept complaining that / didn't have enough space. The event that kicked me into action was that /boot ran out of space and some update message threatened imminent system failure.

I employed the services of Ubuntu's excellent disk usage analyzer. In my zeal to seek and destroy large space hoggers, I deleted several large files (more on which files these were and why they were left around, later), and several older kernels from /boot *manually*, e.g.

pushd /boot
sudo rm init*12.04-02*.
sudo rm vmlinuz*12.04-02*.

While this was sound in theory because I had newer kernels around, I still shot myself in the foot when I rebooted. I got the following message:

GRUB: error 15 file not found

Clearly, I had deleted something that had an entry in grub's menu.lst. Weirdly enough, none of the newer kernels which were left behind were updated in menu.lst. I think the reason /boot got consumed that rapidly was that Update manager failed to update menu.lst, but left the kernels in place.

So how to recover?

1. Downloaded UBUNTU rescue remix 12.04 iso and burnt it to DVD
2. boot with DVD
3. examine all hardware in system using lshw
4. determine which class of systems were present

sudo lshw -short

5. lshw -class disk

From the list presented, i saw that /dev/sda2 (ext3) matched my /boot in size and type, so I went ahead and mounted it:
6. sudo mkdir /mnt/sda2
7. sudo mount -t ext3 /dev/sda2 /mnt/sda2
8. pushd /mnt/sda2/boot/grub
9. sudo cp menu.lst menu.lst~
10. sudo vi menu.lst
11. for the very first boot entry (my default was set to 0, so changing the first would work), change init* and vmlinuz* occurrences to appropriate (latest) kernel image from /dev/sda2/boot
12. save file and reboot

Viola! working system. :-)
Joy to the world.


References:
lshw man page
ubuntu rescue remix 12.04

No comments: