Tuesday, April 20, 2010

When "recovery" mode fails to boot

*** Don't try this unless you understand what you are doing ***
The recent changes to the boot process with the integration of the graphical boot loader and logger (Plymouth) and the continuous development of upstart and associated jobs brought a significant improvement to Ubuntu's Lucid boot experience.
Because the the code is new and is rapidly changing there are some good chances of bugs which may lead to an non bootable system.
Some of the bugs found (promptly fixed, or being worked) on the development stage shown that if you have an issue related to plymouth/mountall the rescue option on the boot menu will not work.
Rescue mode as implemented now depends on plymouth/mountall being started in the first place.

To work around this you will need to be able to boot without automatically starting upstart, you can do this by appending using the init kernel option, from the boot menu, press e for edit, search for the linux line, append the init=/sbin/sulogin, press ctrl-x to boot. If you don't get a root prompt then something was wrong during kernel load/initialization, or while the root filesystem was mounted (removing the quiet option from the linux kernel line might provide some hint on the problem).

So now we are in a root shell, what's next ? On my case I am going to set the console keyboard layout: loadkeys pt .
We can't yet start services, we are running from an sulogin process and we need upstart to be able to start services. We will just backup and disable the upstart services first:
# Remount the root fs in read-write
mount -orw,remount /
# Backup the upstart jobs config
cp -a /etc/init /etc/init.orig
# Nothing will start
rm /etc/init/*
# Add 2 console virtual terminals
echo start on startup > /etc/init/sulogin8.conf
echo exec openvt -c 8 >> /etc/init/sulogin8.conf
echo start on startup > /etc/init/sulogin9.conf
echo exec openvt -c 9 >> /etc/init/sulogin9.conf
Now we can start upstart: exec /sbin/init .
The console will just hang because nothing was started, except our virtual terminals 8 and 9, now let's swtch to VT8 with with CTRL-ALT-F8.
Now list the the available services: initctl list, a bit boring, only our sulogins, let's put the regular jobs in place: cp -p /etc/init.orig/* /etc/init .
Try initctl list again, a long list of services now, which you can start with "start service".

Please note Upstart is an event driven job management system, starting a service will most likely trigger other services which depend on events emitted by the first one. My next research will be on how to avoid the chain reaction, or if not possible request such a feature :)

I hope you don't need this instructions :)

2 comments:

  1. I have a similar problem. Not really sure if this problem is due to the reasons mentioned by you.

    I had upgraded successfully from 9.10 to 10.04 Beta 1 and it was working fine till 2 to 3 days back.
    There were nearly 200 updates waiting in Update manager and I went a head and installed the updates.
    But after the updates, my Ubuntu fails to boot!. I get a black screen after the initial Ubuntu logo screen.

    After that I did a clean installation of 10.04 Beat2 and system showed me that there were many updates waiting, so I went a head and updated and again the same problem.

    Any idea what is the problem? Can you please help me ?

    Thanks & Best regards,
    Subbu.

    ReplyDelete
  2. Hey Jao, you saved my day today (I experienced a power cut during my Ubuntu 10.0.10 migration and was unable to boot the recovery mode, broken CD player...). So a BIG thank for the trick! I was then able to restart the upgrade sequence, starting from apt-get -f install....

    ReplyDelete