Parted bug during kernel upgrade

[djfw]

Hi,
I am running pclinuxos KDE on a liveusb. Original install was 2010. I have kept it up to date with the most recent update just before Christmas. I did not realise kernel upgrading is a manual process as described in: http://www.pclinuxos.com/forum/index.php/topic,92863.0.html until I found the article while searching for something else.

When I attempted to upgrade from 2.6.33.5-pclos1.bfs I received this information:

While installing package kernel-2.6.38.8-pclos3.bfs-1-1pclos2011:

A bug has been detected in GNU Parted.  Refer to the web site of parted
http://www.gnu.org/software/parted/parted.html for more information of what
could be useful for bug submitting!  Please email a bug report to
address@hidden containing at least the version (2.3) and the following
message:  Assertion (head_size <= 63) at dos.c:661 in function
probe_partition_for_geom() failed.
Backtrace has 11 calls on stack:

with the backtrace listed after that.

The install seemed to continue alright after that. However, when I rebooted there was no delay to do anything. When I reboot now the kernel that boots is 2.6.33.5-pclos1.bfs. When I look in Synaptic both versions of the kernel are marked as installed. Also, libparted0 is listed as version 2.3-1pclos2010 and up to date and parted is listed as not installed.

I reported the parted issue to http://lists.gnu.org/archive/html/bug-parted/ and received this advice:

This is fixed in parted v2.4, or you can fix it by re-partitioning the
device using parted. Usually this is an issue with factory USB sticks
that have not been re-partitioned.

I am unsure of what to do next to sort things out. I have no idea what parted was up to at the start of the kernel upgrade. Also, I do not know what to do about Synaptic reporting two kernels being installed at the same time while the system still boots into the old version.

Thanks for any help!

[AS]

Hi,
I am running pclinuxos KDE on a liveusb.

Hi and welcome to PCLinuxOS forum! 

As first thing there is to note that it's not possible to update the kernel on a liveUSB system (this is actually a limitation of Live systems due the layered structure of LiveUSB filesystem (which is made up from two overlapping filesystems, one read-only and one read-write).

Original install was 2010. I have kept it up to date with the most recent update just before Christmas. I did not realise kernel upgrading is a manual process as described in: http://www.pclinuxos.com/forum/index.php/topic,92863.0.html until I found the article while searching for something else.

When I attempted to upgrade from 2.6.33.5-pclos1.bfs I received this information:

While installing package kernel-2.6.38.8-pclos3.bfs-1-1pclos2011:

A bug has been detected in GNU Parted.  Refer to the web site of parted
http://www.gnu.org/software/parted/parted.html for more information of what
could be useful for bug submitting!  Please email a bug report to
address@hidden containing at least the version (2.3) and the following
message:  Assertion (head_size <= 63) at dos.c:661 in function
probe_partition_for_geom() failed.
Backtrace has 11 calls on stack:

with the backtrace listed after that.

The install seemed to continue alright after that. However, when I rebooted there was no delay to do anything. When I reboot now the kernel that boots is 2.6.33.5-pclos1.bfs. When I look in Synaptic both versions of the kernel are marked as installed. Also, libparted0 is listed as version 2.3-1pclos2010 and up to date and parted is listed as not installed.

I reported the parted issue to http://lists.gnu.org/archive/html/bug-parted/ and received this advice:

This is fixed in parted v2.4, or you can fix it by re-partitioning the
device using parted. Usually this is an issue with factory USB sticks
that have not been re-partitioned.

Thanks for the info, the bug is known as several users have encountered the same issue at various stage, however it was unclear the real cause of this bug. As I see you have updated info on the subject, I would be grateful if you could point out the exact thread where the above phrase is reported.

see this thread as example about previous occurrence of the same bug:
http://www.pclinuxos.com/forum/index.php/topic,101457.msg865537.html#msg865537

I am unsure of what to do next to sort things out. I have no idea what parted was up to at the start of the kernel upgrade. Also, I do not know what to do about Synaptic reporting two kernels being installed at the same time while the system still boots into the old version.

Thanks for any help!

I would greatly suggest you a complete reinstall from scratch, starting from a recent ISO.
Due to the LiveCD limitation, you may want to evaluate a plain install on a USB stick instead of a Live System, depending on your real needs.

AS

[kjpetrie]

Or you could install your current system on an HDD or large Flash drive, then re-install the newer kernel, then boot into the newer kernel and remaster again. Finally copy the isolinux, boot, and livecd.sqfs back to the liveusb and empty the persistence partition completely before booting it.

[djfw]

Hi and thanks for the welcome,
Been doing my homework but am still at a bit of a loss with some detail.

@as

note that it's not possible to update the kernel on a liveUSB system (this is actually a limitation of Live systems due the layered structure of LiveUSB filesystem

Didn't know, or find, that.  Maybe it needs to be somewhere in the kernel upgrade info and / or the LiveUSB info...

I see you have updated info on the subject

Here is the thread reply, but it is what I quoted:
http://lists.gnu.org/archive/html/bug-parted/2012-01/msg00024.html
I could ask for more info, but given it is a resolved issue I might not get too much more than a polite "read the change log".  Given this is occurring more than once and the answer is parted v2.4 and parted is now at v3.0, maybe the PCLOS team could look at upgrading the parted package...

see this thread as example about previous occurrence of the same bug

Missed it because I searched inside this forum not all forums. My bad!

complete reinstall from scratch

I was dreading that is the answer!

evaluate a plain install on a USB stick instead of a Live System

that is a better answer for me. Again, for the life of me I don't recall seeing that as documented anywhere a noob would find it. If I had, that's how I would have gone. Not to worry. Maybe I'll look at getting a wiki password and adding some stuff to the LiveUSB entries 

So to the reinstall. I have my 2011.9 KDE LiveCD and now know I can install directly onto my USB, rather than just create a LiveUSB. I have read up on remastering and will do that next. So, my question is, is there an entry somewhere that tells me how to get the remaster to update the new install on the USB with all my added software, settings, etc. and not install the messed up double kernel install? Or is there a procedure for upgrading the kernel on a LiveUSB that could be modified? If not, can anyone help me out with that?

@kjpetrie
Reading your procedure, there maybe a detail step or two I would check before I attempted it. However, I think that it is supposed result in a persistent LiveUSB with the new kernel and all my added software, settings, etc in place.  If so, could you check my reply to @as and see if you can guide me through what I need to get it done as a USB install with all the old elements and a new kernel?

Thanks everyone for your help!

[kjpetrie]

There is no way to upgrade the kernel directly on a liveUSB because the actual system is read-only and compressed. Changes are stored to the changes file system which will reside either in a separate partition or a special file, and the kernel merges the two whenever it reads a file - but it can't do that with itself! With the amount of upgrading you've done I would expect that to be quite slow, as almost everything will now be changed.

What you can do is upgrade the kernel on an installed system, and you can also install your liveUSB system to a hard drive or another USB drive, provided it's large enough to take the uncompressed files, and you can also remaster an installed system and make it into a liveUSB, which you must already done.

What you need:

1. Your current liveUSB,
2. A computer with an Internet connection,
3. Either enough blank space (around 20GB or so should be ample) on your computer's hard drive to make a partition to install to, or another USB drive 16GB or bigger.
4. draklive-install. This needs to be on your liveUSB when you start, so open Synaptic and install it if it's not there.

Procedure:

1. boot into the livecd, open a terminal and su to root and run draklive-install
2. Select "custom disk partitioning" and navigate to the place you have chosen to install. If it is not a partition click "create", select partition type (ext4 is fine) and choose / as its mount point. If it is a partition select "Advanced Settings", click "Change type" and select ext4 and choose / as the mount point. Click Done.
3. You will be warned it is going to format the partition. Click OK.
4. Continue with the install. At the end, you will have to choose where to put the bootloader. that will depend on where you've installed to and your current set-up. If you installed to a USB drive you can safely select the MBR of that drive. If to a hard disc, it depends what is on the MBR of the hard disk - if it's another Linux you could install the bootloader to the partition you've just used and then modify the other bootloader to "chain load" your new system. If it's Windows you probably don't want to mess with it and you might want a small ext/2/3/4-formatted USB stick to install it to. You will need to decide based on what is best for you.
5. Once install is complete, shut down the system and remove the liveUSB.
6. Boot into the installed system. Ensure it is connected to the Internet. Open Synaptic, click reload and upgrade it fully. Then remove the new (partially-installed) kernel. Then mark the kernel you want for installation and install it.
7. Reboot into the installed system. If all is well you could remove the old kernel and its kernel-devel. That's not essential but it will make the remaster smaller and the old kernel will be useless there.
8. If you just want a system on a USB stick which you intend to keep upgraded, and that is where you installed it, you could just stop there. Otherwise, check how much free space it has. It needs just over twice the size of the final ISO, or about 2/3 the size of the install free. If your install takes less than about 60% of the space you should be OK.
9. Use mylivecd to create an ISO image of your system.
10. Follow the same procedure you used to create your original liveUSB from the new ISO.

If all is well you will now have an upgraded and newly compressed liveUSB.

[djfw]

I have done some research and experimentation and am reporting the results with possible follow up.

1) Resolve mistaken attempt at LiveUSB Kernel upgrade by making USB into a full install with upgraded kernel:

A) remaster LiveUSB as fall-back to disc 1, this remaster has both versions of the kernel marked as present;
B) open Synaptic, mark the upgrade version of the kernel for removal, apply changes, the USB has only the old version of the kernel marked in Synaptic, but I found it was not particularly stable;
C) remaster LiveUSB to disc 2, this remaster has only the old version of the kernel marked in Synaptic, but *may* still have elements of the upgrade kernel present;
D) shut down system;
E) boot system from disc 2;
F) install PCLinuxOS from disc 2, at this stage the USB is an installed version of the old version of the kernel;
G) open Synaptic, mark the upgrade kernel for installation, apply changes.
H) shut down the system and remove disc 2;
I) boot the system from the USB, wait for the upgrade to complete. The USB is now is an installed PCLinuxOS system with an upgraded kernel.
I cold boot at this stage so the system has a "clean" load.

Nothing amazing, but I have demonstrated that the failed kernel upgrade to a LiveUSB, at least in my case of 2.6.33.5-pclos1.bfs to 2.6.38.8-pclos3.bfs, does not prevent recovery and force a clean install. I think that topic http://www.pclinuxos.com/forum/index.php/topic,92863.0.html could be updated to mention that LiveUSB systems cannot be upgraded in place, but need to be converted to an installation or go through a tested process. I happy to assist with testing and / or documenting.

Now I have an upgraded kernel, is it recommended to remove the back rev kernel. If so, do I remove or completely remove it?

After that, I will reconvert to a LiveUSB to reduce the write wear and tear on my USB that an installed version causes compared to a LiveUSB version. And I will always have a fresh-ish remaster on hand because you are only as good as your last successful back-up! 


2) Remaster parameters create a logical loop and perpetual process

There have been a couple of instances of this reported already. I fell for it, went looking for reports in the fora and found it mentioned. The most common scenario appears to be someone with a tight / partition and lots of room on their /home partition and using the iso file specification to point to /home.

I have not yet looked at the iso creation program but I suspect that, having watched it through two iterations doing what looks like this, it compresses / to /home then starts to compress /home, is alerted to /home changing (that would be the first block or so being added to the iso file) qand decides to start again from scratch. I have seen disc checking software do the same thing. I will look at the iso program and report back.

I am happy to assist with updating existing fora entries, wiki entries and mylivecd coding and testing to deal with this.


3) draklive-install crash (@as please note)

My earlier reported issue is a hard, reproducible issue.  I had a single partition device, formatted to FAT32 attached to my system when I ran draklive-install. On every occasion it returned "Assertion (head_size <= 63) at dos.c:661 in function probe_partition_for_geom() failed.". This is caused by a regression bug introduced into libparted 1.9.0 and fixed in parted 2.4. This is a hard error that can be corrected by removing the device from the system (in my case a factory formatted USB). See http://anonscm.debian.org/gitweb/?p=parted/parted.git;a=commitdiff;h=616a2a1659d89ff90f9834016a451da8722df509. I have not upgraded parted on my system, but will do that next and report back. May take a week...

Parted is now at 3.0 since 2011-05-30, see http://anonscm.debian.org/gitweb/?p=parted/parted.git;a=shortlog;pg=1
I suggest that documentation, e.g. fora, wiki, etc be updated to note this issue, and upgrading parted to 3.0 in the repositories be looked into. As with other items, I am happy to assist with upgrading fora items, wiki, testing etc.

If someone can put me into contact with the appropriate people for the various tasks I would be grateful.

Thanks for your help and cheers!

[Just17]

Changes are stored to the changes file system which will reside either in a separate partition or a special file

Just a minor point ..... changes can only be stored to a Linux filesystem.
If the LiveOS is running on a Linux FS then the changes can be stored on the same partition.

This is what LiveUSB Creator does by default.

***

IIRC ...  you can have as many kernel versions as you like installed but the one in use when the remaster is made is the one that will be used for booting the LiveOS.

2) Remaster parameters create a logical loop and perpetual process

Only if the incorrect or insufficient options are used.
If a user wishes to use other than the default parameters then is it not reasonable that they find out what needs to be done to implement those wishes? == use the necessary command options.

I think that topic http://www.pclinuxos.com/forum/index.php/topic,92863.0.html could be updated to mention that LiveUSB systems cannot be upgraded in place

I do not see the necessity in that topic .....  LiveUSB/LiveHDD/LiveCD is a different fish altogether.
The fact that a lot of the LiveOS can be updated is a big plus .....  but is after all a 'work-around'.

regards

[wedgetail]

A bit of an add on to my topic linked above.

I am feeling relieved now, this is the confirmation I had been looking for (not very hard) that there are USB FAT formatted drives in circulation that knocks the dust out of you.  I have now two of those, 4 GB Toshiba, and anytime they are connected trying to use parted -l you get instant confirmation you have problem USB stick.

Following Old-Polack's  experience I have never heard him having problems with fdisk, my problem sticks format fine using fdisk and then I can leave the stick plugged in, make my notes or copy past as the draklive-install takes place.  What drove me up the wall was I had a number of these sticks and used them mixed and the other FAT formatted USB sticks worked fine.  You can then used parted 2.3 based liveCD no problem if the USB is plugged in.

In my case it is worth as I have similar problems with hard drives on this system, this got mixed up in my battle.  I am happy that the USB side has been nailed for me though.

When the USB stick is formatted ext3 then of course no go in a Windows machine.

[djfw]

Hi,

Long time for this one. Apologies, but I have had an extended illness.

Libparted0 has been updated in PCLinuxOS as of March 2012 to version 2.4. This resolves the issue with reporting on the pre-formatted fat16 USB discs through liparted0. This includes the crash in PCC when attempting to access the disc management component, that I forgot to mention earlier.

I will follow up with the appropriate developers and documenters to resolve the side issues I reported.

Many thanks to all the package managers for PCLinuxOS. I love this rolling distro! It and you rock!!!

Cheers