Bootloader Limitation Problem

[Just17]

http://www.pclinuxos.com/forum/index.php/topic,108512.msg927537.html#msg927537

I think it is the sheer number of partitions you have that's the problem. If you look at bootloader.pm you will see that the add_entry subroutine contains a for loop (line 575)  which will only run until the index variable reaches 10. After that it will move to line 604 which dies with an error message. Clearly, this is intended to trap a runaway loop condition and the programmer never envisaged the loop would complete 10 increments. However, as you have a very large number of installs, I suspect your loop goes round more times than the programmer considers reasonable and hits this limit. As an experiment, before running draklive-install, open the file as root and change that 10 on line 575 to 100 or even 1000. If it now installs we've found the reason.

Whether that's a bug or whether you have exceeded reasonable limits is a matter for conjecture

The above thread and quoted post refers to a problem with the bootloader.pm.

The edit above fixes the problem.

Most other locations of limitation on partitions etc are already at 99 IIRC, so having this limitation at 10 seems unduly unreasonable.

It seems a simple edit to implement; it fixes the problem.

Is it possible that this can be permanently incorporated into the PCLOS installs?

[wedgetail]

Just17
Thanks for bringing this up again, I forgot about the problem when battling with CUPS in the KDE64 test05 and ran out of of time.  I am sure I would have made the first re-install a failure as I would not have remembered.  Now I will remember for another 14 days.   

As I still have many partitions I naturally volunteer to test if needed.

[wedgetail]

I was going to make a quick install of yet another  PCLinuxOS for some testing. I went right into the trap again. It was a KDE32 system and it failed at the bootloader stage.  I then edited line 575 in bootloader.pm to test 100 times instead of 10.  this time a successful install.

The exercise made me realise that I should not be too quick to delete my LiveHDD copies of downloaded ISO files because once the bootloader.pm has been edited it will be ready for another successful install. I know I have already tested the KDE32 by running up a LiveHDD first then editing the bootloader before starting (as written up here in this thread) the install but I was also a bit quick to delete the LiveHDD (Or I have lost track of where the trial KDE32 LiveHDD went).

This time I have made a better title of the LiveHDD and will be using this as a poor man's fix for the time being.   

[wedgetail]

Well it was my own fault I could not remember where this note was, tried to install KDE 2012.12 on a partition to high.

I am about to check by editing said line and do it again.    

Errrrh, a CD is not writable so I can not edit the file, well I have to install a LiveHDD and edit in that  

1.. Is there a way I can modify the ISOs before burning them to DVD?

2.. I believe the actual file transfer has taken place, there is a pretty complete / filesystem installed. I remembered that a couple of years ago I noticed this and I think I tried just to point a stanza the the partition.  This did not work as far as I recall.

Is there a way to resume when the counter in line 575 is increased from 10 to an arbitrary 99/100?  I have in this install found the bootloader.pm and edited the value to 100.  

3.. A wild guess make a new squashfs of the partition with the edited bootloader.pm and then use that inside an ISO (I don really know what I am talking about)  
 

[Just17]

1.. Is there a way I can modify the ISOs before burning them to DVD?

...  not something I have done recently, but imagine it can be done easily enough ......

mount the ISO so its contents are accessible

copy the contents to a local /partition/directory

unsquash  the   livecd.sqfs  file

make the edit

recreate the squash file

replace the original .sqfs file with the edited one

create a new ISO from the edited contents


Check out the tools   mksquashfs & unsquashfs

I think the gzip compression is the default, and for your purposes this should be sufficient as space is not a constraint. But that is just conjecture ....  

EDIT:
            This link might be of help

http://www.tldp.org/HOWTO/html_single/SquashFS-HOWTO/#unsquashing

[wedgetail]

Just17
Ummm, good stuff. Certainly put me on the right track.

Added two new mount points

Code: [Select]

[root@KDE32-test05 gert]# mkdir /za /zz

ISO file located here

/media/B-18/130108-pclinuxos-kde-2012.12.iso      

Loop mounting

mount -o loop  /media/B-18/130108-pclinuxos-kde-2012.12.iso /za

Just checking to make sure there is a squashfile on board

Code: [Select]

[root@KDE32-test05 gert]# ls -l /za
total 1370030
drwxr-xr-x 3 root root       2048 Dec  7 04:38 boot/
drwxr-xr-x 2 root root       8192 Dec  7 05:08 isolinux/
-rw-r--r-- 1 root root 1402900480 Dec  7 05:08 livecd.sqfs
[root@KDE32-test05 gert]#
Loop mounted my ISO and successful.

Listing the files in livecd.sqfs  to a file which shows about 140,000 files  
[root@KDE32-test05 gert]# unsquashfs -l /za/livecd.sqfs >>/home/gert/130115-isorepair.glk

In the listing there is one file:

squashfs-root/usr/lib/libDrakX/bootloader.pm
        line 36501

So I know the file of interest is there, and I would like to just extract this one file.  I was going so well but Ihave been unable to figure out how to unsquashfs -f... no matter what combinations of options and positions from man unsquash I get the same error, example here:

Code: [Select]

[root@KDE32-test05 gert]# unsquashfs -l /za/livecd.sqfs   /squashfs-root/usr/lib/libDrakX/bootloader.pm

Could not open /za/livecd.sqfs/squashfs-root/usr/lib/libDrakX/bootloader.pm, because Not a directory


Sorry I am doing some educational digging here, trying to handle one file.  I did notice you said just to unravel the livecd.sqfs in a directory do the edit and squash again etc...

Have I misunderstood the unsquashfs command, I am treating it a bit like zip file where you can extract/zip single files from the bunch. ?

Here are instructions I am focused on, the second line is giving me trouble.

[gert@KDE32-test05 ~]$ man unsquashfs

[gert@KDE32-test05 ~]$ unsquashfs --help
SYNTAX: unsquashfs [options] filesystem [directories or files to extract]
.....
...
        -l (s)                   list filesystem, but don't unsquash                                    
.....
       -e[f] <extract file>    list of directories or files to extract.
...........
....
One per line
Decompressors available:
        gzip
        xz
[gert@KDE32-test05 ~]$ man unsquashfs

Had to modify [*s] option above and here adding *, the expression seems to spook editor doing strike through to the end of topic. Interesting.

Added-1:
Getting a bit further, with some trepidation I started the following ad I really did not want 140,000 files on this system.   

Code: [Select]

[root@KDE642-test05 gert]# unsquashfs /za/livecd.sqfs /squashfs-root/usr/lib/libDrakX/bootloader.pm
Parallel unsquashfs: Using 2 processors
0 inodes (0 blocks) to write


created 0 files
created 1 directories
created 0 symlinks
created 0 devices
created 0 fifos
[root@KDE642-test05 gert]# ls -al

Found the directory: /squashfs-root but despite praying that the single file would be inside

[root@KDE642-test05 squashfs-root]# ls -al
total 8
drwxr-xr-x  2 root root 4096 Dec  7 04:36 ./
drwxr-xr-x 34 gert gert 4096 Jan 15 23:26 ../
[root@KDE642-test05 squashfs-root]#

Hmmmm, it is as if only one level was done

[root@KDE642-test05 squashfs-root]# unsquashfs -l /za/livecd.sqfs /usr/lib/libDrakX/bootloader.pm
Parallel unsquashfs: Using 2 processors
1 inodes (1 blocks) to write

squashfs-root
squashfs-root/usr
squashfs-root/usr/lib
squashfs-root/usr/lib/libDrakX
squashfs-root/usr/lib/libDrakX/bootloader.pm
[root@KDE642-test05 squashfs-root]#

This was only the listing, pheeeww, could not find the file 

Code: [Select]

[root@KDE642-test05 squashfs-root]# unsquashfs /za/livecd.sqfs /usr/lib/libDrakX/bootloader.pm
Parallel unsquashfs: Using 2 processors
1 inodes (1 blocks) to write

[========================================================================/] 1/1 100%
created 1 files
created 4 directories
created 0 symlinks
created 0 devices
created 0 fifos
[root@KDE642-test05 squashfs-root]#


[Just17]

I suggest you do as I indicated earlier ......  unsquash the .sqfs file into a directory on your local partition.
You can then edit the .pm file (and any other you wish) in that directory, and mksquash the same directory into a new .sqfs file.

[wedgetail]

Just17
Errrh I was going to do that but as you can see in my add on in previous post, I have been scratching around a bit.  I am on the trail and just could not leave this single file question at that stage.

As mentioned I am only doing this as an eduction, I can see I more likely to go back to follow your suggestion if I am going to work more in this area.

The last information indicated that 1 file was written

Code: [Select]

[root@KDE642-test05 squashfs-root]# updatedb

[root@KDE642-test05 squashfs-root]# locate bootloader.pm
/home/gert/squashfs-root/squashfs-root/usr/lib/libDrakX/bootloader.pm

/usr/lib/libDrakX/bootloader.pm
[root@KDE642-test05 squashfs-root]#


[root@KDE642-test05 squashfs-root]# chown gert:root /home/gert/squashfs-root/squashfs-root/usr/lib/libDrakX/bootloader.pm
[root@KDE642-test05 squashfs-root]#

Sorry for highlighting so boldly but when I higlight the file shown in root konsole I get a pop up allowing to open file with either user kwrite or root kwrite

Notice that I change owner of the file so I could get write permission, that was not the smartest, I should have opened the file by choosing the option root kwrite this way I would not have messed up the permissions. It is very likely I will forget to set it back to root owner another time.

I don't think I can resist the temptation to explore this a lot, it beats finding out that nano <filename> is not in the 64 bit system or typing kwrite  <filename> in the konsole.  I think vi is available but that is crushing, I have tried to explore that a number of times.Hmmm old dogs new tricks  .

The above file I have now set the counter to 99 in line 575 and it is ZZZzz... time where I live.

Big question, can I squash that file back into the parent file?

[Just17]

While I well understand the route you have taken ----  been there done that with lots of things  ----- I would have done the fix the 'easy' way and then moved on to see if I could do it the way you are trying 

I look forward to seeing the results of your efforts 

[wedgetail]

Well getting closer, I have the fixed livecd.sqfs ready. Now what 

Looking inside the loopmounted ISO source file I see /boot and /isolinux where do they come from?

Are they kindly supplied by mkiso?

[Just17]

The procedure is

mount the ISO
copy the contents to a local directory
Do all the work from there

unsquash, edit and re-squash the livecd file and replace the original livecd.sqfs with the edited one (in the working directory)

make a new ISO from the working directory contents which now has the edited squash file.

[wedgetail]

Just17
Yes I ended up doing this, found an almost empty partition, looked like I got it right but the ISO fails to work, more later. Here is my path.

[root@KDE32-test05 gert]# mount -o loop  /media/B-18/130108-pclinuxos-kde-2012.12.iso /za

Could not help doing ls -l /za just to make sure the livecd.sqfs was there

root@KDE32-test05 gert]# unsquashfs /za/livecd.sqfs -d /media/B-20/comp/

My aim was just to run unsquashfs with no regard to where pwd was, just using long pathnames and specify destination. Failed could not figure out what was the problem, something about this squashfs-root directory already exist, not that I could see.  So I relented and made the working directory where files were copied to and did.

Code: [Select]

[root@KDE64-test05 comp]# unsquashfs  /za/livecd.sqfs
Parallel unsquashfs: Using 2 processors
128223 inodes (129153 blocks) to write

[============================================================================================================================\ ] 128486/129153  99%
create_inode: socket squashfs-root/var/lib/portreserve/portreserve.sock ignored
[============================================================================================================================\ ] 128634/129153  99%
create_inode: socket squashfs-root/var/run/acpid.socket ignored

create_inode: socket squashfs-root/var/run/audispd_events ignored
[============================================================================================================================\ ] 128648/129153  99%
create_inode: socket squashfs-root/var/run/avahi-daemon/socket ignored
[============================================================================================================================\ ] 128648/129153  99%
create_inode: socket squashfs-root/var/run/cups/cups.sock ignored
[============================================================================================================================\ ] 128648/129153  99%
create_inode: socket squashfs-root/var/run/dbus/system_bus_socket ignored
[============================================================================================================================\ ] 128648/129153  99%
create_inode: socket squashfs-root/var/run/nscd/socket ignored
[============================================================================================================================\ ] 128649/129153  99%
create_inode: socket squashfs-root/var/run/saslauthd/mux ignored
[============================================================================================================================\ ] 128649/129153  99%
create_inode: socket squashfs-root/var/run/sdp ignored
[============================================================================================================================\ ] 128649/129153  99%
create_inode: socket squashfs-root/var/run/uuidd/request ignored
[============================================================================================================================\ ] 128649/129153  99%
create_inode: socket squashfs-root/var/run/xdmctl/dmctl/socket ignored
[============================================================================================================================\ ] 128649/129153  99%
create_inode: socket squashfs-root/var/run/xdmctl/dmctl-:0/socket ignored
[============================================================================================================================\ ] 129141/129153  99%
created 119349 files
created 12613 directories
created 8208 symlinks
created 29 devices
created 0 fifos
[root@KDE64-test05 comp]#

Now I had the file system unraveled, found the bootloader.pm and set the counter to 99 in line 575.

Code: [Select]

[root@KDE64-test05 comp]# mksquashfs squashfs-root/ livecd.sqfs
Parallel mksquashfs: Using 2 processors
Creating 4.0 filesystem on livecd.sqfs, block size 131072.
[=============================================================================================================================|] 144487/144487 100%
Exportable Squashfs 4.0 filesystem, gzip compressed, data block size 131072
        compressed data, compressed metadata, compressed fragments, compressed xattrs
        duplicates are removed
Filesystem size 1771787.84 Kbytes (1730.26 Mbytes)
        39.41% of uncompressed filesystem size (4495231.13 Kbytes)
Inode table size 1419290 bytes (1386.03 Kbytes)
        30.02% of uncompressed inode table size (4727808 bytes)
Directory table size 1419414 bytes (1386.15 Kbytes)
        40.83% of uncompressed directory table size (3476602 bytes)
Number of duplicate files found 12869
Number of inodes 140199
Number of files 119349
Number of fragments 8481
Number of symbolic links  8208
Number of device nodes 29
Number of fifo nodes 0
Number of socket nodes 0
Number of directories 12613
Number of ids (unique uids + gids) 33
Number of uids 14
        root (0)
        messagebus (499)
        rpcuser (485)
        daemon (2)
        gert (500)
        uuidd (492)
        rpc (488)
        lp (4)
        xfs (486)
        rpm (494)
        usbmux (493)
        haldaemon (495)
        saned (487)
        avahi (498)
Number of gids 28
        root (0)
        messagebus (499)
        sys (3)
        unknown (479)
        daemon (2)
        disk (6)
        shadow (25)
        gert (500)
        lpadmin (488)
        unknown (480)
        polkituser (496)
        lp (7)
        adm (4)
        xgrp (486)
        man (15)
        tty (5)
        chkpwd (26)
        mysql (490)
        utmp (24)
        rtkit (489)
        haldaemon (495)
        rpc (482)
        saned (481)
        sasl (417)
        uucp (14)
        avahi (498)
        unknown (477)
        mail (12)
[root@KDE64-test05 comp]#
The livecd.sqfs I moved into a directory /media/B-20/comp/ISO/ where I also had the /boot and /isolinux with content.

Code: [Select]

[root@localhost gert]# mkisofs -o /media/B-18/gert_livecd_image.iso /media/B-20/comp/ISO/
Making the ISO did not seem to be big deal, I have the record fully but edited out most of the repeats in the following

Code: [Select]

[root@localhost gert]# mkisofs -o /media/B-18/gert_livecd_image.iso /media/B-20/comp/ISO/
I: -input-charset not specified, using utf-8 (detected in locale settings)
  0.56% done, estimate finish Wed Jan 16 19:21:25 2013
 ................................
 99.12% done, estimate finish Wed Jan 16 19:22:09 2013
 99.68% done, estimate finish Wed Jan 16 19:22:09 2013
Total translation table size: 0
Total rockridge attributes bytes: 0
Total directory bytes: 6338
Path table size(bytes): 38
Max brk space used 1e000
892868 extents written (1743 MB)
[root@localhost gert]#
I have done a quick loop mount below

Code: [Select]

[root@localhost gert]# mount -t iso9660 /media/B-18/gert_livecd_image.iso /mnt -o loop
mount: warning: /mnt seems to be mounted read-only.
[root@localhost gert]#

[root@localhost gert]# ls -l /mnt
total 1771794
dr-xr-xr-x 1 root root       2048 Dec  7 04:38 grub/
dr-xr-xr-x 1 root root       4096 Dec  7 05:08 isolinux/
-r-xr-xr-x 1 root root 1814310912 Jan 16 14:53 livecd.sqf*
[root@localhost gert]#

[root@localhost gert]# umount /mnt
[root@localhost gert]#

Here is the snag which I did not notice until liveHDD failed, despite I was told it was ok, so I spent fair time trying to figure the problem.  When I had a log for liveHDD creation I zoomed in on the livecd.sqfs.

When I listed my ISO content, I actually noticed the star at the end .sqf* I believe this may be why I am in trouble with liveHDD creation.

-r-xr-xr-x 1 root root 1814310912 Jan 16 14:53 livecd.sqf*

Have no idea why my new livecd.sqfs file changed name slightly after placement in the ISO.  Did I perhaps miss an option when doing mkisofs?   This is hard work  

Added:
This is from the liveHDD logging, at this stage I picked up when I read it for the 3rd time.  

....
line 809:  mv: cannot stat `/media/DELTA-07/livetmp/livecd.sqfs': No such file or directory
.....

[Just17]

Have no idea why my new livecd.sqfs file changed name slightly after placement in the ISO.  Did I perhaps miss an option when doing mkisofs?   This is hard work

I have no idea either ....  did you ensure you had only one livecd.sqfs in the directory and that the extracted squashfs was not included?

If it was easy, everyone would be doing it 

Also, to make your ISO bootable you will (I think) need to include a boot image .....  check the man page of mkisofs for info.

I went through the full exercise here a few minutes ago without any problems.
It produced an ISO (I did not include a boot image so it is not bootable) and had no problems like you described with file names.


Of course, most if not all of this can be done with something like  acetoneiso from the repository.

[wedgetail]

Just17

The following is what is in the directory:

Code: [Select]

[gert@localhost ~]$ ls -l /media/B-20/comp/ISO/
total 1773532
drwxrwxr-x 2 gert gert       4096 Dec  7 04:38 grub/
drwxrwxr-x 2 gert gert       4096 Dec  7 05:08 isolinux/
-rw-r--r-- 1 root root 1814310912 Jan 16 14:53 livecd.sqfs
[gert@localhost ~]$


Code: [Select]

[root@localhost gert]# mkisofs -o /media/B-18/gert_livecd_image.iso /media/B-20/comp/ISO/
I was concerned about getting extras, this why I moved to livecd.sqfs to the ISO directory.

I did notice that my final iso file had grown about 400 MB. Wondering if I needed some extra options when squashing again.

My exercise did include some levels in the pathnames, though I am floundering at the moment.

For good measure here is mkisofs version:

Code: [Select]

[root@localhost gert]# mkisofs -version
mkisofs 2.01 is not what you see here. This line is only a fake for too clever
GUIs and other frontend applications. In fact, this program is:
genisoimage 1.1.11 (Linux)
[root@localhost gert]#
Added:

-r-xr-xr-x 1 root root 1814310912 Jan 16 14:53 livecd.sqf*   <<--------------- my version
-rw-r--r-- 1 root root 1402900480 Dec  7 05:08 livecd.sqfs<<------------------------------------ original ISO from mirror

Notice my livecd.sqfs permissions changed during the mkisofs process.

Any chance to have a look at your ISO's ls -l of course I do wonder what you are doing different to me  

ZZZzz...

[kjpetrie]

This file is in drakxtools-backend, so the way to change it in future ISOs would be to submit an updated version of that package. Would a package request for the change be legitimate?