[SOLVED] Persistence and changes boot code

[Was_Just19]

This is another one of those threads about something that is impacting on variations in usage of USB and other types of live booting.

As I understand things presently, and I may be wrong of course as I am just basing this on what I think I have observed, the folowing applies:-

when using the changes_dev=  boot code, the destination partition, with an ext filesystem, is given in some form that is usable. That form may be  /dev/sdx,y, LABEL or UUID. On recognising this boot code the booting system will create, if it does not already exist, a  changes  directory on the specified partition, within which will be stored the various system folders which hold changes made during the session.
If the changes directory already exists, then the contents are written into the booting system and are thus present when booted.

the changes_file= boot code is somewhat similar. This boot code accepts the path to a specially prepared file. The file must contain an ext filesystem, and is loop mounted in the booting system, for reading previously made changes and writing new changes to the system directories which it will contain.
Unfortunately, unlike a changes directory, a changes file size must be predetermined when creating it, so it always takes up the same space whether used or not. Not very efficient. Nonetheless it has one attractive feature for some .....  it can be stored on most filesystems and is not limited to ext like the changes directory, because of course the ext filesystem is contained within the file.

So most usage situations are covered, even if it is a bit awkward to implement sometimes.

The most awkward situation, in my view, is where the user wants to save all changes from maybe booting different versions of PCLOS ( or other distro Boo! ) on one single partition, to keep all such changes in one location for convenience.

I expect this to become a more common situation in the future, as USB flash sticks become more used and the utilities for their use as boot devices for different distros becomes more common.

I have been up against this problem for quite some time.
I have not found an easy solution.
Most of the time I resort to putting the changes directory for different versions on different ext partitions, which keeps them separated. I have tried using the changes_file= boot code but found it less than satisfactory for my use.

While up against this problem again today I began to wonder ......  and such musings brings me to the following .....

Is there any real reason why two separate boot codes are needed for this purpose?
Would it not be more convenient, more flexible, and ultimately more useful, if there was just the one such boot code?

I have in mind a boot code of  changes=  which can be followed by a path, and if requird a file name.

So to replace a boot code like this

changes_dev=LABEL=LivePCLOS

which will create (or use if it exists) a changes directory on the specified partition
we would write

changes=LABEL=LivePCLOS/changes

In other words we would write the complete path to the directory where the changes are to be stored.

If we were using the changes_file= boot code this might be what happens ....

changes_file=LABEL=LivePCLOS/<file name>

which would be changed to

changes=LABEL=LivePCLOS/changes/<file name>

It should be a simple matter to determine if the path given contains a file name at the end or not, and thus determine whether such a file would be loop mounted or a directory would be used, as done presently with two different boot codes.

Of course such a scheme would also give the user much more flexibility about the naming of the directory into which the changes would be saved.
That would allow changes for multiple distros/versions to reside without interfering with each other, on a single partition.
So the user might have the following changes directories on their specified partition ...
/kde_full/
/minime/
/gnome/
etc etc ...  or any other scheme of locating their changes they decide.

I have not gone so far as to investigate where (in what file) these boot codes are determined.
I suspect it should not be a big problem to edit the code to eliminate one of the boot codes, as described, and allow the user to determine the location of the saved changes.
Presently it seems that the saved changes are hard coded to the changes directory on the specified partition.
What I propose would overcome that limitation and allow the user to determine what such a directory is named.
That alone would allow multiple changes to be saved on a single partition using the path to a specified directory.
If using a non-ext filesystem there would be very little change .....  the path to the file and file name would be specified.

***

Your challenge, should you decide to accept it, is to discuss the above and comment on why it is a good or bad idea ....  or maybe impossible to implement for some reason I have not considered.

This thread should not, I hope, self-destruct!

Over to you      

[Was_Just19]

I have been thinking a little about this and wonder if something like this might work .....

Code: [Select]

CHANGES="`cmdline_value changes`"
if [ "$CHANGES" != "" ]; then
if [ -d "$CHANGES" ] ;
then
echo "Path is directory only"
# Do what is presently done for changes_dev= boot code
else
echo "Path is not a directory "
# Do what is presently done for changes_file= boot code
# presulably checking that the file exists and can be loop mounted etc.
fi
fi
I still don't know where all this happens though ......  anyone got ideas on that?

[Neal ManBear]

This seems a good idea. Hmm...... how to implement, if possible.....

[Was_Just19]

This seems a good idea. Hmm...... how to implement, if possible.....

First things first I guess .........  where is this controlled?

Is there a file somewhere that might be editable?

[Neal ManBear]

Time to search the system. It will be in there somewhere.

[Was_Just19]

Time to search the system. It will be in there somewhere.

I have checked the mylivecd package and linuxrc as I thought those might yield some indication .... unfortunately not.

Any suggestions where to look next?

Think I have it!

rc.sysinit  in  /usr/share/mylivecd

[Neal ManBear]

Hmm...... yes, that does appear to be it.

[Was_Just19]

Hmm...... yes, that does appear to be it.

Yep .....  in the "do_union" function.

Now to try to figure out the easiest means of making the changes_dev apply a full path when it is given and not just the device.

That may take me some time - if at all - so anyone/everyone with suggestions please jump in here ...

[Was_Just19]

There may be a problem with the thought process outlined above ...  although not insurmountable.

Where a device is specified ....  such as using UUID ..... the device is mounted in the booting system.

If we were to specify a path in the changes code then that path would either need to be mounted or specifically mounted in the code ......  I guess just like the device is at present.

So not so much a problem I suppose, as something else that needs to be considered.

***

I have been doing some tests this afternoon, and have succeeded in saving changes to a specific directory on the same partition that the OS is booting from.

In truth this would cover quite a lot of what I set out to do --- if I can tidy it up.

It does not help, of coirse, if I want to save all changes in different directories, on maybe an internal HDD ......  like having a Home location for the various OS versions.

Still testing .......  so will report back again if anything further to report.


regards.

[Was_Just19]

OK, some progress hase been made in relation to this whole thing.

If you use the PCLinux-liveusb  utility from the repository to do the installs of the live OSs the following should hold true.

It should be simple matter to include this in a future version of the utility. I will look into that later.

***

The utility renames the OS files that it extracts from an ISO, so that many different versions of PCLOS can reside on a single partition.

To allow each of those installs to be able to save changes (Persistence) the boot code can be changed in the Grub menu.lst, without doing any other changes that I can see. In addition a suitably named directory must be created on the partition.

As an example, I am again using the KDE Mini ISO for tests .......... being more familiar with KDE.

I named the install "My_Minie2" as it was a remaster ........  and misspelled it deliberately.
The OS files were automatically renamed to "My_Minie2_0" by the utility, and I created a directory in the root of the partition called "My_Minie2".

I created an extra boot stanza in menu.lst on the partition as this was a second OS install on the partition and no persistent stanza was automatically created.
The boot stanza looks like this .........

title   My_Minie2 with Persistence
kernel (hd0,0)/My_Minie2_0/vmlinuz livecd=My_Minie2_0 fromusb root=UUID=e6cfaeb0-772e-4a5e-a6d9-7851b558f571 acpi=on vga=791 changes_dev=/initrd/cdrom/My_Minie2
initrd (hd0,0)/My_Minie2_0/initrd.gz

The result of this is that the saved changes are stored on the partition that booted, in the following directory .........

/My_Minie2/changes/

Changes for the other OS installed were originally stored on the partition in

/changes/

Altering the boot code for the other OS installed allows them both to have their own changes saved without interfering with each other.

***

If others would care to test this on their devices and hopefully with different DE versions of PCLOS, that would be greatly helpful.

Your results posted here can confirm (or not) that this is a universal solution of part of the problem, at least.

It certainly is sufficient for the present, to allow more flexible use of the USB devices ...........   for which I am happy!

Over to the testers to confirm this so that the utility might be adjusted to accomodate it.

regards.

PS.      The extra functionality is not being forgotten, but I think it can remain on the back burner until this part is confirmed.

.

[Was_Just19]

I have been testing this since I posted and have had no problems so far.

I then made changes to the PCLinuxOS-liveusb utility to incorporate the things needed to use this method and have checked that out also ...  but in limited circumstances -- such as I only ran it from an installed OS and used only an ISO as the source for the OS.
The changes will need to be tested both from a liveCD & liveUSB, and using both an ISO as well as the files from the live media.
But that is down the line somewhat I guess.

For now it seems this works for the liveUSB situation, which was the primary concern.

Still looking for some who will try this on their own setup and report back.

regards.

[Was_Just19]

Having done a further series of tests, the limitations of this method are beginning to necome apparent.

As is failry obvious, the destination of the changes must be a mounted file system.

If Copy2RAM is used in conjunction with the changes code above, the destination is not mounted and is thus not usable. 

So, back to the drawing board it seems.

Back to the first idea again of specifying the 'device' via UUID/LABEL etc but making it so that a directory on that device can be used and not just the root of the device.

So it seems here .....  other people's tests will likely confirm this I guess ......  so look forward to seeing the results.

regards.

[Was_Just19]

I believe a solution has been found to the problem of maintaining independent changes files for different versions of PCLOS, on the one partition.
This will also allow anyone using the liveusb creator utility to have several boot options for each installed OS. Those would be

Standard boot ....  running from the USB device; all changes made forgotten at shutdown.

Copy2RAM .....  same as Standard, except the loop image is copied to RAM if there is enough, and runs from there, allowing the USB device to be withdrawn if required.

Persistent ......  same as Standard except that all changes made while running are saved on a user specified partition, and in a sub-directory of a "changes" directory, named from the name of the loop image, which itself is dependent on the name supplied by the user at installation.

Copy2RAM with Persistence ......  a combination of both ...  runs from RAM and changes saved to a specified partition.

To get the functionality a couple of small edits are needed to the rc.sysinit file in the mylivecd directory.
Granted the testing of these changes has not been exhaustive, but they appear to be solid so far.
Having made the changes, a remaster is required to incorporate those changes in the ISO.

The detail:

Code: [Select]

1.
Line 412 presently reads

CHANGES=/initrd/changes/changes/


Change it to

CHANGES=/initrd/changes/changes/$BASEIMG



2.
Insert after line 475 this extra line

livecd=*) BASEIMG="${x#livecd=}"; [ -z "$BASEIMG" ] &&  BASEIMG="livecd"; ;;

This is just a copy of the line in the Linuxrc file which does the same thing ....  gets the
name of the loop image file and applies it to the variable $BASEIMG, which is then used
to create the subdirectory in the /changes directory on the partition.
After that it is merely a matter of editing the menu.lst file on the usb device to include the other options (as above) that become available.

Should this prove to be successful, and acceptable for PCLOS, there will be an update of the liveusb creator package which will automatically create the new boot options on use.

The next round of quarterly ISOs should then automatically have those options available.

That is the hope.

Now the waiting starts to see if there are any undesirable results .....  so if anyone should decide to test this, please post here so that we know what happens.

Here is an example of my boot stanzas in menu.lst from one of the tests I did .....

Code: [Select]

title   KDE
kernel (hd0,0)/KDE_0/vmlinuz livecd=KDE_0 fromusb root=UUID=d70f741a-f35e-41b4-a68e-5b5a7707fcdb acpi=on vga=791 splash=verbose
initrd (hd0,0)/KDE_0/initrd.gz
 
title  KDE with Persistence
kernel (hd0,0)/KDE_0/vmlinuz livecd=KDE_0 fromusb root=UUID=d70f741a-f35e-41b4-a68e-5b5a7707fcdb acpi=on fstab=rw,noauto vga=791 changes_dev=UUID=d70f741a-f35e-41b4-a68e-5b5a7707fcdb splash=verbose
initrd (hd0,0)/KDE_0/initrd.gz
 
title  KDE with Persistence & Copy2RAM
kernel (hd0,0)/KDE_0/vmlinuz livecd=KDE_0 fromusb copy2ram root=UUID=d70f741a-f35e-41b4-a68e-5b5a7707fcdb acpi=on fstab=rw,noauto vga=791 changes_dev=UUID=d70f741a-f35e-41b4-a68e-5b5a7707fcdb splash=verbose
initrd (hd0,0)/KDE_0/initrd.gz


and one where I tested the "changes_file=" boot code

Code: [Select]

title  KDE with Persistence by File
kernel (hd0,0)/KDE_0/vmlinuz livecd=KDE_0 fromusb root=UUID=d70f741a-f35e-41b4-a68e-5b5a7707fcdb acpi=on fstab=rw,noauto vga=791 changes_file=UUID=d70f741a-f35e-41b4-a68e-5b5a7707fcdb/kde splash=verbose
initrd (hd0,0)/KDE_0/initrd.gz

NB:  The UUID numbers above will be different for other partitions. Generate and use your own.


The changes file has of course to be created first, made a particular size and then an ext filesystem written in it, so that it can be used.

I believe that is all the info there is available at this time.

Hopefully this will be an acceptable patch for the mylivecd utility and we can incorporate those facilities into the liveusb creator soon.

Do post if you check this out as feedback could be invaluable. Thanks.

regards.

[Neal ManBear]

Excellent!

[Neal ManBear]

Is johnboy, Just19 and Neal the same person(s)?   

I am me! Separate! Individual!