mylivecd compression

[Was_Just19]

I had an opportunity tonight to do a small test with mylivecd.
I made a remaster of my system ....  7.8GB installed ......  using first the gzip compression.
It produced an ISO of 2.8GB in about 25 mins.

So I then tried the (default?) xz compression.
It is now almost 2 hours 20 mins later, and it is still building the ISO .....  at the moment it is 2.3GB in size and the terminal is showing some 71% complete.

For me this is an unacceptable time ......  I should be in bed!  I had allowed an hour for the process.
I did not expect the time difference to be so great ......  and then for so little gain in size .. whatever that might be eventually!

It makes me wonder at the value of having the xz compression as default ......  I know I will be using gzip for all future remasters.

regards.

[Archie]

I too would be using the gzip compression. The trade off in time for a better compression is not worth the wait.

But as Tex did post that software are getting bigger and less are getting packed for a standard CD. I guess the difference in what we are doing in our remasters is the more or less official ISO to be distributed and our personal remasters. For personal remasters, although the size is also important, we can use DVDs. But for general distribution, size would be very important. The amount of software bundled in the OS either be more or less functionality for the end-user.

Perhaps it would be better to revert to the gzip compression for the mylivecd because for the benefit of the end-users. The devs can instead use the xz compression when they build ISOs for eventual distribution.

My 0.02 cents.

[Texstar]

Why can't you just type mylivecd --gzip new.iso ? Is it really that hard to remember?

[Archie]

Tex, I didn't mean me in particular but other users, especially the new ones that might just use mylivecd new.iso.

[Linuxera]

If you begin the process and then just head off to bed, to have a shiney new iso waiting for you in the morning, would that be too hard?  I had to redo Spring Fairy a dozen times to get it the way I wanted, and each time I started the process, then went on to do other things..   Have we become such a 'microwave, want it now' society that an hour or two to compress an image is such a big deal?  Seems totally unreasonable to gripe about a process like this..

[wedgetail]

Hey chaps, 

What about some facts?

How big systems are you re-mastering and how long did they take using whatever option?
On my systems when run I get a couple of lines indicating amount and time taken.

[Linuxera]

Hey chaps, 

What about some facts?

How big systems are you re-mastering and how long did they take using whatever option?
On my systems when run I get a couple of lines indicating amount and time taken.

I'm doing my remasters inside of VBox on an AMD Phenom x 3.  Only 1 CPU and 1.5BG RAM is dedicated to VBox.  I am running about a 655 MB ISO when all is said and done because I want to produce a CD, not a DVD from the 2.3 GB install.     It takes roughly one hour or over to produce that 655 MB ISO.

[wedgetail]

Your figures got me to delve into some records I have been keeping.  I do remember vaguely seeing something about changing compression I guess it was for mylivecd.  I do not know much about mylivecd I just blunder through using something like: mylivecd 110422-minime-backup.iso from root terminal, if I am adventurous I my battle through and make the full pass to its real resting place.

I am using a  twin engined Intel 2.4GHz with 4GB RAM

Tue April 19 2011
Bit of long time not much happening since last update, but have recovered my /root/.kde4 and /home/gert/.kde4
My customsettings are all back I think so I am ready for another big update

First a backup

[root@localhost gert]# mylivecd 110419-minime.iso
mylivecd, version 0.9.3, http://pclinuxos.com/
Copyright (C) 2010, Texstar <texstar at gmail.com>

Creating initrd:                                   [ 19.66% 00:00:00/00:00:00]install: cannot stat `/lib/i686/libpthread-2.10.1.so': No such file or directory
[==============                                    [ 21.37% 00:00:00/00:00:00]install: cannot stat `/lib/i686/librt-2.10.1.so': No such file or directory
               ==                                  [100.00% 00:00:04/00:00:04]
Setting filesystem parameters:                     [100.00% 00:00:02/00:00:02]
Creating compressed image:                         [100.00% 00:11:42/00:11:42]
Creating isolinux boot:                            [100.00% 00:00:00/00:00:00]
Creating final iso:                                [100.00% 00:00:43/00:00:43]
Embedding MD5 checksum:                            [100.00% 00:00:26/00:00:26]
                                                                             
Restoring Services on the installed system

Created '110419-minime.iso' (2225,225,728 bytes) in 00:12:59

[root@localhost gert]#

The reason for the big size is I have forgotten to re-locate one backup iso outside the system. Not important to me as in a short time I am not likely to need it.
This system was fully backed up on:

Commit Log for Thu Mar 24 17:17:43 2011

This update cost me my custom settings and I left it well alone.

Since I did very little Linux work, house maintenance called (still does, but a little test job sneaked in last few days). This testing on LiveHDD installs had one benefit to very smartly in matter of minutes recover my custom settings.  (Happy me  )

As a follow on the testing deserved to be done on a fully updated system. Update done on 19 April.  One other change took place, due to unbelievable problem with radiotray (not working at all) I also added Clementine (worked just great)

Further I removed the ISO before I made another mylivecd, to be used with the LiveHDD testing.

[root@localhost gert]# mylivecd 110422-minime_backup.iso
mylivecd, version 0.9.3, http://pclinuxos.com/
Copyright (C) 2010, Texstar <texstar at gmail.com>


Creating initrd:                                   [100.00% 00:00:04/00:00:04]
Setting filesystem parameters:                     [100.00% 00:00:04/00:00:04]
Creating compressed image:                         [100.00% 00:21:58/00:21:58]
Creating isolinux boot:                            [100.00% 00:00:00/00:00:00]
Creating final iso:                                [100.00% 00:00:20/00:00:20]
Embedding MD5 checksum:                            [100.00% 00:00:13/00:00:13]
                                                                             
Restoring Services on the installed system

Created '110422-minime_backup.iso' (1331,068,928 bytes) in 00:22:39

[root@localhost gert]#

I am just curious and had not really taken notice earlier that apparently a much smaller system now doubled the time for iso generation.  I do not understand most of all this I just happened to make detailed records for the MiniMe system I am testing with at present. 

[Archie]

If you begin the process and then just head off to bed, to have a shiney new iso waiting for you in the morning, would that be too hard?  I had to redo Spring Fairy a dozen times to get it the way I wanted, and each time I started the process, then went on to do other things..   Have we become such a 'microwave, want it now' society that an hour or two to compress an image is such a big deal?  Seems totally unreasonable to gripe about a process like this..

@Linuxera, it is not me who have problems. I know how to add parameters and I will be using the gzip compression. There will be other users who are gonna be posting ... four hours remastering an ISO and it didn't work! Bad enough with the remaster not working but the four hours? Yo, four hours is just an example.

I guess we can always tell them, xz compression does better compression but if you're not too concern about that, you can always try adding --gzip.

Anyway, I do what Tex already had posted with the --nofile --nodir --workdir --tmp, etc. and I am fine with the mylivecd.

@wedgetail, did you check /tmp to see if there are previous files that were saved from your previous remasters?

[Was_Just19]

Why can't you just type mylivecd --gzip new.iso ? Is it really that hard to remember?

I can; I did; and I said I would in the future.  Was my post that hard to understand?

As Archie pointed out, I can see the benefit of the xz compression when trying to eke out the last bit of space for a CD release.
The time it takes for a personal remaster is going to cause a lot of questions and complaints about the time it takes, by users who use the default   mylivecd new.iso  , IMO.
If you are happy with that, then all is OK.

If you begin the process and then just head off to bed, to have a shiney new iso waiting for you in the morning, would that be too hard?  ......

Only problem is, the PC does not run without power to it ........  and all such electrical appliances are switched off at night time.

For anyone interested, the ISO I made was 2.4GB in size using the xz compression (a little over 2.5 hours), 2.8GB with gzip (25 mins). I did not try it with lzma, but guess it would be somewhere between the two.
They were done on my 3.2Ghz P4 HT 4GB memory with nothing else in use except a browser. Both 'CPUs' were maxed out for the 3 hours.

regards.

[Neal ManBear]

Perhaps a how to for new mylivecd users might be good to have. Then it could be linked to in replies.

[wedgetail]

Archie

@wedgetail, did you check /tmp to see if there are previous files that were saved from your previous remasters?

No, and I often forget to turn off external mounted devices. I only remember that the last I read about making iso files was when mklivecd or something like that where I had to insert options, worked well but everytime I had to find my notes and study what I did . An accidental use of mylivecd and I was converted, did not look up anything, so I just assume mylivecd strips off anything not needed to work optimal. About 10minutes for a MiniMe system with my customisation and a bit under 1GB for final ISO I am happy.  

I am more interested in getting LiveHDD install to work so it is only a few minutes to install and iso to LiveHDD run it and fish out what I need (ie from my backup iso files)  

Neal
Yes, I actually assumed there was such a thread but do not have time to look for it when what I am doing seems to work. Anyway I love it.  Had just got used to rsync -a etc but now I can stake several LiveHDD installs into the same partition it is hard to beat.

[Neal ManBear]

Neal
Yes, I actually assumed there was such a thread but do not have time to look for it when what I am doing seems to work. Anyway I love it.  Had just got used to rsync -a etc but now I can stake several LiveHDD installs into the same partition it is hard to beat.

There is a "how to" found in, of all places the LiveCD, MyliveCD, LiveUSB and Remastering section.
http://www.pclinuxos.com/forum/index.php/topic,74418.0.html

What I was talking about however is something geared to the new user.

BTW, before remastering there are 2 things to remember to do. First, do a clean up - bleachbit and bleachbit root. Or you can do it manually, if you prefer. Second, unmount unwanted partitions/drives - as root, umount -a. (not a misspelling)

[scoundrel]

whats the difference between xs and gzip .??  not asking about time .. and what happened to "remasterme" ?

[Was_Just19]

whats the difference between xs and gzip .??  not asking about time .. and what happened to "remasterme" ?

xz and gzip are different compression formats which mylivecd can use ......  it can also use lzma compression.

remasterme was a small script which prepared the user's environment for remastering.
It did things like unmount all but root and home partitions and a couple of things of that nature, and then it ran the mklivecd script.
mylivecd script has replaced mklivecd script.

So if you wish you could use a new remasterme script to run mylivecd as was done previously with mklivecd.

Hope that makes some sense