Author Topic: mediacheck?? (Read 3769 times)

AS · « **Reply #15 on:** October 09, 2011, 04:04:52 PM »

A step further:

# implantisomd5 pclinuxos-kde-fullmonty-2011.09.iso

then run the iso image from Vbox, at grub screen press F6, select default, add the option md5sum (it's the same thing that was done from the MediaCheck grub menu entry), kernel will load and mediacheck will start, result PASS.

AS

pinoc · « **Reply #16 on:** October 10, 2011, 08:03:36 AM »

ok, some update:
there seems to be a difference between
a) implanting md5sum on an existing iso and running media check (works, I can confirm the tests by AS)
b) implanting md5sum when doing a remaster and doing the media check from the grub-screen of the live-session (fails)

what I did: added back Media Check and embedding md5sum in mylivecd, then remastered my current FM (2010.10)
what I get: a fully functional FM, the embedded md5sum is the same as the one calculated from the remastered iso. Running media check from the grub of the live-session starts perfectly fine, tests and completes to 100% and the result is FAIL, then you can click on OK (you have no other option anyway) and the live-session nicely continues to a fully functional live-session desktop...

Conclusion: I see nothing wrong in the way the md5sum is calculated and implemented by mylivecd on the remastered iso. The same is true for media check. media check on a remastered iso does work for a CD-remaster and it does not work for a DVD-remaster. I don't know why it does not work on a DVD-remaster: maybe because of the iso-size; or some limitation of squashfs, mediacheck, etc.; or probably because the live-session environment is not the same as that of an installed system; and maybe it even does work for a DVD-remaster but not when remastering the FM for whatever reason, I have no idea. I have not yet found out what to add/change to have it working when remastering a DVD. media check on a live-session is a nice feature to have but only if it works. It is not essential and I don't want to spend more time on this, if someone finds a fix I'd be more than happy to add it back on the next FM.

best regards,
-p.

AS · « **Reply #17 on:** October 10, 2011, 10:38:26 AM »

Quote from: pinoc on October 10, 2011, 08:03:36 AM

ok, some update:
there seems to be a difference between
a) implanting md5sum on an existing iso and running media check (works, I can confirm the tests by AS)
b) implanting md5sum when doing a remaster and doing the media check from the grub-screen of the live-session (fails)

As far as I can see, to do mediacheck from command line or from grub screen lead to the same result, I have to think that something is going wrong at time of "implanting the md5sum".

Quote

what I did: added back Media Check and embedding md5sum in mylivecd, then remastered my current FM (2010.10)
what I get: a fully functional FM, the embedded md5sum is the same as the one calculated from the remastered iso.

This is interesting, because from my tests the embedded/implanted md5sum it's always different from the md5sum calculated from the whole iso, unless I'm misunderstanding "the one calculated from the remastered iso."

Quote

Running media check from the grub of the live-session starts perfectly fine, tests and completes to 100% and the result is FAIL, then you can click on OK (you have no other option anyway) and the live-session nicely continues to a fully functional live-session desktop...

Conclusion: I see nothing wrong in the way the md5sum is calculated and implemented by mylivecd on the remastered iso. The same is true for media check. media check on a remastered iso does work for a CD-remaster and it does not work for a DVD-remaster. I don't know why it does not work on a DVD-remaster: maybe because of the iso-size; or some limitation of squashfs, mediacheck, etc.; or probably because the live-session environment is not the same as that of an installed system; and maybe it even does work for a DVD-remaster but not when remastering the FM for whatever reason, I have no idea. I have not yet found out what to add/change to have it working when remastering a DVD. media check on a live-session is a nice feature to have but only if it works. It is not essential and I don't want to spend more time on this, if someone finds a fix I'd be more than happy to add it back on the next FM.

best regards,
-p.

I fully agree, it's useful only if work, and understand that this is a time consuming task. As reported before, actually I have had only PASS results, so I'm going to test fullmonty directly, will report back about results as soon as possible.

AS

pinoc · « **Reply #18 on:** October 10, 2011, 11:04:29 AM »

Hi AS,
a quick follow up: maybe stating the obvious but on a correct iso-image with an implanted md5sum, that implanted md5sum must be the same as the one calculated from a comand-line, like md5sum xx.iso. Implanting a md5sum later into an existing iso will obviously change the md5sum of the original iso.

If you want to test media check and implanting the md5sum on the FM then you would need to a) add the mediacheck boot option in mylivecd, b) remaster an installed FM with this modified mylivecd (and using the default highest compression, maybe that causes the problem???), and c) test the media check boot option in the live-session of that remaster. This is what I did this afternoon and the result is fail. This is quite some work, are you sure you want to do that?

The problem with mediacheck seems to only appear when remastering with mylivecd

best,
-p.

Just17 · « **Reply #19 on:** October 10, 2011, 11:39:06 AM »

I have had a strange result ..... using zen-mini 2011.09 ISO ...

First I tried mediacheck on the ISO ==> failed to find embedded MD5sum

Second I took the md5sum of the ISO ==> 7a00c456cd4eb8b6be47275490bbe633

Next I implanted an MD5sum ==> the output says md5 = eb6fcfbc6ae64dbb0f447f01cc253e51

So then I check the MD5sum again ==> 93bcd00c0224e9151b1acd8308e3f6f1

Thinking something is surely wrong, I did a mediacheck on the ISO ==> PASS!!!

Code: [Select]

[user@AMD ~]$ mediacheck ~/pclinuxos-Zen-Mini-2011.09.12.iso
[user@AMD ~]$ ==>> FAIL  couldn't find embedded md5sum <<==
[user@AMD ~]$ md5sum ~/pclinuxos-Zen-Mini-2011.09.12.iso
7a00c456cd4eb8b6be47275490bbe633  ~/pclinuxos-Zen-Mini-2011.09.12.iso
[user@AMD ~]$
[user@AMD ~]$ implantisomd5 ~/pclinuxos-Zen-Mini-2011.09.12.iso
Read     0 MB
Read    50 MB
Read   100 MB
Read   150 MB
Read   200 MB
Read   250 MB
Read   300 MB
Read   350 MB
Inserting md5sum into iso image...
md5 = eb6fcfbc6ae64dbb0f447f01cc253e51
Done!
[user@AMD ~]$ md5sum ~/pclinuxos-Zen-Mini-2011.09.12.iso
93bcd00c0224e9151b1acd8308e3f6f1  ~/pclinuxos-Zen-Mini-2011.09.12.iso
[user@AMD ~]$ mediacheck ~/pclinuxos-Zen-Mini-2011.09.12.iso
[user@AMD ~]$ ==>>  PASS <<==

What am I missing here? ...... this is surely wrong somewhere?

The md5sum reported from the implantiso command should be that which the ISO had after the implantation (or else before) ........ but it is neither.

... ??

pinoc · « **Reply #20 on:** October 10, 2011, 01:12:23 PM »

Hi Just18,

just checked the same steps on a tiny iso-image which did not have a md5sum included.
When inserting the md5sum implantisomd5 reports a checksum which is different to the one before as well as after the insertion, meaning I can confirm your findings and indeed, weird stuff. After that, doing a mediacheck as well as checkisomd5 provides a positive answer. While mediacheck only says the iso is OK, checkisomd5 is more verbose and prints out the imbedded and the checked md5sums stating that they match. Here, the md5sum is the one reported by implantisomd5 during the insertion. BUT this md5sum echoed by checkisomd5 is not the same as the one obtained from the cmd-line, doing md5sum xx.iso. And that is really not what I expected, so there are two md5sums for the same iso-file?? One for md5sum and one for checkisomd5, oh boy...

BUT, AS your tests make me think of an idea how to get mediacheck working on the FM:
Since mediacheck does not show any md5sums but only reports FAIL or OK, I can insert the md5sum later on the FM-remaster overwriting the existing one, and after that mediacheck should work, as tested with the small iso right now. Will test this tomorrow and report back. Heck, what a mess...
-p.

AS · « **Reply #21 on:** October 10, 2011, 01:17:28 PM »

Quote from: pinoc on October 10, 2011, 11:04:29 AM

Hi AS,
a quick follow up: maybe stating the obvious but on a correct iso-image with an implanted md5sum, that implanted md5sum must be the same as the one calculated from a comand-line, like md5sum xx.iso. Implanting a md5sum later into an existing iso will obviously change the md5sum of the original iso.

Hi pinoc,

thank for the follow up.

I have observed the same behavior as reported from Just18, one md5sum before implanting the md5sum, another different as reported from implantisomd5, a third different as calculated from md5sum on the iso with implanted checksum. Still the result of mediacheck is PASS (and also the result of checkisomd5)

The following result of checkisomd5 on my 4 Gb remaster, the md5sum shown is the same as reported from implantisomd5,
and on the same iso mediacheck result is PASS. (and it is different from md5sum as.iso before and after the implanting)

Quote

# md5sum as.iso (before implanting)
27b28090b47122e069fa8acf7d60ced7 as.iso

# implantisomd5 as.iso
Read 0 MB
Read 50 MB
...
Read 3750 MB
Read 3800 MB
Read 3850 MB
Inserting md5sum into iso image...
md5 = 16bf0ceff844ff38fa7993d1e543afe5
Done!

#md5sum as.iso (after implanting)
c8f7206c20e976dc5f36b9d9d006f2ab as.iso

# checkisomd5 as.iso
...
Read 3750 MB
Read 3800 MB
Read 3850 MB
16bf0ceff844ff38fa7993d1e543afe5
16bf0ceff844ff38fa7993d1e543afe5
Md5sums match.

If you are sure that the implanted md5sum must be equal to the calculated md5 before implanting, that clearly means that implantisomd5 is broken, only that at this point I should consider broken also mediacheck and checkisomd5

Quote

If you want to test media check and implanting the md5sum on the FM then you would need to a) add the mediacheck boot option in mylivecd, b) remaster an installed FM with this modified mylivecd (and using the default highest compression, maybe that causes the problem???), and c) test the media check boot option in the live-session of that remaster. This is what I did this afternoon and the result is fail. This is quite some work, are you sure you want to do that?

The problem with mediacheck seems to only appear when remastering with mylivecd

best,
-p.

It's true that I have used --gzip compression on my 4 Gb remaster, will try using max (default) compression, however it appear to be working on the latest kde-2011-09 official iso. I fully understand that I'm going to spend some time around the issue, but it will not be wasted, I'm really interested on mylivecd, also it's one of the few area where hopefully I can provide some (2 cents

) little help. You also should know that I'm working on a related project

As a side info, because I saw mksquashfs mentioned in a previous post, there is no 4 Gb limit on mksquashfs, we have (me and Just18) already verified that it's possible to successfully build a liveUSB containing a file.sqfs larger than 4 Gb.

Thanks also to Just18 for confirmation of results.

Regards
AS

Quote

BUT, AS your tests make me think of an idea how to get mediacheck working on the FM:
Since mediacheck does not show any md5sums but only reports FAIL or OK, I can insert the md5sum later on the FM-remaster overwriting the existing one, and after that mediacheck should work, as tested with the small iso right now. Will test this tomorrow and report back. Heck, what a mess...
-p.

One thing I have observed is that if the md5sum is implanted a second time (using -f option) then the next mediacheck will FAIL

Regards
AS

Just17 · « **Reply #22 on:** October 10, 2011, 01:21:35 PM »

What if the problem is with the command md5sum only?

Code: [Select]

[user@AMD ~]$ checkisomd5  ~/pclinuxos-Zen-Mini-2011.09.12.iso
Read     0 MB
Read    50 MB
Read   100 MB
Read   150 MB
Read   200 MB
Read   250 MB
Read   300 MB
Read   350 MB
eb6fcfbc6ae64dbb0f447f01cc253e51
eb6fcfbc6ae64dbb0f447f01cc253e51
Md5sums match.
[user@AMD ~]$ md5sum ~/pclinuxos-Zen-Mini-2011.09.12.iso
93bcd00c0224e9151b1acd8308e3f6f1  ~/pclinuxos-Zen-Mini-2011.09.12.iso
[user@AMD ~]$

Mediacheck says all is OK

The only thing out of step seems to be md5sum

mediacheck says all is OK; implantisomd5 reports the same md5sum as checkisomd5 .......

Possibility ...... two different checks going on? ..... one including the embedded md5 and another not ..... ??

EDIT:
When attempting to burn the ISO above with K3b ...... it reported the same md5 as md5sum command ....
Looking more like implant and check do not include the embedded md5 ...... which would make mdsum correct ..... but what then is mediacheck doing?

...... too confusing ....

EDIT2
If mediacheck is using the embedded md5, and checking that in the same manner as implant and checkisomd5 that would explain how it passes ISOs after the md5 has been implanted.
If mylivecd is implanting the md5, calculated slightly differently, that would explain why such ISOs fail.

Then to make matters worse ..... after implanting and checking an md5 in bonsai this is what I get ....

Code: [Select]

[user@AMD ~]$ md5sum -c ~/pclinuxos-openbox-bonsai-test-2011-06-18_2.iso
md5sum: ~/pclinuxos-openbox-bonsai-test-2011-06-18_2.iso: no properly formatted MD5 checksum lines found

Seems there are two different md5 schemes at play ......

EDIT 3
Just checked .... mylivecd uses implantisomd5 ...... so now completely confused ...... I give up

Last comment ..... I note at line 1181 of mylivecd, that checkisomd5 was previously used ..... maybe going back to it might resolves the problem ......

AS · « **Reply #23 on:** October 10, 2011, 02:08:06 PM »

Quote from: Just18 on October 10, 2011, 01:21:35 PM

What if the problem is with the command md5sum only?

no, thank you.

for sure I have obtained the same results as described here: http://en.wikipedia.org/wiki/MD5
(chapter MD5 hashes, the most simple check is to try on an empty files ...), md5sum is OK.

BTW, I have used the command md5sum to verify other media ... coming from outside linux world ...

Quote

Code: [Select]
[user@AMD ~]$ md5sum -c ~/pclinuxos-openbox-bonsai-test-2011-06-18_2.iso md5sum: ~/pclinuxos-openbox-bonsai-test-2011-06-18_2.iso: no properly formatted MD5 checksum lines found

Confirmed here too.

Quote

Just checked .... mylivecd uses implantisomd5 ...... so now completely confused ...... I give up

Yes, it use implantisomd5. I will continue to search what is going wrong

EDIT:

Quote

Last comment ..... I note at line 1181 of mylivecd, that checkisomd5 was previously used ..... maybe going back to it might resolves the problem ......

So far, for me at least, mediacheck and checkisomd5 are providing always the same results.

Just17 · « **Reply #24 on:** October 10, 2011, 02:12:27 PM »

http://permalink.gmane.org/gmane.comp.handhelds.meego.scm/26350

Quote

State: new 2011-07-27T19:09:56 guichen

I thought this might hold some info about a recent update to implantisomd5 or or related package ..... if it does I failed to notice it

AS · « **Reply #25 on:** October 10, 2011, 03:06:17 PM »

Quote from: Just18 on October 10, 2011, 02:12:27 PM

http://permalink.gmane.org/gmane.comp.handhelds.meego.scm/26350

Quote
State: new 2011-07-27T19:09:56 guichen

I thought this might hold some info about a recent update to implantisomd5 or or related package ..... if it does I failed to notice it

reading the patch I noticed:
- the support for the latest gcc 4.6,
- the addition of a sparc64 flag
- some verbose output additions
and the most suspect and interesting changes are related to a couple of arrays, size increased by one byte, possibly to provide space for a "string terminator" ...:

Code: [Select]

-    char fragmentsums[FRAGMENT_SUM_LENGTH];
-    char thisfragsum[FRAGMENT_SUM_LENGTH];
+    char fragmentsums[FRAGMENT_SUM_LENGTH+1];
+    char thisfragsum[FRAGMENT_SUM_LENGTH+1];

AS · « **Reply #26 on:** October 10, 2011, 05:25:12 PM »

Another weird thing

I have a separate /tmp partition, size 10 Gb, usually i build my remaster as /tmp/as.iso. I was going to need rebuild my ISO using maximum compression, thus requiring some time, so I launched the following commands:

[root@dv1710 tmp]# mylivecd as.iso ; mylivecd --gzip --md5sum as1.iso

the intent was to build the first as.iso at maximum compression, and a second one implanting the md5sum, look at output:

Code: [Select]

[root@dv1710 tmp]# mylivecd as.iso ; mylivecd --gzip --md5sum as1.iso
mylivecd, version 0.9.4, http://pclinuxos.com/
Copyright (C) 2010, Texstar <texstar at gmail.com>

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

Disabling Services not needed on the LiveCD

running: /sbin/chkconfig --list
running: /sbin/chkconfig --list
running: /sbin/chkconfig --list
running: /sbin/chkconfig --list
running: /sbin/chkconfig --list


Creating initrd:                                   [100.00% 00:00:06/00:00:06]
Setting filesystem parameters:                     [100.00% 00:00:18/00:00:18]
Creating compressed image:                         [100.00% 01:30:01/01:30:01]
Creating isolinux boot:                            [100.00% 00:00:00/00:00:00]
Creating final iso:                                [100.00% 00:01:15/00:01:15]

Restoring Services on the installed system


Created 'as.iso' (3357,247,488 bytes) in 01:31:45

mylivecd, version 0.9.4, http://pclinuxos.com/
Copyright (C) 2010, Texstar <texstar at gmail.com>

This program is free software; you can redistribute it and/or
modify it under the terms of the GNU General Public License
as published by the Free Software Foundation; either version 2
of the License, or (at your option) any later version.

Disabling Services not needed on the LiveCD

running: /sbin/chkconfig --list
running: /sbin/chkconfig --list
running: /sbin/chkconfig --list
running: /sbin/chkconfig --list
running: /sbin/chkconfig --list


Creating initrd:                                   [100.00% 00:00:07/00:00:07]
Setting filesystem parameters:                     [100.00% 00:00:13/00:00:13]
Creating compressed image:                         [100.00% 00:13:52/00:13:52]
Creating isolinux boot:                            [100.00% 00:00:00/00:00:00]
Creating final iso:                                [100.00% 00:01:10/00:01:10]
Embedding MD5 checksum:                            [100.00% 00:00:37/00:00:37]

Restoring Services on the installed system


Created 'as1.iso' (2939,576,320 bytes) in 00:16:03

The first thing to notice is that the second iso, as1.iso is smaller the the first one, which used the max compression.
What happened ?
the first iso was build successful, later I have implanted the md5sum and mediacheck result with PASS.
While building the second iso my /tmp partition runned out of space, but see..., there is no error reported back
Even worst, because the second iso was built with the option --md5sum, mediacheck is returning PASS but clearly this is a corrupted (incomplete) iso, should have been approx. 3.9 GB.

This behavior, (running out of space), need to be addressed, IMHO, tests continue

AS

EDIT: runned Media check from grub testing both ISO in Vbox, both resulted as PASS, but the second (as1.iso), clearly failed while starting the live system.

Just17 · « **Reply #27 on:** October 10, 2011, 06:02:59 PM »

I recall errors being reported because of lack of space ..... so why not this time ..... ? .... puzzle ..... another one

AS · « **Reply #28 on:** October 10, 2011, 06:10:01 PM »

Quote from: Just18 on October 10, 2011, 06:02:59 PM

I recall errors being reported because of lack of space ..... so why not this time ..... ? .... puzzle ..... another one

there are two stage: one while building livecd.sqfs, another while building the iso, apparently I got the ~~error~~ corruption while building the iso:
10 Gb - 3.2 GB as.iso - ~ 3.9 GB livecd.sqfs (as1.iso) = ~ 3 GB, the size of the corrupted as1.iso

AS · « **Reply #29 on:** October 10, 2011, 06:22:52 PM »

Another note, while testing an iso image in virtualbox I noticed an error message while booting:

Quote

Starting udev: udevd[518]: unable to move watches dir '/run/udev/watch', old watches will not be restored: Invalid cross-device link

I believe that /run should be excluded from the remastered iso.

PCLinuxOS-Forums

News:

Author Topic: mediacheck?? (Read 3769 times)

AS

Re: mediacheck??

pinoc

Re: mediacheck??

AS

Re: mediacheck??

pinoc

Re: mediacheck??

Just17

Re: mediacheck??

pinoc

Re: mediacheck??

AS

Re: mediacheck??

Just17

Re: mediacheck??

AS

Re: mediacheck??

Just17

Re: mediacheck??

AS

Re: mediacheck??

AS

Re: mediacheck??

Just17

Re: mediacheck??

AS

Re: mediacheck??

AS

Re: mediacheck??