rsync -av produced different sized results

[littlenoodles]

Hi.  In a prior thread, I managed to copy my root filesystem to a new partition using rsync -av.  After some screw-ups, everything's fine now.  Just one thing.  The original partition, /mnt/sdb6, has 4.5GB used, and the copy, /mnt/sdb3, has 5.8GB.  Not a serious problem, just want to understand it and make sure I didn't do something wrong.

$ df
Filesystem            Size  Used Avail Use% Mounted on
/dev/sdb6             5.6G  4.5G  882M  84% /mnt/sdb6
/dev/sdb3              17G  5.8G   11G  37% /mnt/sdb3

I've tried comparing the contents several ways, and it looks like everything's there..  There are a hundred or so more files in the copy than the original, but I'll chalk that up to the fact that I'm running in the copy, and may have some more stuff in /tmp.  When I look at some specific directories, though, the contents and individual file sizes seem to be the same, but ls -l reports different 'total' values.

< /mnt/sdb6/tmp/kdecache-michael/kpc:
< total 8
< -rw-r--r-- 1 michael users 16240640 Aug  2 09:03 kde-icon-cache.data
< -rw-r--r-- 1 michael users  4296704 Aug  2 09:03 kde-icon-cache.index
---
> /mnt/sdb3/tmp/kdecache-michael/kpc:
> total 20056
> -rw-r--r-- 1 michael users 16240640 Jul 12 11:31 kde-icon-cache.data
> -rw-r--r-- 1 michael users  4296704 Jul 12 11:31 kde-icon-cache.index

What gives?  Both filesystems are ext4 (I think - at least fstab says so).  Could I have accidentally created the new filesystem with a different block size than the original (tune2fs reports 4K blocks for both)?  I used the gparted live CD to create the copy, and the PCLOS installer to create the original.  Could there be that much extra overhead due to the larger physical size of sdb3?  Here's a diff between the relevant outputs of tune2fs -l for each (the large difference between reserved block counts seems particularly big):

$ diff tune2fs.sdb6 tune2fs.sdb3
< Inode count:              371680
< Block count:              1485532
< Reserved block count:     74276
< Free blocks:              299970
< Free inodes:              222838
---
> Inode count:              1115728
> Block count:              4460288
> Reserved block count:     223014
> Free blocks:              2873868
> Free inodes:              966274
21c21
< Reserved GDT blocks:      362
---
> Reserved GDT blocks:      1022
24,25c24,25
< Inodes per group:         8080
< Inode blocks per group:   505
---
> Inodes per group:         8144
> Inode blocks per group:   509

[/font]

[muungwana]

what file systems do both partitions use?

[T6]

he says that ext4

"Could I have accidentally created the new filesystem with a different block size than the original (tune2fs reports 4K blocks for both)"

i ignore if this is possible, what command can verify this?

i think that pclinux control center can tell you this on the partitions section

have you verified your ram modules? problems on them can give you errors like this one

[AS]

< /mnt/sdb6/tmp/kdecache-michael/kpc:
< total 8
< -rw-r--r-- 1 michael users 16240640 Aug  2 09:03 kde-icon-cache.data
< -rw-r--r-- 1 michael users  4296704 Aug  2 09:03 kde-icon-cache.index
---
> /mnt/sdb3/tmp/kdecache-michael/kpc:
> total 20056
> -rw-r--r-- 1 michael users 16240640 Jul 12 11:31 kde-icon-cache.data
> -rw-r--r-- 1 michael users  4296704 Jul 12 11:31 kde-icon-cache.index

The total unit is 1 KByte blocks, so the 20056 appear to be the correct value, and the total 8 look strange ....
However this is due to this specific files, because they are so called "sparse files"
http://en.wikipedia.org/wiki/Sparse_file
Basically the file size is set to some value i.e. 16240640 while only one 4 Kb block is actually used/written, the same for the second file, and therefore a total of 8 is the result.
Upon copy (rsync -av) the file on the copy is filled, all blocks are duplicated, and not allocated blocks are copied as blocks of zeroes.

strange things the "sparse files"  

AS

[littlenoodles]

@as   Sparse files was one of the things I'd considered, but I assumed that rsync -av was smart enough to copy sparse files as sparse.  Otherwise it shouldn't have been recommended for the purpose of cloning my whole root fs.

Maybe I needed another command line option to get rsync to preserve sparseness?  I had a similar problem with gunzip at work a while back.  When you zipped and then unzipped a sparse file, it would come back non-sparse (all the zeros taking up real disk space).  I made my own gunzip with a --sparse option that we use at work - submitted the change to the GNU folks, but they never responded 

[littlenoodles]

Oh boy.  I just did a 'man rsync', and it does have a --sparse option.  I had assumed -a was a catch-all that said 'duplicate everything the way it was in the original', but it doesn't even preserve hard links (another possible space waster).  I guess I need to rerun the copy.  Anybody have a suggestion of a better way to copy my root fs from one partition to another?  Or at least a suggestion for the right combination of rsync flags to get an exact copy?  Will 'rsync -avH --sparse' do it?

       -a, --archive
              This is equivalent to -rlptgoD. It is a quick way of saying you want recursion and  want
              to preserve almost everything (with -H being a notable omission).  The only exception to
              the above equivalence is when --files-from  is  specified,  in  which  case  -r  is  not
              implied.

              Note  that  -a  does  not  preserve  hardlinks, because finding multiply-linked files is
              expensive.  You must separately specify -H.

[AS]

@as   Sparse files was one of the things I'd considered, but I assumed that rsync -av was smart enough to copy sparse files as sparse.  Otherwise it shouldn't have been recommended for the purpose of cloning my whole root fs.

rsync can handle sparse file, but you need to add the option -S, see man rsync.
EDIT (you found this yourself) 

About cloning: cloning is different from copy, in that when you copy a file you meant to duplicate the content, i.e. you can copy one file from ext3 fs to an xfs filesystem obtaining a duplicate of content but for sure you will have a different underlying structure. rsync is not a cloning tool. Cloning tools works at blocks level which is under the filesystem layout, i.e. dd, you can clone a disk.

That said, generally there is no need to backup /tmp, because by definition it contains only temporary files.
You can loose it's content at any time without any problem.

BTW, if your intent is mirroring some content, rsync surely can do that, in addition to the option -av (and -S) you may need to use also the --delete option, so that when you update your backup (that's the reason to use rsync), you may want that files deleted on the source will be deleted on the destination.

Maybe I needed another command line option to get rsync to preserve sparseness?  I had a similar problem with gunzip at work a while back.  When you zipped and then unzipped a sparse file, it would come back non-sparse (all the zeros taking up real disk space).  I made my own gunzip with a --sparse option that we use at work - submitted the change to the GNU folks, but they never responded 

Take a few time, make a few tests, and you will find your way with rsync.

Enjoy! 

AS