USB stick performance [ANSWERED]

[AS]

I have written my own test program, it will not be published here because it could be very dangerous; if wrongly used, potentially can destroy a device layout (partitions + contents) within few seconds.  

However the test program is freely available, PM me if you want it. (or may be I will publish it in Advanced user sections).

How is performed the test ? The original question was around what was the optimal format for a USB stick, in turn this may greatly depend from the the read-write unit used, a.k.a block size.

Each filesystem type has it's own algorithms and allocations strategy to write on device, so I decided to get rid of all fancy (and great) features offered from kernel and filesystem, and go directly to test the raw device.

How is done ? very simple: the device is opened for writing using O_SYNC flag, that will assure that each written block will be really written on disk and will not stay in buffers.

The program perform 6 tests, for read and write operations, using the followings block sizes: 4K, 8K, 16K, 32K, 64K, and 128K, for a total of 128 Mb.

Things now start becoming interesting, here some results:

EDIT: results modified because of a slightly improved version of the test program

Trascend 2 Gb:

Code: [Select]

scsi 14:0:0:0: Direct-Access     JetFlash TS2GJFV30        8.07 PQ: 0 ANSI: 2

Disk /dev/sdc: 2055 MB, 2055207936 bytes
64 heads, 62 sectors/track, 1011 cylinders, total 4014078 sectors

# ./testflash /dev/sdc
/dev/sdc: block   4096: Read  11.42 secs  11.21 MB/s, Write  36.61 secs   3.50 MB/s
/dev/sdc: block   8192: Read  11.42 secs  11.21 MB/s, Write  31.94 secs   4.01 MB/s
/dev/sdc: block  16384: Read  11.47 secs  11.16 MB/s, Write  29.54 secs   4.33 MB/s
/dev/sdc: block  32768: Read  11.52 secs  11.11 MB/s, Write  28.21 secs   4.54 MB/s
/dev/sdc: block  65536: Read  11.52 secs  11.11 MB/s, Write  28.18 secs   4.54 MB/s
/dev/sdc: block 131072: Read  11.54 secs  11.09 MB/s, Write  28.23 secs   4.53 MB/s

Kingston 2 Gb:

Code: [Select]

scsi 15:0:0:0: Direct-Access     Kingston DataTraveler 2.0 PMAP PQ: 0 ANSI: 0 CCS

Disk /dev/sdc: 2031 MB, 2031091712 bytes
63 heads, 62 sectors/track, 1015 cylinders, total 3966976 sectors

# ./testflash /dev/sdc
/dev/sdc: block   4096: Read   6.51 secs  19.66 MB/s, Write 102.43 secs   1.25 MB/s
/dev/sdc: block   8192: Read   6.52 secs  19.65 MB/s, Write  30.36 secs   4.22 MB/s
/dev/sdc: block  16384: Read   6.53 secs  19.59 MB/s, Write  25.59 secs   5.00 MB/s
/dev/sdc: block  32768: Read   6.54 secs  19.56 MB/s, Write  22.64 secs   5.65 MB/s
/dev/sdc: block  65536: Read   6.53 secs  19.61 MB/s, Write  21.54 secs   5.94 MB/s
/dev/sdc: block 131072: Read   6.53 secs  19.61 MB/s, Write  21.27 secs   6.02 MB/s

Verbatim 4 Gb:

Code: [Select]

scsi 16:0:0:0: Direct-Access     General  USB Flash Disk   1100 PQ: 0 ANSI: 0 CCS

Disk /dev/sdc: 4001 MB, 4001366016 bytes
124 heads, 62 sectors/track, 1016 cylinders, total 7815168 sectors

# ./testflash /dev/sdc
/dev/sdc: block   4096: Read   8.90 secs  14.39 MB/s, Write 101.54 secs   1.26 MB/s
/dev/sdc: block   8192: Read   8.58 secs  14.93 MB/s, Write  60.41 secs   2.12 MB/s
/dev/sdc: block  16384: Read   8.38 secs  15.27 MB/s, Write  37.37 secs   3.43 MB/s
/dev/sdc: block  32768: Read   8.45 secs  15.15 MB/s, Write  38.26 secs   3.35 MB/s
/dev/sdc: block  65536: Read   8.35 secs  15.32 MB/s, Write  34.95 secs   3.66 MB/s
/dev/sdc: block 131072: Read   8.32 secs  15.38 MB/s, Write  41.30 secs   3.10 MB/s

HP 4Gb:

Code: [Select]

scsi 17:0:0:0: Direct-Access     HP       v100w            PMAP PQ: 0 ANSI: 0 CCS

Disk /dev/sdc: 4007 MB, 4007657472 bytes
124 heads, 62 sectors/track, 1018 cylinders, total 7827456 sectors

# ./testflash /dev/sdc
/dev/sdc: block   4096: Read   7.06 secs  18.13 MB/s, Write 225.12 secs   0.57 MB/s
/dev/sdc: block   8192: Read   7.06 secs  18.14 MB/s, Write 136.26 secs   0.94 MB/s
/dev/sdc: block  16384: Read   7.06 secs  18.13 MB/s, Write  83.80 secs   1.53 MB/s
/dev/sdc: block  32768: Read   6.88 secs  18.60 MB/s, Write  11.86 secs  10.79 MB/s
/dev/sdc: block  65536: Read   6.87 secs  18.63 MB/s, Write   8.12 secs  15.77 MB/s
/dev/sdc: block 131072: Read   6.89 secs  18.58 MB/s, Write   8.09 secs  15.81 MB/s

Kingston3G, 8Gb:

Code: [Select]

scsi 18:0:0:0: Direct-Access     Kingston DataTraveler G3  PMAP PQ: 0 ANSI: 0 CCS

Disk /dev/sdc: 8011 MB, 8011120640 bytes
247 heads, 62 sectors/track, 1021 cylinders, total 15646720 sectors

# ./testflash /dev/sdc
/dev/sdc: block   4096: Read   6.26 secs  20.43 MB/s, Write 109.36 secs   1.17 MB/s
/dev/sdc: block   8192: Read   6.26 secs  20.44 MB/s, Write  56.98 secs   2.25 MB/s
/dev/sdc: block  16384: Read   6.26 secs  20.44 MB/s, Write  41.33 secs   3.10 MB/s
/dev/sdc: block  32768: Read   6.10 secs  20.99 MB/s, Write  13.22 secs   9.68 MB/s
/dev/sdc: block  65536: Read   6.14 secs  20.86 MB/s, Write  11.50 secs  11.13 MB/s
/dev/sdc: block 131072: Read   6.10 secs  20.97 MB/s, Write  11.36 secs  11.27 MB/s

interesting things to note:
- the read spead practically doesn't change when you change the block size
- the HP 4Gb device, result in both the best and the worse performance depending on block size
- nearly all perform better in writing using large blocks

... to be continued with filesystems related tests

[djohnston]

http://en.wikipedia.org/wiki/XFS
Not necessarily true - the wiki may be a little aged... 

I've used xfs on one box for several months now, for all partitions except the swap. Have had no problems.

[JALU]

Using AS updated version 0.2 test script (in a LiveOS USB session)>

Toshiba 4G:
Disk /dev/sdc: 4007 MB, 4007657472 bytes
124 heads, 62 sectors/track, 1018 cylinders, total 7827456 sectors

Code: [Select]

[root@localhost owner]# ./testflash /dev/sdc
/dev/sdc: block   4096: Read   6.59 secs  19.43 MB/s, Write  86.76 secs   1.48 MB/s
/dev/sdc: block   8192: Read   6.56 secs  19.50 MB/s, Write  43.99 secs   2.91 MB/s
/dev/sdc: block  16384: Read   6.57 secs  19.48 MB/s, Write  18.42 secs   6.95 MB/s
/dev/sdc: block  32768: Read   6.57 secs  19.48 MB/s, Write  14.77 secs   8.66 MB/s
/dev/sdc: block  65536: Read   6.58 secs  19.45 MB/s, Write  13.28 secs   9.64 MB/s
/dev/sdc: block 131072: Read   6.57 secs  19.50 MB/s, Write  16.95 secs   7.55 MB/s

Corsair Voyager 8G:
Disk /dev/sdc: 8019 MB, 8019509248 bytes
247 heads, 62 sectors/track, 1022 cylinders, total 15663104 sectors

Code: [Select]

/dev/sdc: block   4096: Read   4.66 secs  27.46 MB/s, Write  70.96 secs   1.80 MB/s
/dev/sdc: block   4096: Read   4.63 secs  27.67 MB/s, Write  71.03 secs   1.80 MB/s
/dev/sdc: block   8192: Read   4.61 secs  27.75 MB/s, Write  33.76 secs   3.79 MB/s
/dev/sdc: block  16384: Read   4.66 secs  27.46 MB/s, Write  15.48 secs   8.27 MB/s
/dev/sdc: block  32768: Read   4.66 secs  27.47 MB/s, Write  14.99 secs   8.54 MB/s
/dev/sdc: block  65536: Read   4.61 secs  27.75 MB/s, Write  12.95 secs   9.89 MB/s
/dev/sdc: block 131072: Read   4.64 secs  27.58 MB/s, Write  16.23 secs   7.89 MB/s

Cages

(edit:  reran the test a few times to get a better average)

[AS]

Using AS updated version 0.2 test script (in a LiveOS USB session)>

Thanks for the report. 

Note that if you used the LiveCd to prevent damage on your hard disk in case of errors, this is not going to save you unless you physically disconnect the hard disk. Even from LiveCD you can access / potentially destroy your hard disk layout.

Interesting to note that even in case of 30 MB/s read speed the write performance are relatively low ... 
Also, it appear that nearly for all devices, until now, the best write performance are obtained using a 64Kb block.

[JALU]

Using AS updated version 0.2 test script (in a LiveOS USB session)>

Note that if you used the LiveCd to prevent damage on your hard disk in case of errors, this is not going to save you unless you physically disconnect the hard disk. Even from LiveCD you can access / potentially destroy your hard disk layout.

 yes this is true...

but I'm running the tests on a Windoz box booted to a LiveOS USB session..  so who cares!!  
I'm NOT going anywhere near my PCLinux box with the weird things..
Cages  

[JALU]

Here are my 'real world' tests comparing different file system types:
Toshiba 4GB, new DOS table, fresh format, write 1.2 GB .avi file for test.
type          Data          time (secs)       approx data rate MB/s

FAT16        1.2GB         154                    7.97
ext2           1.2GB         218                   5.64
ext3           1.2GB         226                   5.44
ext4           1.2GB         130                   9.45

also, using previously mentioned code>
mke2fs -t ext4 -E stripe-width=32 -m 0 /dev/sdc1
results vary little from the raw ext4 partition/format

(Data=1.2GB)               time (secs)       approx data rate MB/s
stripe-width=8                131                  9.38
stripe-width=16              131                  9.38
stripe-width=32              128                  9.6
stripe-width=64              129                  9.53

My question is >  Last I checked, the LiveUSB creation was limited to ext2/3      Why cannot ext4 be used for this purpose?
.Cages

[Just17]

My question is >  Last I checked, the LiveUSB creation was limited to ext2/3      Why cannot ext4 be used for this purpose?
.Cages

It can and has been done, but the update has not yet been released .....  trying to sort another bug which is proving difficult.

[AS]

My question is >  Last I checked, the LiveUSB creation was limited to ext2/3      Why cannot ext4 be used for this purpose?
.Cages

It can and has been done, but the update has not yet been released .....  trying to sort another bug which is proving difficult.

the improved version (+ ext4) should be released for itself ... unless the bug is strictly related. 

[djohnston]

My question is >  Last I checked, the LiveUSB creation was limited to ext2/3      Why cannot ext4 be used for this purpose?
.Cages

It can and has been done, but the update has not yet been released .....  trying to sort another bug which is proving difficult.

I just tried it out, and it works beautifully with ext4. Only a slight modification to the script was needed. Unless I'm missing something.

[Just17]

My question is >  Last I checked, the LiveUSB creation was limited to ext2/3      Why cannot ext4 be used for this purpose?
.Cages

It can and has been done, but the update has not yet been released .....  trying to sort another bug which is proving difficult.

I just tried it out, and it works beautifully with ext4. Only a slight modification to the script was needed. Unless I'm missing something.

You're not missing anything     ....  just adjust the checks for fs types. The update to allow this was done in mylivecd.

regards.

[AS]

interesting things to note:
- the read spead practically doesn't change when you change the block size
- the HP 4Gb device, result in both the best and the worse performance depending on block size
- nearly all perform better in writing using large blocks

... to be continued with filesystems related tests

I'm closing this thread, and will not perform further tests. The results so far obtained show very clearly that the USB sticks performance are strictly related to the device brand/model. About the result that show that a large block perform better, I could add that no matter what filesystem is in use on the USB stick and no matter what block/stripe parameters was used to format the filesystem, the kernel itself is intelligent enough to perform a lot of optimization, reaching nearly the maximum raw transfer rate when writing. Conclusion: buy good media. 

Thanks to all who have posted their contributions to the discussion. 

AS