Solutions for Portable Security

From time to time customers ask us what the transfer speeds are of our secure USB storage devices. It’s a legitimate question but one that is difficult to give a straight answer to. Vendors typically quote read/write transfer speeds on their product boxes and datasheets and you’ll see numbers anywhere in the range of 5 MB/s to 29 MB/s. Hopefully speed isn’t the only criteria you use to select which secure USB stick to deploy, but if speed is an important factor you definitely should not make a decision based on the quoted speeds on the box.

If you read them carefully most transfer speeds are qualified with an “up to”, or a footnote disclaimer saying that speeds vary based on host hardware, software, and usage. These are very important qualifiers because your “real world” usage may result in significant (even orders of magnitude) differences from what is quoted.

To illustrate the disparity I have benchmarked a number of devices using a “real world” file copy test in a Windows environment. I only focused on writes because they are generally slower than reads in flash technology. Knowing that smaller file transfers are much less efficient than large file transfers, my tests involved copying many files of a given size and seeing what kind of throughput the devices achieved as the file size changed. The results are shown in the graph below. I also pulled out some samples and put them in a table so you can see the lower end numbers.

The trend is very clear. You get better throughput when you transfer large files, than if you transfer lots of small files. In fact the difference can be astounding. Let’s look at Vendor A for example. If you were to copy 128 MB of data onto vendor A’s product, the theoretical best case time for the copy would be 128 MB /(19 MB/s) = 6.74 seconds. The reality however is that it would take almost 4.5 hours if that 100 MB were made up of 2 KB files. The best case (from the table) is that this 128 MB consists of four 32 MB files to copy and would take just over 15 seconds.

Why is there such a discrepancy? It turns out that there is a lot of overhead in copying many files. File copies are not just about writing the data contained in the files, but also require updates to the file system data structures to keep track of the files and pieces of files. The result is that the device is hit with many extra reads and writes. The more files you have, the more overhead there is and when the files are very small, the overhead becomes significant. That being said, some devices handle the overhead much better than others. Vendor C for could handle the 2 KB file set in 39 minutes. I’d rather wait 39 minutes than 4.5 hours.

You might think that having so many 2 KB files is unrealistic so I decided to check my Internet Explorer Temporary Internet Files cache to see a sample of real data. It reported 130 MB in 2,268 files which is an average file size of about 60 KB. Rounding up to 64 KB and using the table here is how the vendors would stack up if I were to copy this directory onto each device:

Vendor A: 8.3 minutes
Vendor B: 2.2 minutes
Vendor C: 52 seconds
Vendor D: 3.0 minutes
Vendor E: 3.5 minutes

Your day to day information comes in all sizes. Browser cookies are on the order of hundreds of bytes while my Outlook .pst file is pushing 2 GB. It’s important to realize that such a range can have a drastic effect on transfer speeds.

Unfortunately, the only numbers you probably see quoted are optimal numbers, derived in contrived test situations that in most cases are far from the reality that you work in. It reminds me of some car commercials where you see a single car driving at enormous speed on an infinite, flat, empty plain. It looks fantastic, except you know that you’ll be stuck in traffic jams every day and the mileage and experience won’t be the same.

You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.