As a part of building out the Cloud in a Box, I wanted some storage for Docker images, as well as data.
Based upon my previous experience, I believed that a SSD would be faster than a Micro SD, but I hadn’t tested it as yet. The challenge from Dieter Reuter (@Quintus23M), asking how I’d hooked up the SSD as well as whether it was faster than the Micro SD was a good motivator. I did find a couple of surprises along the way.
SSD Config
The SSD are (at the moment) in a USB 2.0 case such as Inland 2.5″ SATA to High-Speed USB 2.0 External Hard Drive Enclosure 434746 – Micro Center. There wasn’t much reason to go with USB 3 as the Pi doesn’t support it. I mainly wanted it for the cirucuit — a case with circuit and cable was $5 USD, whereas a conversion cable is over $10.
At some point I am thinking about building a sata to USB converter supporting multiple drives — the
JM20337 is an inexpensive chip and appears to be what the converter is using.
|
1 2 3 4 5 6 |
+----+ +-----+ +-----+ | | | | | | | Pi | <---> | Hub | <---> | SSD | | | | | | | +----+ +-----+ +-----+ |
The Pi, even when plugged into a 2 Amp power supply, didn’t have enough juice to run the drive. Consequently I needed a powered hub. I think it’s quite possible that another, more expensive case/circuit might not need to have a powered USB hub.
Devices
| Device | Mount Point | Type | Notes |
|---|---|---|---|
| /dev/mmcblk0 | / | MicroSD | Class 10, 16GB in Pi Micro SD slot |
| /dev/sdb | /data | SSD | 240GB Sata 3 SSD |
| /dev/sda | /opt2 | Spinning Disk | 2.5″ 5400 RPM, 160GB |
Testing
In order to make as accurate a test as possible, the buffers and cache are dumped prior to every run:
|
1 2 |
echo 3 | sudo tee /proc/sys/vm/drop_caches |
Additionally, the data is sync‘d in order to ensure that the reads/writes are finished and measured as a part of the time elapsed in the test.
The hdparm tests have an implicit dump of the cache/buffers.
Sequential Write Test
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 |
$ for i in /data/test_ssd /tmp/test_microsd /opt2/test_spinning; do sync; echo 3 | sudo tee /proc/sys/vm/drop_caches; echo $i; time (dd if=/dev/zero of=$i count=400K;sync;sync;sync); done 3 /data/test_ssd 409600+0 records in 409600+0 records out 209715200 bytes (210 MB) copied, 14.4097 s, 14.6 MB/s real 0m15.125s user 0m0.900s sys 0m12.530s 3 /tmp/test_microsd 409600+0 records in 409600+0 records out 209715200 bytes (210 MB) copied, 20.4993 s, 10.2 MB/s real 0m24.579s user 0m0.970s sys 0m11.820s 3 /opt2/test_spinning 409600+0 records in 409600+0 records out 209715200 bytes (210 MB) copied, 14.4149 s, 14.5 MB/s real 0m15.848s user 0m0.970s sys 0m12.270s |
Sequential Read Test
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 |
$ for i in /data/test_ssd /tmp/test_microsd /opt2/test_spinning; do sync; echo 3 | sudo tee /proc/sys/vm/drop_caches; echo $i; time (dd if=$i of=/dev/null count=400K;sync;sync;sync); done 3 /data/test_ssd 409600+0 records in 409600+0 records out 209715200 bytes (210 MB) copied, 6.67669 s, 31.4 MB/s real 0m6.823s user 0m0.830s sys 0m2.900s 3 /tmp/test_microsd 409600+0 records in 409600+0 records out 209715200 bytes (210 MB) copied, 11.9394 s, 17.6 MB/s real 0m12.082s user 0m0.660s sys 0m3.070s 3 /opt2/test_spinning 409600+0 records in 409600+0 records out 209715200 bytes (210 MB) copied, 6.97501 s, 30.1 MB/s real 0m7.354s user 0m0.550s sys 0m3.140s |
hdparm Buffered Disk Reads
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 |
$ for i in sda sdb mmcblk0; do echo $i;for j in `seq 3`; do hdparm -t /dev/$i; done; done sda /dev/sda: Timing buffered disk reads: 86 MB in 3.06 seconds = 28.08 MB/sec /dev/sda: Timing buffered disk reads: 90 MB in 3.01 seconds = 29.89 MB/sec /dev/sda: Timing buffered disk reads: 90 MB in 3.03 seconds = 29.68 MB/sec sdb /dev/sdb: Timing buffered disk reads: 92 MB in 3.00 seconds = 30.66 MB/sec /dev/sdb: Timing buffered disk reads: 94 MB in 3.04 seconds = 30.91 MB/sec /dev/sdb: Timing buffered disk reads: 94 MB in 3.05 seconds = 30.86 MB/sec mmcblk0 /dev/mmcblk0: Timing buffered disk reads: 52 MB in 3.09 seconds = 16.81 MB/sec /dev/mmcblk0: Timing buffered disk reads: 52 MB in 3.09 seconds = 16.81 MB/sec /dev/mmcblk0: Timing buffered disk reads: 52 MB in 3.09 seconds = 16.81 MB/sec |
hdparm Cached Disk Reads
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
$ for i in sda sdb mmcblk0; do echo $i;for j in `seq 3`; do hdparm -T /dev/$i; done; done sda /dev/sda: Timing cached reads: 690 MB in 2.00 seconds = 345.00 MB/sec /dev/sda: Timing cached reads: 690 MB in 2.00 seconds = 344.72 MB/sec /dev/sda: Timing cached reads: 698 MB in 2.00 seconds = 348.89 MB/sec sdb /dev/sdb: Timing cached reads: 728 MB in 2.00 seconds = 364.04 MB/sec /dev/sdb: Timing cached reads: 726 MB in 2.00 seconds = 363.00 MB/sec /dev/sdb: Timing cached reads: 726 MB in 2.00 seconds = 363.26 MB/sec mmcblk0 /dev/mmcblk0: Timing cached reads: 716 MB in 2.00 seconds = 357.65 MB/sec /dev/mmcblk0: Timing cached reads: 714 MB in 2.00 seconds = 357.02 MB/sec p /dev/mmcblk0: Timing cached reads: 714 MB in 2.00 seconds = 357.10 MB/sec |
Results
The main limitation of I/O on the Pi is the speed of the USB Bus. Given it’s limited to 480Mbs, the bus is much slower than the speed which the hard drive and SSD support.
By comparison, here is the HDParms results for a SSD on a desktop:
|
1 2 3 4 5 6 7 8 9 10 |
$ sudo hdparm -t /dev/sda /dev/sda: Timing buffered disk reads: 372 MB in 3.01 seconds = 123.64 MB/sec $ sudo hdparm -T /dev/sda /dev/sda: Timing cached reads: 4554 MB in 2.00 seconds = 2277.27 MB/sec |
and a 5400 spinning disk on a laptop (doing other work at the same time):
|
1 2 3 4 5 6 7 8 9 10 |
<br /># hdparm -t /dev/sda /dev/sda: Timing buffered disk reads: 204 MB in 3.01 seconds = 67.75 MB/sec # hdparm -T /dev/sda /dev/sda: Timing cached reads: 2386 MB in 2.00 seconds = 1193.32 MB/sec |
The Micro SD device, however, is half the speed of the other two devices — I believe that this is to spec. “Difference between Speed Class, UHS Speed Class, and Speed Ratings (performance) for SD/SDHC/SDXC cards“, indicates that the class refers to the minimum speed that a card supports.
The Best microSD Card states that
It’s important to test SD cards via USB 3.0 to prevent bottlenecks, since USB 2.0 tops out around 33 MB/s
This is the approximate speed being seen for the SSD and Spinning Disk. Since the Micro SD is half the speed of the others, I’ve probably reached the limit of the card — approximately 1.6 times the minimum speed for reads and the spec for writes.
There are UHS cards which are faster, however. One of these should be able to match or exceed the performance of the SSD or spinning disk. For instance, the Samsung EVO Micro SDXC claims, depending on the version, speeds up to
48MB/s or 90MB/s.
Interestingly enough, the hdparm tests for cached reads has all three roughly within range of each other. The buffered tests, however, are similar to the Sequential Read test in the reported rates.
On writes, the spinning disk is slightly slower than the SSD. However, the SSD pulls a bit ahead on reads — ~1 MB/sec for buffered and then ~20 MB/sec on the cached reads.
At the end of the day, the SSD and Spinning Disk are faster than the Micro SD — however, this might be due to the Micro SD card I’m using.
6 comments
Skip to comment form ↓
Dieter Reuter (@Quintus23M)
April 5, 2015 at 5:18 am (UTC -5) Link to this comment
Well done and quick comparison, thanks!
Dieter Reuter (@Quintus23M)
April 5, 2015 at 7:04 am (UTC -5) Link to this comment
Matt, here is a super-detailed post about what performance we could get on a RPi with SD cards, USB drives or Ethernet: http://raspberrypi.stackexchange.com/questions/1262/what-is-the-highest-performing-hardware-configuration. In short: SD card 15-20MB/sec, USB drive 30MB/sec.
I just run your tests on Pi 2 with a SanDisk Extreme Pro 32GB SD card, which can handle up to 90MByte/sec R/W (tested on my 2015 MacBookPro).
Sequential Writes: 14.3 MB/s
Sequential Reads: 19.1 MB/s
Buffered Reads: 18.25 MB/sec
Cached Reads: 362.55 MB/sec
Matt Williams
April 6, 2015 at 5:57 pm (UTC -5) Link to this comment
Thank you for the challenge. I’d made some assumptions and hadn’t validated them!
Matt Williams
April 6, 2015 at 5:58 pm (UTC -5) Link to this comment
Thank you for this. I’m definitely going to check it out.
I do think the main limiting factor is the USB 2.0 bus at this point…..
reza
March 4, 2016 at 1:54 am (UTC -5) Link to this comment
recently RP3 with USB 3.0 just came out. Can anyone suggest how we can repeat the experiments to connect RP3 to SSD to get the highest possible data transfer speed?
Matt Williams
March 4, 2016 at 9:32 am (UTC -5) Link to this comment
Actually, according the the raspberry pi site, https://www.raspberrypi.org/magpi/raspberry-pi-3-specs-benchmarks/, it’s still USB 2.0:
The major differences for the Pi 3 are the wireless & bluetooth as well as the upgraded processor.