Discussion:
Amazing finding about SSD
(too old to reply)
root
2019-07-17 20:16:51 UTC
Permalink
I just upgraded my m2 SSD from 250GB to 1TB. This
morning I was playing around testing the speed
vs SATA drives. I have a SATA3 spinning disk
on /dev/sda1 and a Sandisk SSD on /dev/sdb1

I copy 5GB from either of these 2 to a
ram disk. If I mount the SSD as:
mount /dev/sdb1 /sdb1
The 5GBcopy takes 48 seconds.

Mounting the spinning SATA3 drive in the
same way the copy takes 53 seconds.

These results are repeatable.

Now here is the amazing finding. If I make
an entry in /etc/fstab:

/dev/sdb1 /sdb1 ext4 defaults,relatime,nodiratime,errors=remount-ro 1 1
and then execute mount -a

The 5GB copy takes just under 3 seconds.

In other words, mounting the SSD via fstab results
in a 16 fold increase in speed.

I repeated the results twice and verified the copy.
Sylvain Robitaille
2019-07-17 21:11:51 UTC
Permalink
Post by root
mount /dev/sdb1 /sdb1
The 5GBcopy takes 48 seconds.
...
Now here is the amazing finding. If I make
/dev/sdb1 /sdb1 ext4 defaults,relatime,nodiratime,errors=remount-ro 1 1
and then execute mount -a
The 5GB copy takes just under 3 seconds.
In other words, mounting the SSD via fstab results
in a 16 fold increase in speed.
I fully expect that "relatime,nodiratime" are where your speedup is
happening, not whether the mount was configured into fstab.
--
----------------------------------------------------------------------
Sylvain Robitaille ***@encs.concordia.ca

Systems analyst / AITS Concordia University
Faculty of Engineering and Computer Science Montreal, Quebec, Canada
----------------------------------------------------------------------
Grant Taylor
2019-07-17 21:21:24 UTC
Permalink
Post by Sylvain Robitaille
I fully expect that "relatime,nodiratime" are where your speedup is
happening, not whether the mount was configured into fstab.
Agreed.

Try unmounting (optionally removing / commenting fstab) and running the
following command:

mount -o relatime,nodiratime /dev/sdb1 /sdb1
--
Grant. . . .
unix || die
root
2019-07-18 00:54:00 UTC
Permalink
Post by Grant Taylor
Post by Sylvain Robitaille
I fully expect that "relatime,nodiratime" are where your speedup is
happening, not whether the mount was configured into fstab.
Agreed.
Try unmounting (optionally removing / commenting fstab) and running the
mount -o relatime,nodiratime /dev/sdb1 /sdb1
That was what I was looking for, but get this:
mount -o relatime,nodiratime /dev/sdb1 /sdb1
time cp -aR /sdb1/root/magic .

real 0m48.165s
user 0m2.710s
sys 0m16.786s
root
2019-07-18 00:49:50 UTC
Permalink
Post by Sylvain Robitaille
Post by root
In other words, mounting the SSD via fstab results
in a 16 fold increase in speed.
I fully expect that "relatime,nodiratime" are where your speedup is
happening, not whether the mount was configured into fstab.
Exactly. I don't know how to add those to a simple mount
command. Do you?
Aragorn
2019-07-18 01:35:22 UTC
Permalink
Post by root
Post by Sylvain Robitaille
Post by root
In other words, mounting the SSD via fstab results
in a 16 fold increase in speed.
I fully expect that "relatime,nodiratime" are where your speedup is
happening, not whether the mount was configured into fstab.
Exactly. I don't know how to add those to a simple mount
command. Do you?
# mount -t fstype -o relatime,nodiratime /dev/sth /somewhere

Relace "fstype" with the appropriate filesystem type or "auto". You
can also omit the "-t" option completely if the filesystem is already
in /etc/fstab.

$ man mount

... gives you more information.
--
With respect,
= Aragorn =
Richard Kettlewell
2019-07-17 22:02:30 UTC
Permalink
Post by root
I just upgraded my m2 SSD from 250GB to 1TB. This
morning I was playing around testing the speed
vs SATA drives. I have a SATA3 spinning disk
on /dev/sda1 and a Sandisk SSD on /dev/sdb1
I copy 5GB from either of these 2 to a
mount /dev/sdb1 /sdb1
The 5GBcopy takes 48 seconds.
Mounting the spinning SATA3 drive in the
same way the copy takes 53 seconds.
These results are repeatable.
Now here is the amazing finding. If I make
/dev/sdb1 /sdb1 ext4 defaults,relatime,nodiratime,errors=remount-ro 1
1
and then execute mount -a
The 5GB copy takes just under 3 seconds.
In other words, mounting the SSD via fstab results
in a 16 fold increase in speed.
I repeated the results twice and verified the copy.
Did you clear the cache before doing each test?
--
https://www.greenend.org.uk/rjk/
root
2019-07-18 01:01:36 UTC
Permalink
Post by Richard Kettlewell
Post by root
I repeated the results twice and verified the copy.
Did you clear the cache before doing each test?
I ran the commands twice in a row.


As suggested earlier, I used the mount command with -o
options.

time cp -aR /sdb1/root/magic .

real 0m47.711s
user 0m2.819s
sys 0m16.203s
The second time I got:

aa:/ram>time cp -aR /sdb1/root/magic .

real 0m2.947s
user 0m0.508s
sys 0m2.431s

So cache explains the speedup. Oops.
Grant Taylor
2019-07-18 01:57:27 UTC
Permalink
Post by root
So cache explains the speedup. Oops.
;-)

(Unexpected) observations can be a gateway to learning if you allow them
to be. I consider it to be a bad day if I don't learn something.
--
Grant. . . .
unix || die
Sylvain Robitaille
2019-07-18 15:28:55 UTC
Permalink
.... I consider it to be a bad day if I don't learn something.
Words to live by ...
--
----------------------------------------------------------------------
Sylvain Robitaille ***@encs.concordia.ca

Systems analyst / AITS Concordia University
Faculty of Engineering and Computer Science Montreal, Quebec, Canada
----------------------------------------------------------------------
notbob
2019-07-18 16:20:49 UTC
Permalink
Post by Sylvain Robitaille
Words to live by ...
.....until you get old, then it's a matter of keeping what ya' got.
;)

nb
Mike Spencer
2019-07-18 19:48:07 UTC
Permalink
Post by notbob
Post by Sylvain Robitaille
Words to live by ...
.....until you get old, then it's a matter of keeping what ya' got.
;)
In addition to "what ya' got" fading, they keep *changing* stuff. New
& (allegedly) better interface on $AP or config on $SYSTEM is all
different. Replacement $PRODUCT in plastic means relearning what you
knew about working $PRODUCT in metal. Worst of all, new and more
exacting research, especially in physiology and medicine, means much
of the (putatively) factual science is to be found on the rubbish
heap.

"Life-long learning" is supposed to be about learning *new* stuff, not
about learning the same stuff over and over. Dang!
--
Mike Spencer Nova Scotia, Canada

My electric toaster, used every day in summer, is 106 years old,
The user interface is the same as it was in 1913.
Jim Diamond
2019-07-21 15:31:50 UTC
Permalink
Post by Mike Spencer
Post by notbob
Post by Sylvain Robitaille
Words to live by ...
.....until you get old, then it's a matter of keeping what ya' got.
;)
In addition to "what ya' got" fading, they keep *changing* stuff. New
& (allegedly) better interface on $AP or config on $SYSTEM is all
different. Replacement $PRODUCT in plastic means relearning what you
knew about working $PRODUCT in metal. Worst of all, new and more
exacting research, especially in physiology and medicine, means much
of the (putatively) factual science is to be found on the rubbish
heap.
"Life-long learning" is supposed to be about learning *new* stuff, not
about learning the same stuff over and over. Dang!
My electric toaster, used every day in summer, is 106 years old,
The user interface is the same as it was in 1913.
Mike,

have you considered upgrading that with an Arduino or RPi (or similar)
and making it available over wifi in your house so that you could
initiate toasting from another room? Surely after 106 years you want
to do a UI upgrade on it. No?
Mike Spencer
2019-07-24 06:21:39 UTC
Permalink
Post by Jim Diamond
Post by Mike Spencer
Post by notbob
Post by Sylvain Robitaille
Words to live by ...
.....until you get old, then it's a matter of keeping what ya' got.
;)
In addition to "what ya' got" fading, they keep *changing* stuff. New
& (allegedly) better interface on $AP or config on $SYSTEM is all
different. Replacement $PRODUCT in plastic means relearning what you
knew about working $PRODUCT in metal. Worst of all, new and more
exacting research, especially in physiology and medicine, means much
of the (putatively) factual science is to be found on the rubbish
heap.
"Life-long learning" is supposed to be about learning *new* stuff, not
about learning the same stuff over and over. Dang!
My electric toaster, used every day in summer, is 106 years old,
The user interface is the same as it was in 1913.
Mike,
have you considered upgrading that with an Arduino or RPi (or similar)
and making it available over wifi in your house so that you could
initiate toasting from another room? Surely after 106 years you want
to do a UI upgrade on it. No?
Ha! I was just looking at a HOWTO file on controling relays through
the parallel port and I *do* have an Arduino around here somewhere
that I bought for the mouse exercise wheel tachometer project and
never used.

But no, the 1913 toaster requires manual flip from side to side. So
I'd need to have...lessee...a detector for when to flip (a cheap smoke
alarm shd do) and an actuator -- two, actually -- to do the flipping.
A turn signal unit from a 1953 VW would work but I don't have one.
Maybe the tray eject mechanism from a CD drive?

So many projects and so little time and so few gumpties! I want an
infra-red heater lamp in the outhouse; new engine on the rototiller; I
still haven't actually *made* anything with the 300# air hammer I
spent $2,000 and countless hours getting working; and I've just
managed to get a script working that tells me how many bytes are
running through the spiffy, new Oxygen-3/SIM card data link that will,
it is to be hoped, replace dialup. The carrots need weeding, the
squash bugs need attention (with extreme prejudice) and next winter's
wood, all sawn and split, has to be put into the shed.

Okay, cool idea. Low on the priority list. :-)

When this toaster was young, this was a popular joke and it's still
relevant if you don't pay attention:

Alice (to neighbor): Oh, Martha, George is just helpless in the
kitchen. He can't even make toast!

George: C'mon, Alice, I make toast the same way you do: put
it in the toaster and burn it, take it to the sink
and scrape it.
--
Michael Spencer Nova Scotia, Canada .~.
/V\
/( )\
^^-^
Jim Diamond
2019-07-30 23:54:25 UTC
Permalink
Post by Mike Spencer
Post by Jim Diamond
Post by Mike Spencer
My electric toaster, used every day in summer, is 106 years old,
The user interface is the same as it was in 1913.
Mike,
have you considered upgrading that with an Arduino or RPi (or similar)
and making it available over wifi in your house so that you could
initiate toasting from another room? Surely after 106 years you want
to do a UI upgrade on it. No?
Ha! I was just looking at a HOWTO file on controling relays through
the parallel port and I *do* have an Arduino around here somewhere
that I bought for the mouse exercise wheel tachometer project and
never used.
But no, the 1913 toaster requires manual flip from side to side. So
I'd need to have...lessee...a detector for when to flip (a cheap smoke
alarm shd do) and an actuator -- two, actually -- to do the flipping.
A turn signal unit from a 1953 VW would work but I don't have one.
Maybe the tray eject mechanism from a CD drive?
Aren't you a blacksmith? Surely you could (*cough*) hammer something
together?
Post by Mike Spencer
So many projects and so little time and so few gumpties! I want an
infra-red heater lamp in the outhouse; new engine on the rototiller; I
still haven't actually *made* anything with the 300# air hammer I
spent $2,000 and countless hours getting working;
Here's your chance ;-)
Post by Mike Spencer
and I've just managed to get a script working that tells me how many
bytes are running through the spiffy, new Oxygen-3/SIM card data
link that will, it is to be hoped, replace dialup. The carrots need
weeding, the squash bugs need attention (with extreme prejudice) and
next winter's wood, all sawn and split, has to be put into the shed.
Sure. But what about after lunch?

Jim
Pascal Hambourg
2019-07-18 17:34:17 UTC
Permalink
Post by root
I have a SATA3 spinning disk
on /dev/sda1 and a Sandisk SSD on /dev/sdb1
I copy 5GB from either of these 2 to a
mount /dev/sdb1 /sdb1
The 5GBcopy takes 48 seconds.
104 MB/s. This is a rather poor speed for an modern SSD.
Post by root
Mounting the spinning SATA3 drive in the
same way the copy takes 53 seconds.
94 MB/s. This is an average speed for a hard disk.
Aren't you puzzled that the hard disk is almost as fast as the SSD ?
Post by root
Now here is the amazing finding. If I make
/dev/sdb1 /sdb1 ext4 defaults,relatime,nodiratime,errors=remount-ro 1 1
and then execute mount -a
The 5GB copy takes just under 3 seconds.
1,6 GB/s. This is an impossible speed with SATA. The maximum speed is
600 MB/s (6 Gbit/s). This should have raised an alarm in your mind.
root
2019-07-19 15:57:21 UTC
Permalink
Post by Pascal Hambourg
Post by root
I have a SATA3 spinning disk
on /dev/sda1 and a Sandisk SSD on /dev/sdb1
I copy 5GB from either of these 2 to a
mount /dev/sdb1 /sdb1
The 5GBcopy takes 48 seconds.
104 MB/s. This is a rather poor speed for an modern SSD.
Post by root
Mounting the spinning SATA3 drive in the
same way the copy takes 53 seconds.
94 MB/s. This is an average speed for a hard disk.
Aren't you puzzled that the hard disk is almost as fast as the SSD ?
Post by root
Now here is the amazing finding. If I make
/dev/sdb1 /sdb1 ext4 defaults,relatime,nodiratime,errors=remount-ro 1 1
and then execute mount -a
The 5GB copy takes just under 3 seconds.
1,6 GB/s. This is an impossible speed with SATA. The maximum speed is
600 MB/s (6 Gbit/s). This should have raised an alarm in your mind.
That blazing speed was the result of reading from cache.

I apologize for my screw up.

Here are my findings:

Copy 4.8GB from SATA3 spinning disk to ram:
real 1m29.908s
user 0m4.217s
sys 0m21.366s

Same copy from Sandisk sata SSD:
real 0m51.052s
user 0m3.367s
sys 0m18.422s

Same copy from Sandisk M2 ssd:

real 0m16.992s
user 0m1.270s
sys 0m7.670s

The spinning disk time has sometimes risen
to about 2Min 30sec.

After a number of such tests I conclude that
the SATA3 SSD is three times as fast as a
SATA3 spinning disk, and that an M2 PCIe
SSD is three times as fast as the SATA SSD.
Pascal Hambourg
2019-07-19 18:44:55 UTC
Permalink
Post by root
That blazing speed was the result of reading from cache.
I know.
Post by root
real 1m29.908s
user 0m4.217s
sys 0m21.366s
What do you mean by "to ram" ?
Is this one single big file or multiple files (how many) ? Or do you
read the raw block device ?
53 MB/s is not a very good speed for a single unfragmented file on a
recent hard disk drive.
Post by root
real 0m51.052s
user 0m3.367s
sys 0m18.422s
94 MB/s.
Post by root
real 0m16.992s
user 0m1.270s
sys 0m7.670s
300 MB/s. Does this SSD have an NVMe, SATA or AHCI interface ?
An NMVe interface should do much better. Even a SATA 3.0 interface could
do better.
root
2019-07-20 03:10:51 UTC
Permalink
Post by Pascal Hambourg
What do you mean by "to ram" ?
I mount tmpfs to /ram, and all the copies were
to a file in that directory.
Post by Pascal Hambourg
Is this one single big file or multiple files (how many) ? Or do you
read the raw block device ?
The copy was an ordinary copy command, for example:

cp -aR source dest

This is representative of how the devices are used.
Post by Pascal Hambourg
53 MB/s is not a very good speed for a single unfragmented file on a
recent hard disk drive.
The source drive was unfragmented.
Post by Pascal Hambourg
Post by root
real 0m16.992s
user 0m1.270s
sys 0m7.670s
300 MB/s. Does this SSD have an NVMe, SATA or AHCI interface ?
An NMVe interface should do much better. Even a SATA 3.0 interface could
do better.
The M2 device is an NMVe Sandisk 970.
Rich
2019-07-20 03:57:21 UTC
Permalink
Post by root
cp -aR source dest
This is representative of how the devices are used.
You can also use the 'hdparm' command to perform device read speed
timings:

$ man hdparm

-t Perform timings of device reads for benchmark and comparison
purposes. For meaningful results, this operation should be
repeated 2-3 times on an otherwise inactive system (no other
active processes) with at least a couple of megabytes of free
memory. This displays the speed of reading through the buffer
cache to the disk without any prior caching of data. This
measurement is an indication of how fast the drive can sustain
sequential data reads under Linux, without any filesystem
overhead. To ensure accurate measurements, the buffer cache is
flushed during the processing of -t using the BLKFLSBUF ioctl.

-T Perform timings of cache reads for benchmark and comparison
purposes. For meaningful results, this operation should be
repeated 2-3 times on an otherwise inactive system (no other
active processes) with at least a couple of megabytes of free
memory. This displays the speed of reading directly from the
Linux buffer cache without disk access. This measurement is
essentially an indication of the throughput of the processor,
cache, and memory of the system under test.
root
2019-07-20 15:46:32 UTC
Permalink
Post by Rich
Post by root
cp -aR source dest
This is representative of how the devices are used.
You can also use the 'hdparm' command to perform device read speed
$ man hdparm
-t Perform timings of device reads for benchmark and comparison
purposes. For meaningful results, this operation should be
repeated 2-3 times on an otherwise inactive system (no other
active processes) with at least a couple of megabytes of free
memory. This displays the speed of reading through the buffer
cache to the disk without any prior caching of data. This
measurement is an indication of how fast the drive can sustain
sequential data reads under Linux, without any filesystem
overhead. To ensure accurate measurements, the buffer cache is
flushed during the processing of -t using the BLKFLSBUF ioctl.
Good advice. I think the -t operation more accurately reflects normal
operation. Here is what I find:

Spinning SATA3 disk:
Timing buffered disk reads: 440 MB in 3.01 seconds = 146.30 MB/sec

Sandisk SATA3 SSD:
Timing buffered disk reads: 1610 MB in 3.00 seconds = 536.65 MB/sec

Sandisk M2 PCIe SSD:
Timing buffered disk reads: 8878 MB in 3.00 seconds = 2958.75 MB/sec
root
2019-07-20 20:17:55 UTC
Permalink
Post by root
Post by Rich
Post by root
cp -aR source dest
This is representative of how the devices are used.
You can also use the 'hdparm' command to perform device read speed
$ man hdparm
-t Perform timings of device reads for benchmark and comparison
purposes. For meaningful results, this operation should be
repeated 2-3 times on an otherwise inactive system (no other
active processes) with at least a couple of megabytes of free
memory. This displays the speed of reading through the buffer
cache to the disk without any prior caching of data. This
measurement is an indication of how fast the drive can sustain
sequential data reads under Linux, without any filesystem
overhead. To ensure accurate measurements, the buffer cache is
flushed during the processing of -t using the BLKFLSBUF ioctl.
Good advice. I think the -t operation more accurately reflects normal
Timing buffered disk reads: 440 MB in 3.01 seconds = 146.30 MB/sec
Timing buffered disk reads: 1610 MB in 3.00 seconds = 536.65 MB/sec
Timing buffered disk reads: 8878 MB in 3.00 seconds = 2958.75 MB/sec
To which I want to add for hdparm -t /dev/ram1 the result is:

/dev/ram1:
Timing buffered disk reads: 16 MB in 0.01 seconds = 2419.11 MB/sec

So, in this case the M2 SSD is faster than reading from ram?
Pascal Hambourg
2019-07-20 22:27:56 UTC
Permalink
Post by root
Post by root
Timing buffered disk reads: 8878 MB in 3.00 seconds = 2958.75 MB/sec
Timing buffered disk reads: 16 MB in 0.01 seconds = 2419.11 MB/sec
So, in this case the M2 SSD is faster than reading from ram?
No. The size of this ramdisk is just not enough for an accurate speed
measurement.
Pascal Hambourg
2019-07-20 08:00:00 UTC
Permalink
Post by root
I mount tmpfs to /ram, and all the copies were
to a file in that directory.
Be aware that the contents of tmpfs may be swapped out, which implies
extra writes to the swap device with a performance impact.
Post by root
Post by Pascal Hambourg
Is this one single big file or multiple files (how many) ? Or do you
read the raw block device ?
cp -aR source dest
This is representative of how the devices are used.
But this is heavily affected by the filesystem.
This probably explains the poor speed compared to reading the raw devices.
Post by root
Post by Pascal Hambourg
53 MB/s is not a very good speed for a single unfragmented file on a
recent hard disk drive.
The source drive was unfragmented.
It is if the source consists of multiple files. Each file and
sub-directory counts as at least one fragment.
root
2019-07-20 15:50:46 UTC
Permalink
Post by Pascal Hambourg
Post by root
I mount tmpfs to /ram, and all the copies were
to a file in that directory.
Be aware that the contents of tmpfs may be swapped out, which implies
extra writes to the swap device with a performance impact.
Post by root
Post by Pascal Hambourg
Is this one single big file or multiple files (how many) ? Or do you
read the raw block device ?
cp -aR source dest
This is representative of how the devices are used.
But this is heavily affected by the filesystem.
This probably explains the poor speed compared to reading the raw devices.
I use ext4 for all my storage.

I have been afraid to try "exotic" file systems. It took me a long
time to move away from ext2. Now I wouldn't consider anything other
than a journaling file system.
Post by Pascal Hambourg
Post by root
Post by Pascal Hambourg
53 MB/s is not a very good speed for a single unfragmented file on a
recent hard disk drive.
The source drive was unfragmented.
It is if the source consists of multiple files. Each file and
sub-directory counts as at least one fragment.
I said it was unfragmented because I had just copied over my
entire system to a freshly formatted spinning drive before the
test.
Pascal Hambourg
2019-07-20 22:44:20 UTC
Permalink
Post by root
Post by Pascal Hambourg
Post by root
Post by Pascal Hambourg
Is this one single big file or multiple files (how many) ? Or do you
read the raw block device ?
cp -aR source dest
This is representative of how the devices are used.
But this is heavily affected by the filesystem.
This probably explains the poor speed compared to reading the raw devices.
I use ext4 for all my storage.
I did not mean the filesystem type but the filesystem organization : how
many files, which sizes...
Post by root
Post by Pascal Hambourg
Post by root
Post by Pascal Hambourg
53 MB/s is not a very good speed for a single unfragmented file on a
recent hard disk drive.
The source drive was unfragmented.
It is
I meant "it is fragmented"
Post by root
if the source consists of multiple files. Each file and
Post by Pascal Hambourg
sub-directory counts as at least one fragment.
I said it was unfragmented because I had just copied over my
entire system to a freshly formatted spinning drive before the
test.
Even though, a set of multiple unfragmented files is not stored the same
way as a single unfragmented file. If ext4 allocates extents for each
file, files are not stored in consecutive blocks. Also multiple files
may not be read in physical order. This is why each file and directory
should be considered as a fragment.
tom
2019-07-31 20:35:31 UTC
Permalink
On Wed, 17 Jul 2019 20:16:51 +0000 (UTC)
Post by root
I just upgraded my m2 SSD from 250GB to 1TB. This
morning I was playing around testing the speed
vs SATA drives. I have a SATA3 spinning disk
on /dev/sda1 and a Sandisk SSD on /dev/sdb1
I copy 5GB from either of these 2 to a
mount /dev/sdb1 /sdb1
The 5GBcopy takes 48 seconds.
Mounting the spinning SATA3 drive in the
same way the copy takes 53 seconds.
These results are repeatable.
Now here is the amazing finding. If I make
/dev/sdb1 /sdb1 ext4
defaults,relatime,nodiratime,errors=remount-ro 1 1 and then
execute mount -a
The 5GB copy takes just under 3 seconds.
In other words, mounting the SSD via fstab results
in a 16 fold increase in speed.
I repeated the results twice and verified the copy.
Interesting, just to make sure, can you test barrier options to your
mount options? The system could be presenting the transaction as done
to the virtual filesystem, but the underlying hardware transfer is
still happening. the barrier mount option has changed a lot in version
but it's usually something along the lines of barrier=0 or nobarrier
Loading...