Ubuntu: A bunch of relatively small images are reading at 1 MB/s via Samba



Question:

I have a bunch of images of total size of 6Gb, but each of which is relatively small. I am trying to copy these images over LAN and the speed of copying is approximately 1 MB/s.

It is slower than Internet nowadays.

My LAN has gigabit speed. Looks like this is happens only with lots of small files. When I am copying one large file, speed is more than 100 MB/s.

Is it possible to make samba faster for a bunch of small files?


Solution:1

You can't make Samba faster for small files, because Samba is not the issue.

The best way to transfer a large amount of small files is to compress them into an archive first, and then transfer the entire archive. You may even find the entire process of compress>transfer>extract to be faster than transferring all the files.

The reason is two parts, overhead, and hard drive operations. There's more overhead on the network when transferring multiple files compared to a single file since each file has many attributes that go along with it. And, your hard drive works harder to allocate space to write each packet from its buffer. If you have a mechanical drive, then you also have to factor in seek time added to it, and the hard drive's operation of fitting the data into sectors. SSD's aren't affected as much.

You'll find the same problem when transferring files from HDD to HDD, or from USB drive to HDD. It's the overhead of many individual files.


Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »