All issues

SFTP performance tuning

theincogtion     Jun 2 10:31AM 2018 CLI

I am using duplicacy 2.1.0 on my qnap with 2 arm cores. My upload speed is 3,1MB/s (25Mbit/s) I am running a VPS on which I want to upload my files using duplicacy. The upload only has 1,5MB/s in average, exactly half the speed I could actually use. It never even reaches 3,1MB/s as peak. What it did to find the problem: running iperf between server and client -> full speed 3,1MB/s possible uploading sftp files from my windows client to my VPS -> with 5 parallel streams I get 3,1MB/s using duplicacy between my 2 local NASes gave me 6MB/s

So my conclusion is that no throtteling is active (if so my windows pc wouldn't be able to gain the upload speed) and also every computer in the setup can reach this speeds. What I tried with duplicacy: increasing chunk size from 4M to 8MB -> 1,5MB/s increasing threads to 5 -> 1,5MB/s

That I discoreved that the Ram usage of duplicati is very high: performance

top top

Any ideas how to increase the upload speed? I am using encryption, is there any way to lower the compression level?

edit: I could lower the ram usage by using chunk size 1M and DUPLICACY_ATTRIBUTE_THRESHOLD=1

Nevertheless the speed with 4.5 and 6 threads was as low as before.

gchen    Jun 4 10:55AM 2018

If you lower the RAM usage and still get the same upload speed, then the RAM isn't the bottoleneck.

Did you mean when you ran Duplicacy on the same qnap and backed up to a local NAS the upload speed was 6MB/s?