Alex Apr 23 11:20PM 2018 CLI
Hi All, I am a new Duplicacy convert from CrashPlan and I am seeing very slow backup speeds but cannot figure out what the bottleneck is.
I have read through a number of different threads including this github issue and this optimization thread. From the discussion in the second link, I realized I should use only a single thread since I am backing up from a hard drive and I did see about a 50% speed improvement at steady state. However, I am still only getting about 150kB/s at the moment and my modest 400 GB backup says it is going to take 20 days.
A couple notes about my setup - I am running Ubuntu 16.04 with Duplicacy running inside a docker container. I have a dual core CPU (which hovers around 2% usage) with 8 GB of RAM and am backing up from a 2x2TB Raid 1 array (software raid).
As far as I can tell, there are plenty of resources available to the backup engine and I am stumped as to what would be causing such low speeds, especially when some of the other threads are reporting speeds in the 1-10 MB/s range.
Any suggestions would be appreciated!
gchen Apr 24 7:44AM 2018
If you're backing up to B2 then you should actually use as many threads as permitted by your system. If you use only one thread then you are only uploading to one B2 upload sever which may be very slow if you're unlucky.
The recommendation to use one thread in https://duplicacy.com/issue?id=5670666258874368 was mainly for local disk backends. I don't even know it is correct since it turned out the main issue there was using a storage originally initialized by Vertical Backup for Duplicacy.
Alex Apr 24 12:06PM 2018
Oh interesting. Thanks for the explanation. I will try increasing the thread count when I get home this evening and see if that improves things for me.
towerbr Apr 25 9:14AM 2018
Gilbert, what would be a good thread setup for Wasabi?
gchen Apr 25 9:14PM 2018
Wasabi is much faster than B2 in my testing (https://github.com/gilbertchen/cloud-storage-comparison) so you don't need to use as many threads as B2. I don't know a good number though. You can try increasing the number gradually until your upload bandwidth is saturated.
Alex Apr 26 7:41AM 2018
Bumping the number of threads up to 16 definitely gave me some improvement, but I was occasionally getting b2 errors 408 and 500. I tried decreasing the threads to 8 and this seems to be working without giving me errors. My speed is looking better an hovers closer to 1.5 MB/s after letting the backup run overnight. Looks like it is still going to take a couple days to backup, but that is better than the 20 days I was seeing before.