Multiple concurrent upload/download threads

nzevth     Nov 6 5:26PM 2016 CLI

Is it currently possible to use multiple upload and/or download threads? The CLI help only lists a -threads option while backing up, and only for Backblaze.

If not currently supported, is this planned at all for other providers in the future? Transferring a single chunk at a time seems to be a limiting factor for faster connection speeds. Being able to make use of maybe 2-3 simultaneous transfer threads would speed things up greatly. I know I can increase chunk size, though from what I understand this will lower the effectiveness of deduplication.


gchen    Nov 7 2:31PM 2016

Currently only uploading to Backblaze allows multiple threads. We'll add multi-threaded uploading to other storages if there is enough user demand. What is the storage backend you're using?


nzevth    Nov 7 10:08PM 2016

Google Drive.

I find using the Google Drive API in other products there is generally a 2s delay before a download begins - single threaded speeds are generally fine for individual large files in this case, but when there are a number of smaller files, such as Duplicacy's chunks, multiple threads can make quite a difference - especially noticeable on a faster connection.

This is more of an issue with restores, as Google Drive's response times are a lot lower for uploads, although backups would also benefit from multiple threads.