All issues

Errors while backing up to B2

Moshnic     Sep 22 10:49AM 2017 CLI

Hello,

While backing up my data to Backblaze B2 I encountered several error messages (not many, around 50 errors for 40GB of data).

2017-09-20 05:02:03.586 INFO BACKBLAZE_UPLOAD URL request 'https://pod-000-1728-07.backblaze.com/b2api/v1/b2_upload_file/f7da76fd6add26c35ded0e1a/c021_v0001038_t0321' returned status code 408

and

URL request 'https://pod-000-1304-04.backblaze.com/b2api/v1/b2_upload_file/f7da77ed6ade27c35ded2c1b/c001_v0001090_t8010' returned status code 503

My questions are:

  • Does Duplicacy resend the chunk after it encounters an error?
  • How can I make sure that the backup is correct? The duplicacy check command only verifies that the chunks exist and duplicacy check -files requires a huge download which I'd like to avoid. Is there an option to verify only a single file or a small group of files?

Thank again.

gchen    Sep 22 2:49PM 2017

Yes, there is a retry mechanism to resend the chunk if the B2 server returns an error. These error message are normal as long as the backup finishes. According to Backblaze, some of these errors are intentionally introduced.

If you want to verify only a subset of files then you can try a restore. Create an empty directory and initialize it with the same repository id and store url. Then run the restore command but specify only the files you want to check. Duplicacy verify file hashes on restore so no error means all files passed the hash verification.

dgcom    Sep 25 9:30PM 2017

Are there any plans to add support for verifying file hashes without download?

B2 (and many others) support this and other apps (like rclone) can do this...

gchen    Sep 27 9:51AM 2017

B2 keeps a sha1 hash for each file while Duplicacy uses blake2. So that would require Duplicacy to save the sha1 hashes in a local database during backup and then compare them with what B2 returns when running the check command if such a database is found.

This is definitely something doable but it may take a while.

dgcom    Sep 27 11:07PM 2017

Thank you. I know, the idea of Duplicacy is to not keep much data locally. Things like this won't break backup if missing, but will provide a lot of help if implemented... Similar idea can be applied to other supported backends - there is a list of hashes/providers here: https://rclone.org/overview/