Trouble restoring from B2 backup

stefandz    Jun 15 4:49PM 2017 CLI

I am test driving duplicacy right now and having trouble restoring from a backup. Here's what I have done:

  1. Successfully backed up 6GB of data from a Raspberry Pi to a B2 bucket - backup is named test_data
  2. Downloaded a snapshot of that bucket to my Mac and unzipped.
  3. Made a new folder on my Mac and run duplicacy init test_data /path/to/download to init that folder and link it to the unzipped data from B2.
  4. In that folder run duplicacy restore -r 1 but instead of success got Chunk {64-bit hash} can't be found

Any hints? My guess is that the key used for encrypting / decrypting chunk files is incorrect on the new machine (for data backed up from the Pi) but how do I pass this value across? How are others restoring backups from B2 in the scenario where the entirety of the original machine is destroyed (let's say stolen or fire damaged as opposed to just a HDD failure).

Any help gratefully received!

Many thanks


gchen    Jun 15 7:43PM 2017

File-based storage (as well as SFTP and Dropbox) uses a hierarchical structure for the chunks directory , while all other storages use a flat one. Therefore, if you copy all files stored under the B2 storage to a local disk, you'll need change the directory hierarchy accordingly. For example, chunks/dba7b45ae9d4e3ca6f578563f47e40cdcf604ce628b394292559469bb731b3b7 needs to be moved to chunks/db/a7/b45ae9d4e3ca6f578563f47e40cdcf604ce628b394292559469bb731b3b7.

Alternatively, you can use the copy command to move backups from one storage to another. To do this, run the add command in the original repository to add an additional storage:

duplicacy add -copy default mac test_data sftp://<mac ip>/path/to/download

And then run the copy command:

duplicacy copy -from default -to mac

This will save chunks in the correct way locally to your mac. Of course, you can also create a new repository on your mac with the same repository id test_data and the B2 bucket as the default storage, and run the add command to add path/to/download as a file-based storage.

Please note that to run the copy command you'll need to build from the latest source. There was a bug that I just fixed but it hasn't got into the latest binary releases.

gchen    Jun 15 8:58PM 2017

I saw your questions on twitter -- I'll just answer them here since this is easier...

If your purpose is to restore the test_data repository on you Mac, you'll just need to create a new repository with the same repository id and storage url:

duplicacy init test_data b2://bucket

And then duplicacy restore -r 1 should work just fine.

stefandz    Jun 16 5:51AM 2017

Thank you so much for the response, Gilbert. This works perfectly, thank you - and duplicacy restore recovers well from network interruptions.

Loving Duplicacy - let me know if there's a way I can contribute!

stefandz    Jun 16 7:30AM 2017

If anyone ever wants to try the "unflattening of the folder structure" method mentioned above, I wrote a small (and VERY basic) Node.js script to do exactly that, and put it here.

I wouldn't recommend this method, however - restoring directly from the B2 bucket is fully supported and tolerant of network mess, so only use this if for whatever reason you have a flat file structure copy that needs unflattening.

gchen    Jun 16 11:33AM 2017

Thanks. That tool can be useful.

stefandz    Jun 17 5:34AM 2017

No problem. Shame I don't have the skills to have written it in Go!

Log in to comment
Copyright © Acrosync LLC 2016-2017