maccra Oct 16 9:39PM 2017 CLI
Hello, I am having an issue with the prune command. I run the following command:
duplicacy prune -a -storage default -keep 0:14 -keep 1:3 -exclusive
but I receive following error;
Chunk 55381f9bf500ed80a204abd202f52ad3250b115e38af34a36371ab92726cf97c can't be found
and the prune just aborts and does not complete. All the snapshots which should be deleted are still on the storage location. I check by doing a List.
I looked in the docs, but I could not find any information about what I need to do to solve it. Please advise.
Thanks
gchen Oct 16 10:13PM 2017
Prune log files are kept under .duplicacy/logs. You may be able to find out when this chunk was deleted from these logs files.
Of course first double check the chunk doesn't exist in the storage, otherwise it is a different issue.
To correct the state of the storage, you need to find out which snapshots reference this missing chunk by running the check command, and then manually remove these snapshot files from the storage.
After that, run duplicacy prune -exclusive -exhaustive to clean up the storage. Remember that no other backup can be running when you run this command (due to the use of -exclusive). This missing chunk is very likely to be the result of running the prune command with the -exclusive while another backup was running.
maccra Oct 17 6:08AM 2017
Ok.. Thanks for the instructions.
I found the log file where it was deleted. It does not tell me much except that it was deleted under -exclusive mode as you guessed.
I checked the storage and the missing chunk does not exist. good here.
I then ran the Check command and found out that the first few (i.e. the oldest snapshots) were referencing the missing chunks. So I went ahead and deleted them manually from the storage location, as you explained.
I now have snapshots from 400 to 511
I ran the Check command on these and received response that all chunks for all snapshots 400-511 exist;
C:\Duplicacy\Vaults>duplicacy check -r 400-511 Storage set to C:\BDup Listing all chunks All chunks referenced by snapshot SP4-Vaults at revision 400 exist All chunks referenced by snapshot SP4-Vaults at revision 401 exist .. .. .. All chunks referenced by snapshot SP4-Vaults at revision 509 exist All chunks referenced by snapshot SP4-Vaults at revision 510 exist All chunks referenced by snapshot SP4-Vaults at revision 511 exist
I then ran the Prune command with -exclusive and -exhaustive and received an error that another chunk can't be found;
C:\Duplicacy\Vaults>duplicacy prune -r 400-511 -exclusive -exhaustive Storage set to C:\BDup Deleting snapshot SP4-Vaults at revision 400 Deleting snapshot SP4-Vaults at revision 401 .. .. .. Deleting snapshot SP4-Vaults at revision 509 Deleting snapshot SP4-Vaults at revision 510 Deleting snapshot SP4-Vaults at revision 511 Chunk e067d67852ad05910a06320d4912eaa314293239d71a2f0f5ff3660adf5e6f99 can't be found
Now this is a different chunk from above. The Prune command aborts again and the snapshots are not deleted. I check the logs and the chunk e067d67....6f99 has also been deleted in the log under -exclusive mode and also this chunk does not exist in the storage location.
But now I am baffled what to do. Check command says all chunks exist while Prune aborts because of missing chunks.
Please advise what to do now. Thanks PS: sorry if it hard to read, but I can't seem to get the formatting right to make it easier to read.
gchen Oct 17 10:26AM 2017
There was a bug in the local cache cleaning code that happens if you attempt to delete the latest snapshot with the -exclusive option. This bug had been reported and fixed: https://github.com/gilbertchen/duplicacy/issues/182
You'll need to build from the latest source to have the fix. But since the bug occurred when Duplicacy tried to clean the local cache as the last step of the prune command, you can also just nuke the entire local cache under .duplicacy/cache and everything should be working fine.
maccra Oct 17 11:39AM 2017
Thanks for the response. I have no idea how to build stuff so I will need to wait for that.
But I did what you suggested. I deleted the local cache under .duplicacy/cache. However I still get the same error and the snapshots folder and snapshots 400 to 511 get rewritten to the cache folder.
C:\Duplicacy\Vaults>duplicacy prune -r 400-511 -exclusive -exhaustive Storage set to C:\BDup Chunk e067d67852ad05910a06320d4912eaa314293239d71a2f0f5ff3660adf5e6f99 can't be found
There are multiple repositories backing up to the same storage location if that would have any influence (which at the time of the prune command, none were performing any backup though). For info, storage location is local. System is Win 10 Pro.
Still no luck. cheers.
gchen Oct 17 1:10PM 2017
Run duplicacy check -a to see which snapshots reference this e067 chunk. It could be a snapshot created from a different repository.
You'll then need to manually remove any snapshot files under the snapshots directory from the storage that reference this chunk (and all other missing chunks), until duplicacy check -a doesn't report an error. And run this command to clean up the storage afterwards:
duplicacy -exclusive -exhaustive
maccra Oct 17 1:51PM 2017
Ok thanks. I finally could get it to work and prune the storage. Needed to check all repositories for missing chunks and deleted the individual snapshots as you said.
For some storage locations I have 10-12 repositories backing up to it. This could a lot of work if this happens again and I think it would if I am not careful. I realise now that I need to be more careful using the -exclusive option in Prune.
Is there any way you could build a comprehensive build function into your software to make it automatic. Something to consider, as a feature request.
Thanks again for the instructions.