Etienne May 30 3:09PM 2017 CLI
I'm slowly designing my new backup system based on duplicacy.
I though about creating the repository on the NAS and add a symbolic link to the real directory to back up, but due to the Volume mapping of docker, I doubt following symbolic links inside the container would be OK.
For those reasons, I'd like to be able to store the .duplicacy directory outside of the repository.
Else I though about the forking ducplicacy-cli and preparing a pull request with
Any comments ?
Is there any interest beside me to add this extra feature ?
Thanks for your time reading me ;-) Etienne
gchen May 30 8:30PM 2017
I think it is a good idea to add an option to specify the location of the .duplicacy directory. But is there a better name than
-confdir? Something like
-pref-dir. I would avoid using anything similar to
config, as it is the name of the file used by the storage to store settings.
Etienne May 31 2:18AM 2017
Thanks for your answer !
I'd go for -pref-dir then.
I'll study the code more deeply and prepare a pull-request "asap" ;-)
Etienne Jun 1 2:21PM 2017
I'm trying to build the duplicacy-cli but build instructions do not work, probably due to
Any advice ?
gchen Jun 1 3:13PM 2017
Follow the build instructions on the github page:
git clone https://github.com/gilbertchen/duplicacy-cli.git ~/go/src/github.com/gilbertchen/duplicacy cd ~/go/src/github.com/gilbertchen/duplicacy/src go get ./... go build main/duplicacy_main.go
I made a change to
main/duplicacy_main.go to change the import path from
github.com/gilbertchen/duplicacy/src. You need this change for the build instructions to work.
Etienne Jun 2 6:18AM 2017
I'll check tonight ( I'm at work right now)
IMHO, it would be "better" to stick with golang culture
My 0.02€ Etienne
gchen Jun 2 3:39PM 2017
I wanted the README.md page to look neat so I moved all source files to the
src directory. But now that so many people have difficulties building from source, I think this may be a bad decision. So I just moved them back and renamed the github repository to
duplicacy. Here are the updated build instructions:
git clone https://github.com/gilbertchen/duplicacy.git cd duplicacy go get ./... go build main/duplicacy_main.go
Etienne Jun 3 3:19AM 2017
I'll redo my fork and "really" get started !!
Thanks a lot for listening feedback ! Have a nice week-end Etienne
Etienne Jun 4 5:42AM 2017
Need feedback ;-)
After studying the code a little bit more here is what I found out
Here is my proposition
Add the -pref-dir only to the init command: if the option is not specified -> unchanged behavior. If the option is specified,
-find .duplicacy directory:
In my opinion:
Any comment/suggestion is very welcome
Have a nice Day Etienne
gchen Jun 4 10:04AM 2017
This .duplicacy-location is basically a symbolic link. So my question is, why not make .duplicacy a symbolic link if the -pref-dir option is passed to the init command? The logic would be much simpler this way.
Etienne Jun 4 11:17AM 2017
Indeed .duplicacy-location is basically a symboly link. BUT...
First let me explain my use cases:
The data I want to back-up are stored on my NAS (QNAP). duplicacy will only run on the NAS itself (in a docker container) and backs up data on a remote cloud provider.
My users (my kids: I don't want them fiddling with duplicacy UI) use software like "Syncthing"  and QSync to sync data on their laptops. This means the "repository"( data to back up) will be replicated on laptops unaware of duplicacy.
In this use case, having a .duplicacy-location being a simple file without any "semantic for the OS" ( eg: symbolic links) is more "kid-proof" because there won't be any link to inexistant directories replicated on the laptops.
In my case, I'd prefer having a normal hidden file in my repository than a symbolic link pointing to a (sometimes) inexistant location...
Is it more clear ? Thanks for you patience... Etienne PS: two options -pref-dir (my idea ;-)) and pref-dir-link (yours) can always be implemented ....
gchen Jun 4 8:16PM 2017
How about making .duplicacy a file containing the real preference location if you pass the -pref-dir option to the init command? This way we don't need to deal with another special file like .duplicacy-location.
The -pref-dir-link isn't really needed I think, because users can create the symbolic link themselves if this is what they want.
Etienne Jun 5 3:29AM 2017
I'll study the .duplicacy idea more deeply but it might cause compatibility issues in case you use a old binary with a new repository format.
Scenario: external pref dir with a older client "unaware" of the pref-dir. with .repository-location the old binary will just fail ( not finding the .duplicacy directory) ( error: repo not initialized)
with the .duplicacy possibly being a file or a directory, how will a older client fail in such case is not yet clear in my mind..
I'll keep you posted..
Etienne PS: Please have a look at syncthing, they have the .stfolder marker file ( empty file) and they have .stversions directory for syncthing internal purpose ( revision tracking) PS2/ I'm very open to alternatives ways/ file names but I think the solution must be either compatible with older client versions, either fail safely in a predictive way and without side effects...
Etienne Jun 5 4:35PM 2017
I just pushed an 'early adopter' branch on my fork
It's not ready for prime time yet, but, gchen, would you mind having a look checking if the coding style and overall logic is OK ? ( it of course needs more tests...)
Could you check file duplicacy_shadowcopy_windows.go ? path building code is different.. Is this intentional ?
Thanks for your time Etienne PS: This is my first golang code... please be indulgent ;-)
gchen Jun 5 7:22PM 2017
GetDotDuplicacyPathName, and the variable holding the return value
GetDuplicationPreferencePathcan simply return the global variable.
\\is used when creating the symlink, but maybe we should just keep it, unless you have a test that verifies the change. The reason for creating the symlink is that Go can't handle the UNC path required to access the shadow copy, but I don't know if this is still the case with Go 1.8. We need to revisit this issue later.
Thank you for the good work. I think it is in good shape.
Etienne Jun 6 2:47AM 2017
Thanks for the feedback,
I'll update my branch asap !
BTW do you have anything like an integration test suite ? else I'll try to enhance the bash script I wrote. Etienne
gchen Jun 6 10:51AM 2017
I don't have an integration test other than those unit tests in the *_test.go files. Your bash script may be a good start.
Etienne Jun 7 1:52PM 2017
... GetDuplicationPreferencePath rather than GetDotDuplicacyPathName, and the variable holding the return value preferencePath.
Are you sure ? Or did you mean GetDuplicacyPreferencePath ?
gchen Jun 7 2:03PM 2017
Sorry. Yes, it should be GetDuplicacyPreferencePath.
Etienne Jun 11 7:13AM 2017
I rebased my pref-dir on the current master ( 2.0.2). I update the GUIDE.md document.
From my side, it look ready to be merged.
I currently use a custom built duplicacy on my backup scripts and report any issue.
Feel free to request here for changes needed for you to merge into the "main code"..
gchen Jun 11 7:34PM 2017
I'll take a look tomorrow. Thank you for your work!
gchen Jun 12 10:14AM 2017
It looks good to me. I think it is ready for a PR.
Etienne Jun 12 12:34PM 2017
Here is the pull request ...
Just rebased on the very latest master ( main directory renamed)
Etienne Jun 21 3:34PM 2017
I just found a regression in the -pref-dir behavior and I'm asking for advice :
Look like the line
err = os.Rename(temporaryPath, fullPath) in duplicacy_backupmanager around line 1228 fails because os.Rename can't move files across filesystems. 
What's your preferred way to fix this ?
Sorry for the regression ...
Etienne PS: do you want me to file a bug in github directly ?
gchen Jun 21 7:57PM 2017
I think the best option is to enforce the in-place mode if the preference path is not under the repository. Just set the variable
inPlace to true and it should work.
Let me fix this as I've made some changes to the GetDuplicacyPreference function in my branch. Yes, please file a bug in github.
Etienne Jun 22 5:34AM 2017
Issue created !
gchen Jul 7 9:28PM 2017
Sorry about the delay. I've been busy working on the performance study of Duplicacy and finally had some time to fix bugs accumulating over the past two weeks!
Let me know if the fix works.