dotphiles / dotsync

Sync dotfiles between multiple machines from a git repo or push using rsync

dotsync

dotsync keeps your local dotfiles in sync with a git repository and keeps multiple remote machines in sync, either with them pulling from the git repo or pushed via rsync

Master servers can be assigned, for cases where groups of machines are behind firewalls or only accessible from a certain location.

Requirements

dotsync assumes that you have ssh setup correctly, with ssh-agent and configs for correct usernames etc, if you can't 'ssh hostname' it probably wont work and if your asked for passwords it will take hours with any number of machines.

Installation

If you already have your dotfiles in a git repo...

cd ~/.dotfiles
git submodule add [email protected]:dotphiles/dotsync.git
git submodule update --init

If you dont...

Fork the main project on github

git clone --recursive [email protected]:*username*/dotphiles.git ~/.dotfiles

Copy your dotfiles into ~/.dotfiles without the dot...

List your dotfiles in dotsyncrc without the dot...

Checkin your changes...

git commit -a -m "Initial Commit"
git push

Then, symlink your dotfiles into place...

~/.dotfiles/dotsync/bin/dotsync -L

See the Configuration section for configuration info.

Then add hosts into dotsyncrc and dotsync -I -H hostname (at your own risk!)

Usage

-I          - Initialise a machine using dotsync
-L          - Symlink available dotfiles into $HOME
-u          - Update to the latest copy of dotfiles
-U          - Update to the latest copy of dotfiles inc submodules
-P          - Push any local changes back to the repo (git only)
-H host     - Perform action against host listed in config, can be 'ALL'
-a          - Updates dotfiles on all known machines
-A          - Updates dotfiles and submodules on all known machines
-r          - Use rsync instead of git
-f conf     - Config file, defaults to '~/.dotsyncrc' or '$DOTSYNCRC'
-d dotfiles - Location of dotfiles, defaults to '~/.dotfiles'
-l          - List configured hosts and dotfiles to symlink
-c          - Run an arbitrary shell command on all known machines
-g          - Same as -c, but only on git configured machines
-v          - Verbose
-h          - Show help message

Configuration

~/.dotsyncrc

This config determines which hosts to sync, how, where from and which dotfiles to symlink into your homedir.

This file can be included in the repo, and dotsync will use it if its not already symlinked.

See templates/dotsyncrc

~/.dotfiles/.gitignore

Controls which files are ignored from being added to the git repo, add any temporary and 'secret' files here.

See templates/gitignore

~/dotfiles/.rsyncignore

Controls which files are ignored when rsyncing a remote host, add any temporary and 'secret' files here.

See templates/rsyncignore

Custom Dotfiles

Sometimes its handy to link a specific file on a specific machine, or to have a local dotfile in place instead.

If you have the following in dotsyncrc

[files]
dotfile
[endfiles]

dotsync will look for

~/$DOTFILES/dotfile.d/localhost
~/$DOTFILES/dotfile.d/$HOSTNAME
~/$DOTFILES/dotfile.d/$DOMAIN

And link the first one it finds instead of the standard dotfile. The localhost dotfile should be excuded from your repo.

Machine specific Dotfiles

Sometimes you have a dotfile thats only for one machine and not symlinked on any others.

If you add the following to dotsyncrc

[hosts]
hostname.example.com        git=ANY             file=dotfile1/dotfile1,dotfile2:.linkto
[endhosts]

dotsync will link the following on hostname only

~/$DOTFILES/dotfile1/dotfile1 to ~/.dotfile1
~/$DOTFILES/dotfile1 to ~/.linkto

Remote Machines

WARNING! This is the least tested and resilient code, i use it to sync a few hundred machines without problem but you can easily wipe out your ssh keys on remote machines if anything goes wrong....be prepared

dotsync is based around the idea of having lots of remote machines, that you want to sync your dotfiles to.

e.g.

         *origin*
      ----github------   remoteservers:r
    /         |        \      |
desktop:g----laptop:g----workdesktop:g
                |             |
            mac-mini:r   workservers:r

and the following dotsyncrc

[hosts]
laptop                       git=NONE
desktop                      git=laptop
mac-mini                     rsync=laptop
workdesktop                  git=laptop
workserver1                  rsync=workdesktop
workserver2                  rsync=workdesktop
workserver3                  rsync=workdesktop
remoteserver1                rsync=workdesktop
remoteserver1                rsync=workdesktop
remoteserver3                rsync=workdesktop
[endhosts]

Runing a dotsync -A on laptop would

• git pull from github
• connect to workdesktop & desktop & git pull dotfiles from github
• rsync dotfiles to mac-mini

Runing a dotsync -A on workdesktop would

• git pull from github
• rsync dotfiles to workserver{1,2,3} & remoteserver{1,2,3}

These can all be done with dotsync -A -H ALL which when run on laptop, should

• git pull from github
• connect to workdesktop & desktop & git pull dotfiles from github
• rsync dotfiles to mac-mini
• ssh to workdesktop & rsync dotfiles to workserver{1,2,3} & remoteserver{1,2,3}

Backups

Existing dotfiles will be backed up in ~/.backup/dotfiles/ Existing dotfile repos will be backed up as ~/.backup/dotfiles.old/

License

Copyright (c) 2012 dotsync contributers

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

dotsync's Issues

Reconsider the ping test

Currently the script attempts to ping a remote $host, and if unreachable it fails.

I haven't made a patch for this yet, but I suggest changing that to test the connection using a no-op ssh attempt.

Why? Because "ssh my-alias" is sufficient for dotsync to do it's job even though "ping my-alias" is guaranteed to always fail.

Symlink under windows

It is possible on windows to use the bash (e.g., after installing the git commandline). Dotsync also runs fine on windows.

My issue:
Windows does not support symlinks on its filesystem

My temp-fix:
Replace all ln -s with cp -r

Better solution:
Detect operating-system or provide a flag, which does the same as -L but for copying instead of linking
Or use something like this: https://stackoverflow.com/questions/18641864/git-bash-shell-fails-to-create-symbolic-links

Non-existing parent folder

Currently dotsync -L fails when the parent folder for the link destination doesn't exist.
This, however, may happen sometimes in newly installed systems.
E.g. the folder $HOME/.config/xfce4/terminal for the terminal configuration for xfce4, in $HOME/.config/xfce4/terminal/terminalrc, doesn't seem to exist until we manually adjust some custom property in the preferences.
So, would it be acceptable to fix this by applying mkdir -p ... on the parent path?

[hosts] files directive not working?

I'm probably missing something obvious here, but I the files=dotfile:linkto doesn't seem to do anything for me in the [hosts] configuration.

Here's an example of what I put in my dotsyncrc:

newton                     git=ANY     files=platforms/ubuntu:local
remus                      git=ANY     files=platforms/mac:local

So, I want newton to have a symlink from ~/local -> ~/.dotfiles/platforms/ubuntu
and remus to have ~/local -> ~/.dotfiles/platforms/mac

As far as I can tell, dotsync is ignoring these directives entirely.

What am I missing?

remote host name in dotsyncrc has to match with actual server name

Issue

Need to use an actual server name(remote machine's HOSTNAME) as a remote machine name for remote-rsync-update with a machine specific dot file,

sample environment:

machines:

• 192.168.0.1(localhost)
• 192.168.0.2(remote host)

/etc/hosts:

192.168.0.2  bar  #alias on my local machine

hostname on 192.168.0.2:

foo

~/$DOTFILES/dotsyncrc:

[hosts]
bar
[endhosts]

~/$DOTFILES/some-dotfile.d/bar:

• a dot file specific to the remote server foo, but referenced as bar on local machine

Status Quo:

executing dotrsync -u -r -H bar on local will not create a dotfile for "some-dotfile" in remote machine foo.

This is because after rsync, the script invokes "dotsync -L" on remote server via ssh.
When "symlink()" function finds target file("getrealdotfile()" function), it uses "$HOSTNAME" variable.

On the remote server, "$HOSTNAME" is "foo" because "bar" is an alias only on local machine.
Thus, "~/$DOTFILES/some-dotfile.d/bar" will not be resolved on remote server.

This is very inconvenient if the remote servers don't have good readable name, for example hostname returns "ip-192.168.0.2".

Is it possible to sync at the sub-directory level?

In my dotsyncrc file I have the following configuration:

[files]
config/fish/functions

Which I was hoping would create that same structure when it gets synced to my home directory, but instead of ~/.config/fish/functions I ended up with ~/.functions.

Is there a way to make it re-create the same structure that is in the dotsyncrc file?

Thanks!

Option -d is confusing

The option -d states:

-d dotfiles - Location of dotfiles, defaults to '~/.dotfiles'
But however inside the script the dotfiles path is built from $HOME/$DOTFILES, so to customize the dotfiles folder path one has to give the path relative to $HOME,
e.g. dotsync -L -f ~/.cronsync/dotsyncrc -d .cronsync
Consider changing the help message to avoid issues when using this option. :)

-f flag doesn't work

I added this as a git submodule to my dotfiles repo ~/dotfiles, then trying to execute dotsync -f ~/dotfiles/dotsyncrc fails with error:

*** /Users/jamie/.dotsyncrc doesnt exist, exiting ***

This seems to be because getconfig is called before options in the script

How to specify entry for .emacs.d ?

Hi

I tried to use this tools to synch my ~/dotfiles directory
I run into problems with my ~/dotfiles/.emacs.d directory - it is jsut ignored
Renaming it to emacs_d and adding a line
emacs_d:emacs.d
results in a link
~/emacs.d
Which is not right.
Changing to
emacs_d:.emacs.d
is also silently ignored.
So how should I specify my .emacs.d folder to make it work?

Cheers,

Daniel

Instructions possibly for old version of git

I'm new to dotsync and am trying to get everything to work for me. When I try to pull down my dotfiles from my dotphiles repo, I am following the instructions provided, but for this step:

git submodule --init update

git just shows a help message. It seems for my version of git (2.8.1) the equivalent seems to be

git submodule update --init