1 #+TITLE: Putting Your org Files Under Version Control.
3 #+EMAIL: ian@manor-farm.org
6 #+OPTIONS: H:3 num:nil toc:t \n:nil @:t ::t |:t ^:t -:t f:t *:t TeX:t LaTeX:t skip:nil d:(HIDE) tags:not-in-toc
11 Since I discovered org mode it has gradually become my "Information
12 sink" and most of my life seems to have moved into it. I wanted a
13 system that allowed me to move my org files around between different
14 computers which was easy to use and kept my files under version control.
16 I use bzr for my version control, but the system I describe should
17 work with any of the distributed VCS systems, such as Mercurial and
18 git. Since org mode is maintained in git and many org users are using
19 git, I have tried to give examples using both bzr and git.
21 This isn't intended to be an in depth discussion of version control,
22 but more of a quick overview of how you can use a version control
23 system with org to keep various computer in sync and as a backup system.
25 At home I normally work with my org files on an nfs share using one of
26 our desktop computers. The org files themselves are stored on my local
27 server. They are also automatically mirrored with a repository on my Web
30 If I go away before I set off, I will update the local branch of my
31 org files that I keep on my notebook using bzr. If I forget to update
32 my local copy, I can update my notebook repository from my home
33 server, or my web server as long as I have an Internet connection.
35 If I can't connect to the Internet, the org files on my notebook are
36 not usually more than 24 hours out of sync with the copy on my
37 server. I can carry on working and then merge my changes when I get
38 home, or can access the Internet.
41 I am assuming you already have Emacs and org installed. If you are
42 using a Debian based system installing git and bzr is simply a matter
45 #+BEGIN_SRC emacs-shell
46 apt-get install bzr bzrtools
50 If you want to push changes to a remote server, you might also want to
51 install the ncftp client.
54 ** Creating a Repository.
55 The first step is to create a repository in the directory where you
57 #+BEGIN_SRC emacs-shell
64 #+BEGIN_SRC emacs-shell
68 *** Excluding Files from the Repository.
69 You might have things, such as emacs backup files, in your org
70 directory that you don't want included in your VCS. A simple way to
71 avoid them being added is to create a .bzrignore file in the org file
74 In my .bzrignore I simply have:
76 #+BEGIN_SRC emacs-shell
80 With git you can create a .gitignore file, which has a very similar
83 ** Adding Files to Your Repository.
84 Now you need to add your org files to the repository. In the directory
85 where you keep you org files:
87 #+BEGIN_SRC emacs-shell
93 #+BEGIN_SRC emacs-shell
97 ** Committing Changes.
98 Once you have added your files to the repository you need to commit
99 the changes to update the repository:
101 #+BEGIN_SRC emacs-shell
102 bzr commit -m "Imported sources."
107 #+BEGIN_SRC emacs-shell
108 git commit -a -m "Imported sources."
111 Commit makes a snapshot of the working state of the files. You can
112 choose to commit changes at any time. There are several Emacs
113 packages, which will let you do this directly from Emacs. I also run a
114 cron job which commits any changes automatically each evening. The
115 shell script I use is:
117 #+BEGIN_SRC emacs-shell
120 bzr commit -m "automatic commit via cron."
123 If you want to commit files selectively, rather than all uncommited
124 files, you can specify the file names on the command line:
126 #+BEGIN_SRC emacs-shell
127 bzr commit -m "Made some changes." myfile.org another_file.org
130 * Working With Your org Files On More Than One Computer.
131 You will probably want to work on your org files on more than one
132 computer. The traditional method would be to copy all your org files
133 between your various computers. This is error prone and you
134 might end up accidentally overwriting new files with older versions.
136 A better way to synchronise files between your computers is to use
137 your version control system. Not only do your org files get updated,
138 but you also get their complete version history. The steps you need to
139 take to maintain versions of your org files on different computers are
142 Note that there are several methods which you could use, the one I
143 describe works for me.
145 ** Creating a Local Branch.
146 First create a local copy of your org files by making a branch. You
147 can then continue working on your files locally and then use bzr or git to
148 merge changes back to your main copy of org when required.
150 To create an initial working branch on your local computer:
152 #+BEGIN_SRC emacs-shell
153 bzr branch ~nfs/firewall/Documents/org ~/Documents/org
157 #+BEGIN_SRC emacs-shell
158 git clone ~nfs/firewall/Documents/org ~/Documents/org
161 This command creates a local branch on my notebook from the repository
162 on my file server. Note that bzr and git will create the directory for
163 the branch. If the directory already exists you will get an error.
165 The general format of the branch command is:
167 bzr branch location directory
169 bzr understands a wide variety of transport protocols, which you
170 could use to create your initial branch:
172 |------------+-------------------------------------------------------------|
173 | aftp:// | Access using active FTP. |
174 | bzr:// | Fast access using the Bazaar smart server. |
175 | bzr+ssh:// | Fast access using the Bazaar smart server over SSH. |
176 | file:// | Access using the standard filesystem (default) |
177 | ftp:// | Access using passive FTP. |
178 | http:// | Read-only access of branches exported on the web. |
179 | https:// | Read-only access of branches exported on the web using SSL. |
180 | sftp:// | Access using SFTP (most SSH servers provide SFTP). |
181 |------------+-------------------------------------------------------------|
183 You now have a complete copy of your org files from the file server
184 and their version history in ~/Documents/org. You can simply fire up
185 Emacs and work on the org files in that directory.
187 If you use git the syntax is very similar. This example uses ssh:
189 #+BEGIN_SRC emacs-shell
190 git clone url directory
191 git clone ssh://ian@mycomputer.co.uk/~/Documents/org ~/Documents/org
194 At any point you can commit your changes to the local repository using
197 #+BEGIN_SRC emacs-shell
198 bzr commit -m "Committing changes made on my notebook."
201 ** Merging Changes from a Local Branch to Your Server.
202 When you return back home or to the office, you will probably want to
203 update the copy of org on your local server from your notebook:
205 - First you need to commit any local changes that you have made.
207 - Next you need to use the push command to send your changes back to
208 the server. The basic command is:
210 #+BEGIN_SRC emacs-shell
214 If you don't specify a destination location the push is made to the
215 branch that this branch originated from. If you are the only person
216 working with your org files, the push should update your remote copy
217 without requiring any further action. If there are any conflicts take
218 a look at the bzr manual on Resolving Conflicts for more information
219 on how to fix the conflicts.
221 Note that once the push has completed the it needs to be
222 committed on the server to update your working tree:
224 #+BEGIN_SRC emacs-shell
225 cd ~/nfs/firewall/Documents/org
226 bzr commit -m "Merged changes from my notebook."
229 Up to the point of committing the merge you can choose to throw away
230 any changes by using the revert option in bzr.
232 Git uses the same command:
234 #+BEGIN_SRC emacs-shell
238 Unlike bzr, git will auto commit changes after a merge. If you don't
239 like this behaviour use the --no-commit option to make it behave like
242 ** Updating an Existing Branch of Your org Files.
243 Once you have a local branch on your computer, you can update it from
244 your server at any time using the following command:
246 #+BEGIN_SRC emacs-shell
250 By default pull uses the location where you originally branched
251 from. this will pull any changes from the remote location and merge
252 them with your local copy.
254 The command with git is the same
256 #+BEGIN_SRC emacs-shell
260 * Creating a Central Repository.
261 In order to give your system maximum resilience you may want to create
262 a central repository somewhere off site that is accessible from the
263 Internet. Using an off site server will also increase your backup
266 Typically you will only want to store history and not working copies
267 of your files in your central repository. So you can create the repository with the
270 #+BEGIN_SRC emacs-shell
271 bzr init-repo --no-trees sftp://myserver.com/bzr/myrepo
274 You can now use bzr to push your local branch to the central
277 #+BEGIN_SRC emacs-shell
279 bzr push sftp://myserver.com/bzr/myrepo/org-files
282 Git is a bit more complicated. The goal is to create a bare repository
283 on the remote server. If you have ssh access to the server you can do
286 #+BEGIN_SRC emacs-shell
293 Alternatively create the directory on your local computer and move it
294 to your server via ftp or some other method.
296 Now you need to push your local repository to the server:
298 #+BEGIN_SRC emacs-shell
299 cd ~/my_local_git_repo/.git
303 Edit the config file in the .git directory and add something like:
305 #+BEGIN_SRC emacs-shell
307 url = me@myserver.com:~/repositories/myrepo.git
308 fetch = +refs/heads/*:refs/remotes/origin/*
311 If you don't want to edit the config file directly you can add the
312 required information like this:
314 #+BEGIN_SRC emacs-shell
315 git remote add origin me@myserver.com:~/repositories/myrepo.git
318 Note that you can call "origin" anything you like. Assuming that you
319 have ssh access to your server you should now be able to push your
320 changes to the server:
322 #+BEGIN_SRC emacs-shell
323 git-push push origin master
326 ** Using a USB Stick as a Central Repository.
327 You can create a central repository on a USB stick. You might want to
328 do this if you take files between your home and work computers and
329 can't use the internet to synchronize your files because there is a
332 There are some special considerations required when using a usb stick:
334 - You need to specify that you can used mixed case in filenames, or
335 you will have problems creating names like HEAD with git. Note this
336 is only a problem with usb sticks formatted as FAT or VFAT.
338 If you are using the gnome desktop fire up gconf-editor and navigate to
339 system->storage->default\_options->vfat. Edit the key "mount_options"
340 and change its value to [shortname=mixed,uid=].
342 This option is the default with recent versions of Ubuntu such as
345 *** Creating the Repository on the USB Stick.
346 Using bzr to create a bare repository:
348 #+BEGIN_SRC emacs-shell
349 bzr init-repo --no-trees /media/disk/bzr-repo
352 Now populate the repository with your org files:
354 #+BEGIN_SRC emacs-shell
355 bzr push file:///media/disk/bzr-repo
358 Using git the command you need to create the directory and a bare repo
361 #+BEGIN_SRC emacs-shell
368 Now edit the config file in the .git directory of your local repo and
371 #+BEGIN_SRC emacs-shell
373 url = /media/53141017-f3e3-407f-a08f-20b91064b7f3/repos/org.git/
374 fetch = +refs/heads/*:refs/remotes/usbstick/*
377 You can now push to the usb stick:
378 #+BEGIN_SRC emacs-shell
379 git push usbstick master
382 You may wish to mount your usb stick in /etc/fstab using the disk's
383 UUID to ensure that the mount point is always the same.
385 Once you have made the initial push you can sync your changes to the
388 #+BEGIN_SRC shell-script
391 # bzr will default to using the saved initial location,
392 # so you don't need to specify a location.
396 #+BEGIN_SRC emacs-shell
397 git push usbstick master
400 The git example assumes that the remote on the usb stick is named
401 "origin" (the default) and the branch that you want to push from is
402 named "master" (the default).
404 *** Getting Your org Files from the USB Stick.
405 To get your org files from the usb stick on another computer for the
410 #+BEGIN_SRC emacs-shell
411 bzr branch /media/usbdisk/bzr-repo ~/Documents/org
415 #+BEGIN_SRC emacs-shell
416 git clone /media/disk/org.git ~/Documents/org
419 In subsequent sessions, once the local repository has been populated,
420 you should use the "pull" command.
422 *** Getting Your Local Changes onto the USB Stick.
423 When you have finished working on your local org files you need to
424 commit your changes to the local repository:
426 #+BEGIN_SRC emacs-shell
427 bzr commit -m "Commit before push to usb stick."
430 #+BEGIN_SRC emacs-shell
431 git commit -m "Commit before push to usb stick."
434 To update the repo on your USB stick you need to push your changes:
436 #+BEGIN_SRC emacs-shell
440 #+BEGIN_SRC emacs-shell
444 Assuming that you have successfully set up a repo on your usb stick and
445 have repos on your work and home computers your work flow would look
450 Commit your changes (=bzr commit=)
451 Push the changes to your usb stick (=bzr push=)
454 - Plug in your usb stick.
455 - Get the changes - =bzr pull=
456 - Work on your files in Emacs.
457 - Commit the changes you made - bzr commit -m "Commit before push to usb stick."
458 - Push your changes to the usb stick - =bzr push=
459 - Unplug the usb stick and go home.
461 ** Update the Local Branch on Your Notebook from the Central Repository.
462 Sometimes you might want to update your local branch from your central
463 repository, rather than from your local server. One reason for wanting
464 to do this is if you are away from home and your local computer's
465 branch is quite out of date. If you automatically update your central
466 repository daily from a cron script, the files on there should never
467 be more than one day old, so if you have an Internet connection you
468 can get a more recent version of your files from the central repository.
470 The pull command will do this, but you need to specify the location on
473 #+BEGIN_SRC emacs-shell
474 bzr pull sftp://myserver.co.uk/my_repo/org
478 I have three different scenarios for using my system:
481 My org files are on my home server in a directory accessible via
482 nfs. I can either load my files via an nfs share, or run Emacs in a
483 terminal on the file server.
486 Update the files on my laptop before I leave:
488 #+BEGIN_SRC emacs-shell
493 Launch Emacs using a different .emacs, which has its org configuration
494 set to point to the files on my local hard disk, rather than the nfs
497 When I get back home I can update the files on my server by doing a
499 #+BEGIN_SRC emacs-shell
504 Note that push updates the repository, but not the working copy of the
505 files. To update the working copy on your server you need to run bzr
506 update. Alternatively, you can use the Push and Update plugin from
507 https://launchpad.net/bzr-push-and-update , which automates this
510 You can set up anacron to update the org files on your laptop when you
511 startup, or shutdown which keeps the files on your local hard drive up
512 to date in most circumstances.
514 ** Away from Home and I Forgot to Update My Files Before Leaving.
515 If the files on my laptop aren't up to date, I can use bzr to pull a
516 copy from my web server via my mobile phone. If I can't get a signal,
517 or find a wifi hotspot, the files on my laptop shouldn't be more than
518 24 hrs out of date. I can just work on my local copy and merge the
519 changes when I get back home.
522 So far my system has proved quite robust. it's also useful having the full version
523 history of my org files on each computer.
524 * Further Information.
525 You can find out more about bzr at:
526 http://doc.bazaar-vcs.org/bzr.dev/en/user-reference/bzr_man.html and
527 http://doc.bazaar-vcs.org/latest/en/mini-tutorial/index.html .
529 Git's home page can be found at: http://git.or.cz/
531 ** Useful bzr Plugins.
532 | rspush | http://bazaar-vcs.org/BzrPushExample | Pushes changes using rsync. Useful if your collection of files is very large. |
533 | fastimport | https://launchpad.net/bzr-fastimport | Import exports streams to bzr repositories. |
534 | automirror | https://launchpad.net/bzr-automirror | Automatically mirror the project's current state to another branch. |
536 ** Moving Data Between bzr and git.
537 While I was writing this tutorial I needed to move my data between my
538 bzr repository and a git repository, so I could test out the various
539 git commands. In the process I learnt quite a bit about moving data
540 between bzr and git, which I thought I would record here in case
541 someone finds it useful.
543 *** Moving Your Data from bzr to git.
544 Like most things this is easy once you know how. You need the bzr
545 fast-import plugin. You can get this from
546 https://launchpad.net/bzr-fastimport . Note that although the plugin
547 is named fastimport it does fast exporting too!
549 First you need to create an empty git repo:
551 #+BEGIN_SRC shell-script
552 mkdir ~/devel/org-git
557 Now import your bzr repo into the empty git repo:
559 #+BEGIN_SRC emacs-shell
560 bzr fast-export --export-marks=.git/bzr.mark ~/Documents/org/.bzr | git-fast-import --export-marks=.git/git.mark
563 If all goes well all your revisions will be imported into the git
564 repo. To start working on your org files you first need to check them
567 #+BEGIN_SRC emacs-shell
571 You should also be able to import new bzr changes incrementally into
574 #+BEGIN_SRC emacs-shell
575 bzr fast-export --import-marks=.git/bzr.mark --export-marks=.git/bzr.mark ~/Documents/org.bzr/trunk | git fast-import --import-marks=.git/git.mark --export-marks=.git/git.mark
578 *** Moving Data from git to bzr.
579 You can also move your data from a git repository to a bzr one. As
580 above you will need to install the bzr fastimport plugin. The
581 procedure is based on the one described in the bzr documentation: http://bazaar-vcs.org/Scenarios/ConvertFromGit
583 Create an empty bzr repository:
585 #+BEGIN_SRC emacs-shell
586 bzr init-repo bzr-org
589 Now use git's fast-export to export all the branches and pipe the
590 output to bzr's fast-import.
592 #+BEGIN_SRC emacs-shell
593 $ git fast-export -M --all | (cd bzr-org; bzr fast-import -)
596 Using git fast-export's -M flag embeds git's inferred filenames into
597 the output. If you don't want this, drop the -M flag.
599 I find screencasts a good way of learning things. Unfortunately there
600 don't seem to be any for bzr at the time of writing. However, there
601 are several excellent ones for git:
603 - [[http://www.youtube.com/watch?v%3D4XpnKHJAok8][Linus Torvalds (author of git) on Git]]
605 - [[http://www.youtube.com/watch?v%3D8dhZ9BXQgc4][Randal Schwartz on Git]]\\
606 Randal's screencast gives an excellent overview of git, without getting
607 too bogged down in the details.
609 - [[http://gitcasts.com/][Gitcasts]]\\
610 Gitcasts are a series of short screencasts (approximately 5 mins),
611 each covering a particular aspect of git.
615 Maybe a full versioning system is *not* what you're looking for and
616 backups are enough. [[http://www.20seven.org][Greg Newman]] has a blog entry about [[http://www.20seven.org/journal/2009/09/backing-up-org-mode-files.html][backing up
617 Org-mode files]] under Mac.