1 TopGit - A different patch queue manager
7 TopGit aims to make handling of large amounts of interdependent topic
8 branches easier. In fact, it is designed especially for the case where
9 you maintain a queue of third-party patches on top of another (perhaps
10 Git-controlled) project and want to easily organize, maintain and submit
11 them - TopGit achieves that by keeping a separate topic branch for each
12 patch and providing some tools to maintain the branches.
23 Why not use something like StGIT or Guilt or 'rebase -i' for maintaining
24 your patch queue? The advantage of these tools is their simplicity;
25 they work with patch _series_ and defer to the reflog facility for
26 version control of patches (reordering of patches is not
27 version-controlled at all). But there are several disadvantages - for
28 one, these tools (especially StGIT) do not actually fit well with plain
29 Git at all: it is basically impossible to take advantage of the index
30 effectively when using StGIT. But more importantly, these tools
31 horribly fail in the face of a distributed environment.
33 TopGit has been designed around three main tenets:
35 (i) TopGit is as thin a layer on top of Git as possible. You
36 still maintain your index and commit using Git; TopGit will only
37 automate a few indispensable tasks.
39 (ii) TopGit is anxious about _keeping_ your history. It will
40 never rewrite your history, and all metadata is also tracked by Git,
41 smoothly and non-obnoxiously. It is good to have a _single_ point when
42 the history is cleaned up, and that is at the point of inclusion in the
43 upstream project; locally, you can see how your patch has evolved and
44 easily return to older versions.
46 (iii) TopGit is specifically designed to work in a distributed
47 environment. You can have several instances of TopGit-aware
48 repositories and smoothly keep them all up-to-date and transfer your
51 As mentioned above, the main intended use-case for TopGit is tracking
52 third-party patches, where each patch is effectively a single topic
53 branch. In order to flexibly accommodate even complex scenarios when
54 you track many patches where many are independent but some depend on
55 others, TopGit ignores the ancient Quilt heritage of patch series and
56 instead allows the patches to freely form graphs (DAGs just like Git
57 history itself, only "one level higher"). For now, you have to manually
58 specify which patches the current one depends on, but TopGit might help
59 you with that in the future in a darcs-like fashion.
61 A glossary plug: The union (i.e. merge) of patch dependencies is called
62 a _base_ of the patch (topic branch).
64 Of course, TopGit is perhaps not the right tool for you:
66 (i) TopGit is not complicated, but StGIT et al. are somewhat
67 simpler, conceptually. If you just want to make a linear purely-local
68 patch queue, deferring to StGIT instead might make more sense.
70 (ii) When using TopGit, your history can get a little hairy
71 over time, especially with all the merges rippling through. ;-)
77 ## Create and evolve a topic branch
78 $ tg create t/gitweb/pathinfo-action
79 tg: Automatically marking dependency on master
80 tg: Creating t/gitweb/pathinfo-action base from master...
86 ## Create another topic branch on top of the former one
87 $ tg create t/gitweb/nifty-links
88 tg: Automatically marking dependency on t/gitweb/pathinfo-action
89 tg: Creating t/gitweb/nifty-links base from t/gitweb/pathinfo-action...
93 ## Create another topic branch on top of master and submit
94 ## the resulting patch upstream
95 $ tg create t/revlist/author-fixed master
96 tg: Creating t/revlist/author-fixed base from master...
100 tg: Sent t/revlist/author-fixed
102 To: git@vger.kernel.org
103 Cc: gitster@pobox.com
104 Subject: [PATCH] Fix broken revlist --author when --fixed-string
106 ## Create another topic branch depending on two others non-trivially
107 $ tg create t/whatever t/revlist/author-fixed t/gitweb/nifty-links
108 tg: Creating t/whatever base from t/revlist/author-fixed...
109 tg: Merging t/whatever base with t/gitweb/nifty-links...
111 tg: Please commit merge resolution and call: tg create
112 tg: It is also safe to abort this operation using `git reset --hard`
113 tg: but please remember you are on the base branch now;
114 tg: you will want to switch to a different branch.
118 tg: Resuming t/whatever setup...
122 ## Update a single topic branch and propagate the changes to
124 $ git checkout t/gitweb/nifty-links
127 $ git checkout t/whatever
129 Topic Branch: t/whatever (1 commit)
130 Subject: [PATCH] Whatever patch
132 Depends: t/revlist/author-fixed t/gitweb/nifty-links
134 t/gitweb/nifty-links (1 commit)
136 tg: Updating base with t/gitweb/nifty-links changes...
138 tg: Please commit merge resolution and call `tg update` again.
139 tg: It is also safe to abort this operation using `git reset --hard`,
140 tg: but please remember you are on the base branch now;
141 tg: you will want to switch to a different branch.
145 tg: Updating t/whatever against new base...
147 tg: Please resolve the merge and commit. No need to do anything else.
148 tg: You can abort this operation using `git reset --hard` now
149 tg: and retry this merge later using `tg update`.
153 ## Update a single topic branch and propagate the changes
154 ## further through the dependency chain
155 $ git checkout t/gitweb/pathinfo-action
158 $ git checkout t/whatever
160 Topic Branch: t/whatever (1/2 commits)
161 Subject: [PATCH] Whatever patch
163 Depends: t/revlist/author-fixed t/gitweb/nifty-links
165 t/gitweb/pathinfo-action (<= t/gitweb/nifty-links) (1 commit)
167 tg: Recursing to t/gitweb/nifty-links...
168 [t/gitweb/nifty-links] tg: Updating base with t/gitweb/pathinfo-action changes...
170 [t/gitweb/nifty-links] tg: Please commit merge resolution and call `tg update` again.
171 [t/gitweb/nifty-links] tg: It is also safe to abort this operation using `git reset --hard`,
172 [t/gitweb/nifty-links] tg: but please remember you are on the base branch now;
173 [t/gitweb/nifty-links] tg: you will want to switch to a different branch.
174 [t/gitweb/nifty-links] tg: You are in a subshell. If you abort the merge,
175 [t/gitweb/nifty-links] tg: use `exit` to abort the recursive update altogether.
176 [t/gitweb/nifty-links] $ ..resolve..
177 [t/gitweb/nifty-links] $ git commit
178 [t/gitweb/nifty-links] $ tg update
179 [t/gitweb/nifty-links] tg: Updating t/gitweb/nifty-links against new base...
181 [t/gitweb/nifty-links] tg: Please resolve the merge and commit.
182 [t/gitweb/nifty-links] tg: You can abort this operation using `git reset --hard`.
183 [t/gitweb/nifty-links] tg: You are in a subshell. After you either commit or abort
184 [t/gitweb/nifty-links] tg: your merge, use `exit` to proceed with the recursive update.
185 [t/gitweb/nifty-links] $ ..resolve..
186 [t/gitweb/nifty-links] $ git commit
187 [t/gitweb/nifty-links] $ exit
188 tg: Updating base with t/gitweb/nifty-links changes...
189 tg: Updating t/whatever against new base...
191 ## Clone a TopGit-controlled repository
194 $ tg remote --populate origin
199 ## Add a TopGit remote to a repository and push to it
200 $ git remote add foo URL
204 ## Update from a non-default TopGit remote
213 The 'tg' tool has several subcommands:
217 Our sophisticated integrated help facility. Mostly duplicates
218 what is below, except for adding summary Usage lines.
222 # to get help for a particular command:
227 Create a new TopGit-controlled topic branch of the given name
228 (required argument) and switch to it. If no dependencies are
229 specified (by extra arguments passed after the first one), the
230 current branch is assumed to be the only dependency.
232 After `tg create`, you should insert the patch description into
233 the '.topmsg' file, which will already contain some prefilled
234 bits. You can set the 'topgit.to', 'topgit.cc' and 'topgit.bcc'
235 git configuration variables (see `man git-config`) in order to
236 have `tg create` add these headers with the given default values
239 The main task of `tg create` is to set up the topic branch base
240 from the dependencies. This may fail due to merge conflicts.
241 In that case, after you commit the conflict resolution, you
242 should call `tg create` again (without any arguments); it will
243 detect that you are on a topic branch base ref and resume the
244 topic branch creation operation.
246 In an alternative use case, if '-r BRANCH' is given instead of a
247 dependency list, the topic branch is created based on the given
252 Remove a TopGit-controlled topic branch of the given name
253 (required argument). Normally, this command will remove only an
254 empty branch (base == head) without dependendents; use '-f' to
255 remove a non-empty branch or a branch that is depended upon by
258 The '-f' option is also useful to force removal of a branch's
259 base, if you used `git branch -D B` to remove branch B, and then
260 certain TopGit commands complain, because the base of branch B
263 IMPORTANT: Currently, this command will _NOT_ remove the branch
264 from the dependency list in other branches. You need to take
265 care of this _manually_. This is even more complicated in
266 combination with '-f' - in that case, you need to manually
267 unmerge the removed branch's changes from the branches depending
270 TODO: '-a' to delete all empty branches, depfix, revert
274 Change the dependencies of a TopGit-controlled topic branch.
275 This should have several subcommands, but only `add` is
278 The `add` subcommand takes an argument naming a topic branch to
279 be added, adds it to '.topdeps', performs a commit and then
280 updates your topic branch accordingly. If you want to do other
281 things related to the dependency addition, like adjusting
282 '.topmsg', prepare them in the index before calling `tg depend
285 TODO: Subcommand for removing dependencies, obviously
289 List files changed by the current or specified topic branch.
292 -i list files based on index instead of branch
293 -w list files based on working tree instead of branch
297 Show summary information about the current or specified topic
302 Generate a patch from the current or specified topic branch.
303 This means that the diff between the topic branch base and head
304 (latest commit) is shown, appended to the description found in
307 The patch is simply dumped to stdout. In the future, `tg patch`
308 will be able to automatically send the patches by mail or save
309 them to files. (TODO)
312 -i base patch generation on index instead of branch
313 -w base patch generation on working tree instead of branch
317 Send a patch from the current or specified topic branch as
320 Takes the patch given on the command line and emails it out.
321 Destination addresses such as To, Cc and Bcc are taken from the
324 Since it actually boils down to `git send-email`, please refer
325 to the documentation for that for details on how to setup email
326 for git. You can pass arbitrary options to this command through
327 the '-s' parameter, but you must double-quote everything. The
328 '-r' parameter with a msgid can be used to generate in-reply-to
329 and reference headers to an earlier mail.
331 WARNING: be careful when using this command. It easily sends
332 out several mails. You might want to run
334 git config sendemail.confirm always
336 to let `git send-email` ask for confirmation before sending any
340 -i base patch generation on index instead of branch
341 -w base patch generation on working tree instead of branch
343 TODO: 'tg mail patchfile' to mail an already exported patch
344 TODO: mailing patch series
345 TODO: specifying additional options and addresses on command
350 Register the given remote as TopGit-controlled. This will create
351 the namespace for the remote branch bases and teach `git fetch`
352 to operate on them. However, from TopGit 0.8 onwards you need to
353 use `tg push`, or `git push --mirror`, for pushing
354 TopGit-controlled branches.
356 `tg remote` takes an optional remote name argument, and an
357 optional '--populate' switch. Use '--populate' for your
358 origin-style remotes: it will seed the local topic branch system
359 based on the remote topic branches. '--populate' will also make
360 `tg remote` automatically fetch the remote, and `tg update` look
361 at branches of this remote for updates by default.
365 Show overview of all TopGit-tracked topic branches and their
366 up-to-date status ('>' marks the current topic branch,
367 '0' indicates that it introduces no changes of its own,
368 'l'/'r' indicates respectively whether it is local-only or has a remote mate,
369 'L'/'R' indicates respectively if it is ahead or out-of-date with respect to its remote mate,
370 'D' indicates that it is out-of-date with respect to its dependencies,
371 '!' indicates that it has missing dependencies [even if they are recursive ones], and
372 'B' indicates that it is out-of-date with respect to its base).
374 This can take a long time to accurately determine all the
375 relevant information about each branch; you can pass '-t' to get
376 just a terse list of topic branch names quickly. Alternately,
377 you can pass '--graphviz' to get a dot-suitable output to draw a
378 dependency graph between the topic branches.
380 You can also use the '--sort' option to sort the branches using
381 a topological sort. This is especially useful if each
382 TopGit-tracked topic branch depends on a single parent branch,
383 since it will then print the branches in the dependency order.
384 In more complex scenarios, a text graph view would be much more
385 useful, but that has not yet been implemented.
387 The --deps option outputs dependency information between
388 branches in a machine-readable format. Feed this to `tsort` to
389 get the output from --sort.
392 -i Use TopGit metadata from the index instead of the branch
393 -w Use TopGit metadata from the working tree instead of the branch
395 TODO: Speed up by an order of magnitude
396 TODO: Text graph view
400 Export a tidied-up history of the current topic branch and its
401 dependencies, suitable for feeding upstream. Each topic branch
402 corresponds to a single commit or patch in the cleaned up
403 history (corresponding basically exactly to `tg patch` output
404 for the topic branch).
406 The command has three possible outputs now - either a Git branch
407 with the collapsed history, a Git branch with a linearized
408 history, or a quilt series in new directory.
410 In the case where you are producing collapsed history in a new
411 branch, you can use this collapsed structure either for
412 providing a pull source for upstream, or for further
413 linearization e.g. for creation of a quilt series using git log:
415 git log --pretty=email -p --topo-order origin..exported
417 To better understand the function of `tg export`, consider this
418 dependency structure:
420 origin/master - t/foo/blue - t/foo/red - master
421 `- t/bar/good <,----------'
422 `- t/baz ------------'
424 (where each of the branches may have a hefty history). Then
426 master$ tg export for-linus
428 will create this commit structure on the branch 'for-linus':
430 origin/master - t/foo/blue -. merge - t/foo/red -.. merge - master
431 `- t/bar/good <,-------------------'/
432 `- t/baz ---------------------'
434 In this mode, `tg export` works on the current topic branch, and
435 can be called either without an option (in that case,
436 '--collapse' is assumed), or with the '--collapse' option, and
437 with one mandatory argument: the name of the branch where the
438 exported result will be stored.
440 WARNING: The branch will be silently overwritten if it exists
441 already! If you make a mistake, use `git reflog` to recover.
443 When using the linearize mode:
445 master$ tg export --linearize for-linus
447 you get a linear history respecting the dependencies of your
448 patches in a new branch 'for-linus'. The result should be more
449 or less the same as using quilt mode and then reimporting it
450 into a Git branch. (More or less because the topological order
451 can usually be extended in more than one way into a total order,
452 and the two methods may choose different ones.) The result
453 might be more appropriate for merging upstream, as it contains
456 Note that you might get conflicts during linearization because
457 the patches are reordered to get a linear history.
459 When using the quilt mode,
461 master$ tg export --quilt for-linus
463 would create the following directory 'for-linus':
465 for-linus/t/foo/blue.diff
466 for-linus/t/foo/red.diff
467 for-linus/t/bar/good.diff
475 With '--quilt', you can also pass '-b' parameter followed by
476 a comma-separated explicit list of branches to export. This
477 mode of operation is currently not supported with '--collapse'.
479 In '--quilt' mode the patches are named like the originating
480 topgit branch. So usually they end up in subdirectories of the
481 output directory. With the '--flatten' option the names are
482 mangled so that they end up directly in the output dir (slashes
483 are substituted by underscores). With the '--strip[=N]' option
484 the first 'N' subdirectories (all if no 'N' is given) get
485 stripped off. Names are always '--strip'ped before being
486 '--flatten'ed. With the option '--numbered' (which implies
487 '--flatten') the patch names get a number as prefix to allow
488 getting the order without consulting the series file, which
489 eases sending out the patches.
491 TODO: Make stripping of non-essential headers configurable
492 TODO: Make stripping of [PATCH] and other prefixes configurable
493 TODO: --mbox option for other mode of operation
494 TODO: -a option to export all branches
495 TODO: For quilt exporting, export the linearized history created in a
496 temporary branch---this would allow producing conflict-less
501 Import commits within the given revision range into TopGit,
502 creating one topic branch per commit. The dependencies are set
503 up to form a linear sequence starting on your current branch -
504 or a branch specified by the '-d' parameter, if present.
506 The branch names are auto-guessed from the commit messages and
507 prefixed by 't/' by default; use '-p <prefix>' to specify an
508 alternative prefix (even an empty one).
510 Alternatively, you can use the '-s NAME' parameter to specify
511 the name of the target branch; the command will then take one
512 more argument describing a _single_ commit to import.
516 Update the current, specified or all topic branches with respect
517 to changes in the branches they depend on and remote branches.
518 This is performed in two phases - first, changes within the
519 dependencies are merged to the base, then the base is merged
520 into the topic branch. The output will guide you on what to do
521 next in case of conflicts.
523 When '-a' is specifed, updates all topic branches matched by
524 <pattern>s (see `git-for-all-refs(1)` for details), or all if
525 no <pattern> is given.
527 After the update, if a single topic branch was specified, it is
528 left as the current one; if '-a' was specified, it returns to
529 the branch which was current at the beginning.
531 If your dependencies are not up-to-date, `tg update` will first
532 recurse into them and update them.
534 If a remote branch update brings in dependencies on branches
535 that are not yet instantiated locally, you can either bring in
536 all the new branches from the remote using `tg remote
537 --populate`, or only pick out the missing ones using `tg create
538 -r` (`tg summary` will point out branches with incomplete
539 dependencies by showing an '!' next to them).
541 TODO: tg update -a -c to autoremove (clean) up-to-date branches
545 pushes a TopGit-controlled topic branch - the current branch, if
546 you don't specify which - to a remote repository. By default,
547 the remote gets all the dependencies (both TopGit-controlled and
548 non-TopGit-controlled) and bases pushed to it too.
552 Prints the base commit of the current topic branch. Silently
553 exits with exit code 1 if you are not working on a TopGit
558 Prints the git log of the named topgit branch - or the current
559 branch, if you don't specify a name.
561 NOTE: if you have merged changes from a different repository, this
562 command might not list all interesting commits.
566 Outputs the direct dependencies for the current or named branch.
569 -i show dependencies based on index instead of branch
570 -w show dependencies based on working tree instead of branch
574 Outputs all branches which directly depend on the current or
578 -i show dependencies based on index instead of branch
579 -w show dependencies based on working tree instead of branch
586 TopGit stores all the topic branches in the regular 'refs/heads/'
587 namespace (so we recommend distinguishing them with the 't/' prefix).
588 Apart from that, TopGit also maintains a set of auxiliary refs in
589 'refs/top-*'. Currently, only refs/top-bases/ is used, containing the
590 current _base_ of the given topic branch - this is basically a merge of
591 all the branches the topic branch depends on; it is updated during `tg
592 update` and then merged to the topic branch, and it is the base of a
593 patch generated from the topic branch by `tg patch`.
595 All the metadata is tracked within the source tree and history of the
596 topic branch itself, in .top* files; these files are kept isolated
597 within the topic branches during TopGit-controlled merges and are of
598 course omitted during `tg patch`. The state of these files in base
599 commits is undefined; look at them only in the topic branches
600 themselves. Currently, two files are defined:
602 '.topmsg': Contains the description of the topic branch in a
603 mail-like format, plus the author information, whatever Cc headers you
604 choose or the post-three-dashes message. When mailing out your patch,
605 basically only a few extra mail headers are inserted and then the patch
606 itself is appended. Thus, as your patches evolve, you can record
607 nuances like whether the particular patch should have To-list /
608 Cc-maintainer or vice-versa and similar nuances, if your project is into
609 that. 'From' is prefilled from your current `GIT_AUTHOR_IDENT`; other
610 headers can be prefilled from various optional 'topgit.*' git config
613 '.topdeps': Contains the one-per-line list of branches this
614 branch depends on, pre-seeded by `tg create`. A (continuously updated)
615 merge of these branches will be the "base" of your topic branch.
617 IMPORTANT: DO NOT EDIT '.topdeps' MANUALLY!!! If you do so, you need to
618 know exactly what are you doing, since this file must stay in sync with
619 the Git history information, otherwise very bad things will happen.
621 TopGit also automagically installs a bunch of custom commit-related
622 hooks that will verify whether you are committing the '.top*' files in a
623 sane state. It will add the hooks to separate files within the 'hooks/'
624 subdirectory, and merely insert calls to them to the appropriate hooks
625 and make them executable (but will make sure the original hook's code is
626 not called if the hook was not executable beforehand).
628 Another automagically installed piece is a '.git/info/attributes'
629 specifier for an 'ours' merge strategy for the files '.topmsg' and
630 '.topdeps', and the (intuitive) 'ours' merge strategy definition in
637 There are two remaining issues with accessing topic branches in remote
640 (i) Referring to remote topic branches from your local repository
641 (ii) Developing some of the remote topic branches locally
643 There are two somewhat contradictory design considerations here:
645 (a) Hacking on multiple independent TopGit remotes in a single
647 (b) Having a self-contained topic system in local refs space
649 To us, (a) does not appear to be very convincing, while (b) is quite
650 desirable for 'git-log topic' etc. working, and increased conceptual
653 Thus, we choose to instantiate all the topic branches of given remote
654 locally; this is performed by `tg remote --populate`. `tg update` will
655 also check if a branch can be updated from its corresponding remote
656 branch. The logic needs to be somewhat involved if we are to "do the
657 right thing". First, we update the base, handling the remote branch as
658 if it was the first dependency; thus, conflict resolutions made in the
659 remote branch will be carried over to our local base automagically.
660 Then, the base is merged into the remote branch and the result is merged
661 to the local branch - again, to carry over remote conflict resolutions.
662 In the future, this order might be adjustable on a per-update basis, in
663 case local changes happen to be diverging more than the remote ones.
665 All commands by default refer to the remote that `tg remote --populate`
666 was called on the last time (stored in the 'topgit.remote' git
667 configuration variable). You can manually run any command with a
668 different base remote by passing '-r REMOTE' _before_ the subcommand
675 The following references are useful to understand the development of
676 topgit and its subcommands.
679 http://lists-archives.org/git/688698-add-list-and-rm-sub-commands-to-tg-depend.html