1 hg-fast-export.(sh|py) - mercurial to git converter using git-fast-import
2 =========================================================================
7 Most hg-* scripts are licensed under the [MIT license] and were written
8 by Rocco Rutte <pdmef@gmx.net> with hints and help from the git list and
9 \#mercurial on freenode. hg-reset.py is licensed under GPLv2 since it
10 copies some code from the mercurial sources.
12 The current maintainer is Frej Drejhammar <frej.drejhammar@gmail.com>.
14 [MIT license]: http://www.opensource.org/licenses/mit-license.php
19 If you have problems with hg-fast-export or have found a bug, please
20 create an issue at the [github issue tracker]. Before creating a new
21 issue, check that your problem has not already been addressed in an
22 already closed issue. Do not contact the maintainer directly unless
23 you want to report a security bug. That way the next person having the
24 same problem can benefit from the time spent solving the problem the
27 [github issue tracker]: https://github.com/frej/fast-export/issues
32 This project depends on Python 2.7 or 3.5+, and the Mercurial >= 4.6
33 package (>= 5.2, if Python 3.5+). If Python is not installed, install
34 it before proceeding. TheMercurial package can be installed with
35 `pip install mercurial`.
37 On windows the bash that comes with "Git for Windows" is known to work
43 Using hg-fast-export is quite simple for a mercurial repository <repo>:
46 mkdir repo-git # or whatever
49 hg-fast-export.sh -r <local-repo>
53 Please note that hg-fast-export does not automatically check out the
54 newly imported repository. You probably want to follow up the import
55 with a `git checkout`-command.
57 Incremental imports to track hg repos is supported, too.
59 Using hg-reset it is quite simple within a git repository that is
60 hg-fast-export'ed from mercurial:
63 hg-reset.sh -R <revision>
66 will give hints on which branches need adjustment for starting over
69 When a mercurial repository does not use utf-8 for encoding author
70 strings and commit messages the `-e <encoding>` command line option
71 can be used to force fast-export to convert incoming meta data from
72 <encoding> to utf-8. This encoding option is also applied to file names.
74 In some locales Mercurial uses different encodings for commit messages
75 and file names. In that case, you can use `--fe <encoding>` command line
76 option which overrides the -e option for file names.
78 As mercurial appears to be much less picky about the syntax of the
79 author information than git, an author mapping file can be given to
80 hg-fast-export to fix up malformed author strings. The file is
81 specified using the -A option. The file should contain lines of the
82 form `"<key>"="<value>"`. Inside the key and value strings, all escape
83 sequences understood by the python `unicode_escape` encoding are
84 supported; strings are otherwise assumed to be UTF8-encoded.
85 (Versions of fast-export prior to v171002 had a different syntax, the
86 old syntax can be enabled by the flag `--mappings-are-raw`.)
88 The example authors.map below will translate `User
89 <garbage<tab><user@example.com>` to `User <user@example.com>`.
92 -- Start of authors.map --
93 "User <garbage\t<user@example.com>"="User <user@example.com>"
94 -- End of authors.map --
97 If you have many Mercurial repositories, Chris J Billington's
98 [hg-export-tool] allows you to batch convert them.
100 Tag and Branch Naming
101 ---------------------
103 As Git and Mercurial have differ in what is a valid branch and tag
104 name the -B and -T options allow a mapping file to be specified to
105 rename branches and tags (respectively). The syntax of the mapping
106 file is the same as for the author mapping.
108 When the -B and -T flags are used, you will probably want to use the
109 -n flag to disable the built-in (broken in many cases) sanitizing of
110 branch/tag names. In the future -n will become the default, but in
111 order to not break existing incremental conversions, the default
112 remains with the old behavior.
114 By default, the `default` mercurial branch is renamed to the `master`
115 branch on git. If your mercurial repo contains both `default` and
116 `master` branches, you'll need to override this behavior. Use
117 `-M <newName>` to specify what name to give the `default` branch.
122 hg-fast-export supports filtering the content of exported files.
123 The filter is supplied to the --filter-contents option. hg-fast-export
124 runs the filter for each exported file, pipes its content to the filter's
125 standard input, and uses the filter's standard output in place
126 of the file's original content. The prototypical use of this feature
127 is to convert line endings in text files from CRLF to git's preferred LF:
130 -- Start of crlf-filter.sh --
132 # $1 = pathname of exported file relative to the root of the repo
133 # $2 = Mercurial's hash of the file
134 # $3 = "1" if Mercurial reports the file as binary, otherwise "0"
136 if [ "$3" == "1" ]; then cat; else dos2unix; fi
137 -- End of crlf-filter.sh --
144 hg-fast-export supports plugins to manipulate the file data and commit
145 metadata. The plugins are enabled with the --plugin option. The value
146 of said option is a plugin name (by folder in the plugins directory),
147 and optionally, and equals-sign followed by an initialization string.
149 There is a readme accompanying each of the bundled plugins, with a
150 description of the usage. To create a new plugin, one must simply
151 add a new folder under the `plugins` directory, with the name of the
152 new plugin. Inside, there must be an `__init__.py` file, which contains
156 def build_filter(args):
160 def __init__(self, args):
162 #Or don't pass, if you want to do some init code here
165 Beyond the boilerplate initialization, you can see the two different
166 defined filter methods in the [dos2unix](./plugins/dos2unix) and
167 [branch_name_in_commit](./plugins/branch_name_in_commit) plugins.
170 commit_data = {'branch': branch, 'parents': parents, 'author': author, 'desc': desc, 'revision': revision, 'hg_hash': hg_hash, 'committer': 'committer'}
172 def commit_message_filter(self,commit_data):
174 The `commit_message_filter` method is called for each commit, after parsing
175 from hg, but before outputting to git. The dictionary `commit_data` contains the
176 above attributes about the commit, and can be modified by any filter. The
177 values in the dictionary after filters have been run are used to create the git
181 file_data = {'filename':filename,'file_ctx':file_ctx,'d':d}
183 def file_data_filter(self,file_data):
185 The `file_data_filter` method is called for each file within each commit.
186 The dictionary `file_data` contains the above attributes about the file, and
187 can be modified by any filter. `file_ctx` is the filecontext from the
188 mercurial python library. After all filters have been run, the values
189 are used to add the file to the git commit.
193 See README-SUBMODULES.md for how to convert subrepositories into git
199 hg-fast-export supports multiple branches but only named branches with
200 exactly one head each. Otherwise commits to the tip of these heads
201 within the branch will get flattened into merge commits. Chris J
202 Billington's [hg-export-tool] can help you to handle branches with
204 Alternatively, you can use the [head2branch plugin](./plugins/head2branch)
205 to create a new named branch from an unnamed head.
207 hg-fast-export will ignore any files or directories tracked by mercurial
208 called `.git`, and will print a warning if it encounters one. Git cannot
209 track such files or directories. This is not to be confused with submodules,
210 which are described in README-SUBMODULES.md.
212 As each git-fast-import run creates a new pack file, it may be
213 required to repack the repository quite often for incremental imports
214 (especially when importing a small number of changesets per
217 The way the hg API and remote access protocol is designed it is not
218 possible to use hg-fast-export on remote repositories
219 (http/ssh). First clone the repository, then convert it.
224 hg-fast-export.py was designed in a way that doesn't require a 2-pass
225 mechanism or any prior repository analysis: if just feeds what it
226 finds into git-fast-import. This also implies that it heavily relies
227 on strictly linear ordering of changesets from hg, i.e. its
228 append-only storage model so that changesets hg-fast-export already
229 saw never get modified.
234 Please create a pull request at
235 [Github](https://github.com/frej/fast-export/pulls) to submit patches.
237 When submitting a patch make sure the commits in your pull request:
239 * Have good commit messages
241 Please read Chris Beams' blog post [How to Write a Git Commit
242 Message](https://chris.beams.io/posts/git-commit/) on how to write a
243 good commit message. Although the article recommends at most 50
244 characters for the subject, up to 72 characters are frequently
245 accepted for fast-export.
247 * Adhere to good [commit
248 hygiene](http://www.ericbmerritt.com/2011/09/21/commit-hygiene-and-git.html)
250 When developing a pull request for hg-fast-export, base your work on
251 the current `master` branch and rebase your work if it no longer can
252 be merged into the current `master` without conflicts. Never merge
253 `master` into your development branch, rebase if your work needs
254 updates from `master`.
256 When a pull request is modified due to review feedback, please
257 incorporate the changes into the proper commit. A good reference on
258 how to modify history is in the [Pro Git book, Section
259 7.6](https://git-scm.com/book/en/v2/Git-Tools-Rewriting-History).
265 * git fast-import crashes with: `error: cannot lock ref 'refs/heads/...`
267 Branch names in git behave as file names (as they are just files and
268 sub-directories under `refs/heads/`, and a path cannot name both a
269 file and a directory, i.e. the branches `a` and `a/b` can never
270 exist at the same time in a git repo.
272 Use a mapping file to rename the troublesome branch names.
274 * `Branch [<branch-name>] modified outside hg-fast-export` but I have
275 not touched the repo!
277 If you are running fast-export on a case-preserving but
278 case-insensitive file system (Windows and OSX), this will make git
279 treat `A` and `a` as the same branch. The solution is to use a
280 mapping file to rename branches which only differ in case.
282 * My mapping file does not seem to work when I rename the branch `git
283 fast-import` crashes on!
285 fast-export (imperfectly) mangles branch names it thinks won't be
286 valid. The mechanism cannot be removed as it would break already
287 existing incremental imports that expects it. When fast export
288 mangles a name, it prints out a warning of the form `Warning:
289 sanitized branch [<unmangled>] to [<mangled>]`. If `git fast-import`
290 crashes on `<mangled>`, you need to put `<unmangled>` into the
293 * fast-import mangles valid git branch names which I have remapped!
295 Use the `-n` flag to hg-fast-export.sh.
297 * `git status` reports that all files are scheduled for deletion after
298 the initial conversion.
300 By design fast export does not touch your working directory, so to
301 git it looks like you have deleted all files, when in fact they have
302 never been checked out. Just do a checkout of the branch you want.
304 [hg-export-tool]: https://github.com/chrisjbillington/hg-export-tool