Merge branch 'nd/test-helpers'

[git/debian.git] / Documentation / RelNotes / 2.11.0.txt

Git 2.11 Release Notes
======================

Backward compatibility notes.

 * An empty string used as a pathspec element has always meant
   'everything matches', but it is too easy to write a script that
   finds a path to remove in $path and run 'git rm "$paht"', which
   ends up removing everything.  This release starts warning about the
   use of an empty string that is used for 'everything matches' and
   asks users to use a more explicit '.' for that instead.

   The hope is that existing users will not mind this change, and
   eventually the warning can be turned into a hard error, upgrading
   the deprecation into removal of this (mis)feature.

 * The historical argument order "git merge <msg> HEAD <commit>..."
   has been deprecated for quite some time, and will be removed in the
   next release (not this one).

 * The default abbreviation length, which has historically been 7, now
   scales as the repository grows, using the approximate number of
   objects in the reopsitory and a bit of math around the birthday
   paradox.  The logic suggests to use 12 hexdigits for the Linux
   kernel, and 9 to 10 for Git itself.


Updates since v2.10
-------------------

UI, Workflows & Features

 * Comes with new version of git-gui, now at its 0.21.0 tag.

 * "git format-patch --cover-letter HEAD^" to format a single patch
   with a separate cover letter now numbers the output as [PATCH 0/1]
   and [PATCH 1/1] by default.

 * An incoming "git push" that attempts to push too many bytes can now
   be rejected by setting a new configuration variable at the receiving
   end.

 * "git nosuchcommand --help" said "No manual entry for gitnosuchcommand",
   which was not intuitive, given that "git nosuchcommand" said "git:
   'nosuchcommand' is not a git command".

 * "git clone --resurse-submodules --reference $path $URL" is a way to
   reduce network transfer cost by borrowing objects in an existing
   $path repository when cloning the superproject from $URL; it
   learned to also peek into $path for presense of corresponding
   repositories of submodules and borrow objects from there when able.

 * The "git diff --submodule={short,log}" mechanism has been enhanced
   to allow "--submodule=diff" to show the patch between the submodule
   commits bound to the superproject.

 * Even though "git hash-objects", which is a tool to take an
   on-filesystem data stream and put it into the Git object store,
   allowed to perform the "outside-world-to-Git" conversions (e.g.
   end-of-line conversions and application of the clean-filter), and
   it had the feature on by default from very early days, its reverse
   operation "git cat-file", which takes an object from the Git object
   store and externalize for the consumption by the outside world,
   lacked an equivalent mechanism to run the "Git-to-outside-world"
   conversion.  The command learned the "--filters" option to do so.

 * Output from "git diff" can be made easier to read by selecting
   which lines are common and which lines are added/deleted
   intelligently when the lines before and after the changed section
   are the same.  A command line option is added to help with the
   experiment to find a good heuristics.

 * In some projects, it is common to use "[RFC PATCH]" as the subject
   prefix for a patch meant for discussion rather than application.  A
   new option "--rfc" was a short-hand for "--subject-prefix=RFC PATCH"
   to help the participants of such projects.

 * "git add --chmod=+x <pathspec>" added recently only toggled the
   executable bit for paths that are either new or modified. This has
   been corrected to flip the executable bit for all paths that match
   the given pathspec.

 * When "git format-patch --stdout" output is placed as an in-body
   header and it uses the RFC2822 header folding, "git am" failed to
   put the header line back into a single logical line.  The
   underlying "git mailinfo" was taught to handle this properly.

 * "gitweb" can spawn "highlight" to show blob contents with
   (programming) language-specific syntax highlighting, but only
   when the language is known.  "highlight" can however be told
   to make the guess itself by giving it "--force" option, which
   has been enabled.

 * "git gui" l10n to Portuguese.

 * When given an abbreviated object name that is not (or more
   realistically, "no longer") unique, we gave a fatal error
   "ambiguous argument".  This error is now accompanied by hints that
   lists the objects that begins with the given prefix.  During the
   course of development of this new feature, numerous minor bugs were
   uncovered and corrected, the most notable one of which is that we
   gave "short SHA1 xxxx is ambiguous." twice without good reason.

 * "git log rev^..rev" is an often-used revision range specification
   to show what was done on a side branch merged at rev.  This has
   gained a short-hand "rev^-1".  In general "rev^-$n" is the same as
   "^rev^$n rev", i.e. what has happened on other branches while the
   history leading to nth parent was looking the other way.

 * In recent versions of cURL, GSSAPI credential delegation is
   disabled by default due to CVE-2011-2192; introduce a configuration
   to selectively allow enabling this.
   (merge 26a7b23429 ps/http-gssapi-cred-delegation later to maint).

 * "git mergetool" learned to honor "-O<orderfile>" to control the
   order of paths to present to the end user.

 * "git diff/log --ws-error-highlight=<kind>" lacked the corresponding
   configuration variable to set it by default.

 * "git ls-files" learned "--recurse-submodules" option that can be
   used to get a listing of tracked files across submodules (i.e. this
   only works with "--cached" option, not for listing untracked or
   ignored files).  This would be a useful tool to sit on the upstream
   side of a pipe that is read with xargs to work on all working tree
   files from the top-level superproject.

 * A new credential helper that talks via "libsecret" with
   implementations of XDG Secret Service API has been added to
   contrib/credential/.

 * The GPG verification status shown in "%G?" pretty format specifier
   was not rich enough to differentiate a signature made by an expired
   key, a signature made by a revoked key, etc.  New output letters
   have been assigned to express them.

 * In addition to purely abbreviated commit object names, "gitweb"
   learned to turn "git describe" output (e.g. v2.9.3-599-g2376d31787)
   into clickable links in its output.

 * When new paths were added by "git add -N" to the index, it was
   enough to circumvent the check by "git commit" to refrain from
   making an empty commit without "--allow-empty".  The same logic
   prevented "git status" to show such a path as "new file" in the
   "Changes not staged for commit" section.


Performance, Internal Implementation, Development Support etc.

 * The delta-base-cache mechanism has been a key to the performance in
   a repository with a tightly packed packfile, but it did not scale
   well even with a larger value of core.deltaBaseCacheLimit.

 * Enhance "git status --porcelain" output by collecting more data on
   the state of the index and the working tree files, which may
   further be used to teach git-prompt (in contrib/) to make fewer
   calls to git.

 * Extract a small helper out of the function that reads the authors
   script file "git am" internally uses.
   (merge a77598e jc/am-read-author-file later to maint).

 * Lifts calls to exit(2) and die() higher in the callchain in
   sequencer.c files so that more helper functions in it can be used
   by callers that want to handle error conditions themselves.

 * "git am" has been taught to make an internal call to "git apply"'s
   innards without spawning the latter as a separate process.

 * The ref-store abstraction was introduced to the refs API so that we
   can plug in different backends to store references.

 * The "unsigned char sha1[20]" to "struct object_id" conversion
   continues.  Notable changes in this round includes that ce->sha1,
   i.e. the object name recorded in the cache_entry, turns into an
   object_id.

 * JGit can show a fake ref "capabilities^{}" to "git fetch" when it
   does not advertise any refs, but "git fetch" was not prepared to
   see such an advertisement.  When the other side disconnects without
   giving any ref advertisement, we used to say "there may not be a
   repository at that URL", but we may have seen other advertisement
   like "shallow" and ".have" in which case we definitely know that a
   repository is there.  The code to detect this case has also been
   updated.

 * Some codepaths in "git pack-objects" were not ready to use an
   existing pack bitmap; now they are and as the result they have
   become faster.

 * The codepath in "git fsck" to detect malformed tree objects has
   been updated not to die but keep going after detecting them.

 * We call "qsort(array, nelem, sizeof(array[0]), fn)", and most of
   the time third parameter is redundant.  A new QSORT() macro lets us
   omit it.

 * "git pack-objects" in a repository with many packfiles used to
   spend a lot of time looking for/at objects in them; the accesses to
   the packfiles are now optimized by checking the most-recently-used
   packfile first.
   (merge c9af708b1a jk/pack-objects-optim-mru later to maint).

 * Codepaths involved in interacting alternate object store have
   been cleaned up.

 * In order for the receiving end of "git push" to inspect the
   received history and decide to reject the push, the objects sent
   from the sending end need to be made available to the hook and
   the mechanism for the connectivity check, and this was done
   traditionally by storing the objects in the receiving repository
   and letting "git gc" to expire it.  Instead, store the newly
   received objects in a temporary area, and make them available by
   reusing the alternate object store mechanism to them only while we
   decide if we accept the check, and once we decide, either migrate
   them to the repository or purge them immediately.

 * The require_clean_work_tree() helper was recreated in C when "git
   pull" was rewritten from shell; the helper is now made available to
   other callers in preparation for upcoming "rebase -i" work.

 * "git upload-pack" had its code cleaned-up and performance improved
   by reducing use of timestamp-ordered commit-list, which was
   replaced with a priority queue.

 * "git diff --no-index" codepath has been updated not to try to peek
   into .git/ directory that happens to be under the current
   directory, when we know we are operating outside any repository.

 * Update of the sequencer codebase to make it reusable to reimplement
   "rebase -i" continues.


Also contains various documentation updates and code clean-ups.


Fixes since v2.10
-----------------

Unless otherwise noted, all the fixes since v2.9 in the maintenance
track are contained in this release (see the maintenance releases'
notes for details).

 * Clarify various ways to specify the "revision ranges" in the
   documentation.

 * "diff-highlight" script (in contrib/) learned to work better with
   "git log -p --graph" output.

 * The test framework left the number of tests and success/failure
   count in the t/test-results directory, keyed by the name of the
   test script plus the process ID.  The latter however turned out not
   to serve any useful purpose.  The process ID part of the filename
   has been removed.

 * Having a submodule whose ".git" repository is somehow corrupt
   caused a few commands that recurse into submodules loop forever.

 * "git symbolic-ref -d HEAD" happily removes the symbolic ref, but
   the resulting repository becomes an invalid one.  Teach the command
   to forbid removal of HEAD.

 * A test spawned a short-lived background process, which sometimes
   prevented the test directory from getting removed at the end of the
   script on some platforms.

 * Update a few tests that used to use GIT_CURL_VERBOSE to use the
   newer GIT_TRACE_CURL.

 * "git pack-objects --include-tag" was taught that when we know that
   we are sending an object C, we want a tag B that directly points at
   C but also a tag A that points at the tag B.  We used to miss the
   intermediate tag B in some cases.

 * Update Japanese translation for "git-gui".

 * "git fetch http::/site/path" did not die correctly and segfaulted
   instead.

 * "git commit-tree" stopped reading commit.gpgsign configuration
   variable that was meant for Porcelain "git commit" in Git 2.9; we
   forgot to update "git gui" to look at the configuration to match
   this change.

 * "git add --chmod=+x" added recently lacked documentation, which has
   been corrected.

 * "git log --cherry-pick" used to include merge commits as candidates
   to be matched up with other commits, resulting a lot of wasted time.
   The patch-id generation logic has been updated to ignore merges to
   avoid the wastage.

 * The http transport (with curl-multi option, which is the default
   these days) failed to remove curl-easy handle from a curlm session,
   which led to unnecessary API failures.

 * There were numerous corner cases in which the configuration files
   are read and used or not read at all depending on the directory a
   Git command was run, leading to inconsistent behaviour.  The code
   to set-up repository access at the beginning of a Git process has
   been updated to fix them.
   (merge 4d0efa1 jk/setup-sequence-update later to maint).

 * "git diff -W" output needs to extend the context backward to
   include the header line of the current function and also forward to
   include the body of the entire current function up to the header
   line of the next one.  This process may have to merge to adjacent
   hunks, but the code forgot to do so in some cases.

 * Performance tests done via "t/perf" did not use the same set of
   build configuration if the user relied on autoconf generated
   configuration.

 * "git format-patch --base=..." feature that was recently added
   showed the base commit information after "-- " e-mail signature
   line, which turned out to be inconvenient.  The base information
   has been moved above the signature line.

 * More i18n.

 * Even when "git pull --rebase=preserve" (and the underlying "git
   rebase --preserve") can complete without creating any new commit
   (i.e. fast-forwards), it still insisted on having a usable ident
   information (read: user.email is set correctly), which was less
   than nice.  As the underlying commands used inside "git rebase"
   would fail with a more meaningful error message and advice text
   when the bogus ident matters, this extra check was removed.

 * "git gc --aggressive" used to limit the delta-chain length to 250,
   which is way too deep for gaining additional space savings and is
   detrimental for runtime performance.  The limit has been reduced to
   50.

 * Documentation for individual configuration variables to control use
   of color (like `color.grep`) said that their default value is
   'false', instead of saying their default is taken from `color.ui`.
   When we updated the default value for color.ui from 'false' to
   'auto' quite a while ago, all of them broke.  This has been
   corrected.

 * The pretty-format specifier "%C(auto)" used by the "log" family of
   commands to enable coloring of the output is taught to also issue a
   color-reset sequence to the output.
   (merge 82b83da8d3 rs/c-auto-resets-attributes later to maint).

 * A shell script example in check-ref-format documentation has been
   fixed.

 * "git checkout <word>" does not follow the usual disambiguation
   rules when the <word> can be both a rev and a path, to allow
   checking out a branch 'foo' in a project that happens to have a
   file 'foo' in the working tree without having to disambiguate.
   This was poorly documented and the check was incorrect when the
   command was run from a subdirectory.

 * Some codepaths in "git diff" used regexec(3) on a buffer that was
   mmap(2)ed, which may not have a terminating NUL, leading to a read
   beyond the end of the mapped region.  This was fixed by introducing
   a regexec_buf() helper that takes a <ptr,len> pair with REG_STARTEND
   extension.
   (merge 842a516cb0 js/regexec-buf later to maint).

 * The procedure to build Git on Mac OS X for Travis CI hardcoded the
   internal directory structure we assumed HomeBrew uses, which was a
   no-no.  The procedure has been updated to ask HomeBrew things we
   need to know to fix this.

 * When "git rebase -i" is given a broken instruction, it told the
   user to fix it with "--edit-todo", but didn't say what the step
   after that was (i.e. "--continue").

 * Documentation around tools to import from CVS was fairly outdated.

 * "git clone --recurse-submodules" lost the progress eye-candy in
   recent update, which has been corrected.

 * A low-level function verify_packfile() was meant to show errors
   that were detected without dying itself, but under some conditions
   it didn't and died instead, which has been fixed.

 * When "git fetch" tries to find where the history of the repository
   it runs in has diverged from what the other side has, it has a
   mechanism to avoid digging too deep into irrelevant side branches.
   This however did not work well over the "smart-http" transport due
   to a design bug, which has been fixed.
   (merge 06b3d386e0 jt/fetch-pack-in-vain-count-with-stateless later to maint).

 * In the codepath that comes up with the hostname to be used in an
   e-mail when the user didn't tell us, we looked at ai_canonname
   field in struct addrinfo without making sure it is not NULL first.

 * "git worktree", even though it used the default_abbrev setting that
   ought to be affected by core.abbrev configuration variable, ignored
   the variable setting.  The command has been taught to read the
   default set of configuration variables to correct this.

 * "git init" tried to record core.worktree in the repository's
   'config' file when GIT_WORK_TREE environment variable was set and
   it was different from where GIT_DIR appears as ".git" at its top,
   but the logic was faulty when .git is a "gitdir:" file that points
   at the real place, causing trouble in working trees that are
   managed by "git worktree".  This has been corrected.

 * Codepaths that read from an on-disk loose object were too loose in
   validating what they are reading is a proper object file and
   sometimes read past the data they read from the disk, which has
   been corrected.  H/t to Gustavo Grieco for reporting.

 * The original command line syntax for "git merge", which was "git
   merge <msg> HEAD <parent>...", has been deprecated for quite some
   time, and "git gui" was the last in-tree user of the syntax.  This
   is finally fixed, so that we can move forward with the deprecation.

 * An author name, that spelled a backslash-quoted double quote in the
   human readable part "My \"double quoted\" name", was not unquoted
   correctly while applying a patch from a piece of e-mail.

 * Doc update to clarify what "log -3 --reverse" does.

 * Almost everybody uses DEFAULT_ABBREV to refer to the default
   setting for the abbreviation, but "git blame" peeked into
   underlying variable bypassing the macro for no good reason.

 * The "graph" API used in "git log --graph" miscounted the number of
   output columns consumed so far when drawing a padding line, which
   has been fixed; this did not affect any existing code as nobody
   tried to write anything after the padding on such a line, though.

 * The code that parses the format parameter of for-each-ref command
   has seen a micro-optimization.

 * When we started cURL to talk to imap server when a new enough
   version of cURL library is available, we forgot to explicitly add
   imap(s):// before the destination.  To some folks, that didn't work
   and the library tried to make HTTP(s) requests instead.
   (merge d2d07ab861 ak/curl-imap-send-explicit-scheme later to maint).

 * The ./configure script generated from configure.ac was taught how
   to detect support of SSL by libcurl better.
   (merge 924b7eb1c9 dp/autoconf-curl-ssl later to maint).

 * The command-line completion script (in contrib/) learned to
   complete "git cmd ^mas<HT>" to complete the negative end of
   reference to "git cmd ^master".
   (merge 49416ad22a cp/completion-negative-refs later to maint).

 * The existing "git fetch --depth=<n>" option was hard to use
   correctly when making the history of an existing shallow clone
   deeper.  A new option, "--deepen=<n>", has been added to make this
   easier to use.  "git clone" also learned "--shallow-since=<date>"
   and "--shallow-exclude=<tag>" options to make it easier to specify
   "I am interested only in the recent N months worth of history" and
   "Give me only the history since that version".
   (merge cccf74e2da nd/shallow-deepen later to maint).

 * It is a common mistake to say "git blame --reverse OLD path",
   expecting that the command line is dwimmed as if asking how lines
   in path in an old revision OLD have survived up to the current
   commit.
   (merge e1d09701a4 jc/blame-reverse later to maint).

 * http.emptyauth configuration is a way to allow an empty username to
   pass when attempting to authenticate using mechanisms like
   Kerberos.  We took an unspecified (NULL) username and sent ":"
   (i.e. no username, no password) to CURLOPT_USERPWD, but did not do
   the same when the username is explicitly set to an empty string.
   (merge 5275c3081c dt/http-empty-auth later to maint).

 * "git clone" of a local repository can be done at the filesystem
   level, but the codepath did not check errors while copying and
   adjusting the file that lists alternate object stores.
   (merge 22d3b8de1b jk/clone-copy-alternates-fix later to maint).

 * Documentation for "git commit" was updated to clarify that "commit
   -p <paths>" adds to the current contents of the index to come up
   with what to commit.
   (merge 7431596ab1 nd/commit-p-doc later to maint).

 * A stray symbolic link in $GIT_DIR/refs/ directory could make name
   resolution loop forever, which has been corrected.
   (merge e8c42cb9ce jk/ref-symlink-loop later to maint).

 * The "submodule.<name>.path" stored in .gitmodules is never copied
   to .git/config and such a key in .git/config has no meaning, but
   the documentation described it and submodule.<name>.url next to
   each other as if both belong to .git/config.  This has been fixed.
   (merge 72710165c9 sb/submodule-config-doc-drop-path later to maint).

 * In a worktree connected to a repository elsewhere, created via "git
   worktree", "git checkout" attempts to protect users from confusion
   by refusing to check out a branch that is already checked out in
   another worktree.  However, this also prevented checking out a
   branch, which is designated as the primary branch of a bare
   reopsitory, in a worktree that is connected to the bare
   repository.  The check has been corrected to allow it.
   (merge 171c646f8c dk/worktree-dup-checkout-with-bare-is-ok later to maint).

 * "git rebase" immediately after "git clone" failed to find the fork
   point from the upstream.
   (merge 4f21454b55 jk/merge-base-fork-point-without-reflog later to maint).

 * When fetching from a remote that has many tags that are irrelevant
   to branches we are following, we used to waste way too many cycles
   when checking if the object pointed at by a tag (that we are not
   going to fetch!) exists in our repository too carefully.
   (merge 5827a03545 jk/fetch-quick-tag-following later to maint).

 * Protect our code from over-eager compilers.
   (merge 0ac52a38e8 jk/tighten-alloc later to maint).

 * Recent git allows submodule.<name>.branch to use a special token
   "." instead of the branch name; the documentation has been updated
   to describe it.
   (merge 15ef78008a bw/submodule-branch-dot-doc later to maint).

 * A hot-fix for a test added by a recent topic that went to both
   'master' and 'maint' already.
   (merge 76e368c378 tg/add-chmod+x-fix later to maint).

 * "git send-email" attempts to pick up valid e-mails from the
   trailers, but people in real world write non-addresses there, like
   "Cc: Stable <add@re.ss> # 4.8+", which broke the output depending
   on the availability and vintage of Mail::Address perl module.
   (merge dcfafc5214 mm/send-email-cc-cruft-after-address later to maint).

 * The Travis CI configuration we ship ran the tests with --verbose
   option but this risks non-TAP output that happens to be "ok" to be
   misinterpreted as TAP signalling a test that passed.  This resulted
   in unnecessary failure.  This has been corrected by introducing a
   new mode to run our tests in the test harness to send the verbose
   output separately to the log file.
   (merge 614fe01521 jk/tap-verbose-fix later to maint).

 * Some AsciiDoc formatter mishandles a displayed illustration with
   tabs in it.  Adjust a few of them in merge-base documentation to
   work around them.
   (merge 6750f62699 po/fix-doc-merge-base-illustration later to maint).

 * A minor regression fix for "git submodule" that was introduced
   when more helper functions were reimplemented in C.
   (merge 77b63ac31e sb/submodule-ignore-trailing-slash later to maint).

 * The code that we have used for the past 10+ years to cycle
   4-element ring buffers turns out to be not quite portable in
   theoretical world.
   (merge bb84735c80 rs/ring-buffer-wraparound later to maint).

 * "git daemon" used fixed-length buffers to turn URL to the
   repository the client asked for into the server side directory
   path, using snprintf() to avoid overflowing these buffers, but
   allowed possibly truncated paths to the directory.  This has been
   tightened to reject such a request that causes overlong path to be
   required to serve.
   (merge 6bdb0083be jk/daemon-path-ok-check-truncation later to maint).

 * Other minor doc, test and build updates and code cleanups.
   (merge a94bb68397 rs/cocci later to maint).
   (merge 641c900b2c js/reset-usage later to maint).
   (merge 30cfe72d37 rs/pretty-format-color-doc-fix later to maint).
   (merge d709f1fb9d jc/diff-unique-abbrev-comments later to maint).
   (merge 13092a916d jc/cocci-xstrdup-or-null later to maint).
   (merge 86009f32bb pb/test-parse-options-expect later to maint).
   (merge 749a2279a4 yk/git-tag-remove-mention-of-old-layout-in-doc later to maint).