Merge branch 'sg/t4051-fix'

[git.git] / Documentation / RelNotes / 2.19.0.txt

Git 2.19 Release Notes
======================

Updates since v2.18
-------------------

UI, Workflows & Features

 * "git diff" compares the index and the working tree.  For paths
   added with intent-to-add bit, the command shows the full contents
   of them as added, but the paths themselves were not marked as new
   files.  They are now shown as new by default.

   "git apply" learned the "--intent-to-add" option so that an
   otherwise working-tree-only application of a patch will add new
   paths to the index marked with the "intent-to-add" bit.

 * "git grep" learned the "--column" option that gives not just the
   line number but the column number of the hit.

 * The "-l" option in "git branch -l" is an unfortunate short-hand for
   "--create-reflog", but many users, both old and new, somehow expect
   it to be something else, perhaps "--list".  This step warns when "-l"
   is used as a short-hand for "--create-reflog" and warns about the
   future repurposing of the it when it is used.

 * The userdiff pattern for .php has been updated.

 * The content-transfer-encoding of the message "git send-email" sends
   out by default was 8bit, which can cause trouble when there is an
   overlong line to bust RFC 5322/2822 limit.  A new option 'auto' to
   automatically switch to quoted-printable when there is such a line
   in the payload has been introduced and is made the default.

 * "git checkout" and "git worktree add" learned to honor
   checkout.defaultRemote when auto-vivifying a local branch out of a
   remote tracking branch in a repository with multiple remotes that
   have tracking branches that share the same names.
   (merge 8d7b558bae ab/checkout-default-remote later to maint).

 * "git grep" learned the "--only-matching" option.

 * "git rebase --rebase-merges" mode now handles octopus merges as
   well.

 * Add a server-side knob to skip commits in exponential/fibbonacci
   stride in an attempt to cover wider swath of history with a smaller
   number of iterations, potentially accepting a larger packfile
   transfer, instead of going back one commit a time during common
   ancestor discovery during the "git fetch" transaction.
   (merge 42cc7485a2 jt/fetch-negotiator-skipping later to maint).

 * A new configuration variable core.usereplacerefs has been added,
   primarily to help server installations that want to ignore the
   replace mechanism altogether.

 * Teach "git tag -s" etc. a few configuration variables (gpg.format
   that can be set to "openpgp" or "x509", and gpg.<format>.program
   that is used to specify what program to use to deal with the format)
   to allow x.509 certs with CMS via "gpgsm" to be used instead of
   openpgp via "gnupg".

 * Many more strings are prepared for l10n.

 * "git p4 submit" learns to ask its own pre-submit hook if it should
   continue with submitting.

 * The test performed at the receiving end of "git push" to prevent
   bad objects from entering repository can be customized via
   receive.fsck.* configuration variables; we now have gained a
   counterpart to do the same on the "git fetch" side, with
   fetch.fsck.* configuration variables.

 * "git pull --rebase=interactive" learned "i" as a short-hand for
   "interactive".

 * "git instaweb" has been adjusted to run better with newer Apache on
   RedHat based distros.

 * "git range-diff" is a reimplementation of "git tbdiff" that lets us
   compare individual patches in two iterations of a topic.

 * The sideband code learned to optionally paint selected keywords at
   the beginning of incoming lines on the receiving end.


Performance, Internal Implementation, Development Support etc.

 * The bulk of "git submodule foreach" has been rewritten in C.

 * The in-core "commit" object had an all-purpose "void *util" field,
   which was tricky to use especially in library-ish part of the
   code.  All of the existing uses of the field has been migrated to a
   more dedicated "commit-slab" mechanism and the field is eliminated.

 * A less often used command "git show-index" has been modernized.
   (merge fb3010c31f jk/show-index later to maint).

 * The conversion to pass "the_repository" and then "a_repository"
   throughout the object access API continues.

 * Continuing with the idea to programatically enumerate various
   pieces of data required for command line completion, teach the
   codebase to report the list of configuration variables
   subcommands care about to help complete them.

 * Separate "rebase -p" codepath out of "rebase -i" implementation to
   slim down the latter and make it easier to manage.

 * Make refspec parsing codepath more robust.

 * Some flaky tests have been fixed.

 * Continuing with the idea to programmatically enumerate various
   pieces of data required for command line completion, the codebase
   has been taught to enumerate options prefixed with "--no-" to
   negate them.

 * Build and test procedure for netrc credential helper (in contrib/)
   has been updated.

 * The conversion to pass "the_repository" and then "a_repository"
   throughout the object access API continues.

 * Remove unused function definitions and declarations from ewah
   bitmap subsystem.

 * Code preparation to make "git p4" closer to be usable with Python 3.

 * Tighten the API to make it harder to misuse in-tree .gitmodules
   file, even though it shares the same syntax with configuration
   files, to read random configuration items from it.

 * "git fast-import" has been updated to avoid attempting to create
   delta against a zero-byte-long string, which is pointless.

 * The codebase has been updated to compile cleanly with -pedantic
   option.
   (merge 2b647a05d7 bb/pedantic later to maint).

 * The character display width table has been updated to match the
   latest Unicode standard.
   (merge 570951eea2 bb/unicode-11-width later to maint).

 * test-lint now looks for broken use of "VAR=VAL shell_func" in test
   scripts.

 * Conversion from uchar[40] to struct object_id continues.

 * Recent "security fix" to pay attention to contents of ".gitmodules"
   while accepting "git push" was a bit overly strict than necessary,
   which has been adjusted.

 * "git fsck" learns to make sure the optional commit-graph file is in
   a sane state.

 * "git diff --color-moved" feature has further been tweaked.

 * Code restructuring and a small fix to transport protocol v2 during
   fetching.

 * Parsing of -L[<N>][,[<M>]] parameters "git blame" and "git log"
   take has been tweaked.

 * lookup_commit_reference() and friends have been updated to find
   in-core object for a specific in-core repository instance.

 * Various glitches in the heuristics of merge-recursive strategy have
   been documented in new tests.

 * "git fetch" learned a new option "--negotiation-tip" to limit the
   set of commits it tells the other end as "have", to reduce wasted
   bandwidth and cycles, which would be helpful when the receiving
   repository has a lot of refs that have little to do with the
   history at the remote it is fetching from.

 * For a large tree, the index needs to hold many cache entries
   allocated on heap.  These cache entries are now allocated out of a
   dedicated memory pool to amortize malloc(3) overhead.

 * Tests to cover various conflicting cases have been added for
   merge-recursive.

 * Tests to cover conflict cases that involve submodules have been
   added for merge-recursive.

 * Look for broken "&&" chains that are hidden in subshell, many of
   which have been found and corrected.

 * The singleton commit-graph in-core instance is made per in-core
   repository instance.

 * "make DEVELOPER=1 DEVOPTS=pedantic" allows developers to compile
   with -pedantic option, which may catch more problematic program
   constructs and potential bugs.

 * Preparatory code to later add json output for telemetry data has
   been added.

 * Update the way we use Coccinelle to find out-of-style code that
   need to be modernised.

 * It is too easy to misuse system API functions such as strcat();
   these selected functions are now forbidden in this codebase and
   will cause a compilation failure.

 * Add a script (in contrib/) to help users of VSCode work better with
   our codebase.

 * The Travis CI scripts were taught to ship back the test data from
   failed tests.
   (merge aea8879a6a sg/travis-retrieve-trash-upon-failure later to maint).

 * The parse-options machinery learned to refrain from enclosing
   placeholder string inside a "<bra" and "ket>" pair automatically
   without PARSE_OPT_LITERAL_ARGHELP.  Existing help text for option
   arguments that are not formatted correctly have been identified and
   fixed.
   (merge 5f0df44cd7 rs/parse-opt-lithelp later to maint).

 * Noiseword "extern" has been removed from function decls in the
   header files.

 * A few atoms like %(objecttype) and %(objectsize) in the format
   specifier of "for-each-ref --format=<format>" can be filled without
   getting the full contents of the object, but just with the object
   header.  These cases have been optimized by calling
   oid_object_info() API (instead of reading and inspecting the data).

 * The end result of documentation update has been made to be
   inspected more easily to help developers.

 * The API to iterate over all objects learned to optionally list
   objects in the order they appear in packfiles, which helps locality
   of access if the caller accesses these objects while as objects are
   enumerated.

 * Improve built-in facility to catch broken &&-chain in the tests.

 * The more library-ish parts of the codebase learned to work on the
   in-core index-state instance that is passed in by their callers,
   instead of always working on the singleton "the_index" instance.

 * A test prerequisite defined by various test scripts with slightly
   different semantics has been consolidated into a single copy and
   made into a lazily defined one.
   (merge 6ec633059a wc/make-funnynames-shared-lazy-prereq later to maint).

 * After a partial clone, repeated fetches from promisor remote would
   have accumulated many packfiles marked with .promisor bit without
   getting them coalesced into fewer packfiles, hurting performance.
   "git repack" now learned to repack them.


Fixes since v2.18
-----------------

 * "git remote update" can take both a single remote nickname and a
   nickname for remote groups, and the completion script (in contrib/)
   has been taught about it.
   (merge 9cd4382ad5 ls/complete-remote-update-names later to maint).

 * "git fetch --shallow-since=<cutoff>" that specifies the cut-off
   point that is newer than the existing history used to end up
   grabbing the entire history.  Such a request now errors out.
   (merge e34de73c56 nd/reject-empty-shallow-request later to maint).

 * Fix for 2.17-era regression around `core.safecrlf`.
   (merge 6cb09125be as/safecrlf-quiet-fix later to maint).

 * The recent addition of "partial clone" experimental feature kicked
   in when it shouldn't, namely, when there is no partial-clone filter
   defined even if extensions.partialclone is set.
   (merge cac1137dc4 jh/partial-clone later to maint).

 * "git send-pack --signed" (hence "git push --signed" over the http
   transport) did not read user ident from the config mechanism to
   determine whom to sign the push certificate as, which has been
   corrected.
   (merge d067d98887 ms/send-pack-honor-config later to maint).

 * "git fetch-pack --all" used to unnecessarily fail upon seeing an
   annotated tag that points at an object other than a commit.
   (merge c12c9df527 jk/fetch-all-peeled-fix later to maint).

 * When user edits the patch in "git add -p" and the user's editor is
   set to strip trailing whitespaces indiscriminately, an empty line
   that is unchanged in the patch would become completely empty
   (instead of a line with a sole SP on it).  The code introduced in
   Git 2.17 timeframe failed to parse such a patch, but now it learned
   to notice the situation and cope with it.
   (merge f4d35a6b49 pw/add-p-recount later to maint).

 * The code to try seeing if a fetch is necessary in a submodule
   during a fetch with --recurse-submodules got confused when the path
   to the submodule was changed in the range of commits in the
   superproject, sometimes showing "(null)".  This has been corrected.

 * "git submodule" did not correctly adjust core.worktree setting that
   indicates whether/where a submodule repository has its associated
   working tree across various state transitions, which has been
   corrected.
   (merge 984cd77ddb sb/submodule-core-worktree later to maint).

 * Bugfix for "rebase -i" corner case regression.
   (merge a9279c6785 pw/rebase-i-keep-reword-after-conflict later to maint).

 * Recently added "--base" option to "git format-patch" command did
   not correctly generate prereq patch ids.
   (merge 15b76c1fb3 xy/format-patch-prereq-patch-id-fix later to maint).

 * POSIX portability fix in Makefile to fix a glitch introduced a few
   releases ago.
   (merge 6600054e9b dj/runtime-prefix later to maint).

 * "git filter-branch" when used with the "--state-branch" option
   still attempted to rewrite the commits whose filtered result is
   known from the previous attempt (which is recorded on the state
   branch); the command has been corrected not to waste cycles doing
   so.
   (merge 709cfe848a mb/filter-branch-optim later to maint).

 * Clarify that setting core.ignoreCase to deviate from reality would
   not turn a case-incapable filesystem into a case-capable one.
   (merge 48294b512a ms/core-icase-doc later to maint).

 * "fsck.skipList" did not prevent a blob object listed there from
   being inspected for is contents (e.g. we recently started to
   inspect the contents of ".gitmodules" for certain malicious
   patterns), which has been corrected.
   (merge fb16287719 rj/submodule-fsck-skip later to maint).

 * "git checkout --recurse-submodules another-branch" did not report
   in which submodule it failed to update the working tree, which
   resulted in an unhelpful error message.
   (merge ba95d4e4bd sb/submodule-move-head-error-msg later to maint).

 * "git rebase" behaved slightly differently depending on which one of
   the three backends gets used; this has been documented and an
   effort to make them more uniform has begun.
   (merge b00bf1c9a8 en/rebase-consistency later to maint).

 * The "--ignore-case" option of "git for-each-ref" (and its friends)
   did not work correctly, which has been fixed.
   (merge e674eb2528 jk/for-each-ref-icase later to maint).

 * "git fetch" failed to correctly validate the set of objects it
   received when making a shallow history deeper, which has been
   corrected.
   (merge cf1e7c0770 jt/connectivity-check-after-unshallow later to maint).

 * Partial clone support of "git clone" has been updated to correctly
   validate the objects it receives from the other side.  The server
   side has been corrected to send objects that are directly
   requested, even if they may match the filtering criteria (e.g. when
   doing a "lazy blob" partial clone).
   (merge a7e67c11b8 jt/partial-clone-fsck-connectivity later to maint).

 * Handling of an empty range by "git cherry-pick" was inconsistent
   depending on how the range ended up to be empty, which has been
   corrected.
   (merge c5e358d073 jk/empty-pick-fix later to maint).

 * "git reset --merge" (hence "git merge ---abort") and "git reset --hard"
   had trouble working correctly in a sparsely checked out working
   tree after a conflict, which has been corrected.
   (merge b33fdfc34c mk/merge-in-sparse-checkout later to maint).

 * Correct a broken use of "VAR=VAL shell_func" in a test.
   (merge 650161a277 jc/t3404-one-shot-export-fix later to maint).

 * "git rev-parse ':/substring'" did not consider the history leading
   only to HEAD when looking for a commit with the given substring,
   when the HEAD is detached.  This has been fixed.
   (merge 6b3351e799 wc/find-commit-with-pattern-on-detached-head later to maint).

 * Build doc update for Windows.
   (merge ede8d89bb1 nd/command-list later to maint).

 * core.commentchar is now honored when preparing the list of commits
   to replay in "rebase -i".

 * "git pull --rebase" on a corrupt HEAD caused a segfault.  In
   general we substitute an empty tree object when running the in-core
   equivalent of the diff-index command, and the codepath has been
   corrected to do so as well to fix this issue.
   (merge 3506dc9445 jk/has-uncommitted-changes-fix later to maint).

 * httpd tests saw occasional breakage due to the way its access log
   gets inspected by the tests, which has been updated to make them
   less flaky.
   (merge e8b3b2e275 sg/httpd-test-unflake later to maint).

 * Tests to cover more D/F conflict cases have been added for
   merge-recursive.

 * "git gc --auto" opens file descriptors for the packfiles before
   spawning "git repack/prune", which would upset Windows that does
   not want a process to work on a file that is open by another
   process.  The issue has been worked around.
   (merge 12e73a3ce4 kg/gc-auto-windows-workaround later to maint).

 * The recursive merge strategy did not properly ensure there was no
   change between HEAD and the index before performing its operation,
   which has been corrected.
   (merge 55f39cf755 en/dirty-merge-fixes later to maint).

 * "git rebase" started exporting GIT_DIR environment variable and
   exposing it to hook scripts when part of it got rewritten in C.
   Instead of matching the old scripted Porcelains' behaviour,
   compensate by also exporting GIT_WORK_TREE environment as well to
   lessen the damage.  This can harm existing hooks that want to
   operate on different repository, but the current behaviour is
   already broken for them anyway.
   (merge ab5e67d751 bc/sequencer-export-work-tree-as-well later to maint).

 * "git send-email" when using in a batched mode that limits the
   number of messages sent in a single SMTP session lost the contents
   of the variable used to choose between tls/ssl, unable to send the
   second and later batches, which has been fixed.
   (merge 636f3d7ac5 jm/send-email-tls-auth-on-batch later to maint).

 * The lazy clone support had a few places where missing but promised
   objects were not correctly tolerated, which have been fixed.

 * One of the "diff --color-moved" mode "dimmed_zebra" that was named
   in an unusual way has been deprecated and replaced by
   "dimmed-zebra".
   (merge e3f2f5f9cd es/diff-color-moved-fix later to maint).

 * The wire-protocol v2 relies on the client to send "ref prefixes" to
   limit the bandwidth spent on the initial ref advertisement.  "git
   clone" when learned to speak v2 forgot to do so, which has been
   corrected.
   (merge 402c47d939 bw/clone-ref-prefixes later to maint).

 * "git diff --histogram" had a bad memory usage pattern, which has
   been rearranged to reduce the peak usage.
   (merge 79cb2ebb92 sb/histogram-less-memory later to maint).

 * Code clean-up to use size_t/ssize_t when they are the right type.
   (merge 7726d360b5 jk/size-t later to maint).

 * The wire-protocol v2 relies on the client to send "ref prefixes" to
   limit the bandwidth spent on the initial ref advertisement.  "git
   fetch $remote branch:branch" that asks tags that point into the
   history leading to the "branch" automatically followed sent to
   narrow prefix and broke the tag following, which has been fixed.
   (merge 2b554353a5 jt/tag-following-with-proto-v2-fix later to maint).

 * When the sparse checkout feature is in use, "git cherry-pick" and
   other mergy operations lost the skip_worktree bit when a path that
   is excluded from checkout requires content level merge, which is
   resolved as the same as the HEAD version, without materializing the
   merge result in the working tree, which made the path appear as
   deleted.  This has been corrected by preserving the skip_worktree
   bit (and not materializing the file in the working tree).
   (merge 2b75fb601c en/merge-recursive-skip-fix later to maint).

 * The "author-script" file "git rebase -i" creates got broken when
   we started to move the command away from shell script, which is
   getting fixed now.
   (merge 5522bbac20 es/rebase-i-author-script-fix later to maint).

 * The automatic tree-matching in "git merge -s subtree" was broken 5
   years ago and nobody has noticed since then, which is now fixed.
   (merge 2ec4150713 jk/merge-subtree-heuristics later to maint).

 * "git fetch $there refs/heads/s" ought to fetch the tip of the
   branch 's', but when "refs/heads/refs/heads/s", i.e. a branch whose
   name is "refs/heads/s" exists at the same time, fetched that one
   instead by mistake.  This has been corrected to honor the usual
   disambiguation rules for abbreviated refnames.
   (merge 60650a48c0 jt/refspec-dwim-precedence-fix later to maint).

 * Futureproofing a helper function that can easily be misused.
   (merge 65bb21e77e es/want-color-fd-defensive later to maint).

 * The http-backend (used for smart-http transport) used to slurp the
   whole input until EOF, without paying attention to CONTENT_LENGTH
   that is supplied in the environment and instead expecting the Web
   server to close the input stream.  This has been fixed.
   (merge eebfe40962 mk/http-backend-content-length later to maint).

 * "git merge --abort" etc. did not clean things up properly when
   there were conflicted entries in the index in certain order that
   are involved in D/F conflicts.  This has been corrected.
   (merge ad3762042a en/abort-df-conflict-fixes later to maint).

 * "git diff --indent-heuristic" had a bad corner case performance.
   (merge 301ef85401 sb/indent-heuristic-optim later to maint).

 * The "--exec" option to "git rebase --rebase-merges" placed the exec
   commands at wrong places, which has been corrected.

 * "git verify-tag" and "git verify-commit" have been taught to use
   the exit status of underlying "gpg --verify" to signal bad or
   untrusted signature they found.
   (merge 4e5dc9ca17 jc/gpg-status later to maint).

 * "git mergetool" stopped and gave an extra prompt to continue after
   the last path has been handled, which did not make much sense.
   (merge d651a54b8a ng/mergetool-lose-final-prompt later to maint).

 * Among the three codepaths we use O_APPEND to open a file for
   appending, one used for writing GIT_TRACE output requires O_APPEND
   implementation that behaves sensibly when multiple processes are
   writing to the same file.  POSIX emulation used in the Windows port
   has been updated to improve in this area.
   (merge d641097589 js/mingw-o-append later to maint).

 * "git pull --rebase -v" in a repository with a submodule barfed as
   an intermediate process did not understand what "-v(erbose)" flag
   meant, which has been fixed.
   (merge e84c3cf3dc sb/pull-rebase-submodule later to maint).

 * Recent update to "git config" broke updating variable in a
   subsection, which has been corrected.
   (merge bff7df7a87 sb/config-write-fix later to maint).

 * When "git rebase -i" is told to squash two or more commits into
   one, it labeled the log message for each commit with its number.
   It correctly called the first one "1st commit", but the next one
   was "commit #1", which was off-by-one.  This has been corrected.
   (merge dd2e36ebac pw/rebase-i-squash-number-fix later to maint).

 * "git rebase -i", when a 'merge <branch>' insn in its todo list
   fails, segfaulted, which has been (minimally) corrected.
   (merge bc9238bb09 pw/rebase-i-merge-segv-fix later to maint).

 * "git cherry-pick --quit" failed to remove CHERRY_PICK_HEAD even
   though we won't be in a cherry-pick session after it returns, which
   has been corrected.
   (merge 3e7dd99208 nd/cherry-pick-quit-fix later to maint).

 * Code cleanup, docfix, build fix, etc.
   (merge aee9be2ebe sg/update-ref-stdin-cleanup later to maint).
   (merge 037714252f jc/clean-after-sanity-tests later to maint).
   (merge 5b26c3c941 en/merge-recursive-cleanup later to maint).
   (merge 0dcbc0392e bw/config-refer-to-gitsubmodules-doc later to maint).
   (merge bb4d000e87 bw/protocol-v2 later to maint).
   (merge 928f0ab4ba vs/typofixes later to maint).
   (merge d7f590be84 en/rebase-i-microfixes later to maint).
   (merge 81d395cc85 js/rebase-recreate-merge later to maint).
   (merge 51d1863168 tz/exclude-doc-smallfixes later to maint).
   (merge a9aa3c0927 ds/commit-graph later to maint).
   (merge 5cf8e06474 js/enhanced-version-info later to maint).
   (merge 6aaded5509 tb/config-default later to maint).
   (merge 022d2ac1f3 sb/blame-color later to maint).
   (merge 5a06a20e0c bp/test-drop-caches-for-windows later to maint).
   (merge dd61cc1c2e jk/ui-color-always-to-auto later to maint).
   (merge 1e83b9bfdd sb/trailers-docfix later to maint).
   (merge ab29f1b329 sg/fast-import-dump-refs-on-checkpoint-fix later to maint).
   (merge 6a8ad880f0 jn/subtree-test-fixes later to maint).
   (merge ffbd51cc60 nd/pack-objects-threading-doc later to maint).
   (merge e9dac7be60 es/mw-to-git-chain-fix later to maint).
   (merge fe583c6c7a rs/remote-mv-leakfix later to maint).
   (merge 69885ab015 en/t3031-title-fix later to maint).
   (merge 8578037bed nd/config-blame-sort later to maint).
   (merge 8ad169c4ba hn/config-in-code-comment later to maint).
   (merge b7446fcfdf ar/t4150-am-scissors-test-fix later to maint).
   (merge a8132410ee js/typofixes later to maint).
   (merge 388d0ff6e5 en/update-index-doc later to maint).
   (merge e05aa688dd jc/update-index-doc later to maint).
   (merge 10c600172c sg/t5310-empty-input-fix later to maint).
   (merge 5641eb9465 jh/partial-clone-doc later to maint).
   (merge 2711b1ad5e ab/submodule-relative-url-tests later to maint).