convert: stream from fd to required clean filter instead of mmap
The data is streamed to the filter process anyway. Better avoid
mapping the file if possible. This is especially useful if a clean
filter reduces the size, for example if it computes a sha1 for
binary data, like git media. The file size that the previous
implementation could handle was limited by the available address
space; large files for example could not be handled with (32-bit)
msysgit. The new implementation can filter files of any size as
long as the filter output is small enough.
The new code path is only taken if the filter is required. The
filter consumes data directly from the fd. The original data is not
available to git, so it must fail if the filter fails.
The environment variable GIT_MMAP_LIMIT is used to test that the
expected code path is taken. A related test that exercises required
filters is modified to verify that the data actually has been
modified on its way from the file system to the object store.
Signed-off-by: Steffen Prohaska <prohaska@zib.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>