Completely rewrote object data access to improve performance.
commit42691339476af7c68aac3afc31abfec0ad450d95
authorShawn O. Pearce <spearce@spearce.org>
Thu, 30 Nov 2006 06:42:41 +0000 (30 01:42 -0500)
committerShawn O. Pearce <spearce@spearce.org>
Thu, 30 Nov 2006 06:42:41 +0000 (30 01:42 -0500)
tree74ea0146af4a7bfaf38a0bae2d0e3ad8937aed2f
parentd1636092d5dc8ea65ed73068390818d37db26234
Completely rewrote object data access to improve performance.

According to the profiling that I was able to do we were spending a lot
of our time in the object decompression/delta application portions of
jgit when we were walking a large number of commit objects.  This is
what I feared, which was that our way-too-flexible stack of InputStreams
was simply costing us too much in overhead to be realistic.

So I've written pretty much all of that code and taking the approach
that core Git takes.  We allocate entire byte[] for data anytime we
need to work with something, rather than attempting to stream it from
an InputStream.  I've also pushed the object inflation code down as
close to the ByteWindow as possible, so that the byte window itself
can be offered up to the active Inflater as the input.  This actually
worked out well as it maintains  a nice clean abstraction between
the code that needs the decompressed data and the windows which hold
the compressed data.

So at this point once the data is in memory we are able to access it
without copying, except during decompression.  But if we are using a
mmapped region and Java won't let us get access to the underlying
byte[] (because its mapped MapMode.READ_ONLY) then we are forced to
copy the data into a temporary buffer before we can hand it over to
the deflate routine.

This version of the code shows a roughly a 10x speed improvement for
the case of walking along a chain of 50,000 commits (from the Mozilla
repository).

Unfortunately this version also contains a broken commit parser, an
untested but heavily modified tree tree parser, and a lightly tested
but also heavily modified binary delta apply routine.  More work
still needs to be done on those.

Signed-off-by: Shawn O. Pearce <spearce@spearce.org>
28 files changed:
org.spearce.jgit/src/org/spearce/jgit/lib/BinaryDelta.java [new file with mode: 0644]
org.spearce.jgit/src/org/spearce/jgit/lib/ByteArrayWindow.java
org.spearce.jgit/src/org/spearce/jgit/lib/ByteBufferWindow.java
org.spearce.jgit/src/org/spearce/jgit/lib/ByteWindow.java
org.spearce.jgit/src/org/spearce/jgit/lib/CheckoutTree.java
org.spearce.jgit/src/org/spearce/jgit/lib/Commit.java
org.spearce.jgit/src/org/spearce/jgit/lib/DeltaOfsPackedObjectLoader.java [new file with mode: 0644]
org.spearce.jgit/src/org/spearce/jgit/lib/DeltaOfsPackedObjectReader.java [deleted file]
org.spearce.jgit/src/org/spearce/jgit/lib/DeltaPackedObjectLoader.java [new file with mode: 0644]
org.spearce.jgit/src/org/spearce/jgit/lib/DeltaPackedObjectReader.java [deleted file]
org.spearce.jgit/src/org/spearce/jgit/lib/DeltaRefPackedObjectLoader.java [moved from org.spearce.jgit/src/org/spearce/jgit/lib/DeltaRefPackedObjectReader.java with 52% similarity]
org.spearce.jgit/src/org/spearce/jgit/lib/FileTreeEntry.java
org.spearce.jgit/src/org/spearce/jgit/lib/ObjectId.java
org.spearce.jgit/src/org/spearce/jgit/lib/ObjectLoader.java [moved from org.spearce.jgit/src/org/spearce/jgit/lib/ObjectReader.java with 59% similarity]
org.spearce.jgit/src/org/spearce/jgit/lib/PackFile.java
org.spearce.jgit/src/org/spearce/jgit/lib/PackedObjectLoader.java [copied from org.spearce.jgit/src/org/spearce/jgit/lib/ByteBufferWindow.java with 60% similarity]
org.spearce.jgit/src/org/spearce/jgit/lib/PackedObjectReader.java [deleted file]
org.spearce.jgit/src/org/spearce/jgit/lib/PatchDeltaStream.java [deleted file]
org.spearce.jgit/src/org/spearce/jgit/lib/Repository.java
org.spearce.jgit/src/org/spearce/jgit/lib/Tree.java
org.spearce.jgit/src/org/spearce/jgit/lib/UnpackedObjectLoader.java [new file with mode: 0644]
org.spearce.jgit/src/org/spearce/jgit/lib/UnpackedObjectReader.java [deleted file]
org.spearce.jgit/src/org/spearce/jgit/lib/WholePackedObjectLoader.java [new file with mode: 0644]
org.spearce.jgit/src/org/spearce/jgit/lib/WholePackedObjectReader.java [deleted file]
org.spearce.jgit/src/org/spearce/jgit/lib/WindowedFile.java
org.spearce.jgit/src/org/spearce/jgit/lib/XInputStream.java [deleted file]
org.spearce.jgit/tst/org/spearce/jgit/lib/T0004_PackReader.java
org.spearce.jgit/tst/org/spearce/jgit/lib/XInputStream.java [new file with mode: 0644]