Added an option to limit the amount of commits checked

[git-stats.git] / doc / changelog.txt

Preliminary version of the activityInArea function is done,
can't handle merges yet. The test repo perhaps needs some
TLC to produce merges, have mutliple authors, and mutliple
files. Merges may be handled by splitting the line and
checking if it has more than 1 part.

Finished activityInArea for now, it handles merges as
suggested, but the test repo still needs said TLC. Wrote a
wrapper script, stats.py, that will serve as a command
dispatcher / readability layer. Unhandled at the moment are
directory/file renames.

No work done because of uni project deadline, project is
finished now so more time for gsoc now.

Created a 'touched' command that shows all commits that
touch the same files as the staged changes. Also created
another function that lists all files touched in a commit.
That way instead of allowing it to only work on staged
changes it may also work upon a specific commit.

Created three functions, branchContains, commitmsgMatches,
and commitdiffMatches, to use for metrics later on. The
test repo needs major TLC now, it is rather useless for
testing the above functions. There is no validty checking
yet for any of the current functions, perhaps do this at
dispatch time? The function that dispatches should be able
to do git-rev-check on the arguments if required.

No development done in the morning during due to a
presentation for uni. Couldn't start coding till later in
the day due to going out with the family because the house
is sold. Gave the test repo the TLC it deserves.

Started work on a diff-parser. Turns out this will need
some further thought (what about renames etc), but not now.

Completed the diff parser and added a diff-differ. There
are currently no tests for either, but manual testing has
been done. Next step should probably be the creation of
testcases and perhaps expanding the test repo.

Done only trival work, spent most day studying for
upcomming examns. Tried to decide on how to test things but
came up short.

Decided on using a modified version of test-lib.sh for
testing. Next problem that came up is how to test functions
from the modules. Most obvious solution is to implement
command parsing in stats.py. Another problem is how to get
access to stats.py in the test repo.  Optparse looks nice
for parsing but might be overkill. Main problem now is what
commandline structure to use for dispatching.

Got started on the dispatcher, the 'diff' part actually has
a working dispatcher. All other modules have a no-op method
that excepts any amount of arguments though. Noticed a bug
in commitDiffEquals which will need fixing; it currently is
unable to handle hunks longer than one line, very lame.
This once more points out the need of good test-cases.
Dispatching has to be finished first though, which will
probably be done tomorrow.

Worked mostly on getting dispatching working. There is now
a basic dispatcher that dispatches to the different
modules. In the modules dispatching is handled by means of
an OptParser. So far only diff and commit have such a
dispatcher implemented, although the one for diff is hardly
finished.

Incorperated some spelling fixes by David. Worked on
getting the testcases script resubmitted, for inclusion
this time. Spent a few hours trying to figure out why an
import wouldn't work, very gratifying work... Turned out to
be a cyclic 'from .. import ..', which python doesn't like.
Got annoyed by such a waste of time and spent the rest of
the day studying for school.

Finished the dispatching framework (one day late). Adjusted
the planning accordingly, so that testcases start tomorrow.
The the remainder of the week should be enough to finish
the testscases though. Meanwhile noticed that git has some
things already built in, but not in plumbing! (Such as
'git branch --contains'.) Will probably just use the
porcelain in those cases, perhaps ask at the GML.

Made use of 'stats.py' for the first time in a real life
use-case. Instead of using 'git blame' to find out when the
bug in author.py was introduced, made use of 
'stats.py index' to find out which one it was. Had to
modify commit.prettyPrint to show "-p" instead of
"--name-only", which should probably become a configuration
option at some point.

Finished the branch testcases, noticed another bug, this
time in branch.py. (It wasn't really a bug, the resulting
branches of 'stats.py branch' were ran through prettyPrint
before being displayed. As a result the HEAD commits were
shown instead of the names of the branches.

Worked on a a way to get return values for function calls.
Later found out that the latest git-python has such a
feature, so that work was double. Instead spent time on
getting git-python working 'out of the box' which involved
upgrading to the latest version, in which all trailing
whitespace (including newlines) is removed by default.

Added testcases for commit.py, which pointed out a major
bugs in the --diff-contains filter (noticed it before, but
had not taken the time to investigate). Fixing the bug took
quite some time, mainly because git-python first had to be
extended to support GIT_PYTHON_TRACE=full, which prints the
result of all git calls. With that working, it was a lot
easier to see what was going on.

Spent most of the day trying to get internet at the new
house. Heard the words "sorry, you will have to call the
other company, we can't do this" at least a dozen times.
Hopefully tomorrow will be more productive, today felt
like a day wasted.

Got a lot of work done today, all the subcommands now have
testcases, although some are along the lines of 
'There is no test for ...'. Mostly because the setupRepo
script does not offer the repo that is required to test
that specific functionality. Spent some time toying around
with GitPython, it now has a with_keep_cwd method that will
be usefull for 'commit -r -t'.

Noticed today that http://repo.or.cz is down, as such it is
currently not possible to push out the latest changes. Per
request test-lib.sh no longer requires a git build to be
present, instead it will use the regular git install. 
Together with this change, some functionality was removed
because it was not used anyway. Have been busy most the day
studying for upcomming examns (the first one is monday).

Spore Creature Creator (demo) was leaked a few days early,
spent quite a bit of time playing around with it. Went on
to try and understand computability theory before tomorrows
test. Fixed the bug in "commit -t" where it did not check
if the specified filename is sane.

Also created the missing testcase for 'diff -n', which was
quite trivial with the latest changes to the test repo.
Added the missing test for 'index -a', which involved
expanding the test repo to include a case in which the
switch is usefull. Central today though was the examn, most
of the day revolved around it.

Fixed the last "no testcase for" test, namely "branch -r".
To do so had to add a remote branch in the setupRepo script
with the "git-update-ref" command. Then, while sending a
mail to the list regarding "branch --contains" being a
porcelain command, but with no plumbing equivalent
available, gmail locked down. Not sure why, something about
a possible violation of the Terms of Service.

Got started on the 'belongs to' metric today. It was very
nice, got it right on the first try, the algorithm found
the branch that the test commit belonged to perfectly, and
also marked the diluted branches as desired. It was easy
after that to fetch the best matches. Left to do is to
check the commit message of the merge and see if the
default commit message concurs with what was found to be
the primary parent.

The math examn is closing in fast, wrestled through 5
chapters today, so only a few hours left for coding. 
Hopefully it will be done soon, and the 12h / day hacking
fest will begin! Added a bumpTime that makes the graph
made by gitk --all better readable.

After some silence due to studying, development is picking
up again! Got a lot done today, some of which related to
the latest changes in GitPython. The test suite passes
again now using the latest HEAD of GitPython. Also made
the generated repo's not have all the commits at the same
moment, instead the commit time is bumped with each
'action'.

One and a half day 'lost' to moving, not something to do on
a regular basis. Perhaps if it was as simple as 'git mv'?
Dug into a bug into the branch contains metric now that
there is a proper repo to try it out on. Turned out that
to view the metrics repostitory the git-forest script is
_very_ useful. It seems to be the only visualizer that is
capable of giving a sensible visual representation.
Discovered a corner case that needs to be figured out, in
an attempt to do so factored out the function that
calculates the metric. Too tired now to figure it out,
tomorrow should be good.

Today was good indeed! The problem was that 'git rev-list'
will follow merges (which is good), but that was not taken
in account by the metric (which is bad). There was no easy
way to get it to take that into account (which is bad too),
as a result the entire metric was made to be recursive. As
a result of that rewrite the algorithm took very long to
run, which lead to the first optimization so far. The tests
for this metric run thrice as fast after with the
optimization in place, a good day indeed!

All day birthday party, grandmother turned 75
(congratulations!), couldn't start coding till late in the
evening. Did manage to finish the 'activity in file'
algorithm. Easy to do, but useful in deciding who has been 
working on a certain file.

Worked on the bug.py file which has not been committed yet,
but contains commands related to determining whether a
certain commit was a fix or not. Tried to use git-forest to
pick a commit from git.git from the maint branch, but ran
into trouble with piping to less. Turned out it was a
simple fix, the -R switch was missing. Exporting LESS=FRSX
fixed the problem, although it took quite a bit to figure
that out. When trying stats.py branch -b on the git.git
repo python raised an exception because the recursion
limit was breached. Best fix would be to not use recursion
but go with a stack-based approach instead. Spent a large
part of the day studying for the last test tomorrow.

Went in for the kill and aced today's test, socialized for
the remainder of the day. Busy day tomorrow, hopefully
there will be time for coding tomorrow evening. The 
mid-term evaluation is comming up, it would be nice to have
something to show the mailing list by then.

Working in the night-train since there is a power plug in
the cabin. Managed to rewrite the 'belongs to' algorithm
so that python's recursion limit is not a problem on large
repositories.

Worked a lot on optimizing the 'belongs to' algorithm on
the git.git repository. Now it is usable on large
repositories and can be incorperated in the bug detection
system. Next step now is making the bug detection metrics
configurable through a file or such.

Added aggregation to the 'activity' metric, which is now
capable of showing the activity of all files aggregated
over all developers, or showing the activity for all
developers. 

Made a simple config parser that is capable of parsing
config files that follow the '.git/config' style. It
searches for a '[GitStats]' verse from which it takes all
values and returns them as a dictionary. Used this to feed
the 'bug -t' metric, which is very much usable now. Left to
do now is to aggregate 'bug -t' over all commits.

Many work done on optimizing 'bug -a' which now runs in
under 4 seconds on git-stats.git instead of taking about
1.5 minutes!! It is still not usable on the git.git repo
on which it takes >10 minutes to complete. Will probably
have to cache data in order to get something useful for
such big repo's. Either that, or not run on all commits but
instead only on the most recent n commits or such.
Started to work on a diff matcher per suggestion of
Johannes Schindelin aka Dscho. Would be nice if it works
out and is incorperated into git-log (which would make it
a lot faster, and very useful to others as well).