Update Friday, 30th of January, Anno Domini MMIX, at the hour of the Buffalo

[git/dscho.git] / blog.rss

<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
<channel>
<title>Dscho's blog</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html</link>
<atom:link href="http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=blog.rss" rel="self" type="application/rss+xml"/>
<description>A few stories told by Dscho</description>
<lastBuildDate>Fri, 30 Jan 2009 02:01:26 +0100</lastBuildDate>
<language>en-us</language>
<item>
<title>More valgrind fun</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233277286</link>
<guid>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233277286</guid>
<pubDate>Fri, 30 Jan 2009 02:01:26 +0100</pubDate>
<description><![CDATA[More valgrind fun
</p><p>
So I spent quite a number of hours on that funny zlib/valgrind issue.  The
thing is, zlib people claim that even if their code accesses uninitialized
memory, it does not produce erroneous data (by cutting out the results of the
uninitialized data, which is cheaper than checking for the end of the buffer
in an unaligned manner), so zlib will always be special for valgrind.
</p><p>
However, the bug I was chasing is funny, and different from said issue.  zlib
deflates an input buffer to an output buffer that is exactly 58 bytes long.
But valgrind claims that the 52nd of those bytes is uninitialized, and <u>only</u>
that one.
</p><p>
But it is not.  It must be 0x2c, otherwise zlib refuses to inflate the
buffer.
</p><p>
Now, I went into a debugging frenzy, and finally found out that zlib just
passes fine (with the default suppressions because of the "cute" way it
uses uninitialized memory), <u>except</u> when it is compiled with UNALIGNED_OK
defined.
</p><p>
Which Ubuntu does, of course.  Ubuntu, the biggest forker of all.
</p><p>
The bad part is that it sounds like a bug in valgrind, and I <u>could</u> imagine
that it is an issue of an optimized memcpy() that copies int by int, and
that valgrind misses out on the fact that a part of that int is actually
<u>not</u> uninitialized.
</p><p>
But my debugging session's results disagree with that.
</p><p>
With the help of Julian Seward, the original author of valgrind, I instrumented
zlib's source code so that valgrind checks earlier if the byte is initialized
or not, to find out where the reason of the issue lies.
</p><p>
The sad part is that when I added the instrumentation to both the <u>end</u> of
the while() loop in compress_block() in zlib's trees.c, and just <u>after</u> the
while() loop (whose condition is a plain <i>variable < variable</i> comparison,
nothing fancy, certainly not changing any memory), only the <u>latter</u> catches
a valgrind error.
</p><p>
And that is truly strange.]]></description>
</item>
<item>
<title>Interactive stash</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233193467</link>
<guid>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233193467</guid>
<pubDate>Thu, 29 Jan 2009 02:44:27 +0100</pubDate>
<description><![CDATA[Interactive stash
</p><p>
There is an easy way to split a patch:
</p><p>
<table
                                border=1 bgcolor=white>
                        <tr><td bgcolor=lightblue colspan=3>
                                <pre>                                                                                </pre>
                        </td></tr>
                        <tr><td>
                                <table cellspacing=5 border=0
                                         style="color:black;">
                                <tr><td>
                                        <pre>
$ git reset HEAD^
$ git add -i
$ git commit
$ git diff -R HEAD@{1} | git apply --index
$ git commit
</pre>
                                </td></tr>
                                </table>
                        </td></tr>
                        </table>
</p><p>
but it misses out on the fact that the first of both commits does not
reflect the state of the working directory at any time.
</p><p>
So I think something like an interactive <i>stash</i> is needed.  A method
to specify what you want to keep in the working directory, the rest should
be stashed.  The idea would be something like this:
</p><p>
<ol>
<li>Add the desired changes into a temporary index.
<li>Put the rest of the changes in another temporary index.
<li>Stash the latter index.
<li>Synchronize the working directory with the first index.
<li>Clean up temporary indices.
</ol>
</p><p>
Or in code:
</p><p>
<table
                                border=1 bgcolor=white>
                        <tr><td bgcolor=lightblue colspan=3>
                                <pre>                                                                                </pre>
                        </td></tr>
                        <tr><td>
                                <table cellspacing=5 border=0
                                         style="color:black;">
                                <tr><td>
                                        <pre>
$ cp .git/index .git/interactive-stash-1
$ GIT_INDEX_FILE=.git/interactive-stash-1 git add -i
$ cp .git/index .git/interactive-stash-2
$ GIT_INDEX_FILE=.git/interactive-stash-1 git diff -R |
        (GIT_INDEX_FILE=.git/interactive-stash-2 git apply--index)
$ tree=$(GIT_INDEX_FILE=.git/index git write-tree)
$ commit=$(echo Current index | git commit-tree $tree -p HEAD)
$ tree=$(GIT_INDEX_FILE=.git/interactive-stash-2 git write-tree)
$ commit=$(echo Edited out | git commit-tree $tree -p HEAD -p $commit)
$ git update-ref refs/stash $commit
$ GIT_INDEX_FILE=.git/interactive-stash-1 git checkout-index -a -f
$ rm .git/interactive-stash-1 .git/interactive-stash-2
</pre>
                                </td></tr>
                                </table>
                        </td></tr>
                        </table>
</p><p>
This should probably go into <i>git-stash.sh</i>, maybe even with a switch
to start git-gui to do the interactive adding instead of git-add.]]></description>
</item>
<item>
<title>Splitting topic branches</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233154567</link>
<guid>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233154567</guid>
<pubDate>Wed, 28 Jan 2009 15:56:07 +0100</pubDate>
<description><![CDATA[Splitting topic branches
</p><p>
One might be put off easily by the overarching use of buzzwords in the
description of how <i>Darcs</i> works.  I, for one, do not expect an intelligent
author when I read <i>Theory of patches</i> and <i>based on quantum physics</i>.
</p><p>
The true story, however, is much simpler, and is actually not that dumb:
Let's call two commits "conflicting" when they contain at least one
overlapping change.
</p><p>
The idea is now: Given a list of commits (not a set, as the order is important),
to sort them into smaller lists such that conflicting commits are in the
sublists ("topic branches") and the sublists are minimal, i.e. no two
non-conflicting commits are in the same sublist.
</p><p>
The idea has flaws, of course, as you can have a patch changing the code,
and another changing the documentation, but splitting a list of commits
in that way is a first step to sort out my <i>my-next</i> mess, where I have
a linear perl of not-necessarily-dependent commits.
</p><p>
And actually, my whole rebase revamp aimed at the clean-up for my own
<i>my-next</i> branch, so I am currently writing a script that can be used
as a GIT_EDITOR for git-rebase which implements the Darcs algorithm.  Kind of:
the result is not implicit, but explicit and can be fixed up later.]]></description>
</item>
<item>
<title>Showing off that you're an Alpine user ... priceless!</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233102919</link>
<guid>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233102919</guid>
<pubDate>Wed, 28 Jan 2009 01:35:19 +0100</pubDate>
<description><![CDATA[Showing off that you're an Alpine user ... priceless!
</p><p>
So I was in a hurry to send the patches, and sent all the patches as replies
to the cover-letter, and therefore typed in <i>rnyn</i> all the time, which is the
mantra I need to say to Alpine for <i>Reply</i>, ... include quoted message?
<i>No</i>, ... reply to all recipients? <i>Yes</i>, ... use first role?
<i>No, use default role</i>.
</p><p>
That was pretty embarassing, as it shows everybody that I still do not trust
<i>send-email</i>, and rather paste every single patch by hand.  Which is rather
annoying.
</p><p>
So I started using format-patch today, to output directly to Alpine's
<i>postponed-msgs</i> folder, so that I can do some touchups in the mailer
before sending the patch series on its way.
</p><p>
However, when running format-patch with <i>--thread</i>, it generates Message-ID
strings that Alpine does not like, and therefore replaces.
</p><p>
Oh, well, I'll probably just investigate how the Message-IDs are supposed to
look, and then use sed to rewrite the generated ones by Alpine-friendly ones
during the redirection to <i>postponed-msgs</i>.
</p><p>
But I alread realized that doing it that way is dramatically faster than the
workflow I had before.
</p><p>
And safer: no more <i>rnyn</i>.]]></description>
</item>
<item>
<title>Progress with the interactive rebase preserving merges</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233101919</link>
<guid>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233101919</guid>
<pubDate>Wed, 28 Jan 2009 01:18:39 +0100</pubDate>
<description><![CDATA[Progress with the interactive rebase preserving merges
</p><p>
I thought about the "dropped" commits a bit more, after all, and it is
probably a good thing to substitute them by their parent, as Stephen did it.
</p><p>
Imagine that you have merged a branch with two commits.  One is in upstream,
and you want to rebase (preserving merges) onto upstream.  Then you still
want to merge the single commit.
</p><p>
Even better, if there is no commit left, the <i>$REWRITTEN</i> mechanism will
substitute the commit onto which we are rebasing, so a merge will just
result in a fast-forward!
</p><p>
Oh, another thing: merge commits should not have a patch id, as they have
<u>multiple</u> patches.  However, I borked the code long time ago (9c6efa36)
and merges get the patch-id of their diff to the first parent.  Which is
probably wrong.  So I guess I'll have to fix that with my rebase revamp.
</p><p>
So what about a root commit?  If that was dropped, we will just substitute
it with the commit onto which we rebase (as a root commit did not really
have a parent, but will get the onto-commit as new parent)..
</p><p>
Now that I finally realized that t3410 is so strange because of a bug <u>I</u>
introduced, I can finally go about fixing it.]]></description>
</item>
<item>
<title>Another midnight riddle?</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233099894</link>
<guid>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233099894</guid>
<pubDate>Wed, 28 Jan 2009 00:44:54 +0100</pubDate>
<description><![CDATA[Another midnight riddle?
</p><p>
Okay, here's another riddle: what is the next line?
</p><p>
<pre>
       1
      1 1
      2 1
    1 1 1 2
    3 1 1 2
  2 1 1 2 1 3
...
</pre>
</p><p>
And when does the line get wider than 10 digits?]]></description>
</item>
<item>
<title>Fun with calculus after midnight</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233022809</link>
<guid>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1233022809</guid>
<pubDate>Tue, 27 Jan 2009 03:20:09 +0100</pubDate>
<description><![CDATA[Fun with calculus after midnight
</p><p>
Problem: what is the shortest way of defining a variable consisting of <i>N</i>
spaces?  I.e. for <i>N=80</i> the result will look something like
</p><p>
<table
                                border=1 bgcolor=white>
                        <tr><td bgcolor=lightblue colspan=3>
                                <pre>                                                                                </pre>
                        </td></tr>
                        <tr><td>
                                <table cellspacing=5 border=0
                                         style="color:black;">
                                <tr><td>
                                        <pre>
s='    '
s="$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s$s"
</pre>
                                </td></tr>
                                </table>
                        </td></tr>
                        </table>
</p><p>
Let's see.  Let the minimal number of characters needed be <i>A(N)</i>.  For
simplicity, let's say that we only use one variable.  Then, certainly, <i>A(N)</i>
cannot be larger than <i>5+N</i>, as we could define a variable using 1 character
for the name, 1 for the equal sign, 2 for the quotes, and one for the semicolon
or newline character (whichever).
</p><p>
Now, let's assume <i>N</i> is a product <i>K*L</i>.  Then certainly, <i>A(N)</i> cannot
be larger than <i>A(K)+5+2*L</i>, as we could first define a variable that has
exactly <i>K</i> spaces and then use that to define the end result (in the example
above, <i>K=5</i> and <i>L=20</i>).
</p><p>
So, for which <i>N=K*L</i> is it better to use two definitions instead of one?
</p><p>
Simple calculus says that <i>5+K*L>5+K+5+2*L</i> must hold true, or (after some
scribbling): <i>L>1+7/(K-2)</i>. Which means that it makes no sense to define
a variable with 1 or 2 spaces first, which is kinda obvious (writing '$s'
alone would use two characters, so we could write the spaces right away).
</p><p>
But what for the other values?  For <i>K=3</i>, <i>L</i> must be at least 9 to make
sense (in other words, <i>N</i> must be at least 27).  For <i>K=4</i>, <i>L</i> needs
to be greater or equal to 5 (<i>N>=20</i>), the next pairs are <i>(5,4)</i>,
<i>(6,3)</i>, <i>(7,3)</i>, <i>(8,3)</i>, <i>(9,3)</i> and starting with <i>K=10</i>, any
<i>L>1</i> makes sense.
</p><p>
The second definition can also contain spaces at the end, however, so for any
<i>N=K*L+M</i>, <i>A(N)</i> cannot be larger than <i>A(K)+5+2*L+M</i>.
</p><p>
Not surprisingly, this leads to exactly the same <i>L>1+7/(K-2)</i> (as we can
append the <i>M</i> spaces in the last definition, no matter if we use 1 or
2 definitions).
</p><p>
However, that means that as soon as <i>N>=18</i>, we should use two definitions,
prior to that, it makes no sense.
</p><p>
So for <i>N<18</i>, <i>A(N)=5+N</i>.
</p><p>
But what <i>K</i> should one choose, i.e. how many spaces in the first definition?
In other words, what is <i>A(N)</i> given that we use two definitions?
</p><p>
That will have to wait for another midnight.  Just a teaser: <i>A(80)=36</i>.  Oh,
and with 80 characters, you can define a string of 9900 spaces...]]></description>
</item>
<item>
<title>Valgrind takes a loooong time</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1232997290</link>
<guid>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1232997290</guid>
<pubDate>Mon, 26 Jan 2009 20:14:50 +0100</pubDate>
<description><![CDATA[Valgrind takes a loooong time
</p><p>
Yesterday, I started a run on a fast machine, and it took roughly 5.5
hours by the machine's clock.
</p><p>
And of course, I redirected stdout only... *sigh*
</p><p>
Which triggered a Google search how to force redirection of all the output
in the test scripts to a file and the terminal at the same time.
</p><p>
It seems as if that is not easily done.  I tried
<center><table
                                border=1 bgcolor=white>
                        <tr><td bgcolor=lightblue colspan=3>
                                <pre>                                                                                </pre>
                        </td></tr>
                        <tr><td>
                                <table cellspacing=5 border=0
                                         style="color:black;">
                                <tr><td>
                                        <pre>
exec >(tee out) 2>&1
</pre>
                                </td></tr>
                                </table>
                        </td></tr>
                        </table></center>
</p><p>
but that did not work: it mumbled something about invalid file handles or some
such.
</p><p>
The only solution I found was:
<center><table
                                border=1 bgcolor=white>
                        <tr><td bgcolor=lightblue colspan=3>
                                <pre>                                                                                </pre>
                        </td></tr>
                        <tr><td>
                                <table cellspacing=5 border=0
                                         style="color:black;">
                                <tr><td>
                                        <pre>
mkpipe pipe
tee out < pipe &
exec > pipe 2>&1
</pre>
                                </td></tr>
                                </table>
                        </td></tr>
                        </table></center>
</p><p>
That is a problem for parallel execution, though, so I am still looking for a
better way to do it.
</p><p>
Once I have the output, it is relatively easy to analyze it, as I already
made a script which disects the output into valgrind output and the test
case it came from, then groups by common valgrind output and shows the
result to the user.]]></description>
</item>
<item>
<title>A day full of rebase... and a little valgrind</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1232927812</link>
<guid>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1232927812</guid>
<pubDate>Mon, 26 Jan 2009 00:56:52 +0100</pubDate>
<description><![CDATA[A day full of rebase... and a little valgrind
</p><p>
I think that I am progressing nicely with my rebase -p work, so much so
that I will soon be able to use it myself to work on topic branches <u>and</u>
rebase all the time without much hassle.
</p><p>
In other words, I would like to be able to rebase all my topic branches
to Junio's <i>next</i> branch whenever that has new commits.  With a single
rebase.
</p><p>
And finally, I got the idea of the thing Stephen implemented for dropped
commits; however, I am quite sure I do not like it.
</p><p>
So what are "dropped" commits?
</p><p>
When you rebase, chances are that the upstream already has applied at
least some of your patches.  So we filter those out with <i>--cherry-pick</i>.
Stephen calls those "dropped" commits.
</p><p>
Then he goes on to reinvent the "$REWRITTEN" system: a directory containing
the mappings of old commit names to new commit names.  That is easily fixed.
</p><p>
But worse, he substitutes the dropped commits with their <u>parents</u>, instead
of substituting them with the corresponding commits in upstream.
</p><p>
I guess this will be a medium-sized fight on the mailing list, depending
how much energy Stephen wants to put in to defend his strategy.
</p><p>
Anyway, I finally got to a point where only three of the tests are failing,
t3404, t3410 and t3412.  Somewhat disappointing is t3404, as its name pretends
not to exercize -p at all.  Oh well, I guess I'll see what is broken tomorrow.
</p><p>
Another part of the day was dedicated to the Valgrind patch series, which
should give us yet another level of code quality.
</p><p>
After having confused myself with several diverging/obsolete branches, I did
indeed finally manage to send that patch series off.  Woohoo.]]></description>
</item>
<item>
<title>Regular diff with word coloring (as opposed to word diff)</title>
<link>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1232888842</link>
<guid>http://repo.or.cz/w/git/dscho.git?a=blob_plain;hb=blog;f=index.html#1232888842</guid>
<pubDate>Sun, 25 Jan 2009 14:07:22 +0100</pubDate>
<description><![CDATA[Regular diff with word coloring (as opposed to word diff)
</p><p>
You know, if I were a bit faster with everything I do, I could do so much more!
</p><p>
For example, Junio's idea that you could keep showing a regular diff, only
coloring the words that have been removed/deleted.
</p><p>
Just imagine looking at the diff of a long line in LaTeX source code.  It
should be much nicer to the eye to see the complete removed/added sentences
instead of one sentence with colored words in between, disrupting your read
flow.
</p><p>
Compare these two versions:
</p><p>
Regular diff with colored words:
<blockquote><tt>
-This sentence has a <font color=red>tyop</font> in it.<br>
+This sentence has a <font color=green>typo</font> in it.<br>
</tt></blockquote>
</p><p>
Word diff:
<blockquote><tt>
This sentence has a <font color=red>tyop</font><font color=green>typo</font> in it.<br>
</tt></blockquote>
</p><p>
And it should not be hard to do at all!
</p><p>
In <i>diff_words_show()</i>, we basically get the minus lines as
<i>diff_words->minus</i> and the plus lines as <i>diff_words->plus</i>.  The
function then prepares the word lists and calls the xdiff engine to do all the
hard work, analyzing the result from xdiff and printing the lines in
<i>fn_out_diff_words_aux()</i>.
</p><p>
So all that would have to be changed would be to <u>record</u> the positions
of the removed/added words instead of outputting them, and at the end printing
the minus/plus buffers using the recorded information to color the words.
</p><p>
This would involve
</p><p>
<ul>
<li>adding two new members holding the offsets in the <i>diff_words</i>
struct,
<li>having a special handling for that mode in
<i>fn_out_diff_words_aux()</i> that appends the offsets and
returns,
<li>adding a function <i>show_lines_with_colored_words()</i> that
outputs a buffer with a given prefix ('-' or '+') and coloring the words at
given offsets with a given color,
<li>modify <i>diff_words_show()</i> to call that function for the "special
case: only removal" and at the end of the function, and
<li> disabling the <i>fwrite()</i> at the end of <i>diff_words_show()</i> for that
mode.
</ul>
</p><p>
Of course, the hardest part is to find a nice user interface for that.  Maybe
<i>--colored-words</i>? &#x263a;]]></description>
</item>
</channel>
</rss>