documentation/cvs-regression.sgml

   1   <chapter id="cvs-regression">
   2     <title>How to do regression testing using Cvs</title>
   3
   4     <para>
   5       written by (???)
   6     </para>
   7     <para>
   8       (Extracted from <filename>wine/documentation/bugreports</filename>)
   9     </para>
  10
  11     <para>
  12       A problem that can happen sometimes is 'it used to work
  13       before, now it doesn't anymore...'. Here is a step by step
  14       procedure to try to pinpoint when the problem occured. This is
  15       <emphasis>NOT</emphasis> for casual users.
  16     </para>
  17
  18     <orderedlist>
  19       <listitem>
  20         <para>
  21           Get the 'full cvs' archive from winehq. This archive is
  22           the cvs tree but with the tags controlling the versioning
  23           system. It's a big file (> 15 meg) with a name like
  24           full-cvs-&lt;last update date> (it's more than 100mb
  25           when uncompressed, you can't very well do this with
  26           small, old computers or slow Internet connections).
  27         </para>
  28       </listitem>
  29       <listitem>
  30         <para>
  31           untar it into a repository directory:
  32           <screen>
  33             cd /home/gerard
  34             tar -zxffull-cvs-2000-05-20.tar.gz
  35             mv wine repository
  36           </screen>
  37         </para>
  38       </listitem>
  39       <listitem>
  40         <para>
  41           extract a new destination directory. This directory must
  42           not be in a subdirectory of the repository else
  43           <command>cvs</command> will think it's part of the
  44           repository and deny you an extraction in the repository:
  45           <screen>
  46             cd /home/gerard
  47             mv wine wine_current (-> this protects your current wine sandbox, if any)
  48             export CVSROOT=/home/gerard/repository
  49             cd /home/gerard
  50             cvs -d $CVSROOT checkout wine
  51           </screen>
  52         </para>
  53         <para>
  54           Note that it's not possible to do a checkout at a given
  55           date; you always do the checkout for the last date where
  56           the full-cvs-xxx snapshot was generated.
  57         </para>
  58       </listitem>
  59       <listitem>
  60         <para>
  61           you will have now in the <filename>~/wine</filename>
  62           directory an image of the cvs tree, on the client side.
  63           Now update this image to the date you want:
  64           <screen>
  65             cd /home/gerard/wine
  66             cvs -d $CVSROOT update -D "1999-06-01"
  67           </screen>
  68         </para>
  69         <para>
  70           The date format is <literal>YYYY-MM-DD</literal>.
  71         </para>
  72         <para>
  73           Many messages will inform you that more recent files have
  74           been deleted to set back the client cvs tree to the date
  75           you asked, for example:
  76           <screen>
  77             cvs update: tsx11/ts_xf86dga2.c is no longer in the repository
  78           </screen>
  79         </para>
  80         <para>
  81           <command>cvs update</command> is not limited to upgrade to
  82           a <emphasis>newer</emphasis> version as I have believed for far too long :-(
  83         </para>
  84       </listitem>
  85       <listitem>
  86         <para>
  87           Now proceed as for a normal update:
  88         </para>
  89         <screen>
  90           ./configure
  91           make depend && make
  92         </screen>
  93         <para>
  94           When you have found the exact date when a bug was added to
  95           the cvs tree, use something like :
  96           <screen>
  97             cvs -d $CVSROOT diff -D "1999-07-10" -D "1999-07-12"
  98           </screen>
  99           to get all the differences between the last cvs tree
 100           version known to work and code that first displayed the
 101           misbehavior.
 102         </para>
 103         <note>
 104           <para>
 105             I did not include flags for <command>diff</command>
 106             since they are in my <filename>.cvsrc</filename> file:
 107           </para>
 108           <screen>
 109             cvs -z 3
 110             update -dPA
 111             diff -u
 112           </screen>
 113         </note>
 114         <para>
 115           From this diff file, particularly the file names, and the
 116           <filename>ChangeLog</filename>, it's usually possible to
 117           find the different individual patches that were done at
 118           this time.
 119         </para>
 120         <para>
 121           If any non-programmer reads this, the fastest method to get
 122           at the point where the problem occured is to use a binary
 123           search, that is, if the problem occured in 1999, start at
 124           mid-year, then is the problem is already here, back to 1st
 125           April, if not, to 1st October, and so on.
 126         </para>
 127       </listitem>
 128       <listitem>
 129         <para>
 130           The next step is to start from the last working version
 131           and to dig the individual contributions from
 132           <ulink url="http://www.integrita.com/cgi-local/lwgate.pl/WINE-PATCHES/">
 133             http://www.integrita.com/cgi-local/lwgate.pl/WINE-PATCHES/</ulink>
 134           (where the Wine patches mailing list is archived)
 135         </para>
 136         <para>
 137           If the patch was done by the Wine maintainer or if it was
 138           sent directly to his mail address without going first through
 139           <ulink url="mailto:wine-patches@winehq.com">wine-patches</ulink>,
 140           you are out of luck as you will never find the patch in
 141           the archive. If it is, it's often possible to apply the
 142           patches one by one to last working cvs snapshot, compile and test.
 143           If you have saved the next candidate as
 144           <filename>/home/gerard/buggedpatch1.txt</filename>:
 145         </para>
 146         <screen>
 147           cd /home/gerard/wine
 148           patch -p 0 &lt; /home/gerard/buggedpatch1.txt
 149         </screen>
 150         <para>
 151           Beware that the committed patch is not always identical to
 152           the patch that the author sent to wine-patches, as
 153           sometimes the Wine maintainer changes things a bit.
 154         </para>
 155         <para>
 156           If you find one patch that is getting the cvs source tree to
 157           reproduce the problem, you have almost won; post the problem on
 158           <systemitem>comp.emulators.windows.wine</systemitem> and there
 159           is a chance that the author will jump in to suggest a fix; or
 160           there is always the possibility to look hard at the patch until
 161           it is coerced to reveal where is the bug :-)
 162         </para>
 163       </listitem>
 164     </orderedlist>
 165   </chapter>
 166
 167   <!-- Keep this comment at the end of the file
 168   Local variables:
 169   mode: sgml
 170   sgml-parent-document:("wine-doc.sgml" "set" "book" "part" "chapter" "")
 171   End:
 172   -->