documentation/cvs-regression.sgml

   1   <chapter id="cvs-regression">
   2     <title>How to do regression testing using CVS</title>
   3
   4     <para>
   5       written by Gerard Patel
   6     </para>
   7     <para>
   8       (Extracted from <filename>wine/documentation/bugreports</filename>)
   9     </para>
  10
  11     <para>
  12       A problem that can happen sometimes is 'it used to work
  13       before, now it doesn't anymore...'. Here is a step by step
  14       procedure to try to pinpoint when the problem occurred. This is
  15       <emphasis>NOT</emphasis> for casual users.
  16     </para>
  17
  18     <orderedlist>
  19       <listitem>
  20         <para>
  21           Get the <quote>full CVS</quote> archive from winehq. This archive is
  22           the CVS tree but with the tags controlling the versioning
  23           system. It's a big file (> 40 meg) with a name like
  24           wine-cvsdirs-&lt;last update date> (it's more than 100mb
  25           when uncompressed, you can't very well do this with
  26           small, old computers or slow Internet connections).
  27         </para>
  28       </listitem>
  29       <listitem>
  30         <para>
  31           untar it into a repository directory:
  32           <screen>
  33 cd /home/gerard
  34 tar -zxf cvs-dirs-2003-01-15.tar.gz
  35 mv wine repository
  36           </screen>
  37         </para>
  38       </listitem>
  39       <listitem>
  40         <para>
  41           extract a new destination directory. This directory must
  42           not be in a subdirectory of the repository else
  43           <command>cvs</command> will think it's part of the
  44           repository and deny you an extraction in the repository:
  45           <screen>
  46 cd /home/gerard
  47 mv wine wine_current (-> this protects your current wine sandbox, if any)
  48 export CVSROOT=/home/gerard/repository
  49 cd /home/gerard
  50 cvs -d $CVSROOT checkout wine
  51           </screen>
  52         </para>
  53         <para>
  54           Note that it's not possible to do a checkout at a given
  55           date; you always do the checkout for the last date where
  56           the wine-cvsdirs-xxx snapshot was generated.
  57         </para>
  58         <para>
  59           Note also that it is possible to do all this with a direct
  60           CVS connection, of course. The full CVS file method is less
  61           painful for the Winehq CVS server and probably a bit faster
  62           if you don't have a very good net connection.
  63         </para>
  64         <note>
  65           <para>
  66             If you use CVS directly from the winehq.com server, do not
  67             forget to add to your <filename>.cvsrc</filename> file:
  68           </para>
  69           <screen>
  70 cvs -z 3
  71 update -dPA
  72 diff -u
  73           </screen>
  74          </note>
  75       </listitem>
  76       <listitem>
  77         <para>
  78           you will have now in the <filename>~/wine</filename>
  79           directory an image of the CVS tree, on the client side.
  80           Now update this image to the date you want:
  81           <screen>
  82 cd /home/gerard/wine
  83 cvs -d $CVSROOT update -D "2002-06-01 CST"
  84           </screen>
  85         </para>
  86         <para>
  87           The date format is <literal>YYYY-MM-DD HH:MM:SS</literal>.
  88           Using the CST date format ensure that you will be able to
  89           extract patches in a way that will be compatible with the
  90           wine-cvs archive
  91           <ulink url="http://www.winehq.com/hypermail/wine-cvs">
  92           http://www.winehq.com/hypermail/wine-cvs</ulink>
  93         </para>
  94         <para>
  95           Many messages will inform you that more recent files have
  96           been deleted to set back the client cvs tree to the date
  97           you asked, for example:
  98           <screen>
  99 cvs update: tsx11/ts_xf86dga2.c is no longer in the repository
 100           </screen>
 101         </para>
 102         <para>
 103           <command>cvs update</command> is not limited to upgrade to
 104           a <emphasis>newer</emphasis> version as I have believed for
 105           far too long :-(
 106         </para>
 107       </listitem>
 108       <listitem>
 109         <para>
 110           Now proceed as for a normal update:
 111         </para>
 112         <screen>
 113 ./configure
 114 make depend && make
 115         </screen>
 116         <para>
 117           If any non-programmer reads this, the fastest method to get
 118           at the point where the problem occurred is to use a binary
 119           search, that is, if the problem occurred in 1999, start at
 120           mid-year, then is the problem is already here, back to 1st
 121           April, if not, to 1st October, and so on.
 122         </para>
 123         <para>
 124           If you have lot of hard disk free space (a full compile currently
 125           takes 400 Mb), copy the oldest known working version before
 126           updating it, it will save time if you need to go back. (it's
 127           better to <command>make distclean</command> before going back in
 128           time, so you have to make everything if you don't backup the older
 129           version)
 130         </para>
 131         <para>
 132           When you have found the day where the problem happened, continue
 133           the search using the wine-cvs archive (sorted by date) and a
 134           more precise cvs update including hour, minute, second :
 135           <screen>
 136 cvs -d $CVSROOT update -D "2002-06-01 15:17:25 CST"
 137           </screen>
 138           This will allow you to find easily the exact patch that did it.
 139         </para>
 140       </listitem>
 141       <listitem>
 142         <para>
 143           If you find the patch that is the cause of the problem, you have
 144           almost won; report about it to
 145           <ulink url="http://bugs.winehq.com/">Wine Bugzilla</ulink>
 146           or subscribe to wine-devel and post it there. There is a chance
 147           that the author
 148           will jump in to suggest a fix; or there is always the possibility
 149           to look hard at the patch until it is coerced to reveal where is
 150           the bug :-)
 151         </para>
 152       </listitem>
 153     </orderedlist>
 154   </chapter>
 155
 156   <!-- Keep this comment at the end of the file
 157   Local variables:
 158   mode: sgml
 159   sgml-parent-document:("wine-devel.sgml" "set" "book" "part" "chapter" "")
 160   End:
 161   -->