doc/overview/hints.html

   1 <HTML
   2 ><HEAD
   3 ><TITLE
   4 >Hints On Writing A Test Case</TITLE
   5 ><META
   6 NAME="GENERATOR"
   7 CONTENT="Modular DocBook HTML Stylesheet Version 1.44"><LINK
   8 REL="HOME"
   9 TITLE="DejaGnu"
  10 HREF="book1.html"><LINK
  11 REL="UP"
  12 TITLE="Extending DejaGnu"
  13 HREF="extending.html"><LINK
  14 REL="PREVIOUS"
  15 TITLE="Adding A Test Case To A Test Suite."
  16 HREF="adding.html"><LINK
  17 REL="NEXT"
  18 TITLE="Special variables used by test cases."
  19 HREF="tvariables.html"></HEAD
  20 ><BODY
  21 ><DIV
  22 CLASS="NAVHEADER"
  23 ><TABLE
  24 WIDTH="100%"
  25 BORDER="0"
  26 CELLPADDING="0"
  27 CELLSPACING="0"
  28 ><TR
  29 ><TH
  30 COLSPAN="3"
  31 ALIGN="center"
  32 >DejaGnu: The GNU Testing Framework</TH
  33 ></TR
  34 ><TR
  35 ><TD
  36 WIDTH="10%"
  37 ALIGN="left"
  38 VALIGN="bottom"
  39 ><A
  40 HREF="adding.html"
  41 >Prev</A
  42 ></TD
  43 ><TD
  44 WIDTH="80%"
  45 ALIGN="center"
  46 VALIGN="bottom"
  47 >Chapter 4. Extending DejaGnu</TD
  48 ><TD
  49 WIDTH="10%"
  50 ALIGN="right"
  51 VALIGN="bottom"
  52 ><A
  53 HREF="tvariables.html"
  54 >Next</A
  55 ></TD
  56 ></TR
  57 ></TABLE
  58 ><HR
  59 ALIGN="LEFT"
  60 WIDTH="100%"></DIV
  61 ><DIV
  62 CLASS="SECT1"
  63 ><H1
  64 CLASS="SECT1"
  65 ><A
  66 NAME="HINTS"
  67 >Hints On Writing A Test Case</A
  68 ></H1
  69 ><P
  70 >It is safest to write patterns that match all the output
  71       generated by the tested program; this is called closure.
  72       If a pattern does not match the entire output, any output that
  73       remains will be examined by the next <B
  74 CLASS="COMMAND"
  75 >expect</B
  76 >
  77       command. In this situation, the precise boundary that determines
  78       which <B
  79 CLASS="COMMAND"
  80 >expect</B
  81 > command sees what is very
  82       sensitive to timing between the Expect task and the task running
  83       the tested tool.  As a result, the test may sometimes appear to
  84       work, but is likely to have unpredictable results. (This problem
  85       is particularly likely for interactive tools, but can also
  86       affect batch tools---especially for tests that take a long time
  87       to finish.) The best way to ensure closure is to use the
  88       <TT
  89 CLASS="OPTION"
  90 >-re</TT
  91 > option for the <B
  92 CLASS="COMMAND"
  93 >expect</B
  94 >
  95       command to write the pattern as a full regular expressions; then
  96       you can match the end of output using a <I
  97 CLASS="EMPHASIS"
  98 >$</I
  99 >.
 100       It is also a good idea to write patterns that match all
 101       available output by using <I
 102 CLASS="EMPHASIS"
 103 >.*\</I
 104 > after the
 105       text of interest; this will also match any intervening blank
 106       lines.  Sometimes an alternative is to match end of line using
 107       <I
 108 CLASS="EMPHASIS"
 109 >\r</I
 110 > or <I
 111 CLASS="EMPHASIS"
 112 >\n</I
 113 >, but this is
 114       usually too dependent on terminal settings.</P
 115 ><P
 116 >Always escape punctuation, such as <I
 117 CLASS="EMPHASIS"
 118 >(</I
 119 >
 120       or <I
 121 CLASS="EMPHASIS"
 122 >"</I
 123 >, in your patterns; for example, write
 124       <I
 125 CLASS="EMPHASIS"
 126 >\(</I
 127 >.  If you forget to escape punctuation,
 128       you will usually see an error message like <TABLE
 129 BORDER="0"
 130 BGCOLOR="#E0E0E0"
 131 WIDTH="100%"
 132 ><TR
 133 ><TD
 134 ><PRE
 135 CLASS="PROGRAMLISTING"
 136 >extra
 137       characters after close-quote.</PRE
 138 ></TD
 139 ></TR
 140 ></TABLE
 141 ></P
 142 ><P
 143 >If you have trouble understanding why a pattern does not
 144       match the program output, try using the <TT
 145 CLASS="OPTION"
 146 >--debug</TT
 147 >
 148       option to <B
 149 CLASS="COMMAND"
 150 >runtest</B
 151 >, and examine the debug log
 152       carefully.</P
 153 ><P
 154 >Be careful not to neglect output generated by setup rather
 155       than by the interesting parts of a test case.  For example,
 156       while testing GDB, I issue a send <I
 157 CLASS="EMPHASIS"
 158 >set height
 159       0\n</I
 160 > command.  The purpose is simply to make sure GDB
 161       never calls a paging program.  The <I
 162 CLASS="EMPHASIS"
 163 >set
 164       height</I
 165 > command in GDB does not generate any
 166       output; but running any command makes GDB issue a new
 167       <I
 168 CLASS="EMPHASIS"
 169 >(gdb) </I
 170 > prompt.  If there were no
 171       <B
 172 CLASS="COMMAND"
 173 >expect</B
 174 > command to match this prompt, the
 175       output <I
 176 CLASS="EMPHASIS"
 177 >(gdb) </I
 178 > begins the text seen by the
 179       next <B
 180 CLASS="COMMAND"
 181 >expect</B
 182 > command---which might make that
 183       pattern fail to match.</P
 184 ><P
 185 >To preserve basic sanity, I also recommended that no test
 186       ever pass if there was any kind of problem in the test case.  To
 187       take an extreme case, tests that pass even when the tool will
 188       not spawn are misleading. Ideally, a test in this sort of
 189       situation should not fail either. Instead, print an error
 190       message by calling one of the DejaGnu procedures
 191       <B
 192 CLASS="COMMAND"
 193 >error</B
 194 > or <B
 195 CLASS="COMMAND"
 196 >warning</B
 197 >.</P
 198 ></DIV
 199 ><DIV
 200 CLASS="NAVFOOTER"
 201 ><HR
 202 ALIGN="LEFT"
 203 WIDTH="100%"><TABLE
 204 WIDTH="100%"
 205 BORDER="0"
 206 CELLPADDING="0"
 207 CELLSPACING="0"
 208 ><TR
 209 ><TD
 210 WIDTH="33%"
 211 ALIGN="left"
 212 VALIGN="top"
 213 ><A
 214 HREF="adding.html"
 215 >Prev</A
 216 ></TD
 217 ><TD
 218 WIDTH="34%"
 219 ALIGN="center"
 220 VALIGN="top"
 221 ><A
 222 HREF="book1.html"
 223 >Home</A
 224 ></TD
 225 ><TD
 226 WIDTH="33%"
 227 ALIGN="right"
 228 VALIGN="top"
 229 ><A
 230 HREF="tvariables.html"
 231 >Next</A
 232 ></TD
 233 ></TR
 234 ><TR
 235 ><TD
 236 WIDTH="33%"
 237 ALIGN="left"
 238 VALIGN="top"
 239 >Adding A Test Case To A Test Suite.</TD
 240 ><TD
 241 WIDTH="34%"
 242 ALIGN="center"
 243 VALIGN="top"
 244 ><A
 245 HREF="extending.html"
 246 >Up</A
 247 ></TD
 248 ><TD
 249 WIDTH="33%"
 250 ALIGN="right"
 251 VALIGN="top"
 252 >Special variables used by test cases.</TD
 253 ></TR
 254 ></TABLE
 255 ></DIV
 256 ></BODY
 257 ></HTML
 258 >