| blob | 886cae06419b8025d6e05ae84a4472350ef9cd0f |
1 DejaGnu is a framework for testing other programs. Its purpose is to
2 provide a single front end for all tests. Beyond this, DejaGnu offers
3 several advantages for testing:
5 - The flexibility and consistency of the DejaGnu framework
6 make it easy to write tests for any program.
8 - DejaGnu provides a layer of abstraction which makes all
9 tests (if correctly written) portable to any host or target
10 where a program must be tested. For instance, a test for
11 GDB can run (from any Unix based host) on any target
12 architecture supported by DejaGnu. Currently DejaGnu runs
13 tests on several single board computers, whose operating
14 software ranges from just a boot monitor to a full-fledged,
15 Unix-like realtime OS.
17 - DejaGnu is written in expect, which in turn uses Tcl
18 (Tool command language). The framework comprises two parts:
19 the testing framework and the testsuites themselves. Tests
20 are usually written in expect using Tcl.
22 Bugs can be reported to bug-dejagnu@gnu.org.
24 How To Configure and Build
26 To build DejaGnu, run the ``configure'' script here, e.g.:
28 ./configure MYHOSTTYPE
30 followed by running ``make''. (MYHOSTTYPE is a name for your host computer,
31 for instance "sun4". You can use the script ``config.sub'' to test whether
32 a name is recognized; if it is, config.sub translates it to a triplet
33 specifying CPU, vendor, and OS.) This is used when you plan to
34 configure and build in the source tree.
36 If you use a separate tree for object files, (the recommended way),
37 then the --srcdir option must also be specified. This would also
38 require that the configure script be run from the top level directory.
40 PATH/configure MYHOSTYPE --srcdir PATH/dejagnu
42 where PATH is is the directory that the contains the sources.
44 To configure it so it gets installed somewhere other than the
45 default of /usr/local, use the --prefix option.
47 configure MYHOSTYPE --prefix [PATH]
49 where PATH is the prefix used to install the programs.
51 The configure testing and building will use the native compiler "cc"
52 on your host machine. To change which compiler gets used (like gcc)
53 set a the variable "CC" in your environment to point to it.
55 For csh users: "setenv CC gcc"
56 For bourne shell users: "CC=gcc;export CC"
58 Then when you compile, use "make CC=$CC".
60 See etc/cfg-paper.texi, etc/configure.texi, and/or the README files in
61 various subdirectories, for more details.
63 As DejaGnu is a Tcl program, there is nothing to build. However, the
64 documentation is not built by default. Use these targets:
65 "make info" - Convert the texinfo document to something that can
66 be used the GNU info program or info mode in emacs.
67 "make dvi" - Convert the texinfo document to something that can
68 be printed. This produces dvi output.
69 "make doc" - This builds both.
70 "make ps" - This converts the dvi file into postscript. This
71 requires a copy of dvips.
72 "make install" - This installs DejaGnu based on the --prefix option
73 when configuring. Otherwise it defaults to
74 /usr/local. See the DejaGnu manual for more
75 information on installation.
77 Changes from 1.1.1
78 1. Works with (included in release) Tcl 7.3 and Expect 5.6.
79 2. Much better error trapping and handling, including the
80 execution of sub scripts.
81 3. Re-worked configuration subsystem.
82 4. Default handling for testing unknown targets.
83 5. New testsuite for expect and runtest.
84 6. More debugging procedures.
86 Changes from 1.0
88 1. DejaGnu now conforms to POSIX 1003.3, a standard for
89 testing frameworks.
90 2. A Tcl debugger written by Don Libes has been added.
91 3. Lots of bug fixes.
93 Changes from 0.9
95 1. DejaGnu now installs itself like other utilities.
96 2. 700 G++ tests are included.
97 3. The bugs in the GCC tests have been fixed.
98 4. Testsuites are released separately.
99 5. Testsuite sources now reside with the within each tool's
100 source tree.
102 -------------------------------------------------------------------
103 Here's is a posting on the Free OS testing project. (FROST)
104 -------------------------------------------------------------------
105 If you have built up your own tests for system calls, library
106 routines, networking protocols, or common utilities, this posting
107 offers you an opportunity to put them to good use. You can do a good
108 deed for the Linux community, the Berkeley UNIX community, and the
109 wider world of free software users.
111 Also, if you would like to learn some good test tools and strategies,
112 and devote some time on a volunteer basis to writing tests -- perhaps
113 a couple of weeks full-time, or a couple hours per week over a period
114 of several months -- then here is a project you should get involved
115 in.
117 I am helping to coordinate a test effort for Linux. In the following
118 four sections of this message I will describe the goals, the process,
119 the people trying to do it, and what you can do to help.
121 Goals
123 Linux and BSD share a number of libraries and utilities, both
124 because BSD software was ported to Linux, and because several free
125 software (GNU) utilities have been ported to both operating
126 systems.
128 Linux shows startling differences and failures as you move from one
129 set of hardware to another. People who hope to base their own
130 commercial products on Linux would like to see proof that it is
131 robust, portable, and standard-conforming. The problem is that
132 cross-platform testing is very hard to do, and few decent test
133 suites exist either for free software or for UNIX. (Look at all the
134 differences you find among systems that are SVID-conforming,
135 particularly at higher levels such as the utilities.)
137 Therefore, a number of us who are interested n Linux have decided to
138 enter the modern age of formal, automated software testing -- but in
139 a manner that is proper for the free software community.
141 Using DejaGnu, a test platform developed by Cygnus Support and
142 placed under the GPL, we want to collect the best tests we can from
143 people everywhere, and integrate them into suites that any user can
144 download, run, and interpret with a few commands. DejaGnu already
145 runs under BSD, and Cygnus is porting it to Linux.
147 Our goal is to test as many parts of the system as possible,
148 including system calls (particularly for POSIX 1003.1 compliance),
149 libraries in the official distribution, networking protocols,
150 and utilities. We need help with
152 1) integrating good tests donated by the community (and probably
153 written in a variety of programming languages and command
154 shells) into the DejaGnu framework, and
156 2) writing new tests for major functional areas for which no
157 adequate tests exist.
159 The tests we use will be placed under the GPL and distributed by
160 Cygnus Support along with DejaGnu. H. J. Lu, who maintains the C
161 library for Linux, has offered to run the POSIX 1003.1 tests and as
162 many others as he can.
164 Process
166 First let me summarize the advantages of using DejaGnu, a free
167 software product that will be the umbrella for all the tests, and
168 then explain how we plan to conduct this project.
170 Cygnus Support released DejaGnu to the public on January 3, along
171 with several test suites for GNU language tools. The product is
172 designed for portability and easy cross-platform development and
173 execution. It works more uniformly than typical tests using the
174 UNIX shell, and also supports interactive testing better than most
175 test platforms -- for instance, Cygnus has written over 1300 unit
176 tests for the gdb debugger using it.
178 The implementation of DejaGnu is based on tcl and expect, two simple
179 existing languages for writing commands. You can develop new tests
180 in two major ways: by placing keyword-based comments in C source
181 code, or by writing tests in tcl.
183 While Cygnus is still increasing their body of tests for language
184 tools, they are also turning their resources toward the libraries
185 and GNU utilities. While most of the tests are written at Cygnus,
186 they have reached out to communities of testers and are now porting
187 substantial sets of donated tests.
189 We are hoping to broaden this successful use of collaboration across
190 user communities. We hope that suites of useful tests are sitting
191 in desk drawers out there. We also hope to gather and mobilize
192 people who appreciate the value of formal tests in legitimizing free
193 software, and would like to help write them.
195 I am not an employee of Cygnus, but have volunteered to talk to
196 interested people and do an initial classification of tests, just to
197 offload some of this routine work from them. I will discuss all
198 offers with Cygnus staff to find out what is most needed and decide
199 which tests to incorporate into DejaGnu. There are several criteria
200 for choosing tests, including the degree to which an area is
201 covered, and its importance in the overall stability of the
202 operating system. Some tests may have to be rejected just because
203 they are hard to fit into the DejaGnu model.
205 People
207 Now you can find out why each of us got involved with this project.
209 Cygnus writes and distributes free software, selling support as a
210 means to make money. The company is not directly in the business of
211 supporting operating systems or common UNIX utilities, so this kind
212 of testing is tangential to their main goals. But they may want to
213 support those things in the future. In any case, they would like to
214 see Linux do well, and this big test project will be a good
215 promotion for DejaGnu.
216 H. K. Lu, as a volunteer for Linux, has been using his own system to
217 implement, port, maintain, and ensure ANSI and POSIX compliance for
218 the C library (mostly the GNU C library along with the iostream
219 class from the g++ library). He is looking for ways to validate the
220 work that he and many other contributors have put in.
222 As an editor at a publisher of computer books, my relationship to
223 Linux and BSD is even more distant than that of Cygnus. But we are
224 thinking about putting out books about Linux, from either the Linux
225 Documentation Project or independent authors, and would like to make
226 sure Linux is stable enough to be documented. The testing of
227 utilities is particularly important to me, because it can provide
228 quality assurance for our books, including the BSD version of UNIX
229 in a Nutshell.
231 What you can do
233 We ask people who have written tests in the areas I have described
234 to donate them to this project and see them benefit the public.
235 Don't just send me stuff -- write or call to discuss what you've got
236 in general. You also have to assign the tests to the Free Software
237 Foundation (a simple matter, so long as you are the owner of the
238 tests) so that they can be distributed under the GPL.
240 Also, please don't starting flaming about the GPL. Either join our
241 project and donate your tests, or don't. Personally, I do not take
242 a side in this war.
244 People who are interested in testing as a discipline, and have some
245 time to donate, are invited to contact me to help write tests. Tell
246 me:
248 Your knowledge of computer languages and tools
250 Previous software test efforts you have engaged in, if any
252 How much time you can devote
254 What functional areas you think are important and have a
255 particular interest in
257 Cygnus staff will tell volunteers which areas need testing, and
258 offer guidance while you learn and apply DejaGnu. The most
259 important trait for the project is rigorous thinking along the lines
260 of software quality (so the best preparation is to have done some
261 formal testing before). You do not need to know any particular
262 language or tool; you can learn tcl or other aspects of DejaGnu
263 fairly quickly.
265 Contact information for me is in my signature. I am at the phone
266 number during traditional U.S. business hours. So send mail any
267 time, or try calling from Monday through Friday, 9:00 AM to 5:00 PM,
268 Eastern Daylight Saving Time (four hours later than Greenwich Mean
269 Time).
271 Feel free to distribute and repost this message elsewhere, in its
272 entirety.
274 ----------------------------------------------------------------------
275 Andy Oram O'Reilly & Associates, Inc. andyo@ora.com
276 90 Sherman Street, Cambridge, MA 02140 (617) 354-5800
277 fax (617) 661-1116
278 ----------------------------------------------------------------------
280 There is a new listserv based mailing list for the FROST project. To
281 subscribe, send email to "listserv@cygnus.com" and the body should
282 contain "SUBSCRIBE FROST <your name>". Messages sent to
283 frost@cygnus.com go to the mailing list.