1 HTTP Server for Unit Tests
2 ==========================
4 This page describes the JavaScript implementation of an
5 HTTP server located in ``netwerk/test/httpserver/``.
10 Here are some of the things you can do with the server:
12 - map a directory of files onto an HTTP path on the server, for an
13 arbitrary number of such directories (including nested directories)
14 - define custom error handlers for HTTP error codes
15 - serve a given file for requests for a specific path, optionally with
16 custom headers and status
17 - define custom "CGI" handlers for specific paths using a
18 JavaScript-based API to create the response (headers and actual
20 - run multiple servers at once on different ports (8080, 8081, 8082,
23 This functionality should be more than enough for you to use it with any
24 test which requires HTTP-provided behavior.
29 The server is written primarily for use from ``xpcshell``-based
30 tests, and it can be used as an inline script or as an XPCOM component. The
31 Mochitest framework also uses it to serve its tests, and
32 `reftests <https://searchfox.org/mozilla-central/source/layout/tools/reftest/README.txt>`__
33 can optionally use it when their behavior is dependent upon specific
39 - application update testing
40 - cross-"server" security tests
41 - cross-domain security tests, in combination with the right proxy
42 settings (for example, using `Proxy
43 AutoConfig <https://en.wikipedia.org/wiki/Proxy_auto-config>`__)
44 - tests where the behavior is dependent on the values of HTTP headers
45 (for example, Content-Type)
46 - anything which requires use of files not stored locally
47 - open-id : the users could provide their own open id server (they only
48 need it when they're using their browser)
49 - micro-blogging : users could host their own micro blog based on
50 standards like RSS/Atom
51 - rest APIs : web application could interact with REST or SOAP APIs for
52 many purposes like : file/data storage, social sharing and so on
58 The best and first place you should look for documentation is
59 ``netwerk/test/httpserver/nsIHttpServer.idl``. It's extremely
60 comprehensive and detailed, and it should be enough to figure out how to
61 make the server do what you want. I also suggest taking a look at the
62 less-comprehensive server
63 `README <https://searchfox.org/mozilla-central/source/netwerk/test/httpserver/README>`__,
64 although the IDL should usually be sufficient.
69 From test suites, the server should be importable as a testing-only JS
74 ChromeUtils.import("resource://testing-common/httpd.js");
76 Once you've done that, you can create a new server as follows:
80 let server = new HttpServer(); // Or nsHttpServer() if you don't use ChromeUtils.import.
82 server.registerDirectory("/", nsILocalFileForBasePath);
84 server.start(-1); // uses a random available port, allows us to run tests concurrently
85 const SERVER_PORT = server.identity.primaryPort; // you can use this further on
87 // and when the tests are done, most likely from a callback...
88 server.stop(function() { /* continue execution here */ });
90 You can also pass in a numeric port argument to the ``start()`` method,
91 but we strongly suggest you don't do it. Using a dynamic port allow us
92 to run your test in parallel with other tests which reduces wait times
93 and makes everybody happy. If you really have to use a hardcoded port,
94 you will have to annotate your test in the xpcshell manifest file with
95 ``run-sequentially = REASON``.
96 However, this should only be used as the last possible option.
100 You **must** make sure to stop the server (the last line above)
101 before your test completes. Failure to do so will result in the
102 "XPConnect is being called on a scope without a Components property"
103 assertion, which will cause your test to fail in debug builds, and
104 you'll make people running tests grumbly because you've broken the
110 The server's default error pages don't give much information, partly
111 because the error-dispatch mechanism doesn't currently accommodate doing
112 so and partly because exposing errors in a real server could make it
113 easier to exploit them. If you don't know why the server is acting a
115 `httpd.js <https://searchfox.org/mozilla-central/source/netwerk/test/httpserver/httpd.js>`__
116 and change the value of ``DEBUG`` to ``true``. This will cause the
117 server to print information about the processing of requests (and errors
118 encountered doing so) to the console, and it's usually not difficult to
119 determine why problems exist from that output. ``DEBUG`` is ``false`` by
120 default because the information printed with it set to ``true``
121 unnecessarily obscures tinderbox output.
123 Header Modification for Files
124 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
126 The server supports modifying the headers of the files (not request
127 handlers) it serves. To modify the headers for a file, create a sibling
128 file with the first file's name followed by ``^headers^``. Here's an
129 example of how such a file might look:
133 HTTP 404 I want a cool HTTP description!
134 Content-Type: text/plain
136 The status line is optional; all other lines specify HTTP headers in the
137 standard HTTP format. Any line ending style is accepted, and the file
138 may optionally end with a single newline character, to play nice with
139 Unix text tools like ``diff`` and ``hg``.
144 Any file which ends with a single ``^`` is inaccessible when querying
145 the web server; if you try to access such a file you'll get a
146 ``404 File Not Found`` page instead. If for some reason you need to
147 serve a file ending with a ``^``, just tack another ``^`` onto the end
148 of the file name and the file will then become available at the
149 single-``^`` location.
151 At the moment this feature is basically a way to smuggle header
152 modification for files into the file system without making those files
153 accessible to clients; it remains to be seen whether and how hidden-file
154 capabilities will otherwise be used.
156 SJS: Server-Side Scripts
157 ^^^^^^^^^^^^^^^^^^^^^^^^
159 Support for server-side scripts is provided through the SJS mechanism.
160 Essentially an SJS is a file with a particular extension, chosen by the
161 creator of the server, which contains a function with the name
162 ``handleRequest`` which is called to determine the response the server
163 will generate. That function acts exactly like the ``handle`` function
164 on the ``nsIHttpRequestHandler`` interface. First, tell the server what
165 extension you're using:
169 const SJS_EXTENSION = "cgi";
170 server.registerContentType(SJS_EXTENSION, "sjs");
172 Now just create an SJS with the extension ``cgi`` and write whatever you
177 function handleRequest(request, response)
179 response.setStatusLine(request.httpVersion, 200, "OK");
180 response.write("Hello world! This request was dynamically " +
181 "generated at " + new Date().toUTCString());
184 Further examples may be found `in the Mozilla source
185 tree <https://searchfox.org/mozilla-central/search?q=&path=.sjs>`__
186 in existing tests. The request object is an instance of
187 ``nsIHttpRequest`` and the response is a ``nsIHttpResponse``.
188 Please refer to the `IDL
189 documentation <https://searchfox.org/mozilla-central/source/netwerk/test/httpserver/nsIHttpServer.idl>`
192 Storing Information Across Requests
193 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
195 HTTP is basically a stateless protocol, and the httpd.js server API is
196 for the most part similarly stateless. If you're using the server
197 through the XPCOM interface you can simply store whatever state you want
198 in enclosing environments or global variables. However, if you're using
199 it through an SJS your request is processed in a near-empty environment
200 every time processing occurs. To support stateful SJS behavior, the
201 following functions have been added to the global scope in which a SJS
202 handler executes, providing a simple key-value state storage mechanism:
207 * v : T means v is of type T
208 * function A() : T means A() has type T
211 function getState(key : string) : string
212 function setState(key : string, value : string)
213 function getSharedState(key : string) : string
214 function setSharedState(key : string, value : string)
215 function getObjectState(key : string, callback : function(value : object) : void) // SJS API, XPCOM differs, see below
216 function setObjectState(key : string, value : object)
218 A key is a string with arbitrary contents. The corresponding value is
219 also a string, for the non-object-saving functions. For the
220 object-saving functions, it is (wait for it) an object, or also
221 ``null``. Initially all keys are associated with the empty string or
222 with ``null``, depending on whether the function accesses string- or
223 object-valued storage. A stored value persists across requests and
224 across server shutdowns and restarts. The state methods are available
225 both in SJS and, for convenience when working with the server both via
226 XPCOM and via SJS, XPCOM through the ``nsIHttpServer`` interface. The
227 variants are designed to support different needs.
231 **Warning:** Be careful using state: you, the user, are responsible
232 for synchronizing all uses of state through any of the available
233 methods. (This includes the methods that act only on per-path state:
234 you might still run into trouble there if your request handler
235 generates responses asynchronously. Further, any code with access to
236 the server XPCOM component could modify it between requests even if
237 you only ever used or modified that state while generating
238 synchronous responses.) JavaScript's run-to-completion behavior will
239 save you in simple cases, but with anything moderately complex you
240 are playing with fire, and if you do it wrong you will get burned.
242 ``getState`` and ``setState``
243 '''''''''''''''''''''''''''''
245 ``getState`` and ``setState`` are designed for the case where a single
246 request handler needs to store information from a first request of it
247 for use in processing a second request of it — say, for example, if you
248 wanted to implement a request handler implementing a counter:
253 * Generates a response whose body is "0", "1", "2", and so on. each time a
254 * request is made. (Note that browser caching might make it appear
255 * to not quite have that behavior; a Cache-Control header would fix
256 * that issue if desired.)
258 function handleRequest(request, response)
260 var counter = +getState("counter"); // convert to number; +"" === 0
261 response.write("" + counter);
262 setState("counter", "" + ++counter);
265 The useful feature of these two methods is that this state doesn't bleed
266 outside the single path at which it resides. For example, if the above
267 SJS were at ``/counter``, the value returned by ``getState("counter")``
268 at some other path would be completely distinct from the counter
269 implemented above. This makes it much simpler to write stateful handlers
270 without state accidentally bleeding between unrelated handlers.
274 State saved by this method is specific to the HTTP path,
275 excluding query string and hash reference. ``/counter``,
276 ``/counter?foo``, and ``/counter?bar#baz`` all share the same state
277 for the purposes of these methods. (Indeed, non-shared state would be
278 significantly less useful if it changed when the query string
283 The predefined ``__LOCATION__`` state
284 contains the native path of the SJS file itself. You can pass the
285 result directly to the ``nsILocalFile.initWithPath()``. Example:
286 ``thisSJSfile.initWithPath(getState('__LOCATION__'));``
288 ``getSharedState`` and ``setSharedState``
289 '''''''''''''''''''''''''''''''''''''''''
291 ``getSharedState`` and ``setSharedState`` make up the functionality
292 intentionally not supported by ``getState`` and set\ ``State``: state
293 that exists between different paths. If you used the above handler at
294 the paths ``/sharedCounters/1`` and ``/sharedCounters/2`` (changing the
295 state-calls to use shared state, of course), the first load of either
296 handler would return "0", a second load of either handler would return
297 "1", a third load either handler would return "2", and so on. This more
298 powerful functionality allows you to write cooperative handlers that
299 expose and manipulate a piece of shared state. Be careful! One test can
300 screw up another test pretty easily if it's not careful what it does
301 with this functionality.
303 ``getObjectState`` and ``setObjectState``
304 '''''''''''''''''''''''''''''''''''''''''
306 ``getObjectState`` and ``setObjectState`` support the remaining
307 functionality not provided by the above methods: storing non-string
308 values (object values or ``null``). These two methods are the same as
309 ``getSharedState`` and ``setSharedState``\ in that state is visible
310 across paths; ``setObjectState`` in one handler will expose that value
311 in another handler that uses ``getObjectState`` with the same key. (This
312 choice was intentional, because object values already expose mutable
313 state that you have to be careful about using.) This functionality is
314 particularly useful for cooperative request handlers where one request
315 *suspends* another, and that second request must then be *resumed* at a
316 later time by a third request. Without object-valued storage you'd need
317 to resort to polling on a string value using either of the previous
318 state APIs; with this, however, you can make precise callbacks exactly
319 when a particular event occurs.
321 ``getObjectState`` in an SJS differs in one important way from
322 ``getObjectState`` accessed via XPCOM. In XPCOM the method takes a
323 single string argument and returns the object or ``null`` directly. In
324 SJS, however, the process to return the value is slightly different:
328 function handleRequest(request, response)
330 var key = request.hasHeader("key")
331 ? request.getHeader("key")
334 getObjectState(key, function(objval)
336 // This function is called synchronously with the object value
337 // associated with key.
340 response.write("Keyed object " +
341 (obj && Object.prototype.hasOwnProperty.call(obj, "doStuff")
343 : "does not have ") +
344 "a doStuff method.");
347 This idiosyncratic API is a restriction imposed by how sandboxes
348 currently work: external functions added to the sandbox can't return
349 object values when called within the sandbox. However, such functions
350 can accept and call callback functions, so we simply use a callback
351 function here to return the object value associated with the key.
353 Advanced Dynamic Response Creation
354 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
356 The default behavior of request handlers is to fully construct the
357 response, return, and only then send the generated data. For certain use
358 cases, however, this is infeasible. For example, a handler which wanted
359 to return an extremely large amount of data (say, over 4GB on a 32-bit
360 system) might run out of memory doing so. Alternatively, precise control
361 over the timing of data transmission might be required so that, for
362 example, one request is received, "paused" while another request is
363 received and completes, and then finished. httpd.js solves this problem
364 by defining a ``processAsync()`` method which indicates to the server
365 that the response will be written and finished by the handler. Here's an
366 example of an SJS file which writes some data, waits five seconds, and
367 then writes some more data and finishes the response:
373 function handleRequest(request, response)
375 response.processAsync();
376 response.setHeader("Content-Type", "text/plain", false);
377 response.write("hello...");
379 timer = Cc["@mozilla.org/timer;1"].createInstance(Ci.nsITimer);
380 timer.initWithCallback(function()
382 response.write("world!");
384 }, 5 * 1000 /* milliseconds */, Ci.nsITimer.TYPE_ONE_SHOT);
387 The basic flow is simple: call ``processAsync`` to mark the response as
388 being sent asynchronously, write data to the response body as desired,
389 and when complete call ``finish()``. At the moment if you drop such a
390 response on the floor, nothing will ever terminate the connection, and
391 the server cannot be stopped (the stop API is asynchronous and
392 callback-based); in the future a default connection timeout will likely
393 apply, but for now, "don't do that".
395 Full documentation for ``processAsync()`` and its interactions with
396 other methods may, as always, be found in
397 ``netwerk/test/httpserver/nsIHttpServer.idl``.
399 Manual, Arbitrary Response Creation
400 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
402 The standard mode of response creation is fully synchronous and is
403 guaranteed to produce syntactically correct responses (excluding
404 headers, which for the most part may be set to arbitrary values).
405 Asynchronous processing enables the introduction of response handling
406 coordinated with external events, but again, for the most part only
407 syntactically correct responses may be generated. The third method of
408 processing removes the correct-syntax property by allowing a response to
409 contain completely arbitrary data through the ``seizePower()`` method.
410 After this method is called, any data subsequently written to the
411 response is written directly to the network as the response, skipping
412 headers and making no attempt whatsoever to ensure any formatting of the
413 transmitted data. As with asynchronous processing, the response is
414 generated asynchronously and must be finished manually for the
415 connection to be closed. (Again, nothing will terminate the connection
416 for a response dropped on the floor, so again, "don't do that".) This
417 mode of processing is useful for testing particular data formats that
418 are either not HTTP or which do not match the precise, canonical
419 representation that httpd.js generates. Here's an example of an SJS file
420 which writes an apparent HTTP response whose status text contains a null
421 byte (not allowed by HTTP/1.1, and attempting to set such status text
422 through httpd.js would throw an exception) and which has a header that
423 spans multiple lines (httpd.js responses otherwise generate only
424 single-line headers):
428 function handleRequest(request, response)
430 response.seizePower();
431 response.write("HTTP/1.1 200 OK Null byte \u0000 makes this response malformed\r\n" +
432 "X-Underpants-Gnomes-Strategy:\r\n" +
433 " Phase 1: Collect underpants.\r\n" +
434 " Phase 2: ...\r\n" +
435 " Phase 3: Profit!\r\n" +
441 While the asynchronous mode is capable of producing certain forms of
442 invalid responses (through setting a bogus Content-Length header prior
443 to the start of body transmission, among others), it must not be used in
444 this manner. No effort will be made to preserve such implementation
445 quirks (indeed, some are even likely to be removed over time): if you
446 want to send malformed data, use ``seizePower()`` instead.
448 Full documentation for ``seizePower()`` and its interactions with other
449 methods may, as always, be found in
450 ``netwerk/test/httpserver/nsIHttpServer.idl``.
452 Example Uses of the Server
453 ~~~~~~~~~~~~~~~~~~~~~~~~~~
455 Shorter examples (for tests which only do one test):
457 - ``netwerk/test/unit/test_bug331825.js``
458 - ``netwerk/test/unit/test_httpcancel.js``
459 - ``netwerk/test/unit/test_cookie_header.js``
461 Longer tests (where you'd need to do multiple async server requests):
463 - ``netwerk/test/httpserver/test/test_setstatusline.js``
464 - ``netwerk/test/unit/test_content_sniffer.js``
465 - ``netwerk/test/unit/test_authentication.js``
466 - ``netwerk/test/unit/test_event_sink.js``
467 - ``netwerk/test/httpserver/test/``
469 Examples of modifying HTTP headers in files may be found at
470 ``netwerk/test/httpserver/test/data/cern_meta/``.
475 The server, while very functional, is not yet complete. There are a
476 number of things to fix and features to add, among them support for
477 pipelining, support for incrementally-received requests (rather than
478 buffering the entire body before invoking a request handler), and better
479 conformance to the MUSTs and SHOULDs of HTTP/1.1. If you have
480 suggestions for functionality or find bugs, file them in
481 `Testing-httpd.js <https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=General>`__