1 Network Programming in ANSI Common Lisp with IOLib
2 by: Peter Keller (psilord@cs.wisc.edu)
10 IOLib is a portable I/O library for ANSI Common Lisp. It includes socket
11 interfaces for network programming with IPV4/IPV6 TCP and UDP, an I/O
12 multiplexer that includes nonblocking I/O, a DNS resolver library, and a
18 The current version of IOLib is found here:
20 http://common-lisp.net/project/iolib/download.shtml
22 Please use the repository located in the Live Sources section for the most up
23 to date version of IOLib.
28 This tutorial loosely follows the exposition of network programming in "UNIX
29 Network Programming, Networking APIs: Sockets and XTI 2nd Edition" by W.
30 Richard Stevens. Many examples are derived from the source codes in that book.
31 Major deviations from the C sources include converting the concurrent examples
32 which use fork() into threaded examples which use the portable Bordeaux Threads
33 package, more structured implementations of certain concepts such as data
34 buffers and error handling, and general movement of coding style towards a
35 Common Lisp viewpoint.
37 The scope of this version of the tutorial is:
38 0. Exposition suitable for programmers unfamiliar with ANSI Common Lisp
40 2. Client/Server architecture
41 3. Iterative vs Concurrent (via threading) vs Multiplexed Server Design
42 4. Blocking and nonblocking I/O
44 It is intended, however, that this tutorial grows to contain the entirety of
45 IOLib's API as detailed in the Future Directions section of this tutorial. As
46 newer revisions of this tutorial are released, those gaps will be filled until
47 the whole of the IOLib API has been discussed.
49 Finally, the example code in this tutorial is algorithmically cut from the
50 actual example programs and inserted into the tutorial via a template
51 generation method. The example codes have embedded in them a tiny markup
52 language which facilitates this in the form (on a single line) of ';; ex-NNNb'
53 to begin an example section, and ';; ex-NNNe' to end an example section--NNN
54 stands for an enumeration integer for which each section's begin and end must
60 I would like to greatly thank Stelian Ionescu, the author of IOLib
61 for his exposition of the various features of IOLib and his patience
62 in our sometimes long conversations.
67 The file package.lisp contains a small library of codes used widely in the
68 examples. The supporting code implements:
70 0. The package containing the examples, called :iolib.examples.
72 1. The variables *host* and *port*, set to "localhost" and 9999
73 respectively. This is the default name and port to which
74 client connect and servers listen. Servers usually bind
77 2. A small, but efficient, queue implementation, from "ANSI Common Lisp"
78 by Paul Graham. The interface calls are:
84 3. :iolib.examples currently depends upon IOLib alone and uses
85 packages :common-lisp, :iolib, and :bordeaux-threads.
90 These examples were developed and tested on SBCL 1.0.33.30 running on an x86
91 Ubuntu 8.10 machine. They were ran with two sessions of SBCL running, one
92 acting as a client, and the other as a server.
94 Supposing we'd like to start up the first example of the daytime server and
95 connect to it with the first daytime client example. Initially, the server will
96 bind to *host* and *port* and wait for the client to connect. We connect with
97 the client to *host* and *port*, get the time, and exit.
99 First we'll start up a server:
102 This is SBCL 1.0.33.30, an implementation of ANSI Common Lisp.
103 More information about SBCL is available at <http://www.sbcl.org/>.
105 SBCL is free software, provided as is, with absolutely no warranty.
106 It is mostly in the public domain; some portions are provided under
107 BSD-style licenses. See the CREDITS and COPYING files in the
108 distribution for more information.
109 * (require :iolib.examples) ; much output!
110 * (in-package :iolib.examples)
112 #<PACKAGE "IOLIB.EXAMPLES">
114 Created socket: #<passive IPv4 stream socket, unbound {BF84B99}>[fd=5]
115 Bound socket: #<passive IPv4 stream socket bound to 0.0.0.0/9999 {BF84B99}>
116 Listening on socket bound to: 0.0.0.0:9999
117 Waiting to accept a connection...
118 [ server is waiting for the below client to connect! ]
119 Got a connection from 127.0.0.1:34794!
120 Sending the time...Sent!
124 Now we'll start up the client which connected to the above server:
127 This is SBCL 1.0.33.30, an implementation of ANSI Common Lisp.
128 More information about SBCL is available at <http://www.sbcl.org/>.
130 SBCL is free software, provided as is, with absolutely no warranty.
131 It is mostly in the public domain; some portions are provided under
132 BSD-style licenses. See the CREDITS and COPYING files in the
133 distribution for more information.
134 * (require :iolib.examples) ; much output!
135 * (in-package :iolib.examples)
137 #<PACKAGE "IOLIB.EXAMPLES">
139 Connected to server 127.0.0.1:9999 via my local connection at 127.0.0.1:34794!
144 In each client example, one can specify which host or port to which it should
147 * (run-ex1-client :host "localhost" :port 9999)
148 Connected to server 127.0.0.1:9999 via my local connection at 127.0.0.1:34798!
153 The servers can be told a port they should listen upon and in this tutorial,
154 unless otherwise specified, will always bind to 0.0.0.0:9999 which means across
155 all interfaces on the machine and on port 9999.
160 IPV4 TCP Client/Server
161 Blocking and nonblocking I/O
166 The examples consist of a collection of clients and servers. They are split
167 into two groups: a set of daytime clients and server, and echo clients and
168 servers. In some of the examples, a certain network protocol, suppose
169 end-of-file handling, must be matched between client and server causing further
172 Client protocols are matched to server protocols thusly:
174 Clients: ex1-client, ex2-client, ex3-client, can work
175 with servers: ex1-server, ex2-server, ex3-server, ex4-server.
177 Clients: ex4-client, ex5a-client, can work with servers: ex5-server,
180 Clients: ex5b-client, can work with servers: ex7-server, ex8-server
182 Some clients and servers use the "daytime" series of protocols, those
183 are ex1-client, ex2-client, ex3-client, and ex1-server, ex2-server,
184 ex3-server, and ex4-server.
186 Some clients and servers use the "echo a line" series of protocols,
187 those are ex4-client, ex5a-client, ex5b-client, and ex5-server,
188 ex6-server, ex7-server, and ex8-server.
190 Even though much of the example source is included in the tutorial, it is
191 recommended that the example sources be carefully read and understood in order
192 to gain the most benefit from the tutorial.
197 In this section we show the evolution of a client which connects to a server
198 and gets the time of day. Each example shows some kind of an incremental
199 improvement to the previous one.
201 Daytime Client IVP4/TCP: ex1-client.lisp
202 ----------------------------------------
204 This example is a very simple daytime client program which contacts a server,
205 by default at *host* and *port*, returns a single line of text that is the
206 current date and time, and then exits. It is written in more of a C style just
207 to make it easy to compare with similar simple examples in other languages. It
208 uses blocking, line oriented I/O.
210 The steps this program performs are:
212 0. The ex1-client.lisp entrance call:
214 <example ex1-client:ex-0>
216 1. Create an active TCP socket:
218 The socket creation function (MAKE-SOCKET ...) is the method by which one
219 creates a socket in IOLib. It is very versatile and can be used to both
220 create and initialize the socket in a single call.
222 In this case, we use it simply and create an active IPV4 Internet stream
223 socket which can read or write utf8 text and that understands a particular
224 newline convention in the underlying data.
226 One small, but important, deviation of IOLib sockets from Berkeley sockets
227 is that when a socket is created, it is predestined to forever and
228 unalterably be either an active or passive socket. Active sockets are used
229 to connect to a server and passive sockets are used for a server's
232 <example ex1-client:ex-1>
234 2. Specify the Server's IP address and port and establish a connection
237 This bit of code contains many calls into IOLib and we shall examine each
240 The function LOOKUP-HOSTNAME takes as a string the DNS name
241 for a machine and returns 4 values:
243 B. a list of additional addresses(if existent)
244 C. the canonical name of the host
245 D. an alist of all the host's names with their respective addresses
247 We use only the first return value, the address component, to pass to the
250 The function CONNECT will connect the socket to the address, but to a
251 random port if the :port keyword argument is not specified. The average
252 client codes usually use :wait t to block until the connect can resolve
253 with a connected fd or an error. The exception to always using :wait t is
254 if the client needs to connect to many servers at once, suppose a web
255 client, or if a server is also a client in other contexts and wishes not to
258 The functions REMOTE-HOST and REMOTE-PORT return the ip address and port of
259 the remote connection associated with the connected socket. LOCAL-HOST and
260 LOCAL-PORT return the information of the client's end of the connected
261 socket. Analogous calls REMOTE-NAME and LOCAL-NAME each return two values
262 where the first value is the equivalent of *-host and the second value is
263 the equivalent of *-port.
265 <example ex1-client:ex-2>
267 3. Read and display the server's reply:
269 Now that the socket has been connected to the server, the server will send
270 a line of text to the client. The client uses the standard Common Lisp
271 function READ-LINE to read the information from the socket. The function
272 READ-LINE blocks and will only return when an *entire line* is read. Once
273 read, the line is emitted to *standard-output* via the function call
276 <example ex1-client:ex-3>
280 We close the socket with the standard function CLOSE and return true so the
281 return value of this example is t.
283 <example ex1-client:ex-4>
285 While this program works, it has some major flaws in it. First and foremost is
286 that it doesn't handle any conditions that IOLib signals in common use cases.
287 An example would be to run the ex1-client.lisp example without a daytime server
288 running. In most, if not all, Common Lisp toplevels, you'll be dropped into the
289 debugger on an unhandled SOCKET-CONNECTION-REFUSED-ERROR condition. Secondly,
290 it isn't written in the Common Lisp style.
292 Daytime Client IVP4/TCP: ex2-client.lisp
293 ----------------------------------------
295 In this example, we simply tackle the fact ex1-server.lisp can be shortened
296 with an IOLib form to something where the application writer has less to do
297 concerning cleaning up the socket object. It also uses line oriented blocking
300 The introduced macro WITH-OPEN-SOCKET calls MAKE-SOCKET with the arguments in
301 question and binds the socket to the variable 'socket'. When this form returns,
302 it will automatically close the socket.
304 This shortens the program so much, that the example can be included in its
307 <example ex2-client:ex-0>
309 This shorthand can go even further, if we add this to the WITH-OPEN-SOCKET
312 :remote-host (lookup-hostname host)
315 then the underlying MAKE-SOCKET call will in fact connect the socket directly
316 to the server before it is available for the body of the macro allowing us to
317 remove the connect call entirely! In the early examples, however, we don't
318 utilize IOLib's shorthand notations to this degree in order to make apparent
319 how the library maps into traditional socket concepts. After one gains
320 familiarity with the IOLib API, the situations where application of the
321 shortcuts are useful become much easier to see.
323 Daytime Client IVP4/TCP: ex3-client.lisp
324 ----------------------------------------
326 Now we come to condition handling, which can moderately affect the layout of
327 your IOLib program. Any real program using IOLib must handle IOLib's signaled
328 conditions which are common to the boundary cases of network programming.
329 We've already seen one of these boundary cases when we tried to connect a
330 daytime client to a server that wasn't running. The condition signaled in that
331 case was: SOCKET-CONNECTION-REFUSED-ERROR. The stream interface has a set of
332 conditions which IOLib will signal, and another lower level IOLib layer--which
333 we'll come to in the nonblocking I/O examples have another set of conditions.
334 There is some intersection between them and we will explore that later. For
335 now, we'll just use the conditions associated with a stream.
337 Our rewrite of ex2-client.lisp into ex3-client.lisp (continuing to use line
338 oriented blocking I/O) proceeds thusly:
340 0. We create a helper function which connects to the server and reads the
343 Notice the HANDLER-CASE macro around the portion of the function which
344 reads the date from the server. In looking at the boundary conditions from
345 the server given this protocol, we can receive an END-OF-FILE condition if
346 the client connected, but before the server could respond it exited,
347 closing the connection. Since in this case we're inside of a
348 WITH-OPEN-SOCKET form, we simply note that we got an END-OF-FILE and let
349 the cleanup forms of WITH-OPEN-SOCKET close the connection. If we don't
350 catch this condition, then the program will break into the debugger and
351 that isn't useful. It is usually debatable as to where one should handle
352 conditions: either near to or far away from the generating calls. In these
353 simple examples, no choice has any significant pros or cons. As your IOLib
354 programs become more and more complex, however, it becomes more obvious at
355 what abstraction level to handle signaled conditions.
357 <example ex3-client:ex-0>
359 1. Some conditions which are complete show-stoppers to the functioning of the
360 code are caught at a higher level:
362 Notice we catch the possible SOCKET-CONNECTION-REFUSED-ERROR from the
363 connect inside of the function run-ex3-client-helper.
365 <example ex3-client:ex-1>
367 Here are some common conditions in IOLib (some from ANSI Common Lisp too) and
368 under what situations they are signaled. In any IOLib program, *at least*
369 these conditions should be handled where appropriate.
372 When a stream function such as READ, READ-LINE, etc...(but not
373 RECEIVE-FROM), reads from a socket where the other end has been closed.
376 When writing to a socket with a stream function such as WRITE,
377 FORMAT, etc...(but not SEND-TO), if the socket is closed then this
378 condition is signaled.
380 SOCKET-CONNECTION-RESET-ERROR:
381 When doing I/O on a socket and the other side of the socket sent a
382 RST packet, this condition is signaled. It can also happen with
383 the IOLIb function ACCEPT and similar.
385 SOCKET-CONNECTION-REFUSED-ERROR:
386 Signaled by connect if there is no server waiting to accept the incoming
393 Now that we have completed the evolution of the daytime client, let's look at
396 The exposition of the servers follows in style of the clients.
398 Daytime Server IVP4/TCP: ex1-server.lisp
399 ----------------------------------------
401 This first example is an iterative server which handles a single client and
402 then exits. The I/O is blocking and no error handling is performed. This is
403 similar in scope to the ex1-client.lisp example.
405 0. Create the server socket:
407 We see that the socket is :passive. Every socket in IOLib is predestined to
408 be either an active or passive socket and since this is a server socket, it
409 is passive. Also here we see that we can ask for the underlying fd of the
410 socket with the function SOCKET-OS-FD.
412 <example ex1-server:ex-0>
416 Binding a socket is what gives it an endpoint to which clients can connect.
417 The IOLib constant +IPV4-UNSPECIFIED+ represents 0.0.0.0 and means if a
418 connection arrives on any interface, it will be accepted if it comes to the
419 :port specified. The :reuse-addr keyword represents the socket option
420 SO_REUSEADDR and states (among other things) that if the socket is in the
421 TIME_WAIT state it can be reused immediately. It is recommended that all
422 servers use :reuse-addr on their listening socket.
424 <example ex1-server:ex-1>
426 2. Listen on the socket
428 Listening on a socket allows clients to connect. In this example, we've
429 specified that 5 pending connection can be queued up in the kernel before
430 being accepted by the process.
432 <example ex1-server:ex-2>
434 3. Accept the client connection.
436 Here we finally call the IOLib function ACCEPT-CONNECTION. We would like it
437 to block, so we pass it :wait t. When ACCEPT-CONNECTION returns it will
438 return a new socket which represents the connection to the client.
439 ACCEPT-CONNECTION can return nil under some situations, such as on a slow
440 server when the client sent a TCP RST packet in between the time the kernel
441 sees the connection attempt and ACCEPT-CONNECTION is actually called. We
442 also opt to use the function REMOTE-NAME, which returns two values, the ip
443 address and port of the remote side of the socket.
445 <example ex1-server:ex-3>
447 4. Write the time to the client.
449 Here we've figured out the time string and wrote it to the client. Notice
450 we call the function FINISH-OUTPUT. This ensures that all output is written
451 to the client socket. For streams using blocking I/O, it is recommended
452 that every write to a blocking socket be followed up with a call to
455 <example ex1-server:ex-4>
457 5. Close the connection to the client.
459 We're done writing to the client, so close the connection so the client
460 knows it got everything.
462 <example ex1-server:ex-5>
464 6. Close the server's socket.
466 Since this is a one shot server, we close the listening socket and exit. In
467 this and all other servers we call FINISH-OUTPUT to flush all pending
468 message to *standard-output*, if any.
470 <example ex1-server:ex-6>
472 The above code is the basic idea for how a very simple TCP blocking I/O server
473 functions. Like ex1-client, this server suffers from the inability to handle
474 common signaled conditions such as a HANGUP from the client--which means the
475 client went away before the server could write the time to it.
477 However, one major, and subtle, problem of this particular example is that the
478 socket to the client is *not immediately closed* if the server happens to exit,
479 say by going through the debugger back to toplevel--or a signaled condition,
480 before writing the date to the client. If this happens, it can take a VERY long
481 time for the socket to be garbage collected and closed. In this scenario, the
482 client will hang around waiting for data which will never come until the Lisp
483 implementation closes the socket when it gets around to collecting it. Garbage
484 collection is an extremely nice feature of Common Lisp, but non-memory OS
485 resources in general should be eagerly cleaned up. Clients can suffer from
486 this problem too, leaving open, but unmanipulable, sockets to servers.
488 All clients or servers written against IOLib should either use some IOLib
489 specific macros to handle closing of socket, Common Lisp's condition system
490 like handler-case to catch the signaled conditions, or some other manual
493 Daytime Server IVP4/TCP: ex2-server.lisp
494 ----------------------------------------
496 Similarly to ex2-client, this server uses the macro WITH-OPEN-SOCKET to open
497 the server socket. We introduce WITH-ACCEPT-CONNECTION to accept the client and
498 convert this server from a single shot server to an iterative server which can
499 handle, in a serial fashion only, multiple clients.
501 0. Serially accept and process clients:
503 This portion of ex2-server shows the infinite loop around the accepting of
504 the connection. The macro WITH-ACCEPT-CONNECTION takes the server socket
505 and introduces a new binding: client, which is the accepted connection. We
506 ensure to tell the accept we'd like to be blocking. If for whatever reason
507 we exit the body, it'll clean up the client socket automatically.
509 <example ex2-server:ex-0>
511 For very simple blocking I/O servers like this one, serially accepting and
512 handling client connections isn't so much of a problem, but if the server does
513 anything which takes a lot of time or has to send lots of data back and forth
514 to many persistent clients, then this is a poor design. The means by which you
515 exit this server is by breaking evaluation and returning to the toplevel. When
516 this happens, the WITH-* forms automatically close the connection to the
519 Daytime Server IVP4/TCP: ex3-server.lisp
520 ----------------------------------------
522 In this iterative and blocking I/O server example, we add the handling of the
523 usual signaled conditions in network boundary cases often found with sockets.
524 Like the earlier client where we introduced HANDLER-CASE, this involves a
525 little bit of restructuring of the codes.
527 0. A helper function which opens a passive socket, binds it, and
530 There is nothing new in this portion of the code. We've seen this pattern
531 before. In production code, we could probably shorten this further by
532 having WITH-OPEN-SOCKET do the binding and connecting with appropriate
535 <example ex3-server:ex-0>
537 1. Repeatedly handle clients in a serial fashion:
539 The new material in this function is the HANDLER-CASE around sending the
540 client the time information. The boundary conditions when writing to a
541 client include the server getting a reset (RST) from the client or
542 discovering the client had gone away and there is no-one to which to write.
543 Since the write is contained within the WITH-ACCEPT-CONNECTION form, if any
544 of these conditions happen, we simply notice that they happened and let the
545 form clean up the socket when it exits. If we didn't catch the conditions,
546 however, we'd break into the debugger.
548 One might ask what the value of catching these conditions here is at all
549 since we don't actually do anything with them--other than printing a
550 message and preventing the code from breaking into the debugger. For the
551 purposes of the tutorial, it is intended that the reader induce the
552 boundary cases manually and see the flow of the code and to understand
553 exactly what conditions may be signaled under what conditions and how to
554 structure code to deal with them. In production code where the author might
555 not care about these conditions at all, one might simply ignore all the
556 signaled conditions that writing to the client might cause.
558 Of course, the appropriateness of ignoring network boundary conditions is
559 best determined by context.
561 <example ex3-server:ex-1>
563 2. End of the helper function, returns T to whomever called it:
565 <example ex3-server:ex-2>
567 3. The entry point into this example:
569 We handle the condition SOCKET-ADDRESS-IN-USE-ERROR which is most commonly
570 signaled when we try to bind a socket to address which already has a server
571 running on it or when the address is in the TIME_WAIT state. The latter
572 situation is so common--usually caused by a server just having exited and
573 another one starting up to replace it, that when binding addresses, one
574 should supply the keyword argument :reuse-addr with a true value to
575 BIND-ADDRESS to allow binding a socket to an address in TIME_WAIT state.
577 <example ex3-server:ex-3>
579 Daytime Server IVP4/TCP: ex4-server.lisp
580 ----------------------------------------
582 This is the first of our concurrent servers and the last of our daytime
583 protocol servers. Usually concurrency is introduced (in the UNIX environment)
584 with the fork() library call which creates an entirely new process with
585 copy-on-write semantics to handle the connection to the client. In this
586 tutorial environment, we've chosen to render this idea with the portable
587 threading library Bordeaux Threads. The I/O is still line oriented and
588 blocking, however, when a thread blocks another can run giving the illusion of
589 a server handling multiple clients in a non-blocking fashion.
591 We also introduce UNWIND-PROTECT ensures that various sockets are closed under
592 various boundary conditions in the execution of the server. An UNWIND-PROTECT
593 executes a single form, and after the evaluation, or interruption, of that
594 form, evaluates a special cleanup form. The cleanup form is *always* evaluated
595 and we use this to cleanup non-memory system resources like sockets.
597 Threads present their own special problems in the design of a server. Two
598 important problems are: data races and thread termination. The tutorial tries
599 very hard to avoid any data races in the examples and this problem is
600 ultimately solvable using Bordeaux-Threads mutexes or condition variables. Our
601 simple examples do not need mutexes as they do not share any data between
604 The harder problem is thread termination. Since the tutorial encourages
605 experimentation with the clients and servers in a REPL, threads may leak when
606 the server process' initial thread stops execution and goes back to the REPL.
607 We use three API calls from the Bordeaux Threads: THREAD-ALIVE-P, ALL-THREADS,
608 and DESTROY-THREAD--which are not to be used in normal thread programming. We
609 do this here in order to try and clean up leaked threads so the clients know
610 immediately when the server process stopped and we don't pollute the REPL with
611 an ever increasing number of executing threads. The employed method of
612 destroying the threads, on SBCL specifically, allows the invocation of the
613 thread's UNWIND-PROTECT's cleanup form, which closes the socket to the client
614 before destroying the thread. On other implementations of Common Lisp, we are
615 not guaranteed that the thread's UNWIND-PROTECT cleanup form will be evaluated
618 This method is also extremely heavy handed in that it uses the function
619 IGNORE-ERRORS to ignore any condition that Bordeaux Thread's DESTROY-THREAD may
620 have signaled, including important conditions like HEAP-EXHAUSTED-ERROR, an
621 SBCL specific condition. In a real threaded server, the exiting of the initial
622 thread (which means exiting of the runtime and termination of the entire Lisp
623 process) will destroy all other threads as the process tears itself down and
624 exits. This is the recommended way a threaded server should exit.
626 Since threading is implementation dependent for what guarantees are provided,
627 any non-toy threaded network server will probably use the native implementation
628 of threads for a specific Common Lisp implementation. An example difficult
629 situation would be trying to terminate a thread which is blocked on I/O.
630 Different implementations would handle this in different ways.
632 The two provided examples, ex4-server and ex5-server, provide a general idea
633 for the structuring of the code to utilize threads.
635 Here is the dissection of ex4-server:
637 0. A special variable which will allow the initial thread to pass a client
638 socket to a thread handling said client:
640 <example ex4-server:ex-0>
642 1. A helper function which begins with the usual recipe for a server:
644 <example ex4-server:ex-1>
646 2. Forever more, accept a client connection on the listening socket
647 and start a thread which handles it:
649 There is a lot going on in this piece of code. The first thing to notice is
650 the UNWIND-PROTECT and its cleanup form. The form which UNWIND-PROTECT is
651 guarding is an infinite loop which does a blocking accept to get a client
652 socket, rebinds *default-special-bindings* adding to its assoc list the
653 binding for *ex4-tls-client*, and creates a thread which handles the
656 The cleanup form walks all of the active client threads and destroys them,
657 ignoring any conditions that may have arose while doing so. Destroying the
658 threads prevents them from piling up and eventually causing havoc if many
659 servers start and exit over time. In addition, it forces an eager close on
660 the client sockets allowing any clients to know the server went away
663 <example ex4-server:ex-2>
665 3. The beginning of the thread handling the client:
667 When the thread is born, the aforementioned explicit binding of the client
668 socket to *ex4-tls-client* takes effect via the *default-special-bindings*
669 mechanism. By declaring *ex4-tls-client* ignorable, we inform the compiler
670 that this variable is set "elsewhere" and no warning should be emitted
671 about its possibly undefined value. In our case, this will always be
672 defined at runtime in this server.
674 <example ex4-server:ex-3>
676 4. Send the time to the socket:
678 The UNWIND-PROTECT in this form handles every possible case of leaving the
679 evaluable function such as it completing normally, a condition being
680 signaled, or by thread destruction--on SBCL! In all cases, the socket to
681 the client is closed which cleans up OS resources and lets the client know
682 right away the server has closed the connection. The HANDLER-CASE form here
683 just informs us which of the common IOLib conditions may have been signaled
684 while writing the time to the client.
686 <example ex4-server:ex-4>
688 It is a bit tricky to robustly handle closing of the client socket in the
689 thread. For example, if we bound the special variable *ex4-tls-client* to a
690 lexically scoped variable and then did the UNWIND-PROTECT form to close the
691 lexically scoped variable, then if this thread wakes up and gets destroyed
692 after the lexical binding, but before the UNWIND-PROTECT, we'd lose a
693 socket to a client into the garbage collector.
695 Such incorrect code would look like:
697 ;; This code is incorrect!
698 (defun process-ex4-client-thread ()
699 (declare (ignorable *ex4-tls-client*))
700 (let ((client *ex4-tls-thread*))
701 ;; thread gets destroyed right here! client socket is left open!
706 5. The entry point into this example:
708 Like earlier servers, we call the helper function and catch what happens if
709 :reuse-addr wasn't true in the BIND-ADDRESS function call.
711 <example ex4-server:ex-5>
714 Daytime Client/Server Commentary
715 --------------------------------
717 This concludes the examples using the daytime protocol. We've seen patterns
718 emerge in how the simplest of clients and servers are built and began to reason
719 about how to handle common signaled conditions. Threading, of course, increases
720 the care one must have in order to ensure that data access and control flow is
723 Echo Line Clients and Servers
724 -----------------------------
726 These next examples focus on the echo protocol. This is simply a server that
727 sends back to the client whatever the client wrote to it. A client can request
728 to quit talking to a server (except ex7-server and ex8-server, where this
729 feature isn't implemented) by sending the word "quit", on a line by itself.
730 This tells the server to close the connection to the client once it has
731 finished echoing the line. The closing of the client's read socket lets the
732 client know the connection to the server went away and that it is time to exit.
733 We also introduce the socket multiplexer interface which allows concurrent
734 processing of socket connections. This is similar to how UNIX's select(),
735 epoll(), or kqueue() works. Due to portability concerns on doing nonblocking
736 operations on *standard-input* and *standard-output* (we can't easily do it) we
737 are beholden to some form of blocking I/O in our clients because they interact
738 with a human. We will explore true non-blocking I/O in the ex8-server example
739 since that server only has to converse with connected clients.
744 The echo clients are a group of programs which read a line from
745 *standard-input*, write it to the server, read back the response from the
746 server, and emit the result to *standard-output*. While there is a portable
747 method to read "however much is available" from *standard-input*, there isn't
748 the symmetrical method to write "whatever I'm able" to *standard-output*. For
749 our client design, this means that all of these clients are line oriented and
750 do blocking I/O when reading from *standard-input* and writing to
753 Echo Client IPV4/TCP: ex4-client.lisp
754 --------------------------------------
756 This is a very basic echo client program that handles the usual conditions
757 while talking to the server:
759 0. Connect to the server and start echoing lines:
761 Here we use WITH-OPEN-SOCKET to create an active socket that we then use to
762 connect to the server. We handle HANGUP, for when the server went away
763 before the client could write to it, and END-OF-FILE, for when the server
764 closes down the connection.
766 Notice we call the function ex4-str-cli inside of a HANDLER-CASE macro.
767 This allows us to not check for any signaled conditions in ex4-str-cli and
768 greatly simplifies its implementation.
770 In this specific example, we don't do anything other than notify that the
771 condition happened since after that the socket gets closed via
774 <example ex4-client:ex-0>
776 1. Echo lines to the server:
778 Until the user inputs "quit" on a line by itself, we read a line, send it
779 to the server, read it back, and emit it to stdout. If any of the usual
780 conditions are signaled here, the handler-case in the Step 0 code fires and
781 we deal with it there.
783 When "quit" is entered, the line is sent on the round trip to the server
784 like usual, but this time the server closes the connection to the client.
785 Unfortunately, since the client is doing blocking I/O, we must read another
786 line from *standard-input* before we get any signaled condition when IOLib
787 discovers the socket has been closed by the server.
789 In practice, this means after the server closed the connection, the user
790 must hit <return> in order to drive the I/O loop enough to get the signaled
793 <example ex4-client:ex-1>
795 2. Entry point into the example:
797 We handle the usual connection refused condition, but otherwise this step
800 <example ex4-client:ex-2>
802 Echo Client IPV4/TCP: ex5a-client.lisp
803 --------------------------------------
805 This is the first client to use the socket multiplexer to notice when the
806 socket to the server is ready for reading or writing. While the multiplexer is
807 often used in single threaded servers it can be used for clients--especially
808 clients which may talk to multiple servers like web clients. Use of the
809 multiplexer API will require a significant change in how the code is
810 structured. It is not recommended that the multiplexer and threads be used
811 simultaneously to handle network connections.
813 Keeping in mind the fact that we ALWAYS could block while reading from
814 *standard-input* or writing to *standard-output*, we only attempt to read/write
815 to the standard streams when the multiplexer thinks it can read/write to the
816 server without blocking. This is a change from the traditional examples of how
817 to do this in C because in C one can determine if STDIN or STDOUT are ready in
818 the same manner as a network file descriptor.
820 The first big change from our previous examples is that we stop using
821 WITH-OPEN-SOCKET since now we must manually control when the socket to the
822 server must be closed. This is especially important for clients who use active
823 sockets. The second change is how we do the creation and registering of the
824 handlers for reading and writing to the server socket. The third change is how
825 to unregister a handler and close the socket associated with it under the right
826 conditions. Other changes will be explained as we meet them.
828 The main functions of the multiplexer API are:
829 (make-instance 'iomux:event-base ....)
830 Create an instance of the event-base, and associate some properties
831 with it, such as event-dispatch should return if the multiplexer
832 does not have any sockets it is managing.
834 :exit-when-empty - when no handlers are registered, event-dispatch
838 By default, sit in the multiplexer loop forever and handle I/O
839 requests. It is passed the event-base binding and in addition:
840 :once-only - run the ready handlers once then return.
841 :timeout - when there is no I/O for a certain amount of time return.
844 Associates a handler with a state to be called with a specific socket.
847 :read or :write or :error keyword
850 (remove-fd-handlers ...)
851 Removes a handler for a specific state with a specific socket.
855 one or more of :read t, :write t, :error t
857 Here is the example using this API.
861 The event-base is the object which holds the state of the multiplexer. It
862 must be initialized and torn down as we'll see in the entry function to
865 <example ex5a-client:ex-0>
867 1. A helper function in which we create the active socket:
869 Instead of using WITH-OPEN-SOCKET, we manually create the socket. We do
870 this to better control how to close the socket. WITH-OPEN-SOCKET will try
871 to FINISH-OUTPUT on the socket before closing it. This is bad if the socket
872 had been previously closed or signaled a condition like HANGUP. Trying to
873 write more data to an already hung up socket will simply signal another
874 condition. To prevent layers of condition handling code, we explicitly
875 handle closing of the socket ourselves.
877 <example ex5a-client:ex-1>
879 2. Connect to the server, register the socket handlers:
881 We protect the closing of the socket via UNWIND-PROTECT. We will talk about
882 the ramifications of this decision in the next step which describes the
883 UNWIND-PROTECT's cleanup form. In this section of code, we set up a read
884 and write handler for the socket, and invoke the dispatch function, which
885 will continue calling the handlers associated with the socket until the
886 socket gets closed and the handlers unregistered. When this happens (see
887 the entrance function step for why), EVENT-DISPATCH returns and we continue
888 on to the cleanup form for the UNWIND-PROTECT.
890 Setting up a handler in the multiplexer requires several arguments to
891 the function set-io-handler. Here are what the arguments to that function
894 This is the instance of the multiplexer for which we are setting
896 2. (socket-os-fd socket)
897 This call returns the underlying operating system's file
898 descriptor associated with the socket.
900 This keyword states that we'd like to call the handler when the
901 socket is ready to read. There is also :write and :error.
902 4. (make-ex5a-str-cli-read socket
903 (make-ex5a-client-disconnector socket))
904 The make-ex5a-str-cli-read function returns a closure over the
905 socket and another closure returned by the
906 make-ex5a-client-disconnector function. This function is what will
907 be called when the socket is ready for reading. We will shortly
908 explain the signature of this function and what gets passed to it
909 by the multiplexer. The disconnector function will be called by the
910 returned reader function if the reader function thinks that it
911 needs to close the socket to the server.
913 <example ex5a-client:ex-2>
915 3. Cleanup form for UNWIND-PROTECT:
917 In the cleanup form, we always close the socket and we pass the function
918 close :abort t to try and close the socket in any way possible. If we just
919 tried closing the socket, then we might cause another condition to be
920 signaled if a previous condition, like HANGUP, had already affected the
921 socket. :abort t avoids that case. If the socket is already closed by a
922 handler by the time we get here, closing it again hurts nothing.
924 <example ex5a-client:ex-3>
926 4. Make the writer function for when the socket is ready to write:
928 This function returns a closure which is called by the multiplexer when it
929 is ready to read something from the server. The arguments to the closure
930 are fd, the underlying file descriptor for the ready socket, event, which
931 could be :read, :write, or :error if the handler was registered multiple
932 times, and exception, which is nil under normal conditions, :error under an
933 error with the socket, or :timeout, if we were using timeout operations
934 when dealing with the socket.
936 The closure will read a line with the function READ-LINE and write it to
937 the server. The read will be blocking, but hopefully the write won't be
938 since the multiplexer told us we could perform the write and not block.
939 Obviously, is we write an enormous line, then we might block again, and in
940 this case the FINISH-OUTPUT on the socket will push the data in a blocking
941 I/O fashion until it is done and we return from the handler. So while this
942 closure for the most part writes when ready, there are cases under which
945 In this handler, if there is a signaled condition either reading from
946 *standard-input* (the END-OF-FILE condition) or writing to the server
947 socket (the HANGUP condition), we invoke the disconnector closure and pass
948 it :close. When we get to the description of the disconnector function,
949 you'll see what that means.
951 Once the disconnector closure is invoked, the handler will have been
952 removed and the socket closed. This will make EVENT-DISPATCH return since
953 the only socket it was multiplexing for was closed--because we've told the
954 multiplexer to do so when it was made!
956 <example ex5a-client:ex-4>
958 5. Make the reader function for when the socket is ready to read:
960 This piece of code is very similar to the previous step's code, we just
961 handle the appropriate conditions and after reading the line from the
962 server emit it to *standard-output*. Again, even though we are told we can
963 read from the server without blocking, if the read is large enough we will
964 continue to block until read-line reads the all the data and the newline.
966 <example ex5a-client:ex-5>
968 6. The disconnector function:
970 This function returns a closure which takes an arbitrary number of
971 arguments. If the arguments to the invoked closure contain :read, :write,
972 or :error, the respective handler on the associated socket is removed. If
973 none of those three are supplied, then all handlers for that socket are
974 removed. Additionally if :close is specified, the socket is closed. While
975 not all features of this function is used in this example, this function
976 (or a similar one using the correct event-base special variable) is used
977 whenever we use the multiplexer in an example.
979 The closure is called whenever a handler believes it should unregister
980 itself or another handler, or close the socket. Because we will often close
981 the socket in the disconnector closure, we can't use WITH-OPEN-SOCKET to
982 automatically close the socket because WITH-OPEN-SOCKET may try to flush
983 data on the socket, signaling another condition.
985 <example ex5a-client:ex-6>
987 7. The entry point for this example and setting up the event-base:
989 This function is much more complex than in examples that do not use the
990 multiplexer. Protected by an UNWIND-PROTECT, we first initialize the event
991 base my calling make-instance 'iomux:event-base. Here is where we pass the
992 keyword argument :exit-when-empty t which states that the event-dispatch
993 function should return when there are no more registered handlers. Once
994 that is done, we call the helper, catching a common condition and waiting
997 <example ex5a-client:ex-7>
999 8. The cleanup form for UNWIND-PROTECT:
1001 This cleanup form closes the *ex5a-event-base* instance. IOLib defines a
1002 method for the generic function CLOSE which accepts an event-base and
1003 performs the necessary work to shut it down.
1005 <example ex5a-client:ex-8>
1007 While this program works just fine with human input, it has a failure when
1008 reading batch input. The failure is that when we get the END-OF-FILE condition
1009 when *standard-input* closes, we _immediately_ unregister the read/write
1010 handlers to the server, close the socket and exit the program. This destroys
1011 any in-flight data to/from the server and lines being echoed may be lost.
1014 Echo Client IPV4/TCP: ex5b-client.lisp
1015 --------------------------------------
1017 In order to fix the batch input problem of ex5a-client, we will use the
1018 shutdown function which allows us to inform the server we are done writing
1019 data, but leave the socket open so we can read the rest of the responses from
1020 the server. This effectively closes only one-half of the TCP connection. The
1021 server has to be made aware of this kind of protocol so it doesn't assume the
1022 client completely exited when it gets an END-OF-FILE from the client and shuts
1023 down the whole connection throwing away any queued data for the client.
1025 This client is nearly identical to ex5a-client except we shut down the write
1026 end of the connection to the server when we get END-OF-FILE from
1027 *standard-input* and wait until we get all of the data back from the server.
1028 The server signifies to us that it has sent all of the pending data by closing
1029 the write end of its connection. The client sees the closing of the server's
1030 write end as an END-OF-FILE on the socket connected to the server.
1032 We show this example as a difference to ex5aq-client.
1034 0. Shutdown the write end of the socket to the server:
1036 Here we use the expanded functionality of the disconnector closure. After
1037 we shut down the write end of our TCP connection, we call (funcall
1038 disconnector :write) which states only to remove the write (to the server)
1039 handler, but leave the connection open. After this happens, there is no way
1040 we can read from *standard-input* again. Once the server sends the final
1041 data and the closes its connection to this client, we remove the read
1042 handler, which removes the last handler, and causes the EVENT-DISPATCH
1043 function to return, which ends the client computation.
1045 <example ex5b-client:ex-0>
1047 Be aware that even if both directions on one end of a connection are shutdown,
1048 close still must be called upon the socket in order to release resources held
1049 by the operating system.
1054 The echo servers, paired to clients as per the beginning of this tutorial,
1055 further evolve to using the multiplexer and becoming more fine grained with
1056 respect to when I/O is done until we reach the ability to perform nonblocking
1057 I/O of arbitrary read/write sizes.
1059 Echo Server IPV4/TCP: ex5-server.lisp
1060 -------------------------------------
1062 This threaded server is very similar to ex4-server, but instead of sending only
1063 the time, each thread handles an echo protocol to a client. While this is
1064 still a blocking I/O server, only a single thread talking to a client gets
1065 blocked, not the whole server. Other than the server not honoring batch input
1066 from the client correctly, this is a common model for a class of servers due to
1067 its nonblocking behavior.
1069 0. The special variable used to communicate the client socket to the thread:
1071 <example ex5-server:ex-0>
1073 1. The usual prologue to a server:
1075 <example ex5-server:ex-1>
1077 2. First half of creating the client threads:
1079 <example ex5-server:ex-2>
1081 3. Second half, the cleanup form for the UNWIND-PROTECT:
1083 We make sure to clean up only the client threads!
1085 <example ex5-server:ex-3>
1087 4. Handle the client and deal with signaled conditions:
1089 In this function, we ensure that under all conditions of the execution of
1090 this function, if something goes wrong, we eagerly close the socket to the
1091 client so it is not leaked into the garbage collector. We also handle
1092 numerous conditions the the client could generate while talking to it in
1093 the function str-ex5-echo.
1095 <example ex5-server:ex-4>
1097 5. Actually perform the echo protocol to the client:
1099 Read lines from the client and echo them back. All of this I/O is blocking.
1100 If we see "quit" from the client, then exit the loop, which causes the
1101 UNWIND-PROTECT cleanup form in step 4 to fire and close the connection to
1104 <example ex5-server:ex-5>
1106 6. The entrance function into this example:
1108 <example ex5-server:ex-6>
1111 Echo Server IPV4/TCP: ex6-server.lisp
1112 -------------------------------------
1114 This is the first of the echo servers which use the multiplexer to handle
1115 multiple clients concurrently. It is a single threaded program. As mentioned
1116 before, one shouldn't mix the multiplexer and threads together to handle
1117 network connections.
1119 We explore a new concept with the multiplexer in that the listening server
1120 socket is itself registered with the multiplexer. The read handler (called the
1121 listener handler in this context) associated with this socket becomes ready
1122 when a client has connected to the server address. Thus, once the listening
1123 socket is ready the listener handler accepts the client and associates the line
1124 echo protocol callback with the client's socket in the multiplexer.
1126 The I/O design of this server is such that if the client connection is ready to
1127 read, we read a line, then immediately write the line back to the client in the
1128 same function without waiting to see if it is ready for writing. Since we are
1129 still using blocking I/O, this is ok. The reason for this example's design was
1130 to minimize the complexity of using the multiplexer in order to introduce the
1131 listener handler. Later examples become much more complex as we push the
1132 multiplexer API farther.
1134 0. The variable which holds the multiplexer instance:
1136 <example ex6-server:ex-0>
1138 1. A hash table of client connections:
1140 We record each client that connects to the server into a hash table socket
1141 keyed by the list (ip address port) and associate with it a value of the
1142 client's socket. This is so that under any conditions of the server exiting
1143 we can eagerly close any open connections to clients in a cleanup form.
1145 <example ex6-server:ex-1>
1147 2. Create and bind the server socket:
1149 We protect how we manipulate the server socket with an UNWIND-PROTECT so we
1150 ensure to close the socket at the end of the server's computation or if
1151 something went wrong.
1153 <example ex6-server:ex-2>
1155 3. Register a listener handler on the server socket and start dispatching
1156 events with the multiplexer:
1158 <example ex6-server:ex-3>
1160 4. When the server stops handling clients, we close the server socket:
1162 <example ex6-server:ex-4>
1164 5. The listener handler:
1166 Once the returned closure from this function is called by the multiplexer
1167 on the ready server socket, we accept the client with a blocking accept.
1168 We then save the client connection in our table and register the line echo
1169 closure with the socket. The line echo closure will also contain a
1170 disconnector function as in previous usages of the multiplexer.
1172 <example ex6-server:ex-5>
1174 6. The line echo closure generator:
1176 This function returns a closure which is then bound to a client socket in
1177 the multiplexer. When the socket is ready, we read a line form the client
1178 and write it back to the client immediately. Since this is blocking I/O the
1179 whole server will wait until this transaction is complete. This means that
1180 a client which sends one byte of ASCII that is not a newline can cause the
1181 whole server to block for all clients. This serious defect is remedied with
1182 non-blocking I/O, which we show in a later example.
1184 <example ex6-server:ex-6>
1186 7. The disconnector closure generator:
1188 This function returns a closure that removes all the handlers from the
1189 socket in question and then closes it. Notice that this means this server
1190 is not capable of handling batch input from a client, since when it
1191 receives the END-OF-FILE on the read from a client, will immediately tear
1192 down the connection destroying any in flight data. After closing the
1193 socket, we also remove it from our table of open connections.
1195 <example ex6-server:ex-7>
1197 8. Initialize the event-base, the connection table, and start the server:
1199 This code is the beginning of the UNWIND-PROTECT form which protects the
1200 server's socket resources.
1202 <example ex6-server:ex-8>
1204 9. Cleanup the client connections and close the event-base:
1206 When the server exits we walk the *ex6-server-open-connections* hash and
1207 eagerly close every client we find there. After we are done, we close the
1208 event-base. This ensures every thing is cleaned up properly.
1210 <example ex6-server:ex-9>
1212 This server uses the multiplexer in a simple fashion because only one handler
1213 is registered for a client. That handler reads, then writes the data back to
1214 the client. The scope of the data read from the client never has to leave the
1217 Echo Server IPV4/TCP: ex7-server.lisp
1218 -------------------------------------
1220 This example is different than ex6-server because it fully separates the
1221 reading and writing of data to a client into different handler functions. This
1222 requires an architectural change to the server in order to be able to keep the
1223 data from the client "somewhere" before being able to write it back to the
1224 client when the multiplexer determines it can written to the client. We
1225 introduce an io-buffer object, implemented in terms of a closure and one per
1226 client, which stores the in-flight data until the client is ready to accept the
1227 writes from the server.
1229 Storage of client data introduces a problem in that if the client writes lots
1230 of data to the server but happens to never be ready to accept it back from the
1231 server, the server will consume all memory and run out of resources. We
1232 attempt to prevent this from happening, though not perfectly.
1234 When the io-buffer is created for a client, we state we only would like a
1235 certain number of bytes to be read from the client. Of course, since we're
1236 using read-line with blocking I/O and the client could write a tremendous
1237 amount of data before a newline, we can't completely enforce our storage policy
1238 in this server. If the client, though, is well-behaved in that it sends
1239 reasonable sized lines of text--a rarity in the real world, our implemented
1240 policy is sufficient. When we reach the nonblocking I/O server example, we'll
1241 find that we can perfectly enforce the per client data storage policy.
1243 This server honors batch input from the client. When it sees the END-OF-FILE
1244 from the client, and it still has data to write, the server will attempt to
1245 write the rest of the data out as the multiplexer says the client is ready to
1248 Since this example is quite long the server portion will just be shown as a
1249 difference to ex6-server.
1251 0. The listener handler:
1253 The important code in this function is the call to make-ex7-io-buffer.
1254 This function returns a closure, here called io-buffer, which takes one
1255 argument, either :read-a-line or :write-a-line. When the funcall of
1256 io-buffer with the appropriate argument happens, *another* closure is
1257 returned and this is the closure registered with the appropriate ready
1258 state in the multiplexer.
1260 This returned closure has bound in its lexical scope the storage needed for
1263 Both closures returned by :read-a-line and :write-a-line have access to the
1264 same storage space unique to this object io-buffer. This is the means by
1265 which the client's write handler can get access to the data read by the
1266 client's read handler.
1268 <example ex7-server:ex-0>
1270 1. The disconnector function:
1272 This function is almost identical to a previous example used in
1273 ex5a-client. The only difference is the special variable it references.
1275 Since the io-buffer knows under what conditions it should register or
1276 unregister specific handlers for the client socket, we need to be able to
1277 selectively remove them without disturbing the others.
1279 <example ex7-server:ex-1>
1281 Now we come to the description of the ex7-io-buffer code base. This code base
1282 interacts directly with the event-base multiplexer instance in order to
1283 register and unregister handlers to the client. Handlers are only registered
1284 when there is data to write, or room to read more data up to the buffer size.
1286 0. The io-buffer closure generator and associated lexical storage:
1288 These are the variables closed over which represent the internal state of
1289 the closure and hold the data from the client. In particular note is the
1290 fact we keep track of when a handler is registered (since this object can
1291 register or unregister the handlers in and of itself) and whether or not
1292 we've seen the END-OF-FILE from a client. The line-queue will hold the
1293 actual data from the client.
1295 <example ex7-buffer:ex-0>
1297 1. The read-a-line closure:
1299 This is the function which will ultimately be registered with the
1300 multiplexer hence the arguments it expects. Its job is to read a line from
1301 the client when the multiplexer said the client was readable and then store
1302 the line into the line-queue. If we have read a line, we immediately
1303 register the write-a-line handler with the multiplexer since we need to
1304 know when the client will be ready to accept it. If it turns out there is
1305 more data stored than the high-water mark we set, we unregister the read
1306 handler so we don't continue to keep reading data. If we get END-OF-FILE,
1307 but there is nothing left to write, then this handler performs a small
1308 optimization and closes the socket to the client and unregisters
1309 everything. This prevents a needless loop through the multiplexer in this
1312 The handling of END-OF-FILE is interesting in that we unregister the read
1313 handler, since we won't need it anymore, and mark that we've seen the
1314 END-OF-FILE. At this point, the only thing the multiplexer has to do with
1315 respect to this client is to write all of the lines stored in the
1316 line-queue out to the client and close the connection to the client.
1318 Of the various conditions that can be signaled, the
1319 SOCKET-CONNECTION-RESET-ERROR condition is the one which will shut down the
1320 whole connection by removing all handlers in the multiplexer for this
1321 client and ultimately throw away any in-flight data.
1323 <example ex7-buffer:ex-1>
1325 2. The write-a-line closure:
1327 This function is somewhat symmetrical to read-a-line. It will register and
1328 unregister itself or the read handler based upon how much data is available
1329 to read/write. If the END-OF-FILE is seen and there is nothing left to
1330 write, it will close the connection to the client and unregister
1333 <example ex7-buffer:ex-2>
1335 3. The returned closure, which represents the io-buffer:
1337 This is the actual closure returned by make-ex7-io-buffer and which is used
1338 to gain access into the read-a-line and write-a-line functions. It takes a
1339 single argument, either the keywords :read-a-line or :write-a-line, and
1340 returns a reference to either internal function.
1342 <example ex7-buffer:ex-3>
1344 While this server still uses blocking I/O, we've laid the foundations for
1345 nonblocking I/O and memory storage enforcement. The foundations specifically
1346 are separating the read/write handlers into different pieces and having shared
1347 lexical bindings between them.
1349 Echo Server IPV4/TCP: ex8-server.lisp
1350 -------------------------------------
1352 This server uses nonblocking I/O and the multiplexer to concurrently talk to
1355 Architecturally, it is very similar to ex7-server, but the io-buffer for this
1356 server is implemented with much different internals. Whereas in ex7-server
1357 reading from a client used the stream function READ-LINE, writing used the
1358 stream function FORMAT, and the strings from the client were kept in a queue,
1359 now we use RECEIVE-FROM and SEND-TO along with an array of unsigned-bytes as a
1360 buffer to read/write actual bytes from the socket.
1362 Accessing the socket through the stream API is different than doing it through
1363 the almost raw socket API which we are about to use. RECEIVE-FROM and SEND-TO
1364 are not part of the stream interface. They are a lower level API in IOLib being
1365 closer to the underlying OS abstraction and as a consequence have a somewhat
1366 different set of conditions that they can signal. These different conditions
1367 have the form isys:<unix-errno-name> like: isys:epipe, isys:ewouldblock, etc.
1368 There is some intersection with the condition names signaled by the stream API,
1369 such as: SOCKET-CONNECTION-RESET-ERROR, and SOCKET-CONNECTION-REFUSED.
1371 [TODO figure out complete list!]
1373 An example of the ramifications of this API is RECEIVE-FROM. Comparing against
1374 the stream interface whose READ-LINE will signal an END-OF-FILE when the
1375 reading socket has been closed by the client, the function RECEIVE-FROM will
1376 return 0, signifying the end of file. The stream function FORMAT will signal
1377 HANGUP if it tries to write to a socket where the client has gone away. SEND-TO
1378 might not signal, or otherwise produce, any error at all when writing to a
1379 socket where the client has gone away--usually it is on the next RECEIVE-FROM
1380 that it is discovered the client went away. The bytes that SEND-TO wrote simply
1383 With IOLib, it may surprise you to be told that all underlying fds in the
1384 previous examples have been nonblocking! This is why we specified :wait t for
1385 ACCEPT-CONNECTION and CONNECT.
1387 The IOLib library internally ensures that the stream interface blocks according
1388 to the requirements of ANSI Common Lisp. However, when we use SEND-TO and
1389 RECEIVE-FROM we automatically gain the benefit of the non-blocking status on
1390 the underlying fd. This is why in this example we don't explicitly set the
1391 underlying fd to non-blocking status--it already is!
1393 The server code itself is described as a difference from ex7-server, but the
1394 io-buffer for this nonblocking server (in file ex8-buffer.lisp) will be
1395 described in its entirety. Also, this server honors the batch input requirement
1396 from example client ex-5b-client, which you should use against this server.
1398 The ex8-server codes:
1400 0. The listener handler (first half):
1402 Accept and store the client connection.
1404 <example ex8-server:ex-0>
1406 1. The listener handler (second half):
1408 Like ex7-server, we register the read and write handlers. Notice though
1409 that we changed the keywords to the io-buffer closure to be
1410 :read-some-bytes and :write-some-bytes. This better represents what the
1411 io-buffer is actually doing.
1413 <example ex8-server:ex-1>
1415 The rest of the server is extremely similar to ex7-server.
1417 Now, we'll show the io-buffer specific to ex8-server.
1419 0. The internal state of the io-buffer closure:
1421 The binding echo-buf is an unsigned-byte array of size max-bytes. This is
1422 where data from the client is stored before it is written back to the
1425 The binding read-index keeps track of the beginning of the empty space in
1426 the echo-buf buffer where more data could be stored during a read.
1428 The binding write-index keeps track of how much data has been written to
1429 the client. It moves towards read-index, and when it has the same value as
1430 read-index it means that there is no data left to write to the client.
1432 The bindings read-handler-registered and write-handler-registered allow the
1433 io-buffer to know when it has registered a handler for reading and writing
1436 The binding eof-seen marks when the client has closed its write connection
1437 to the server. The server will push out all data to the client, then close
1438 socket to the client.
1440 <example ex8-buffer:ex-0>
1442 1. Reading bytes form the client:
1444 In this function, we will convert the return value 0 of RECEIVE-FROM on the
1445 read of a closed socket into a signaled END-OF-FILE condition to keep the
1446 structure of our code similar to what has transpired before. Once we read
1447 some bytes, we increment the read-index pointer and ensure to register a
1448 write handler to write the data back out. We optimize the writing process a
1449 little bit and try to write the data out immediately without checking to
1450 see if the socket is ready. Then if there is no more room in the echo-buf
1451 array, we unregister ourselves so we don't try and read more data from the
1452 client until we are ready to accept it (by having written all of the data
1453 back to the client). We mark the END-OF-FILE flag and unregister the read
1454 handler if we see the client has closed its connection. We optimize the
1455 knowledge that if we have no more data to write we just close the
1456 connection to the client.
1458 <example ex8-buffer:ex-1>
1460 2. Writing bytes to the client:
1462 While there are more bytes to write, we write them, keeping track of how
1463 much we wrote. Once we are out of data to write, we unregister the write
1464 handler, since we don't want to be called unnecessarily--usually the client
1465 socket is always ready to write. If we've seen the eof marker and are out
1466 of data, we close the client connection and are done. If we haven't seen
1467 it, then we determine if we are at the end of the buffer, if so, we reset
1468 the indices to the beginning. Either way, we re-register the read handler
1469 to acquire more data.
1471 We handle some new conditions here: isys:ewouldblock is needed because
1472 sometimes the underlying OS will mark an fd as ready to write when in fact
1473 it isn't when we get around to writing it. We might also see this condition
1474 when we tried to optimize the write of the data in the read handler since
1475 we did it outside of the multiplexer--this is idiomatic and saves a trip
1476 through the multiplexer more often than not. Seeing isys:ewouldblock simply
1477 aborts the write and we'll try again later. Under some conditions, send-to
1478 will signal an isys:epipe error, which means the client closed its
1479 connection. It is similar to a HANGUP condition in a format call with the
1480 stream API. We treat it similarly to a HANGUP.
1482 <example ex8-buffer:ex-2>
1484 3. The returned closure of the io-buffer:
1486 Much like make-ex7-io-buffer, we return one of the internal closures which
1487 are appropriate for reading or writing by the multiplexer.
1489 <example ex8-buffer:ex-3>
1495 Of course, more information should go into this tutorial, such as non-blocking
1496 connects/accepts, urgent TCP data, UDP examples, and IPV6. As time permits or
1497 contributions come in, these will be added.
1502 This holds a rough approximation between the sources in this tutorial and the
1503 original sources in the network programming book by Stevens mentioned in the
1504 beginning of the tutorial. Aspects about the implementation of each client or
1505 server are summarized here.
1511 - ex1-client: Blocking I/O, daytime client, C Style
1514 - ex2-client: Blocking I/O, daytime client, Lisp Style
1517 - ex3-client: ex2-client, but with much more error handling
1519 Figure 5.4, 5.5, page 114, 115
1520 - ex4-client: Blocking I/O, line oriented
1522 Figure 6.9, page 157
1523 - ex5a-client: I/O multiplexing with iolib, line oriented, blocking I/O
1524 - note: since this is still blocking I/O, I'm using *standard-input*
1525 and friends. Also note, with batch input, it will close the socket with
1526 in-flight data still present which is incorrect.
1528 Figure 6.13, page 162
1529 - ex5b-client: Same as ex5a-client EXCEPT shutdown is called when the input
1530 reaches end-of-file as to prevent in flight data from being destroyed
1531 on the way to the server.
1536 Figure 4.11, page 101
1537 - ex1-server: Iterative, blocking I/O daytime server, C Style, no
1538 error handling, one shot, line oriented
1540 Figure 4.11, page 101
1541 - ex2-server: Iterative, blocking I/O daytime server, Lisp Style,
1542 no error handling, loop forever, line oriented
1544 Figure 4.11, page 101
1545 - ex3-server: daytime server, ex2-server, but with error handling, line oriented
1547 Figure 4.13, page 105
1548 - ex4-server: daytime server, concurrent, blocking I/O, line oriented
1550 Figure 5.2, 5.3, page 113, 114
1551 - ex5-server: Concurrent, blocking I/O, echo server, line oriented
1553 Figure 6.21,6.22 page 165,166
1554 - ex6-server: I/O multiplexing of clients with iolib, line oriented,
1557 Figure 6.21, 6.22 page 165,166
1558 - ex7-server, ex7-buffer: individual I/O handlers for read/write,
1559 I/O multiplexing of clients with iolib, line oriented, blocking I/O,
1560 has problem with denial of service, page 167.
1562 Figure 15.3, 15.4, 15.5 page 400-403
1563 - ex8-server, ex8-buffer: nonblocking I/O, event-dispatch, send-to, receive-from