| blob | c4d322f0de6abfc9b07cdbdbb35fbfdac555e986 |
1 Network Programming in ANSI Common Lisp with IOLib
2 by: Peter Keller (psilord@cs.wisc.edu)
3 Version 0.0
4 4/02/2010
7 What is IOLib?
8 --------------
10 IOLib is a portable I/O library for ANSI Common Lisp. It includes socket
11 interfaces for network programming with IPV4/IPV6 TCP and UDP, an I/O
12 multiplexer that includes nonblocking I/O, a DNS resolver library, and a
13 pathname library.
15 Where do I get IOLib?
16 ---------------------
18 The current version of IOLib is found here:
20 http://common-lisp.net/project/iolib/download.shtml
22 Please use the repository located in the Live Sources section for the most up
23 to date version of IOLib.
25 Introduction
26 ------------
28 This tutorial loosely follows the exposition of network programming in "UNIX
29 Network Programming, Networking APIs: Sockets and XTI 2nd Edition" by W.
30 Richard Stevens. Many examples are derived from the source codes in that book.
31 Major deviations from the C sources include converting the concurrent examples
32 which use fork() into threaded examples which use the portable Bordeaux Threads
33 package, more structured implementations of certain concepts such as data
34 buffers and error handling, and general movement of coding style towards a
35 Common Lisp viewpoint.
37 The scope of this version of the tutorial is:
38 0. Exposition suitable for programmers unfamiliar with ANSI Common Lisp
39 1. IPV4 TCP
40 2. Client/Server architecture
41 3. Iterative vs Concurrent (via threading) vs Multiplexed Server Design
42 4. Blocking and nonblocking I/O
44 It is intended, however, that this tutorial grows to contain the entirety of
45 IOLib's API as detailed in the Future Directions section of this tutorial. As
46 newer revisions of this tutorial are released, those gaps will be filled until
47 the whole of the IOLib API has been discussed.
49 Finally, the example code in this tutorial is algorithmically cut from the
50 actual example programs and inserted into the tutorial via a template
51 generation method. The example codes have embedded in them a tiny markup
52 language which facilitates this in the form (on a single line) of ';; ex-NNNb'
53 to begin an example section, and ';; ex-NNNe' to end an example section--NNN
54 stands for an enumeration integer for which each section's begin and end must
55 match.
57 Acknowledgements
58 ----------------
60 I would like to greatly thank Stelian Ionescu, the author of IOLib
61 for his exposition of the various features of IOLib and his patience
62 in our sometimes long conversations.
64 Supporting Code
65 ---------------
67 The file package.lisp contains a small library of codes used widely in the
68 examples. The supporting code implements:
70 0. The package containing the examples, called :iolib.examples.
72 1. The variables *host* and *port*, set to "localhost" and 9999
73 respectively. This is the default name and port to which
74 client connect and servers listen. Servers usually bind
75 to 0.0.0.0, however.
77 2. A small, but efficient, queue implementation, from "ANSI Common Lisp"
78 by Paul Graham. The interface calls are:
79 (make-queue)
80 (enqueue obj q)
81 (dequeue q)
82 (empty-queue q)
84 3. :iolib.examples currently depends upon IOLib alone and uses
85 packages :common-lisp, :iolib, and :bordeaux-threads.
87 Running the Examples
88 --------------------
90 These examples were developed and tested on SBCL 1.0.33.30 running on an x86
91 Ubuntu 8.10 machine. They were ran with two sessions of SBCL running, one
92 acting as a client, and the other as a server.
94 Supposing we'd like to start up the first example of the daytime server and
95 connect to it with the first daytime client example. Initially, the server will
96 bind to *host* and *port* and wait for the client to connect. We connect with
97 the client to *host* and *port*, get the time, and exit.
99 First we'll start up a server:
101 Linux black > sbcl
102 This is SBCL 1.0.33.30, an implementation of ANSI Common Lisp.
103 More information about SBCL is available at <http://www.sbcl.org/>.
105 SBCL is free software, provided as is, with absolutely no warranty.
106 It is mostly in the public domain; some portions are provided under
107 BSD-style licenses. See the CREDITS and COPYING files in the
108 distribution for more information.
109 * (require :iolib.examples) ; much output!
110 * (in-package :iolib.examples)
112 #<PACKAGE "IOLIB.EXAMPLES">
113 * (run-ex1-server)
114 Created socket: #<passive IPv4 stream socket, unbound {BF84B99}>[fd=5]
115 Bound socket: #<passive IPv4 stream socket bound to 0.0.0.0/9999 {BF84B99}>
116 Listening on socket bound to: 0.0.0.0:9999
117 Waiting to accept a connection...
118 [ server is waiting for the below client to connect! ]
119 Got a connection from 127.0.0.1:34794!
120 Sending the time...Sent!
121 T
122 *
124 Now we'll start up the client which connected to the above server:
126 Linux black > sbcl
127 This is SBCL 1.0.33.30, an implementation of ANSI Common Lisp.
128 More information about SBCL is available at <http://www.sbcl.org/>.
130 SBCL is free software, provided as is, with absolutely no warranty.
131 It is mostly in the public domain; some portions are provided under
132 BSD-style licenses. See the CREDITS and COPYING files in the
133 distribution for more information.
134 * (require :iolib.examples) ; much output!
135 * (in-package :iolib.examples)
137 #<PACKAGE "IOLIB.EXAMPLES">
138 * (run-ex1-client)
139 Connected to server 127.0.0.1:9999 via my local connection at 127.0.0.1:34794!
140 2/27/2010 13:51:48
141 T
142 *
144 In each client example, one can specify which host or port to which it should
145 connect:
147 * (run-ex1-client :host "localhost" :port 9999)
148 Connected to server 127.0.0.1:9999 via my local connection at 127.0.0.1:34798!
149 2/27/2010 13:53:7
150 T
151 *
153 The servers can be told a port they should listen upon and in this tutorial,
154 unless otherwise specified, will always bind to 0.0.0.0:9999 which means across
155 all interfaces on the machine and on port 9999.
158 CHAPTER 1
159 ---------
160 IPV4 TCP Client/Server
161 Blocking and nonblocking I/O
163 Overview of Examples
164 --------------------
166 The examples consist of a collection of clients and servers. They are split
167 into two groups: a set of daytime clients and server, and echo clients and
168 servers. In some of the examples, a certain network protocol, suppose
169 end-of-file handling, must be matched between client and server causing further
170 delineation.
172 Client protocols are matched to server protocols thusly:
174 Clients: ex1-client, ex2-client, ex3-client, can work
175 with servers: ex1-server, ex2-server, ex3-server, ex4-server.
177 Clients: ex4-client, ex5a-client, can work with servers: ex5-server,
178 ex6-server.
180 Clients: ex5b-client, can work with servers: ex7-server, ex8-server
182 Some clients and servers use the "daytime" series of protocols, those
183 are ex1-client, ex2-client, ex3-client, and ex1-server, ex2-server,
184 ex3-server, and ex4-server.
186 Some clients and servers use the "echo a line" series of protocols,
187 those are ex4-client, ex5a-client, ex5b-client, and ex5-server,
188 ex6-server, ex7-server, and ex8-server.
190 Even though much of the example source is included in the tutorial, it is
191 recommended that the example sources be carefully read and understood in order
192 to gain the most benefit from the tutorial.
194 Daytime Clients
195 ---------------
197 In this section we show the evolution of a client which connects to a server
198 and gets the time of day. Each example shows some kind of an incremental
199 improvement to the previous one.
201 Daytime Client IVP4/TCP: ex1-client.lisp
202 ----------------------------------------
204 This example is a very simple daytime client program which contacts a server,
205 by default at *host* and *port*, returns a single line of text that is the
206 current date and time, and then exits. It is written in more of a C style just
207 to make it easy to compare with similar simple examples in other languages. It
208 uses blocking, line oriented I/O.
210 The steps this program performs are:
212 0. The ex1-client.lisp entrance call:
214 +------------------------------------------------------------------------------+
215 | |
216 (defun run-ex1-client (&key (host *host*) (port *port*))
217 | |
218 +------------------------------------------------------------------------------+
220 1. Create an active TCP socket:
222 The socket creation function (MAKE-SOCKET ...) is the method by which one
223 creates a socket in IOLib. It is very versatile and can be used to both
224 create and initialize the socket in a single call.
226 In this case, we use it simply and create an active IPV4 Internet stream
227 socket which can read or write utf8 text and that understands a particular
228 newline convention in the underlying data.
230 One small, but important, deviation of IOLib sockets from Berkeley sockets
231 is that when a socket is created, it is predestined to forever and
232 unalterably be either an active or passive socket. Active sockets are used
233 to connect to a server and passive sockets are used for a server's
234 listening socket.
236 +------------------------------------------------------------------------------+
237 | |
238 ;; Create a internet TCP socket under IPV4
239 (let ((socket
240 (make-socket
241 :connect :active
242 :address-family :internet
243 :type :stream
244 :external-format '(:utf-8 :eol-style :crlf)
245 :ipv6 nil)))
246 | |
247 +------------------------------------------------------------------------------+
249 2. Specify the Server's IP address and port and establish a connection
250 with the server:
252 This bit of code contains many calls into IOLib and we shall examine each
253 of them.
255 The function LOOKUP-HOSTNAME takes as a string the DNS name
256 for a machine and returns 4 values:
257 A. an address
258 B. a list of additional addresses(if existent)
259 C. the canonical name of the host
260 D. an alist of all the host's names with their respective addresses
262 We use only the first return value, the address component, to pass to the
263 function CONNECT.
265 The function CONNECT will connect the socket to the address, but to a
266 random port if the :port keyword argument is not specified. The average
267 client codes usually use :wait t to block until the connect can resolve
268 with a connected fd or an error. The exception to always using :wait t is
269 if the client needs to connect to many servers at once, suppose a web
270 client, or if a server is also a client in other contexts and wishes not to
271 block.
273 The functions REMOTE-HOST and REMOTE-PORT return the ip address and port of
274 the remote connection associated with the connected socket. LOCAL-HOST and
275 LOCAL-PORT return the information of the client's end of the connected
276 socket. Analogous calls REMOTE-NAME and LOCAL-NAME each return two values
277 where the first value is the equivalent of *-host and the second value is
278 the equivalent of *-port.
280 +------------------------------------------------------------------------------+
281 | |
282 ;; do a blocking connect to the daytime server on the port.
283 (connect socket (lookup-hostname host) :port port :wait t)
284 (format t "Connected to server ~A:~A via my local connection at ~A:~A!~%"
285 (remote-host socket) (remote-port socket)
286 (local-host socket) (local-port socket))
287 | |
288 +------------------------------------------------------------------------------+
290 3. Read and display the server's reply:
292 Now that the socket has been connected to the server, the server will send
293 a line of text to the client. The client uses the standard Common Lisp
294 function READ-LINE to read the information from the socket. The function
295 READ-LINE blocks and will only return when an *entire line* is read. Once
296 read, the line is emitted to *standard-output* via the function call
297 FORMAT.
299 +------------------------------------------------------------------------------+
300 | |
301 ;; read the one line of information I need from the daytime
302 ;; server. I can use read-line here because this is a TCP socket.
303 (let ((line (read-line socket)))
304 (format t "~A" line))
305 | |
306 +------------------------------------------------------------------------------+
308 4. End program:
310 We close the socket with the standard function CLOSE and return true so the
311 return value of this example is t.
313 +------------------------------------------------------------------------------+
314 | |
315 ;; all done
316 (close socket)
317 t))
318 | |
319 +------------------------------------------------------------------------------+
321 While this program works, it has some major flaws in it. First and foremost is
322 that it doesn't handle any conditions that IOLib signals in common use cases.
323 An example would be to run the ex1-client.lisp example without a daytime server
324 running. In most, if not all, Common Lisp toplevels, you'll be dropped into the
325 debugger on an unhandled SOCKET-CONNECTION-REFUSED-ERROR condition. Secondly,
326 it isn't written in the Common Lisp style.
328 Daytime Client IVP4/TCP: ex2-client.lisp
329 ----------------------------------------
331 In this example, we simply tackle the fact ex1-server.lisp can be shortened
332 with an IOLib form to something where the application writer has less to do
333 concerning cleaning up the socket object. It also uses line oriented blocking
334 I/O.
336 The introduced macro WITH-OPEN-SOCKET calls MAKE-SOCKET with the arguments in
337 question and binds the socket to the variable 'socket'. When this form returns,
338 it will automatically close the socket.
340 This shortens the program so much, that the example can be included in its
341 entirety:
343 +------------------------------------------------------------------------------+
344 | |
345 (defun run-ex2-client (&key (host *host*) (port *port*))
347 ;; We introduce with-open-socket here as a means to easily wrap
348 ;; usually synchronous and blocking communication with a form that
349 ;; ensures the socket is closed no matter how we exit it.
350 (with-open-socket
351 (socket :connect :active
352 :address-family :internet
353 :type :stream
354 :external-format '(:utf-8 :eol-style :crlf)
355 :ipv6 nil)
357 ;; Do a blocking connect to the daytime server on the port. We
358 ;; also introduce lookup-hostname, which converts a hostname to an
359 ;; 4 values, but in our case we only want the first, which is an
360 ;; address.
361 (connect socket (lookup-hostname host) :port port :wait t)
362 (format t "Connected to server ~A:~A from my local connection at ~A:~A!~%"
363 (remote-name socket) (remote-port socket)
364 (local-name socket) (local-port socket))
366 ;; read the one line of information I need from the daytime
367 ;; server. I can use read-line here because this is a TCP
368 ;; socket. It will block until the whole line is read.
369 (let ((line (read-line socket)))
370 (format t "~A" line)
371 t)))
372 | |
373 +------------------------------------------------------------------------------+
375 This shorthand can go even further, if we add this to the WITH-OPEN-SOCKET
376 flags
378 :remote-host (lookup-hostname host)
379 :remote-port port
381 then the underlying MAKE-SOCKET call will in fact connect the socket directly
382 to the server before it is available for the body of the macro allowing us to
383 remove the connect call entirely! In the early examples, however, we don't
384 utilize IOLib's shorthand notations to this degree in order to make apparent
385 how the library maps into traditional socket concepts. After one gains
386 familiarity with the IOLib API, the situations where application of the
387 shortcuts are useful become much easier to see.
389 Daytime Client IVP4/TCP: ex3-client.lisp
390 ----------------------------------------
392 Now we come to condition handling, which can moderately affect the layout of
393 your IOLib program. Any real program using IOLib must handle IOLib's signaled
394 conditions which are common to the boundary cases of network programming.
395 We've already seen one of these boundary cases when we tried to connect a
396 daytime client to a server that wasn't running. The condition signaled in that
397 case was: SOCKET-CONNECTION-REFUSED-ERROR. The stream interface has a set of
398 conditions which IOLib will signal, and another lower level IOLib layer--which
399 we'll come to in the nonblocking I/O examples have another set of conditions.
400 There is some intersection between them and we will explore that later. For
401 now, we'll just use the conditions associated with a stream.
403 Our rewrite of ex2-client.lisp into ex3-client.lisp (continuing to use line
404 oriented blocking I/O) proceeds thusly:
406 0. We create a helper function which connects to the server and reads the
407 daytime line:
409 Notice the HANDLER-CASE macro around the portion of the function which
410 reads the date from the server. In looking at the boundary conditions from
411 the server given this protocol, we can receive an END-OF-FILE condition if
412 the client connected, but before the server could respond it exited,
413 closing the connection. Since in this case we're inside of a
414 WITH-OPEN-SOCKET form, we simply note that we got an END-OF-FILE and let
415 the cleanup forms of WITH-OPEN-SOCKET close the connection. If we don't
416 catch this condition, then the program will break into the debugger and
417 that isn't useful. It is usually debatable as to where one should handle
418 conditions: either near to or far away from the generating calls. In these
419 simple examples, no choice has any significant pros or cons. As your IOLib
420 programs become more and more complex, however, it becomes more obvious at
421 what abstraction level to handle signaled conditions.
423 +------------------------------------------------------------------------------+
424 | |
425 (defun run-ex3-client-helper (host port)
427 ;; Create a internet TCP socket under IPV4
428 (with-open-socket
429 (socket :connect :active
430 :address-family :internet
431 :type :stream
432 :external-format '(:utf-8 :eol-style :crlf)
433 :ipv6 nil)
435 ;; do a blocking connect to the daytime server on the port.
436 (connect socket (lookup-hostname host) :port port :wait t)
437 (format t "Connected to server ~A:~A from my local connection at ~A:~A!~%"
438 (remote-name socket) (remote-port socket)
439 (local-name socket) (local-port socket))
441 (handler-case
442 ;; read the one line of information I need from the daytime
443 ;; server. I can use read-line here because this is a TCP
444 ;; socket. It will block until the whole line is read.
445 (let ((line (read-line socket)))
446 (format t "~A" line)
447 t)
449 ;; However, let's notice the signaled condition if the server
450 ;; went away prematurely...
451 (end-of-file ()
452 (format t "Got end-of-file. Server closed connection!")))))
453 | |
454 +------------------------------------------------------------------------------+
456 1. Some conditions which are complete show-stoppers to the functioning of the
457 code are caught at a higher level:
459 Notice we catch the possible SOCKET-CONNECTION-REFUSED-ERROR from the
460 connect inside of the function run-ex3-client-helper.
462 +------------------------------------------------------------------------------+
463 | |
464 ;; The main entry point into ex3-client
465 (defun run-ex3-client (&key (host *host*) (port *port*))
466 (handler-case
468 (run-ex3-client-helper host port)
470 ;; handle a commonly signaled error...
471 (socket-connection-refused-error ()
472 (format t "Connection refused to ~A:~A. Maybe the server isn't running?~%"
473 (lookup-hostname host) port))))
474 | |
475 +------------------------------------------------------------------------------+
477 Here are some common conditions in IOLib (some from ANSI Common Lisp too) and
478 under what situations they are signaled. In any IOLib program, *at least*
479 these conditions should be handled where appropriate.
481 END-OF-FILE:
482 When a stream function such as READ, READ-LINE, etc...(but not
483 RECEIVE-FROM), reads from a socket where the other end has been closed.
485 HANGUP:
486 When writing to a socket with a stream function such as WRITE,
487 FORMAT, etc...(but not SEND-TO), if the socket is closed then this
488 condition is signaled.
490 SOCKET-CONNECTION-RESET-ERROR:
491 When doing I/O on a socket and the other side of the socket sent a
492 RST packet, this condition is signaled. It can also happen with
493 the IOLIb function ACCEPT and similar.
495 SOCKET-CONNECTION-REFUSED-ERROR:
496 Signaled by connect if there is no server waiting to accept the incoming
497 connection.
500 Daytime Servers
501 ---------------
503 Now that we have completed the evolution of the daytime client, let's look at
504 the daytime servers.
506 The exposition of the servers follows in style of the clients.
508 Daytime Server IVP4/TCP: ex1-server.lisp
509 ----------------------------------------
511 This first example is an iterative server which handles a single client and
512 then exits. The I/O is blocking and no error handling is performed. This is
513 similar in scope to the ex1-client.lisp example.
515 0. Create the server socket:
517 We see that the socket is :passive. Every socket in IOLib is predestined to
518 be either an active or passive socket and since this is a server socket, it
519 is passive. Also here we see that we can ask for the underlying fd of the
520 socket with the function SOCKET-OS-FD.
522 +------------------------------------------------------------------------------+
523 | |
524 (defun run-ex1-server (&key (port *port*))
525 ;; Create a passive (server) TCP socket under IPV4 Sockets meant to
526 ;; be listened upon *must* be created passively. This is a minor
527 ;; deviation from the Berkeley socket interface.
528 (let ((socket
529 (make-socket
530 :connect :passive
531 :address-family :internet
532 :type :stream
533 :external-format '(:utf-8 :eol-style :crlf)
534 :ipv6 nil)))
535 (format t "Created socket: ~A[fd=~A]~%" socket (socket-os-fd socket))
536 | |
537 +------------------------------------------------------------------------------+
539 1. Bind the socket
541 Binding a socket is what gives it an endpoint to which clients can connect.
542 The IOLib constant +IPV4-UNSPECIFIED+ represents 0.0.0.0 and means if a
543 connection arrives on any interface, it will be accepted if it comes to the
544 :port specified. The :reuse-addr keyword represents the socket option
545 SO_REUSEADDR and states (among other things) that if the socket is in the
546 TIME_WAIT state it can be reused immediately. It is recommended that all
547 servers use :reuse-addr on their listening socket.
549 +------------------------------------------------------------------------------+
550 | |
551 ;; Bind the socket to all interfaces with specified port.
552 (bind-address socket
553 +ipv4-unspecified+ ; which means INADDR_ANY or 0.0.0.0
554 :port port
555 :reuse-addr t)
556 (format t "Bound socket: ~A~%" socket)
557 | |
558 +------------------------------------------------------------------------------+
560 2. Listen on the socket
562 Listening on a socket allows clients to connect. In this example, we've
563 specified that 5 pending connection can be queued up in the kernel before
564 being accepted by the process.
566 +------------------------------------------------------------------------------+
567 | |
568 ;; Convert the sockxet to a listening socket
569 (listen-on socket :backlog 5)
570 (format t "Listening on socket bound to: ~A:~A~%"
571 (local-host socket) (local-port socket))
572 | |
573 +------------------------------------------------------------------------------+
575 3. Accept the client connection.
577 Here we finally call the IOLib function ACCEPT-CONNECTION. We would like it
578 to block, so we pass it :wait t. When ACCEPT-CONNECTION returns it will
579 return a new socket which represents the connection to the client.
580 ACCEPT-CONNECTION can return nil under some situations, such as on a slow
581 server when the client sent a TCP RST packet in between the time the kernel
582 sees the connection attempt and ACCEPT-CONNECTION is actually called. We
583 also opt to use the function REMOTE-NAME, which returns two values, the ip
584 address and port of the remote side of the socket.
586 +------------------------------------------------------------------------------+
587 | |
588 ;; Block on accepting a connection
589 (format t "Waiting to accept a connection...~%")
590 (let ((client (accept-connection socket :wait t)))
591 (when client
592 ;; When we get a new connection, show who it is from.
593 (multiple-value-bind (who rport)
594 (remote-name client)
595 (format t "Got a connection from ~A:~A!~%" who rport))
596 | |
597 +------------------------------------------------------------------------------+
599 4. Write the time to the client.
601 Here we've figured out the time string and wrote it to the client. Notice
602 we call the function FINISH-OUTPUT. This ensures that all output is written
603 to the client socket. For streams using blocking I/O, it is recommended
604 that every write to a blocking socket be followed up with a call to
605 FINISH-OUTPUT.
607 +------------------------------------------------------------------------------+
608 | |
609 ;; Since we're using a internet TCP stream, we can use format
610 ;; with it. However, we should be sure to call finish-output on
611 ;; the socket in order that all the data is sent. Also, this is
612 ;; a blocking write.
613 (multiple-value-bind (s m h d mon y)
614 (get-decoded-time)
615 (format t "Sending the time...")
616 (format client "~A/~A/~A ~A:~A:~A~%" mon d y h m s)
617 (finish-output client))
618 | |
619 +------------------------------------------------------------------------------+
621 5. Close the connection to the client.
623 We're done writing to the client, so close the connection so the client
624 knows it got everything.
626 +------------------------------------------------------------------------------+
627 | |
628 ;; We're done talking to the client.
629 (close client)
630 (format t "Sent!~%"))
631 | |
632 +------------------------------------------------------------------------------+
634 6. Close the server's socket.
636 Since this is a one shot server, we close the listening socket and exit. In
637 this and all other servers we call FINISH-OUTPUT to flush all pending
638 message to *standard-output*, if any.
640 +------------------------------------------------------------------------------+
641 | |
642 ;; We're done with the server socket too.
643 (close socket)
644 (finish-output)
645 t)))
646 | |
647 +------------------------------------------------------------------------------+
649 The above code is the basic idea for how a very simple TCP blocking I/O server
650 functions. Like ex1-client, this server suffers from the inability to handle
651 common signaled conditions such as a HANGUP from the client--which means the
652 client went away before the server could write the time to it.
654 However, one major, and subtle, problem of this particular example is that the
655 socket to the client is *not immediately closed* if the server happens to exit,
656 say by going through the debugger back to toplevel--or a signaled condition,
657 before writing the date to the client. If this happens, it can take a VERY long
658 time for the socket to be garbage collected and closed. In this scenario, the
659 client will hang around waiting for data which will never come until the Lisp
660 implementation closes the socket when it gets around to collecting it. Garbage
661 collection is an extremely nice feature of Common Lisp, but non-memory OS
662 resources in general should be eagerly cleaned up. Clients can suffer from
663 this problem too, leaving open, but unmanipulable, sockets to servers.
665 All clients or servers written against IOLib should either use some IOLib
666 specific macros to handle closing of socket, Common Lisp's condition system
667 like handler-case to catch the signaled conditions, or some other manual
668 solution.
670 Daytime Server IVP4/TCP: ex2-server.lisp
671 ----------------------------------------
673 Similarly to ex2-client, this server uses the macro WITH-OPEN-SOCKET to open
674 the server socket. We introduce WITH-ACCEPT-CONNECTION to accept the client and
675 convert this server from a single shot server to an iterative server which can
676 handle, in a serial fashion only, multiple clients.
678 0. Serially accept and process clients:
680 This portion of ex2-server shows the infinite loop around the accepting of
681 the connection. The macro WITH-ACCEPT-CONNECTION takes the server socket
682 and introduces a new binding: client, which is the accepted connection. We
683 ensure to tell the accept we'd like to be blocking. If for whatever reason
684 we exit the body, it'll clean up the client socket automatically.
686 +------------------------------------------------------------------------------+
687 | |
688 ;; Keep accepting connections forever.
689 (loop
690 (format t "Waiting to accept a connection...~%")
692 ;; Using with-accept-connection, when this form returns it will
693 ;; automatically close the client connection.
694 (with-accept-connection (client server :wait t)
695 ;; When we get a new connection, show who it is from.
696 (multiple-value-bind (who rport)
697 (remote-name client)
698 (format t "Got a connnection from ~A:~A!~%" who rport))
700 ;; Since we're using a internet TCP stream, we can use format
701 ;; with it. However, we should be sure to finish-output in
702 ;; order that all the data is sent.
703 (multiple-value-bind (s m h d mon y)
704 (get-decoded-time)
705 (format t "Sending the time...")
706 (format client "~A/~A/~A ~A:~A:~A~%" mon d y h m s)
707 (finish-output client)
708 (format t "Sent!~%")
709 (finish-output)
710 t)))))
711 | |
712 +------------------------------------------------------------------------------+
714 For very simple blocking I/O servers like this one, serially accepting and
715 handling client connections isn't so much of a problem, but if the server does
716 anything which takes a lot of time or has to send lots of data back and forth
717 to many persistent clients, then this is a poor design. The means by which you
718 exit this server is by breaking evaluation and returning to the toplevel. When
719 this happens, the WITH-* forms automatically close the connection to the
720 client.
722 Daytime Server IVP4/TCP: ex3-server.lisp
723 ----------------------------------------
725 In this iterative and blocking I/O server example, we add the handling of the
726 usual signaled conditions in network boundary cases often found with sockets.
727 Like the earlier client where we introduced HANDLER-CASE, this involves a
728 little bit of restructuring of the codes.
730 0. A helper function which opens a passive socket, binds it, and
731 listens on it:
733 There is nothing new in this portion of the code. We've seen this pattern
734 before. In production code, we could probably shorten this further by
735 having WITH-OPEN-SOCKET do the binding and connecting with appropriate
736 keyword arguments.
738 +------------------------------------------------------------------------------+
739 | |
740 (defun run-ex3-server-helper (port)
741 (with-open-socket
742 (server :connect :passive
743 :address-family :internet
744 :type :stream
745 :ipv6 nil
746 :external-format '(:utf-8 :eol-style :crlf))
748 (format t "Created socket: ~A[fd=~A]~%" server (socket-os-fd server))
750 ;; Bind the socket to all interfaces with specified port.
751 (bind-address server +ipv4-unspecified+ :port port :reuse-addr t)
752 (format t "Bound socket: ~A~%" server)
754 ;; start listening on the server socket
755 (listen-on server :backlog 5)
756 (format t "Listening on socket bound to: ~A:~A~%"
757 (local-host server)
758 (local-port server))
759 | |
760 +------------------------------------------------------------------------------+
762 1. Repeatedly handle clients in a serial fashion:
764 The new material in this function is the HANDLER-CASE around sending the
765 client the time information. The boundary conditions when writing to a
766 client include the server getting a reset (RST) from the client or
767 discovering the client had gone away and there is no-one to which to write.
768 Since the write is contained within the WITH-ACCEPT-CONNECTION form, if any
769 of these conditions happen, we simply notice that they happened and let the
770 form clean up the socket when it exits. If we didn't catch the conditions,
771 however, we'd break into the debugger.
773 One might ask what the value of catching these conditions here is at all
774 since we don't actually do anything with them--other than printing a
775 message and preventing the code from breaking into the debugger. For the
776 purposes of the tutorial, it is intended that the reader induce the
777 boundary cases manually and see the flow of the code and to understand
778 exactly what conditions may be signaled under what conditions and how to
779 structure code to deal with them. In production code where the author might
780 not care about these conditions at all, one might simply ignore all the
781 signaled conditions that writing to the client might cause.
783 Of course, the appropriateness of ignoring network boundary conditions is
784 best determined by context.
786 +------------------------------------------------------------------------------+
787 | |
788 ;; keep accepting connections forever.
789 (loop
790 (format t "Waiting to accept a connection...~%")
792 ;; Here we see with-accept-connection which simplifies closing
793 ;; the client socket when are done with it.
794 (with-accept-connection (client server :wait t)
795 ;; When we get a new connection, show who it
796 ;; is from.
797 (multiple-value-bind (who rport)
798 (remote-name client)
799 (format t "Got a connnection from ~A:~A!~%" who rport))
801 ;; Since we're using an internet TCP stream, we can use format
802 ;; with it. However, we should be sure to finish-output in
803 ;; order that all the data is sent.
804 (multiple-value-bind (s m h d mon y)
805 (get-decoded-time)
806 (format t "Sending the time...")
808 ;; Catch the condition of the client closing the connection.
809 ;; Since we exist inside a with-accept-connection, the
810 ;; socket will be automatically closed.
811 (handler-case
812 (progn
813 (format client "~A/~A/~A ~A:~A:~A~%" mon d y h m s)
814 (finish-output client))
816 (socket-connection-reset-error ()
817 (format t "Client reset connection!~%"))
819 (hangup ()
820 (format t "Client closed conection!~%")))
822 (format t "Sent!~%"))))
823 | |
824 +------------------------------------------------------------------------------+
826 2. End of the helper function, returns T to whomever called it:
828 +------------------------------------------------------------------------------+
829 | |
830 t))
831 | |
832 +------------------------------------------------------------------------------+
834 3. The entry point into this example:
836 We handle the condition SOCKET-ADDRESS-IN-USE-ERROR which is most commonly
837 signaled when we try to bind a socket to address which already has a server
838 running on it or when the address is in the TIME_WAIT state. The latter
839 situation is so common--usually caused by a server just having exited and
840 another one starting up to replace it, that when binding addresses, one
841 should supply the keyword argument :reuse-addr with a true value to
842 BIND-ADDRESS to allow binding a socket to an address in TIME_WAIT state.
844 +------------------------------------------------------------------------------+
845 | |
846 ;; This is the main entry point into the example 3 server.
847 (defun run-ex3-server (&key (port *port*))
848 (handler-case
850 (run-ex3-server-helper port)
852 (socket-address-in-use-error ()
853 ;; Here we catch a condition which represents trying to bind to
854 ;; the same port before the first one has been released by the
855 ;; kernel. Generally this means you forgot to put ':reuse-addr
856 ;; t' as an argument to bind address.
857 (format t "Bind: Address already in use, forget :reuse-addr t?")))
859 (finish-output))
860 | |
861 +------------------------------------------------------------------------------+
863 Daytime Server IVP4/TCP: ex4-server.lisp
864 ----------------------------------------
866 This is the first of our concurrent servers and the last of our daytime
867 protocol servers. Usually concurrency is introduced (in the UNIX environment)
868 with the fork() library call which creates an entirely new process with
869 copy-on-write semantics to handle the connection to the client. In this
870 tutorial environment, we've chosen to render this idea with the portable
871 threading library Bordeaux Threads. The I/O is still line oriented and
872 blocking, however, when a thread blocks another can run giving the illusion of
873 a server handling multiple clients in a non-blocking fashion.
875 We also introduce UNWIND-PROTECT ensures that various sockets are closed under
876 various boundary conditions in the execution of the server. An UNWIND-PROTECT
877 executes a single form, and after the evaluation, or interruption, of that
878 form, evaluates a special cleanup form. The cleanup form is *always* evaluated
879 and we use this to cleanup non-memory system resources like sockets.
881 Threads present their own special problems in the design of a server. Two
882 important problems are: data races and thread termination. The tutorial tries
883 very hard to avoid any data races in the examples and this problem is
884 ultimately solvable using Bordeaux-Threads mutexes or condition variables. Our
885 simple examples do not need mutexes as they do not share any data between
886 themselves.
888 The harder problem is thread termination. Since the tutorial encourages
889 experimentation with the clients and servers in a REPL, threads may leak when
890 the server process' initial thread stops execution and goes back to the REPL.
891 We use three API calls from the Bordeaux Threads: THREAD-ALIVE-P, ALL-THREADS,
892 and DESTROY-THREAD--which are not to be used in normal thread programming. We
893 do this here in order to try and clean up leaked threads so the clients know
894 immediately when the server process stopped and we don't pollute the REPL with
895 an ever increasing number of executing threads. The employed method of
896 destroying the threads, on SBCL specifically, allows the invocation of the
897 thread's UNWIND-PROTECT's cleanup form, which closes the socket to the client
898 before destroying the thread. On other implementations of Common Lisp, we are
899 not guaranteed that the thread's UNWIND-PROTECT cleanup form will be evaluated
900 when we destroy it.
902 This method is also extremely heavy handed in that it uses the function
903 IGNORE-ERRORS to ignore any condition that Bordeaux Thread's DESTROY-THREAD may
904 have signaled, including important conditions like HEAP-EXHAUSTED-ERROR, an
905 SBCL specific condition. In a real threaded server, the exiting of the initial
906 thread (which means exiting of the runtime and termination of the entire Lisp
907 process) will destroy all other threads as the process tears itself down and
908 exits. This is the recommended way a threaded server should exit.
910 Since threading is implementation dependent for what guarantees are provided,
911 any non-toy threaded network server will probably use the native implementation
912 of threads for a specific Common Lisp implementation. An example difficult
913 situation would be trying to terminate a thread which is blocked on I/O.
914 Different implementations would handle this in different ways.
916 The two provided examples, ex4-server and ex5-server, provide a general idea
917 for the structuring of the code to utilize threads.
919 Here is the dissection of ex4-server:
921 0. A special variable which will allow the initial thread to pass a client
922 socket to a thread handling said client:
924 +------------------------------------------------------------------------------+
925 | |
926 ;; This variable is the means by which we transmit the client socket from
927 ;; the initial thread to the particular thread which will handle that client.
928 (defvar *ex4-tls-client* nil)
929 | |
930 +------------------------------------------------------------------------------+
932 1. A helper function which begins with the usual recipe for a server:
934 +------------------------------------------------------------------------------+
935 | |
936 (defun run-ex4-server-helper (port)
937 (with-open-socket
938 (server :connect :passive
939 :address-family :internet
940 :type :stream
941 :ipv6 nil
942 :external-format '(:utf-8 :eol-style :crlf))
944 (format t "Created socket: ~A[fd=~A]~%" server (socket-os-fd server))
946 ;; Bind the socket to all interfaces with specified port.
947 (bind-address server +ipv4-unspecified+ :port port :reuse-addr t)
948 (format t "Bound socket: ~A~%" server)
950 ;; start listening on the server socket
951 (listen-on server :backlog 5)
952 (format t "Listening on socket bound to: ~A:~A~%"
953 (local-host server)
954 (local-port server))
955 | |
956 +------------------------------------------------------------------------------+
958 2. Forever more, accept a client connection on the listening socket
959 and start a thread which handles it:
961 There is a lot going on in this piece of code. The first thing to notice is
962 the UNWIND-PROTECT and its cleanup form. The form which UNWIND-PROTECT is
963 guarding is an infinite loop which does a blocking accept to get a client
964 socket, rebinds *default-special-bindings* adding to its assoc list the
965 binding for *ex4-tls-client*, and creates a thread which handles the
966 client.
968 The cleanup form walks all of the active client threads and destroys them,
969 ignoring any conditions that may have arose while doing so. Destroying the
970 threads prevents them from piling up and eventually causing havoc if many
971 servers start and exit over time. In addition, it forces an eager close on
972 the client sockets allowing any clients to know the server went away
973 immediately.
975 +------------------------------------------------------------------------------+
976 | |
977 ;; Here we introduce unwind-protect to ensure we properly clean up
978 ;; any leftover threads when the server exits for whatever reason.
979 ;; keep accepting connections forever, but if this exits for
980 ;; whatever reason ensure to destroy any remaining running
981 ;; threads.
982 (unwind-protect
983 (loop ; keep accepting connections...
984 (format t "Waiting to accept a connection...~%")
985 (finish-output)
986 (let* ((client (accept-connection server :wait t))
987 ;; set up the special variable according to the
988 ;; needs of the Bordeaux Threads package to pass in
989 ;; the client socket we accepted to the about to be
990 ;; created thread. *default-special-bindings* must
991 ;; not be modified, so here we just push a new scope
992 ;; onto it.
993 (*default-special-bindings*
994 (acons '*ex4-tls-client* client
995 *default-special-bindings*)))
997 ;; ...and handle the connection!
998 (when client
999 (make-thread #'process-ex4-client-thread
1000 :name 'process-ex4-client-thread))))
1002 ;; Clean up form for uw-p.
1003 ;; Clean up all of the client threads when done.
1004 ;; This code is here for the benefit of the REPL because it is
1005 ;; intended that this tutorial be worked interactively. In a real
1006 ;; threaded server, the server would just exit--destroying the
1007 ;; server process, and causing all threads to exit which then notifies
1008 ;; the clients.
1009 (format t "Destroying any active client threads....~%")
1010 (mapc #'(lambda (thr)
1011 (when (and (thread-alive-p thr)
1012 (string-equal "process-ex4-client-thread"
1013 (thread-name thr)))
1014 (format t "Destroying: ~A~%" thr)
1015 ;; Ignore any conditions which might arise if a
1016 ;; thread happened to finish in the race between
1017 ;; liveness testing and destroying.
1018 (ignore-errors
1019 (destroy-thread thr))))
1020 (all-threads)))))
1021 | |
1022 +------------------------------------------------------------------------------+
1024 3. The beginning of the thread handling the client:
1026 When the thread is born, the aforementioned explicit binding of the client
1027 socket to *ex4-tls-client* takes effect via the *default-special-bindings*
1028 mechanism. By declaring *ex4-tls-client* ignorable, we inform the compiler
1029 that this variable is set "elsewhere" and no warning should be emitted
1030 about its possibly undefined value. In our case, this will always be
1031 defined at runtime in this server.
1033 +------------------------------------------------------------------------------+
1034 | |
1035 ;;; The thread which handles the client connection.
1036 (defun process-ex4-client-thread ()
1037 ;; This variable is set outside of the context of this thread.
1038 (declare (ignorable *ex4-tls-client*))
1039 | |
1040 +------------------------------------------------------------------------------+
1042 4. Send the time to the socket:
1044 The UNWIND-PROTECT in this form handles every possible case of leaving the
1045 evaluable function such as it completing normally, a condition being
1046 signaled, or by thread destruction--on SBCL! In all cases, the socket to
1047 the client is closed which cleans up OS resources and lets the client know
1048 right away the server has closed the connection. The HANDLER-CASE form here
1049 just informs us which of the common IOLib conditions may have been signaled
1050 while writing the time to the client.
1052 +------------------------------------------------------------------------------+
1053 | |
1054 ;; We ensure the client socket is always closed!
1055 (unwind-protect
1056 (multiple-value-bind (who port)
1057 (remote-name *ex4-tls-client*)
1058 (format t "A thread is handling the connection from ~A:~A!~%"
1059 who port)
1061 ;; Prepare the time and send it to the client.
1062 (multiple-value-bind (s m h d mon y)
1063 (get-decoded-time)
1064 (handler-case
1065 (progn
1066 (format t "Sending the time to ~A:~A..." who port)
1067 (format *ex4-tls-client*
1068 "~A/~A/~A ~A:~A:~A~%"
1069 mon d y h m s)
1070 (finish-output *ex4-tls-client*)
1071 (format t "Sent!~%"))
1073 (socket-connection-reset-error ()
1074 (format t "Client ~A:~A reset the connection!~%" who port))
1076 (hangup ()
1077 (format t "Client ~A:~A closed connection.~%" who port)))))
1079 ;; Cleanup form for uw-p.
1080 (format t "Closing connection to ~A:~A!~%"
1081 (remote-host *ex4-tls-client*) (remote-port *ex4-tls-client*))
1082 (close *ex4-tls-client*)))
1083 | |
1084 +------------------------------------------------------------------------------+
1086 It is a bit tricky to robustly handle closing of the client socket in the
1087 thread. For example, if we bound the special variable *ex4-tls-client* to a
1088 lexically scoped variable and then did the UNWIND-PROTECT form to close the
1089 lexically scoped variable, then if this thread wakes up and gets destroyed
1090 after the lexical binding, but before the UNWIND-PROTECT, we'd lose a
1091 socket to a client into the garbage collector.
1093 Such incorrect code would look like:
1095 ;; This code is incorrect!
1096 (defun process-ex4-client-thread ()
1097 (declare (ignorable *ex4-tls-client*))
1098 (let ((client *ex4-tls-thread*))
1099 ;; thread gets destroyed right here! client socket is left open!
1100 (unwind-protect
1101 ( <evaluable form> )
1102 (close client))))
1104 5. The entry point into this example:
1106 Like earlier servers, we call the helper function and catch what happens if
1107 :reuse-addr wasn't true in the BIND-ADDRESS function call.
1109 +------------------------------------------------------------------------------+
1110 | |
1111 ;; The entry point into this example.
1112 (defun run-ex4-server (&key (port *port*))
1113 (handler-case
1115 (run-ex4-server-helper port)
1117 ;; handle some common signals
1118 (socket-address-in-use-error ()
1119 (format t "Bind: Address already in use, forget :reuse-addr t?")))
1121 (finish-output))
1122 | |
1123 +------------------------------------------------------------------------------+
1126 Daytime Client/Server Commentary
1127 --------------------------------
1129 This concludes the examples using the daytime protocol. We've seen patterns
1130 emerge in how the simplest of clients and servers are built and began to reason
1131 about how to handle common signaled conditions. Threading, of course, increases
1132 the care one must have in order to ensure that data access and control flow is
1133 kept consistent.
1135 Echo Line Clients and Servers
1136 -----------------------------
1138 These next examples focus on the echo protocol. This is simply a server that
1139 sends back to the client whatever the client wrote to it. A client can request
1140 to quit talking to a server (except ex7-server and ex8-server, where this
1141 feature isn't implemented) by sending the word "quit", on a line by itself.
1142 This tells the server to close the connection to the client once it has
1143 finished echoing the line. The closing of the client's read socket lets the
1144 client know the connection to the server went away and that it is time to exit.
1145 We also introduce the socket multiplexer interface which allows concurrent
1146 processing of socket connections. This is similar to how UNIX's select(),
1147 epoll(), or kqueue() works. Due to portability concerns on doing nonblocking
1148 operations on *standard-input* and *standard-output* (we can't easily do it) we
1149 are beholden to some form of blocking I/O in our clients because they interact
1150 with a human. We will explore true non-blocking I/O in the ex8-server example
1151 since that server only has to converse with connected clients.
1153 Echo Clients
1154 ------------
1156 The echo clients are a group of programs which read a line from
1157 *standard-input*, write it to the server, read back the response from the
1158 server, and emit the result to *standard-output*. While there is a portable
1159 method to read "however much is available" from *standard-input*, there isn't
1160 the symmetrical method to write "whatever I'm able" to *standard-output*. For
1161 our client design, this means that all of these clients are line oriented and
1162 do blocking I/O when reading from *standard-input* and writing to
1163 *standard-output*.
1165 Echo Client IPV4/TCP: ex4-client.lisp
1166 --------------------------------------
1168 This is a very basic echo client program that handles the usual conditions
1169 while talking to the server:
1171 0. Connect to the server and start echoing lines:
1173 Here we use WITH-OPEN-SOCKET to create an active socket that we then use to
1174 connect to the server. We handle HANGUP, for when the server went away
1175 before the client could write to it, and END-OF-FILE, for when the server
1176 closes down the connection.
1178 Notice we call the function ex4-str-cli inside of a HANDLER-CASE macro.
1179 This allows us to not check for any signaled conditions in ex4-str-cli and
1180 greatly simplifies its implementation.
1182 In this specific example, we don't do anything other than notify that the
1183 condition happened since after that the socket gets closed via
1184 WITH-OPEN-SOCKET.
1186 +------------------------------------------------------------------------------+
1187 | |
1188 (defun run-ex4-client-helper (host port)
1190 ;; Create a internet TCP socket under IPV4
1191 (with-open-socket
1192 (socket :connect :active
1193 :address-family :internet
1194 :type :stream
1195 :external-format '(:utf-8 :eol-style :crlf)
1196 :ipv6 nil)
1198 ;; do a blocking connect to the daytime server on the port.
1199 (connect socket (lookup-hostname host) :port port :wait t)
1201 (format t "Connected to server ~A:~A from my local connection at ~A:~A!~%"
1202 (remote-host socket) (remote-port socket)
1203 (local-host socket) (local-port socket))
1205 (handler-case
1206 (ex4-str-cli socket)
1208 (socket-connection-reset-error ()
1209 (format t "Got connection reset. Server went away!"))
1211 (hangup ()
1212 (format t "Got hangup. Server closed connection on write!~%"))
1214 (end-of-file ()
1215 (format t "Got end-of-file. Server closed connection on read!~%")))))
1216 | |
1217 +------------------------------------------------------------------------------+
1219 1. Echo lines to the server:
1221 Until the user inputs "quit" on a line by itself, we read a line, send it
1222 to the server, read it back, and emit it to stdout. If any of the usual
1223 conditions are signaled here, the handler-case in the Step 0 code fires and
1224 we deal with it there.
1226 When "quit" is entered, the line is sent on the round trip to the server
1227 like usual, but this time the server closes the connection to the client.
1228 Unfortunately, since the client is doing blocking I/O, we must read another
1229 line from *standard-input* before we get any signaled condition when IOLib
1230 discovers the socket has been closed by the server.
1232 In practice, this means after the server closed the connection, the user
1233 must hit <return> in order to drive the I/O loop enough to get the signaled
1234 condition.
1236 +------------------------------------------------------------------------------+
1237 | |
1238 ;; read a line from stdin, write it to the server, read the response, write
1239 ;; it to stdout. If we read 'quit' then echo it to the server which will
1240 ;; echo it back to us and then close its connection to us.
1241 (defun ex4-str-cli (socket)
1242 (loop
1243 (let ((line (read-line)))
1244 ;; send it to the server, get the response.
1245 (format socket "~A~%" line)
1246 (finish-output socket)
1247 (format t "~A~%" (read-line socket)))))
1248 | |
1249 +------------------------------------------------------------------------------+
1251 2. Entry point into the example:
1253 We handle the usual connection refused condition, but otherwise this step
1254 is unremarkable.
1256 +------------------------------------------------------------------------------+
1257 | |
1258 ;; This is the entry point into this example
1259 (defun run-ex4-client (&key (host *host*) (port *port*))
1260 (unwind-protect
1261 (handler-case
1263 (run-ex4-client-helper host port)
1265 ;; handle a commonly signaled error...
1266 (socket-connection-refused-error ()
1267 (format t "Connection refused to ~A:~A. Maybe the server isn't running?~%"
1268 (lookup-hostname host) port)))
1270 ;; Cleanup form
1271 (format t "Client Exited.~%")))
1272 | |
1273 +------------------------------------------------------------------------------+
1275 Echo Client IPV4/TCP: ex5a-client.lisp
1276 --------------------------------------
1278 This is the first client to use the socket multiplexer to notice when the
1279 socket to the server is ready for reading or writing. While the multiplexer is
1280 often used in single threaded servers it can be used for clients--especially
1281 clients which may talk to multiple servers like web clients. Use of the
1282 multiplexer API will require a significant change in how the code is
1283 structured. It is not recommended that the multiplexer and threads be used
1284 simultaneously to handle network connections.
1286 Keeping in mind the fact that we ALWAYS could block while reading from
1287 *standard-input* or writing to *standard-output*, we only attempt to read/write
1288 to the standard streams when the multiplexer thinks it can read/write to the
1289 server without blocking. This is a change from the traditional examples of how
1290 to do this in C because in C one can determine if STDIN or STDOUT are ready in
1291 the same manner as a network file descriptor.
1293 The first big change from our previous examples is that we stop using
1294 WITH-OPEN-SOCKET since now we must manually control when the socket to the
1295 server must be closed. This is especially important for clients who use active
1296 sockets. The second change is how we do the creation and registering of the
1297 handlers for reading and writing to the server socket. The third change is how
1298 to unregister a handler and close the socket associated with it under the right
1299 conditions. Other changes will be explained as we meet them.
1301 The main functions of the multiplexer API are:
1302 (make-instance 'iomux:event-base ....)
1303 Create an instance of the event-base, and associate some properties
1304 with it, such as event-dispatch should return if the multiplexer
1305 does not have any sockets it is managing.
1306 Passed an:
1307 :exit-when-empty - when no handlers are registered, event-dispatch
1308 will return.
1310 (event-dispatch ...)
1311 By default, sit in the multiplexer loop forever and handle I/O
1312 requests. It is passed the event-base binding and in addition:
1313 :once-only - run the ready handlers once then return.
1314 :timeout - when there is no I/O for a certain amount of time return.
1316 (set-io-handler ...)
1317 Associates a handler with a state to be called with a specific socket.
1318 Passed an:
1319 event-base binding
1320 :read or :write or :error keyword
1321 the handler closure
1323 (remove-fd-handlers ...)
1324 Removes a handler for a specific state with a specific socket.
1325 Passed an:
1326 event-base binding
1327 an fd
1328 one or more of :read t, :write t, :error t
1330 Here is the example using this API.
1332 0. The event base:
1334 The event-base is the object which holds the state of the multiplexer. It
1335 must be initialized and torn down as we'll see in the entry function to
1336 this example.
1338 +------------------------------------------------------------------------------+
1339 | |
1340 ;; This will be an instance of the multiplexer.
1341 (defvar *ex5a-event-base*)
1342 | |
1343 +------------------------------------------------------------------------------+
1345 1. A helper function in which we create the active socket:
1347 Instead of using WITH-OPEN-SOCKET, we manually create the socket. We do
1348 this to better control how to close the socket. WITH-OPEN-SOCKET will try
1349 to FINISH-OUTPUT on the socket before closing it. This is bad if the socket
1350 had been previously closed or signaled a condition like HANGUP. Trying to
1351 write more data to an already hung up socket will simply signal another
1352 condition. To prevent layers of condition handling code, we explicitly
1353 handle closing of the socket ourselves.
1355 +------------------------------------------------------------------------------+
1356 | |
1357 (defun run-ex5a-client-helper (host port)
1358 ;; Create a internet TCP socket under IPV4
1359 ;; We specifically do not use with-open-socket here since that form is
1360 ;; more suited for synchronous i/o on one socket. Since we do not use that
1361 ;; form, it is up to the handlers to decide to remove and close the socket
1362 ;; when the connection to the server should be closed.
1363 (let ((socket (make-socket :connect :active
1364 :address-family :internet
1365 :type :stream
1366 :external-format '(:utf-8 :eol-style :crlf)
1367 :ipv6 nil)))
1368 | |
1369 +------------------------------------------------------------------------------+
1371 2. Connect to the server, register the socket handlers:
1373 We protect the closing of the socket via UNWIND-PROTECT. We will talk about
1374 the ramifications of this decision in the next step which describes the
1375 UNWIND-PROTECT's cleanup form. In this section of code, we set up a read
1376 and write handler for the socket, and invoke the dispatch function, which
1377 will continue calling the handlers associated with the socket until the
1378 socket gets closed and the handlers unregistered. When this happens (see
1379 the entrance function step for why), EVENT-DISPATCH returns and we continue
1380 on to the cleanup form for the UNWIND-PROTECT.
1382 Setting up a handler in the multiplexer requires several arguments to
1383 the function set-io-handler. Here are what the arguments to that function
1384 are:
1385 1. *ex5a-event-base*
1386 This is the instance of the multiplexer for which we are setting
1387 up the handler.
1388 2. (socket-os-fd socket)
1389 This call returns the underlying operating system's file
1390 descriptor associated with the socket.
1391 3. :read
1392 This keyword states that we'd like to call the handler when the
1393 socket is ready to read. There is also :write and :error.
1394 4. (make-ex5a-str-cli-read socket
1395 (make-ex5a-client-disconnector socket))
1396 The make-ex5a-str-cli-read function returns a closure over the
1397 socket and another closure returned by the
1398 make-ex5a-client-disconnector function. This function is what will
1399 be called when the socket is ready for reading. We will shortly
1400 explain the signature of this function and what gets passed to it
1401 by the multiplexer. The disconnector function will be called by the
1402 returned reader function if the reader function thinks that it
1403 needs to close the socket to the server.
1405 +------------------------------------------------------------------------------+
1406 | |
1407 (unwind-protect
1408 (progn
1409 ;; do a blocking connect to the echo server on the port.
1410 (connect socket (lookup-hostname host) :port port :wait t)
1412 (format t "Connected to server ~A:~A from my local connection at ~A:~A!~%"
1413 (remote-host socket) (remote-port socket)
1414 (local-host socket) (local-port socket))
1416 ;; set up the handlers for read and write
1417 (set-io-handler *ex5a-event-base*
1418 (socket-os-fd socket)
1419 :read
1420 (make-ex5a-str-cli-read
1421 socket
1422 (make-ex5a-client-disconnector socket)))
1424 (set-io-handler *ex5a-event-base*
1425 (socket-os-fd socket)
1426 :write
1427 (make-ex5a-str-cli-write
1428 socket
1429 (make-ex5a-client-disconnector socket)))
1431 (handler-case
1432 ;; keep processing input and output on the fd by
1433 ;; calling the relevant handlers as the socket becomes
1434 ;; ready. The relevant handlers will take care of
1435 ;; closing the socket at appropriate times.
1436 (event-dispatch *ex5a-event-base*)
1438 ;; We'll notify the user of the client if a handler missed
1439 ;; catching common conditions.
1440 (hangup ()
1441 (format t "Uncaught hangup. Server closed connection on write!%"))
1442 (end-of-file ()
1443 (format t "Uncaught end-of-file. Server closed connection on read!%"))))
1444 | |
1445 +------------------------------------------------------------------------------+
1447 3. Cleanup form for UNWIND-PROTECT:
1449 In the cleanup form, we always close the socket and we pass the function
1450 close :abort t to try and close the socket in any way possible. If we just
1451 tried closing the socket, then we might cause another condition to be
1452 signaled if a previous condition, like HANGUP, had already affected the
1453 socket. :abort t avoids that case. If the socket is already closed by a
1454 handler by the time we get here, closing it again hurts nothing.
1456 +------------------------------------------------------------------------------+
1457 | |
1459 ;; Cleanup expression for uw-p.
1460 ;; Try to clean up if the client aborted badly and left the socket open.
1461 ;; It is safe to call close mutiple times on a socket.
1462 ;; However, we don't want to finish-output on the socket since that
1463 ;; might signal another condition since the io handler already closed
1464 ;; the socket.
1465 (format t "Client safely closing open socket to server.~%")
1466 (close socket :abort t))))
1467 | |
1468 +------------------------------------------------------------------------------+
1470 4. Make the writer function for when the socket is ready to write:
1472 This function returns a closure which is called by the multiplexer when it
1473 is ready to read something from the server. The arguments to the closure
1474 are fd, the underlying file descriptor for the ready socket, event, which
1475 could be :read, :write, or :error if the handler was registered multiple
1476 times, and exception, which is nil under normal conditions, :error under an
1477 error with the socket, or :timeout, if we were using timeout operations
1478 when dealing with the socket.
1480 The closure will read a line with the function READ-LINE and write it to
1481 the server. The read will be blocking, but hopefully the write won't be
1482 since the multiplexer told us we could perform the write and not block.
1483 Obviously, is we write an enormous line, then we might block again, and in
1484 this case the FINISH-OUTPUT on the socket will push the data in a blocking
1485 I/O fashion until it is done and we return from the handler. So while this
1486 closure for the most part writes when ready, there are cases under which
1487 it'll still block.
1489 In this handler, if there is a signaled condition either reading from
1490 *standard-input* (the END-OF-FILE condition) or writing to the server
1491 socket (the HANGUP condition), we invoke the disconnector closure and pass
1492 it :close. When we get to the description of the disconnector function,
1493 you'll see what that means.
1495 Once the disconnector closure is invoked, the handler will have been
1496 removed and the socket closed. This will make EVENT-DISPATCH return since
1497 the only socket it was multiplexing for was closed--because we've told the
1498 multiplexer to do so when it was made!
1500 +------------------------------------------------------------------------------+
1501 | |
1502 (defun make-ex5a-str-cli-write (socket disconnector)
1503 ;; When this next function gets called it is because the event dispatcher
1504 ;; knows the socket to the server is writable.
1505 (lambda (fd event exception)
1506 ;; Get a line from stdin, and send it to the server
1507 (handler-case
1508 (let ((line (read-line)))
1509 (format socket "~A~%" line)
1510 (finish-output socket))
1512 (end-of-file ()
1513 (format t "make-ex5a-str-cli-write: User performed end-of-file!~%")
1514 (funcall disconnector :close))
1516 (hangup ()
1517 (format t
1518 "make-ex5a-str-cli-write: server closed connection on write!~%")
1519 (funcall disconnector :close)))))
1520 | |
1521 +------------------------------------------------------------------------------+
1523 5. Make the reader function for when the socket is ready to read:
1525 This piece of code is very similar to the previous step's code, we just
1526 handle the appropriate conditions and after reading the line from the
1527 server emit it to *standard-output*. Again, even though we are told we can
1528 read from the server without blocking, if the read is large enough we will
1529 continue to block until read-line reads the all the data and the newline.
1531 +------------------------------------------------------------------------------+
1532 | |
1533 (defun make-ex5a-str-cli-read (socket disconnector)
1534 ;; When this next function gets called it is because the event dispatcher
1535 ;; knows the socket from the server is readable.
1536 (lambda (fd event exception)
1537 ;; get a line from the server, and send it to *standard-output*
1538 (handler-case
1539 ;; If we send "quit" to the server, it will close its connection to
1540 ;; us and we'll notice that with an end-of-file.
1541 (let ((line (read-line socket)))
1542 (format t "~A~%" line)
1543 (finish-output))
1545 (end-of-file ()
1546 (format t "make-ex5a-str-cli-read: server closed connection on read!~%")
1547 (funcall disconnector :close)))))
1548 | |
1549 +------------------------------------------------------------------------------+
1551 6. The disconnector function:
1553 This function returns a closure which takes an arbitrary number of
1554 arguments. If the arguments to the invoked closure contain :read, :write,
1555 or :error, the respective handler on the associated socket is removed. If
1556 none of those three are supplied, then all handlers for that socket are
1557 removed. Additionally if :close is specified, the socket is closed. While
1558 not all features of this function is used in this example, this function
1559 (or a similar one using the correct event-base special variable) is used
1560 whenever we use the multiplexer in an example.
1562 The closure is called whenever a handler believes it should unregister
1563 itself or another handler, or close the socket. Because we will often close
1564 the socket in the disconnector closure, we can't use WITH-OPEN-SOCKET to
1565 automatically close the socket because WITH-OPEN-SOCKET may try to flush
1566 data on the socket, signaling another condition.
1568 +------------------------------------------------------------------------------+
1569 | |
1570 (defun make-ex5a-client-disconnector (socket)
1571 ;; When this function is called, it can be told which callback to remove, if
1572 ;; no callbacks are specified, all of them are removed! The socket can be
1573 ;; additionally told to be closed.
1574 (lambda (&rest events)
1575 (format t "Disconnecting socket: ~A~%" socket)
1576 (let ((fd (socket-os-fd socket)))
1577 (if (not (intersection '(:read :write :error) events))
1578 (remove-fd-handlers *ex5a-event-base* fd :read t :write t :error t)
1579 (progn
1580 (when (member :read events)
1581 (remove-fd-handlers *ex5a-event-base* fd :read t))
1582 (when (member :write events)
1583 (remove-fd-handlers *ex5a-event-base* fd :write t))
1584 (when (member :error events)
1585 (remove-fd-handlers *ex5a-event-base* fd :error t)))))
1586 ;; and finally if were asked to close the socket, we do so here
1587 (when (member :close events)
1588 (close socket :abort t))))
1589 | |
1590 +------------------------------------------------------------------------------+
1592 7. The entry point for this example and setting up the event-base:
1594 This function is much more complex than in examples that do not use the
1595 multiplexer. Protected by an UNWIND-PROTECT, we first initialize the event
1596 base my calling make-instance 'iomux:event-base. Here is where we pass the
1597 keyword argument :exit-when-empty t which states that the event-dispatch
1598 function should return when there are no more registered handlers. Once
1599 that is done, we call the helper, catching a common condition and waiting
1600 until we return.
1602 +------------------------------------------------------------------------------+
1603 | |
1604 ;; This is the entry point for this example.
1605 (defun run-ex5a-client (&key (host *host*) (port *port*))
1606 (let ((*ex5a-event-base* nil))
1607 (unwind-protect
1608 (progn
1609 ;; When the connection gets closed, either intentionally in the client
1610 ;; or because the server went away, we want to leave the multiplexer
1611 ;; event loop. So, when making the event-base, we explicitly state
1612 ;; that we'd like that behavior.
1613 (setf *ex5a-event-base*
1614 (make-instance 'iomux:event-base :exit-when-empty t))
1615 (handler-case
1616 (run-ex5a-client-helper host port)
1618 ;; handle a commonly signaled error...
1619 (socket-connection-refused-error ()
1620 (format t "Connection refused to ~A:~A. Maybe the server isn't running?~%"
1621 (lookup-hostname host) port))))
1622 | |
1623 +------------------------------------------------------------------------------+
1625 8. The cleanup form for UNWIND-PROTECT:
1627 This cleanup form closes the *ex5a-event-base* instance. IOLib defines a
1628 method for the generic function CLOSE which accepts an event-base and
1629 performs the necessary work to shut it down.
1631 +------------------------------------------------------------------------------+
1632 | |
1633 ;; Cleanup form for uw-p
1634 ;; ensure we clean up the event base regardless of how we left the client
1635 ;; algorithm
1636 (when *ex5a-event-base*
1637 (close *ex5a-event-base*))
1638 (format t "Client Exited.~%")
1639 (finish-output))))
1640 | |
1641 +------------------------------------------------------------------------------+
1643 While this program works just fine with human input, it has a failure when
1644 reading batch input. The failure is that when we get the END-OF-FILE condition
1645 when *standard-input* closes, we _immediately_ unregister the read/write
1646 handlers to the server, close the socket and exit the program. This destroys
1647 any in-flight data to/from the server and lines being echoed may be lost.
1650 Echo Client IPV4/TCP: ex5b-client.lisp
1651 --------------------------------------
1653 In order to fix the batch input problem of ex5a-client, we will use the
1654 shutdown function which allows us to inform the server we are done writing
1655 data, but leave the socket open so we can read the rest of the responses from
1656 the server. This effectively closes only one-half of the TCP connection. The
1657 server has to be made aware of this kind of protocol so it doesn't assume the
1658 client completely exited when it gets an END-OF-FILE from the client and shuts
1659 down the whole connection throwing away any queued data for the client.
1661 This client is nearly identical to ex5a-client except we shut down the write
1662 end of the connection to the server when we get END-OF-FILE from
1663 *standard-input* and wait until we get all of the data back from the server.
1664 The server signifies to us that it has sent all of the pending data by closing
1665 the write end of its connection. The client sees the closing of the server's
1666 write end as an END-OF-FILE on the socket connected to the server.
1668 We show this example as a difference to ex5aq-client.
1670 0. Shutdown the write end of the socket to the server:
1672 Here we use the expanded functionality of the disconnector closure. After
1673 we shut down the write end of our TCP connection, we call (funcall
1674 disconnector :write) which states only to remove the write (to the server)
1675 handler, but leave the connection open. After this happens, there is no way
1676 we can read from *standard-input* again. Once the server sends the final
1677 data and the closes its connection to this client, we remove the read
1678 handler, which removes the last handler, and causes the EVENT-DISPATCH
1679 function to return, which ends the client computation.
1681 +------------------------------------------------------------------------------+
1682 | |
1683 (defun make-ex5b-str-cli-write (socket disconnector)
1684 ;; When this next function gets called it is because the event dispatcher
1685 ;; knows the socket to the server is writable.
1686 (lambda (fd event exception)
1687 ;; Get a line from stdin, and send it to the server
1688 (handler-case
1689 (let ((line (read-line)))
1690 (format socket "~A~%" line)
1691 (finish-output socket))
1693 (end-of-file ()
1694 (format t
1695 "make-ex5b-str-cli-write: User performed end-of-file!~%")
1696 ;; Shutdown the write end of my pipe to give the inflight data the
1697 ;; ability to reach the server!
1698 (format t
1699 "make-ex5b-str-cli-write: Shutting down write end of socket!~%")
1700 (shutdown socket :write t)
1701 ;; since we've shut down the write end of the pipe, remove this handler
1702 ;; so we can't read more data from *standard-input* and try to write it
1703 ;; it to the server.
1704 (funcall disconnector :write))
1706 (hangup ()
1707 (format t
1708 "make-ex5b-str-cli-write: server closed connection on write!~%")
1709 (funcall disconnector :close)))))
1710 | |
1711 +------------------------------------------------------------------------------+
1713 Be aware that even if both directions on one end of a connection are shutdown,
1714 close still must be called upon the socket in order to release resources held
1715 by the operating system.
1717 Echo Servers
1718 ------------
1720 The echo servers, paired to clients as per the beginning of this tutorial,
1721 further evolve to using the multiplexer and becoming more fine grained with
1722 respect to when I/O is done until we reach the ability to perform nonblocking
1723 I/O of arbitrary read/write sizes.
1725 Echo Server IPV4/TCP: ex5-server.lisp
1726 -------------------------------------
1728 This threaded server is very similar to ex4-server, but instead of sending only
1729 the time, each thread handles an echo protocol to a client. While this is
1730 still a blocking I/O server, only a single thread talking to a client gets
1731 blocked, not the whole server. Other than the server not honoring batch input
1732 from the client correctly, this is a common model for a class of servers due to
1733 its nonblocking behavior.
1735 0. The special variable used to communicate the client socket to the thread:
1737 +------------------------------------------------------------------------------+
1738 | |
1739 ;; The special variable used to hold the client socket for the thread
1740 ;; managing it.
1741 (defvar *ex5-tls-client* nil)
1742 | |
1743 +------------------------------------------------------------------------------+
1745 1. The usual prologue to a server:
1747 +------------------------------------------------------------------------------+
1748 | |
1749 (defun run-ex5-server-helper (port)
1750 (with-open-socket
1751 (server :connect :passive
1752 :address-family :internet
1753 :type :stream
1754 :ipv6 nil
1755 :external-format '(:utf-8 :eol-style :crlf))
1757 (format t "Created socket: ~A[fd=~A]~%" server (socket-os-fd server))
1759 ;; Bind the socket to all interfaces with specified port.
1760 (bind-address server +ipv4-unspecified+ :port port :reuse-addr t)
1761 (format t "Bound socket: ~A~%" server)
1763 ;; start listening on the server socket
1764 (listen-on server :backlog 5)
1765 (format t "Listening on socket bound to: ~A:~A~%"
1766 (local-host server)
1767 (local-port server))
1768 | |
1769 +------------------------------------------------------------------------------+
1771 2. First half of creating the client threads:
1773 +------------------------------------------------------------------------------+
1774 | |
1775 ;; keep accepting connections forever, but if this exits for whatever
1776 ;; reason ensure to destroy any remaining running threads.
1777 (unwind-protect
1778 (loop
1779 (format t "Waiting to accept a connection...~%")
1780 (finish-output)
1781 (let* ((client (accept-connection server :wait t))
1782 ;; set up the special variable to store the client
1783 ;; we accepted...
1784 (*default-special-bindings*
1785 (acons '*ex5-tls-client* client
1786 *default-special-bindings*)))
1788 ;; ...and handle the connection!
1789 (when client
1790 (make-thread #'process-ex5-client-thread
1791 :name 'process-ex5-client-thread))))
1792 | |
1793 +------------------------------------------------------------------------------+
1795 3. Second half, the cleanup form for the UNWIND-PROTECT:
1797 We make sure to clean up only the client threads!
1799 +------------------------------------------------------------------------------+
1800 | |
1801 ;; Clean up form for uw-p.
1802 ;; Clean up all of the client threads when done.
1803 ;; This code is here for the benefit of the REPL because it is
1804 ;; intended that this tutorial be worked interactively. In a real
1805 ;; threaded server, the server would just exit--destroying the
1806 ;; server process, and causing all threads to exit which then notifies
1807 ;; the clients.
1808 (format t "Destroying any active client threads....~%")
1809 (mapc #'(lambda (thr)
1810 (when (and (thread-alive-p thr)
1811 (string-equal "process-ex5-client-thread"
1812 (thread-name thr)))
1813 (format t "Destroying: ~A~%" thr)
1814 ;; Ignore any conditions which might arise if a
1815 ;; thread happened to finish in the race between
1816 ;; liveness testing and destroying.
1817 (ignore-errors
1818 (destroy-thread thr))))
1819 (all-threads)))))
1820 | |
1821 +------------------------------------------------------------------------------+
1823 4. Handle the client and deal with signaled conditions:
1825 In this function, we ensure that under all conditions of the execution of
1826 this function, if something goes wrong, we eagerly close the socket to the
1827 client so it is not leaked into the garbage collector. We also handle
1828 numerous conditions the the client could generate while talking to it in
1829 the function str-ex5-echo.
1831 +------------------------------------------------------------------------------+
1832 | |
1833 ;; The thread which handles the client connection.
1834 (defun process-ex5-client-thread ()
1835 ;; declared ignorable because this dynamic variable is bound outside
1836 ;; of the context of this function.
1837 (declare (ignorable *ex5-tls-client*))
1838 ;; no matter how we get out of the client processing loop, we always
1839 ;; close the connection.
1840 (unwind-protect
1841 (multiple-value-bind (who port)
1842 (remote-name *ex5-tls-client*)
1843 (format t "A thread is handling the connection from ~A:~A!~%"
1844 who port)
1846 (handler-case
1847 ;; perform the actual echoing algorithm
1848 (str-ex5-echo *ex5-tls-client* who port)
1850 (socket-connection-reset-error ()
1851 (format t "Client ~A:~A: connection reset by peer.~%"
1852 who port))
1854 (end-of-file ()
1855 (format t "Client ~A:~A closed connection for a read.~%"
1856 who port)
1857 t)
1859 (hangup ()
1860 (format t "Client ~A:~A closed connection for a write.~%"
1861 who port)
1862 t)))
1864 ;; cleanup form of the unwind-protect
1865 ;; We always close the connection to the client, even if this
1866 ;; thread gets destroyed (at least in SBCL this cleanup form gets
1867 ;; run when this thread is destroyed).
1868 (format t "Closing connection to ~A:~A!~%"
1869 (remote-host *ex5-tls-client*) (remote-port *ex5-tls-client*))
1870 (close *ex5-tls-client*)
1871 t))
1872 | |
1873 +------------------------------------------------------------------------------+
1875 5. Actually perform the echo protocol to the client:
1877 Read lines from the client and echo them back. All of this I/O is blocking.
1878 If we see "quit" from the client, then exit the loop, which causes the
1879 UNWIND-PROTECT cleanup form in step 4 to fire and close the connection to
1880 the client.
1882 +------------------------------------------------------------------------------+
1883 | |
1884 ;; The actual function which speaks to the client.
1885 (defun str-ex5-echo (client who port)
1886 ;; here we let signaled conditions on the boundary conditions of the
1887 ;; client (meaning it closes its connection to us on either a read or
1888 ;; a write) bail us out of the infinite loop
1889 (let ((done nil))
1890 (loop until done
1891 do
1892 (let ((line (read-line client)))
1893 (format t "Read line from ~A:~A: ~A~%" who port line)
1894 (format client "~A~%" line)
1895 (finish-output client)
1896 (format t "Wrote line to ~A:~A: ~A~%" who port line)
1898 ;; Exit the thread when the user requests it with 'quit'.
1899 ;; This forces a close to the client socket.
1900 (when (string= line "quit")
1901 (setf done t))
1902 t))))
1903 | |
1904 +------------------------------------------------------------------------------+
1906 6. The entrance function into this example:
1908 +------------------------------------------------------------------------------+
1909 | |
1910 ;; This just checks for some error conditions so we can print out a nice
1911 ;; message about it.
1912 (defun run-ex5-server (&key (port *port*))
1913 (handler-case
1915 (run-ex5-server-helper port)
1917 ;; handle some common conditions
1918 (socket-address-in-use-error ()
1919 (format t "Bind: Address already in use, forget :reuse-addr t?")))
1921 (finish-output))
1922 | |
1923 +------------------------------------------------------------------------------+
1926 Echo Server IPV4/TCP: ex6-server.lisp
1927 -------------------------------------
1929 This is the first of the echo servers which use the multiplexer to handle
1930 multiple clients concurrently. It is a single threaded program. As mentioned
1931 before, one shouldn't mix the multiplexer and threads together to handle
1932 network connections.
1934 We explore a new concept with the multiplexer in that the listening server
1935 socket is itself registered with the multiplexer. The read handler (called the
1936 listener handler in this context) associated with this socket becomes ready
1937 when a client has connected to the server address. Thus, once the listening
1938 socket is ready the listener handler accepts the client and associates the line
1939 echo protocol callback with the client's socket in the multiplexer.
1941 The I/O design of this server is such that if the client connection is ready to
1942 read, we read a line, then immediately write the line back to the client in the
1943 same function without waiting to see if it is ready for writing. Since we are
1944 still using blocking I/O, this is ok. The reason for this example's design was
1945 to minimize the complexity of using the multiplexer in order to introduce the
1946 listener handler. Later examples become much more complex as we push the
1947 multiplexer API farther.
1949 0. The variable which holds the multiplexer instance:
1951 +------------------------------------------------------------------------------+
1952 | |
1953 ;; This variable represents the multiplexer state.
1954 (defvar *ex6-server-event-base*)
1955 | |
1956 +------------------------------------------------------------------------------+
1958 1. A hash table of client connections:
1960 We record each client that connects to the server into a hash table socket
1961 keyed by the list (ip address port) and associate with it a value of the
1962 client's socket. This is so that under any conditions of the server exiting
1963 we can eagerly close any open connections to clients in a cleanup form.
1965 +------------------------------------------------------------------------------+
1966 | |
1967 ;; This holds any open connections to clients as keys in the table. The values
1968 ;; is a list containing the host and port of the connection. We use this to
1969 ;; close all connections to the clients, if any, when the server exits. This
1970 ;; allows all clients to notice the server had gone away.
1971 (defvar *ex6-server-open-connections*)
1972 | |
1973 +------------------------------------------------------------------------------+
1975 2. Create and bind the server socket:
1977 We protect how we manipulate the server socket with an UNWIND-PROTECT so we
1978 ensure to close the socket at the end of the server's computation or if
1979 something went wrong.
1981 +------------------------------------------------------------------------------+
1982 | |
1983 ;; Set up the server and server clients with the multiplexer
1984 (defun run-ex6-server-helper (port)
1986 ;; We don't use with-open-socket here since we may need to have a
1987 ;; finer control over when we close the server socket.
1988 (let ((server (make-socket :connect :passive
1989 :address-family :internet
1990 :type :stream
1991 :ipv6 nil
1992 :external-format '(:utf-8 :eol-style :crlf))))
1993 (unwind-protect
1994 (progn
1995 (format t "Created socket: ~A[fd=~A]~%" server (socket-os-fd server))
1996 ;; Bind the socket to all interfaces with specified port.
1997 (bind-address server +ipv4-unspecified+ :port port :reuse-addr t)
1998 (format t "Bound socket: ~A~%" server)
2000 ;; start listening on the server socket
2001 (listen-on server :backlog 5)
2002 (format t "Listening on socket bound to: ~A:~A~%"
2003 (local-host server)
2004 (local-port server))
2005 | |
2006 +------------------------------------------------------------------------------+
2008 3. Register a listener handler on the server socket and start dispatching
2009 events with the multiplexer:
2011 +------------------------------------------------------------------------------+
2012 | |
2013 ;; Set up the initial listener handler for any incoming clients
2014 (set-io-handler *ex6-server-event-base*
2015 (socket-os-fd server)
2016 :read
2017 (make-ex6-server-listener-handler server))
2019 ;; keep accepting connections forever.
2020 (handler-case
2021 (event-dispatch *ex6-server-event-base*)
2023 ;; Just in case any handler misses these conditions, we
2024 ;; catch them here.
2025 (socket-connection-reset-error ()
2026 (format t "~A~A~%"
2027 "Caught unexpected reset by peer! "
2028 "Client connection reset by peer!"))
2029 (hangup ()
2030 (format t "~A~A~%"
2031 "Caught unexpected hangup! "
2032 "Client closed connection on write!"))
2033 (end-of-file ()
2034 (format t "~A~A~%"
2035 "Caught unexpected end-of-file! "
2036 "Client closed connection on read!"))))
2037 | |
2038 +------------------------------------------------------------------------------+
2040 4. When the server stops handling clients, we close the server socket:
2042 +------------------------------------------------------------------------------+
2043 | |
2044 ;; Cleanup expression for uw-p.
2045 ;; Ensure the server socket is closed, regardless of how we left
2046 ;; the server.
2047 (close server))))
2048 | |
2049 +------------------------------------------------------------------------------+
2051 5. The listener handler:
2053 Once the returned closure from this function is called by the multiplexer
2054 on the ready server socket, we accept the client with a blocking accept.
2055 We then save the client connection in our table and register the line echo
2056 closure with the socket. The line echo closure will also contain a
2057 disconnector function as in previous usages of the multiplexer.
2059 +------------------------------------------------------------------------------+
2060 | |
2061 ;; When the multiplexer states the server socket is ready for reading
2062 ;; it means that we have a client ready to accept. So we accept it and
2063 ;; then register the accepted client socket back into the multiplexer
2064 ;; with the appropritate echo protocol function.
2065 (defun make-ex6-server-listener-handler (socket)
2066 (lambda (fd event exception)
2068 ;; do a blocking accept, returning nil if no socket
2069 (let* ((client (accept-connection socket :wait t)))
2070 (when client
2071 (multiple-value-bind (who port)
2072 (remote-name client)
2073 (format t "Accepted a client from ~A:~A~%" who port)
2075 ;; save the client connection in case we need to close it later
2076 ;; when the server exits.
2077 (setf (gethash `(,who ,port) *ex6-server-open-connections*) client)
2079 ;; set up an line echo function for the client socket.
2080 (set-io-handler *ex6-server-event-base*
2081 (socket-os-fd client)
2082 :read
2083 (make-ex6-server-line-echoer
2084 client
2085 who
2086 port
2087 (make-ex6-server-disconnector client))))))))
2088 | |
2089 +------------------------------------------------------------------------------+
2091 6. The line echo closure generator:
2093 This function returns a closure which is then bound to a client socket in
2094 the multiplexer. When the socket is ready, we read a line form the client
2095 and write it back to the client immediately. Since this is blocking I/O the
2096 whole server will wait until this transaction is complete. This means that
2097 a client which sends one byte of ASCII that is not a newline can cause the
2098 whole server to block for all clients. This serious defect is remedied with
2099 non-blocking I/O, which we show in a later example.
2101 +------------------------------------------------------------------------------+
2102 | |
2103 ;; This function returns a function that reads a line, then
2104 ;; echoes it right back onto the socket it came from. This is blocking
2105 ;; i/o. This code can suffer denial of service attacks like on page
2106 ;; 167 of "Unix Network Programming 2nd Edition: Sockets and XTI", by
2107 ;; Richard Stevens.
2108 (defun make-ex6-server-line-echoer (socket who port disconnector)
2109 (format t "Creating line-echoer for ~A:~A~%" who port)
2110 (lambda (fd event exception)
2111 (handler-case
2112 (let ((line (read-line socket))) ;; read a line from the client
2113 (format t "Read ~A:~A: ~A~%" who port line)
2114 (format socket "~A~%" line) ;; write it the client
2115 (finish-output socket)
2116 (format t "Wrote ~A:~A: ~A~%" who port line)
2118 ;; close the connection to the client if it asked to quit
2119 (when (string= line "quit")
2120 (format t "Client requested quit!~%")
2121 (funcall disconnector who port)))
2123 (socket-connection-reset-error ()
2124 ;; Handle the usual and common conditions we'll see while
2125 ;; talking to a client
2126 (format t "Client's connection was reset by peer.~%")
2127 (funcall disconnector who port))
2129 (hangup ()
2130 (format t "Client went away on a write.~%")
2131 (funcall disconnector who port))
2133 (end-of-file ()
2134 (format t "Client went away on a read.~%")
2135 (funcall disconnector who port)))))
2136 | |
2137 +------------------------------------------------------------------------------+
2139 7. The disconnector closure generator:
2141 This function returns a closure that removes all the handlers from the
2142 socket in question and then closes it. Notice that this means this server
2143 is not capable of handling batch input from a client, since when it
2144 receives the END-OF-FILE on the read from a client, will immediately tear
2145 down the connection destroying any in flight data. After closing the
2146 socket, we also remove it from our table of open connections.
2148 +------------------------------------------------------------------------------+
2149 | |
2150 ;; If we decide we need to disconnect ourselves from the client, this will
2151 ;; remove all the handlers and remove the record of our connection from
2152 ;; *ex6-server-open-connections*.
2153 (defun make-ex6-server-disconnector (socket)
2154 (lambda (who port)
2155 (format t "Closing connection to ~A:~A~%" who port)
2156 (remove-fd-handlers *ex6-server-event-base* (socket-os-fd socket))
2157 (close socket)
2158 (remhash `(,who ,port) *ex6-server-open-connections*)))
2159 | |
2160 +------------------------------------------------------------------------------+
2162 8. Initialize the event-base, the connection table, and start the server:
2164 This code is the beginning of the UNWIND-PROTECT form which protects the
2165 server's socket resources.
2167 +------------------------------------------------------------------------------+
2168 | |
2169 ;; This is the entrance function into this example.
2170 (defun run-ex6-server (&key (port *port*))
2171 (let ((*ex6-server-open-connections* nil)
2172 (*ex6-server-event-base* nil))
2173 (unwind-protect
2174 (handler-case
2175 (progn
2176 ;; Clear the open connection table and init the event base
2177 (setf *ex6-server-open-connections*
2178 (make-hash-table :test #'equalp)
2180 *ex6-server-event-base*
2181 (make-instance 'event-base))
2183 (run-ex6-server-helper port))
2185 ;; handle a common signal
2186 (socket-address-in-use-error ()
2187 (format t "Bind: Address already in use, forget :reuse-addr t?")))
2188 | |
2189 +------------------------------------------------------------------------------+
2191 9. Cleanup the client connections and close the event-base:
2193 When the server exits we walk the *ex6-server-open-connections* hash and
2194 eagerly close every client we find there. After we are done, we close the
2195 event-base. This ensures every thing is cleaned up properly.
2197 +------------------------------------------------------------------------------+
2198 | |
2199 ;; Cleanup form for uw-p
2200 ;; Close all open connections to the clients, if any. We do this
2201 ;; because when the server goes away we want the clients to know
2202 ;; immediately. Sockets are not memory, and can't just be garbage
2203 ;; collected whenever. They have to be eagerly closed.
2204 (maphash
2205 #'(lambda (k v)
2206 (format t "Closing a client connection to ~A~%" k)
2207 ;; We don't want to signal any conditions on the close...
2208 (close v :abort t))
2209 *ex6-server-open-connections*)
2211 ;; and clean up the multiplexer too!
2212 (when *ex6-server-event-base*
2213 (close *ex6-server-event-base*))
2214 (format t "Server Exited~%")
2215 (finish-output))))
2216 | |
2217 +------------------------------------------------------------------------------+
2219 This server uses the multiplexer in a simple fashion because only one handler
2220 is registered for a client. That handler reads, then writes the data back to
2221 the client. The scope of the data read from the client never has to leave the
2222 handler function.
2224 Echo Server IPV4/TCP: ex7-server.lisp
2225 -------------------------------------
2227 This example is different than ex6-server because it fully separates the
2228 reading and writing of data to a client into different handler functions. This
2229 requires an architectural change to the server in order to be able to keep the
2230 data from the client "somewhere" before being able to write it back to the
2231 client when the multiplexer determines it can written to the client. We
2232 introduce an io-buffer object, implemented in terms of a closure and one per
2233 client, which stores the in-flight data until the client is ready to accept the
2234 writes from the server.
2236 Storage of client data introduces a problem in that if the client writes lots
2237 of data to the server but happens to never be ready to accept it back from the
2238 server, the server will consume all memory and run out of resources. We
2239 attempt to prevent this from happening, though not perfectly.
2241 When the io-buffer is created for a client, we state we only would like a
2242 certain number of bytes to be read from the client. Of course, since we're
2243 using read-line with blocking I/O and the client could write a tremendous
2244 amount of data before a newline, we can't completely enforce our storage policy
2245 in this server. If the client, though, is well-behaved in that it sends
2246 reasonable sized lines of text--a rarity in the real world, our implemented
2247 policy is sufficient. When we reach the nonblocking I/O server example, we'll
2248 find that we can perfectly enforce the per client data storage policy.
2250 This server honors batch input from the client. When it sees the END-OF-FILE
2251 from the client, and it still has data to write, the server will attempt to
2252 write the rest of the data out as the multiplexer says the client is ready to
2253 receive it.
2255 Since this example is quite long the server portion will just be shown as a
2256 difference to ex6-server.
2258 0. The listener handler:
2260 The important code in this function is the call to make-ex7-io-buffer.
2261 This function returns a closure, here called io-buffer, which takes one
2262 argument, either :read-a-line or :write-a-line. When the funcall of
2263 io-buffer with the appropriate argument happens, *another* closure is
2264 returned and this is the closure registered with the appropriate ready
2265 state in the multiplexer.
2267 This returned closure has bound in its lexical scope the storage needed for
2268 the client.
2270 Both closures returned by :read-a-line and :write-a-line have access to the
2271 same storage space unique to this object io-buffer. This is the means by
2272 which the client's write handler can get access to the data read by the
2273 client's read handler.
2275 +------------------------------------------------------------------------------+
2276 | |
2277 ;; Create the listener closure which accepts the client and registers the
2278 ;; buffer functions with it.
2279 (defun make-ex7-server-listener-handler (socket)
2280 (lambda (fd event exception)
2281 ;; do a blocking accept, returning nil if no socket
2282 (let* ((client (accept-connection socket :wait t)))
2283 (when client
2284 (multiple-value-bind (who port)
2285 (remote-name client)
2286 (format t "Accepted a client from ~A:~A~%" who port)
2288 ;; save the client connection in case we need to close it later.
2289 (setf (gethash `(,who ,port) *ex7-open-connections*) client)
2291 ;; We make an io-buffer, which takes care of reading from the
2292 ;; socket and echoing the information it read back onto the
2293 ;; socket. The buffer takes care of this with two internal
2294 ;; handlers, a read handler and a write handler.
2295 (let ((io-buffer
2296 (make-ex7-io-buffer client who port
2297 (make-ex7-server-disconnector client))))
2299 ;; set up an line echo function for the client socket. The
2300 ;; internals of the buffer will perform the appropriate
2301 ;; registration/unregistration of the required handlers at
2302 ;; the right time depending upon data availability.
2304 (set-io-handler *ex7-event-base*
2305 (socket-os-fd client)
2306 :read
2307 (funcall io-buffer :read-a-line))
2309 (set-io-handler *ex7-event-base*
2310 (socket-os-fd client)
2311 :write
2312 (funcall io-buffer :write-a-line))))))))
2313 | |
2314 +------------------------------------------------------------------------------+
2316 1. The disconnector function:
2318 This function is almost identical to a previous example used in
2319 ex5a-client. The only difference is the special variable it references.
2321 Since the io-buffer knows under what conditions it should register or
2322 unregister specific handlers for the client socket, we need to be able to
2323 selectively remove them without disturbing the others.
2325 +------------------------------------------------------------------------------+
2326 | |
2327 (defun make-ex7-server-disconnector (socket)
2328 ;; When this function is called, it can be told which callback to remove, if
2329 ;; no callbacks are specified, all of them are removed! The socket can be
2330 ;; additionally told to be closed.
2331 (lambda (who port &rest events)
2332 (let ((fd (socket-os-fd socket)))
2333 (if (not (intersection '(:read :write :error) events))
2334 (remove-fd-handlers *ex7-event-base* fd :read t :write t :error t)
2335 (progn
2336 (when (member :read events)
2337 (remove-fd-handlers *ex7-event-base* fd :read t))
2338 (when (member :write events)
2339 (remove-fd-handlers *ex7-event-base* fd :write t))
2340 (when (member :error events)
2341 (remove-fd-handlers *ex7-event-base* fd :error t)))))
2342 ;; and finally if were asked to close the socket, we do so here
2343 (when (member :close events)
2344 (format t "Closing connection to ~A:~A~%" who port)
2345 (finish-output)
2346 (close socket)
2347 (remhash `(,who ,port) *ex7-open-connections*))))
2348 | |
2349 +------------------------------------------------------------------------------+
2351 Now we come to the description of the ex7-io-buffer code base. This code base
2352 interacts directly with the event-base multiplexer instance in order to
2353 register and unregister handlers to the client. Handlers are only registered
2354 when there is data to write, or room to read more data up to the buffer size.
2356 0. The io-buffer closure generator and associated lexical storage:
2358 These are the variables closed over which represent the internal state of
2359 the closure and hold the data from the client. In particular note is the
2360 fact we keep track of when a handler is registered (since this object can
2361 register or unregister the handlers in and of itself) and whether or not
2362 we've seen the END-OF-FILE from a client. The line-queue will hold the
2363 actual data from the client.
2365 +------------------------------------------------------------------------------+
2366 | |
2367 (defun make-ex7-io-buffer (socket who port disconnector &key (max-bytes 4096))
2368 (let ((line-queue (make-queue))
2369 (bytes-left-to-write 0)
2370 (read-handler-registered nil)
2371 (write-handler-registered nil)
2372 (eof-seen nil))
2373 | |
2374 +------------------------------------------------------------------------------+
2376 1. The read-a-line closure:
2378 This is the function which will ultimately be registered with the
2379 multiplexer hence the arguments it expects. Its job is to read a line from
2380 the client when the multiplexer said the client was readable and then store
2381 the line into the line-queue. If we have read a line, we immediately
2382 register the write-a-line handler with the multiplexer since we need to
2383 know when the client will be ready to accept it. If it turns out there is
2384 more data stored than the high-water mark we set, we unregister the read
2385 handler so we don't continue to keep reading data. If we get END-OF-FILE,
2386 but there is nothing left to write, then this handler performs a small
2387 optimization and closes the socket to the client and unregisters
2388 everything. This prevents a needless loop through the multiplexer in this
2389 case.
2391 The handling of END-OF-FILE is interesting in that we unregister the read
2392 handler, since we won't need it anymore, and mark that we've seen the
2393 END-OF-FILE. At this point, the only thing the multiplexer has to do with
2394 respect to this client is to write all of the lines stored in the
2395 line-queue out to the client and close the connection to the client.
2397 Of the various conditions that can be signaled, the
2398 SOCKET-CONNECTION-RESET-ERROR condition is the one which will shut down the
2399 whole connection by removing all handlers in the multiplexer for this
2400 client and ultimately throw away any in-flight data.
2402 +------------------------------------------------------------------------------+
2403 | |
2404 (labels
2405 ;; If this function notices that there is data to write, it will
2406 ;; set the io-handler on the socket for the write handler.
2407 ;; If the function notices it has read >= than the max-bytes
2408 ;; it will remove itself from the handler *after* ensuring the
2409 ;; write handler is set up properly.
2410 ((read-a-line (fd event exception)
2411 (handler-case
2412 (let ((line (format nil "~A~%" (read-line socket)))) ; add a \n
2413 (format t "Read from ~A:~A: ~A" who port line)
2414 (enqueue line line-queue)
2415 (incf bytes-left-to-write (length line))
2417 (when (> bytes-left-to-write 0)
2418 ;; If the write handler isn't registered, then do
2419 ;; it now since I have data to write.
2420 (unless write-handler-registered
2421 (set-io-handler *ex7-event-base*
2422 (socket-os-fd socket)
2423 :write
2424 #'write-a-line)
2425 (setf write-handler-registered t)))
2427 ;; Now, if there is more data than I should be
2428 ;; reading, remove myself from the io handler. When
2429 ;; the write handler notices that, after writing some
2430 ;; data, more of it can be read, it will reregister
2431 ;; the io handler for the read socket.
2432 (when (>= bytes-left-to-write max-bytes)
2433 (funcall disconnector who port :read)
2434 (setf read-handler-registered nil)))
2436 (socket-connection-reset-error ()
2437 ;; If the client resets its connection, we close
2438 ;; everything down.
2439 (format t "Client ~A:~A: Connection reset by peer~%" who port)
2440 (funcall disconnector who port :close))
2442 (end-of-file ()
2443 ;; When we get an end of file, that doesn't necessarily
2444 ;; mean the client went away, it could just mean that
2445 ;; the client performed a shutdown on the write end of
2446 ;; its socket and it is expecting the data stored in
2447 ;; the server to be written to it. However, if there
2448 ;; is nothing left to write and our read end is close,
2449 ;; we shall consider it that the client went away and
2450 ;; close the connection.
2451 (format t "Client ~A:~A produced end-of-file on a read.~%"
2452 who port)
2453 (if (zerop bytes-left-to-write)
2454 (funcall disconnector who port :close)
2455 (progn
2456 (funcall disconnector who port :read)
2457 (setf read-handler-registered nil)
2458 (setf eof-seen t))))))
2459 | |
2460 +------------------------------------------------------------------------------+
2462 2. The write-a-line closure:
2464 This function is somewhat symmetrical to read-a-line. It will register and
2465 unregister itself or the read handler based upon how much data is available
2466 to read/write. If the END-OF-FILE is seen and there is nothing left to
2467 write, it will close the connection to the client and unregister
2468 everything.
2470 +------------------------------------------------------------------------------+
2471 | |
2472 ;; This function will notice that if it has written enough bytes to
2473 ;; bring the bytes-left-to-write under max-bytes, it will re-register
2474 ;; the reader io handler. If there is no data to write, it will,
2475 ;; after ensuring the read handler is registered, unregister itself
2476 ;; as to not be called constantly on a write ready socket with no
2477 ;; data to write.
2478 (write-a-line (fd event exception)
2479 (handler-case
2480 (progn
2481 ;; If we have something to write to the client, do so.
2482 (when (> bytes-left-to-write 0)
2483 (let ((line (dequeue line-queue)))
2484 (format socket "~A" line) ;; newline is in the string.
2485 (finish-output socket)
2486 (format t "Wrote to ~A:~A: ~A" who port line)
2487 (decf bytes-left-to-write (length line))))
2489 ;; If we see we've fallen below the max-bytes mark,
2490 ;; re-register the read handler to get more data for
2491 ;; us. However, don't reregister the read handler if
2492 ;; we've seen that the client closed our read end of
2493 ;; our socket.
2494 (when (< bytes-left-to-write max-bytes)
2495 (unless (or eof-seen read-handler-registered)
2496 (set-io-handler *ex7-event-base*
2497 (socket-os-fd socket)
2498 :read
2499 #'read-a-line)
2500 (setf read-handler-registered t)))
2502 ;; If we notice that we don't have any data to write
2503 ;; AND have seen the end of file from the client,
2504 ;; then we close the connection to the client since
2505 ;; it will never speak to us again and we're done
2506 ;; speaking to it.
2507 ;;
2508 ;; If notice we've written all of our data and there
2509 ;; might be more to do later, then unregister the
2510 ;; write handler so we don't get called
2511 ;; unnecesarily. This might mean that sometimes we'll
2512 ;; have to make an extra trip through the
2513 ;; event-dispatcher to perform the write if we read
2514 ;; more from the client and it reregisters us.
2515 (when (zerop bytes-left-to-write)
2516 (if eof-seen
2517 (funcall disconnector who port :close)
2518 (progn
2519 (funcall disconnector who port :write)
2520 (setf write-handler-registered nil)))))
2522 (socket-connection-reset-error ()
2523 ;; If I happen to get a reset, make sure the connection
2524 ;; is closed. I shouldn't get this here, but if you
2525 ;; tinker with the flow of this example, it is a good
2526 ;; guard to have.
2527 (format t "Client ~A:~A: connection reset by peer.~%" who port)
2528 (funcall disconnector who port :close))
2530 (hangup ()
2531 ;; In this server, if the client doesn't accept data,
2532 ;; it also means it will never send us data again. So
2533 ;; close the connection for good.
2534 (format t "Client ~A:~A got hangup on write.~%" who port)
2535 (funcall disconnector who port :close)))))
2536 | |
2537 +------------------------------------------------------------------------------+
2539 3. The returned closure, which represents the io-buffer:
2541 This is the actual closure returned by make-ex7-io-buffer and which is used
2542 to gain access into the read-a-line and write-a-line functions. It takes a
2543 single argument, either the keywords :read-a-line or :write-a-line, and
2544 returns a reference to either internal function.
2546 +------------------------------------------------------------------------------+
2547 | |
2548 ;; This is the actual function returned from make-ex7-io-buffer
2549 ;; which allows us access to the read/writer in the scope of the
2550 ;; closure. We will ask for the correct functions when setting
2551 ;; up the io handlers. NOTE: By simply asking for the handler,
2552 ;; I've assumed it is to be immediately put into an iolib event
2553 ;; handler. This is why they are considered registered at this point.
2554 (lambda (msg)
2555 (cond
2556 ((equalp msg :read-a-line)
2557 (setf read-handler-registered t)
2558 #'read-a-line)
2559 ((equalp msg :write-a-line)
2560 (setf write-handler-registered t)
2561 #'write-a-line)
2562 (t
2563 (error "make-ex7-buffer: Please supply :read-a-line or :write-a-line~%")))))))
2564 | |
2565 +------------------------------------------------------------------------------+
2567 While this server still uses blocking I/O, we've laid the foundations for
2568 nonblocking I/O and memory storage enforcement. The foundations specifically
2569 are separating the read/write handlers into different pieces and having shared
2570 lexical bindings between them.
2572 Echo Server IPV4/TCP: ex8-server.lisp
2573 -------------------------------------
2575 This server uses nonblocking I/O and the multiplexer to concurrently talk to
2576 the clients.
2578 Architecturally, it is very similar to ex7-server, but the io-buffer for this
2579 server is implemented with much different internals. Whereas in ex7-server
2580 reading from a client used the stream function READ-LINE, writing used the
2581 stream function FORMAT, and the strings from the client were kept in a queue,
2582 now we use RECEIVE-FROM and SEND-TO along with an array of unsigned-bytes as a
2583 buffer to read/write actual bytes from the socket.
2585 Accessing the socket through the stream API is different than doing it through
2586 the almost raw socket API which we are about to use. RECEIVE-FROM and SEND-TO
2587 are not part of the stream interface. They are a lower level API in IOLib being
2588 closer to the underlying OS abstraction and as a consequence have a somewhat
2589 different set of conditions that they can signal. These different conditions
2590 have the form isys:<unix-errno-name> like: isys:epipe, isys:ewouldblock, etc.
2591 There is some intersection with the condition names signaled by the stream API,
2592 such as: SOCKET-CONNECTION-RESET-ERROR, and SOCKET-CONNECTION-REFUSED.
2594 [TODO figure out complete list!]
2596 An example of the ramifications of this API is RECEIVE-FROM. Comparing against
2597 the stream interface whose READ-LINE will signal an END-OF-FILE when the
2598 reading socket has been closed by the client, the function RECEIVE-FROM will
2599 return 0, signifying the end of file. The stream function FORMAT will signal
2600 HANGUP if it tries to write to a socket where the client has gone away. SEND-TO
2601 might not signal, or otherwise produce, any error at all when writing to a
2602 socket where the client has gone away--usually it is on the next RECEIVE-FROM
2603 that it is discovered the client went away. The bytes that SEND-TO wrote simply
2604 vanish!
2606 With IOLib, it may surprise you to be told that all underlying fds in the
2607 previous examples have been nonblocking! This is why we specified :wait t for
2608 ACCEPT-CONNECTION and CONNECT.
2610 The IOLib library internally ensures that the stream interface blocks according
2611 to the requirements of ANSI Common Lisp. However, when we use SEND-TO and
2612 RECEIVE-FROM we automatically gain the benefit of the non-blocking status on
2613 the underlying fd. This is why in this example we don't explicitly set the
2614 underlying fd to non-blocking status--it already is!
2616 The server code itself is described as a difference from ex7-server, but the
2617 io-buffer for this nonblocking server (in file ex8-buffer.lisp) will be
2618 described in its entirety. Also, this server honors the batch input requirement
2619 from example client ex-5b-client, which you should use against this server.
2621 The ex8-server codes:
2623 0. The listener handler (first half):
2625 Accept and store the client connection.
2627 +------------------------------------------------------------------------------+
2628 | |
2629 (defun make-ex8-server-listener-handler (socket)
2630 (lambda (fd event exception)
2631 ;; do a blocking accept, returning nil if no socket
2632 (let* ((client (accept-connection socket :wait t)))
2633 (when client
2634 (multiple-value-bind (who port)
2635 (remote-name client)
2636 (format t "Accepted a client from ~A:~A~%" who port)
2638 ;; save the client connection in case we need to close it later.
2639 (setf (gethash `(,who ,port) *ex8-open-connections*) client)
2640 | |
2641 +------------------------------------------------------------------------------+
2643 1. The listener handler (second half):
2645 Like ex7-server, we register the read and write handlers. Notice though
2646 that we changed the keywords to the io-buffer closure to be
2647 :read-some-bytes and :write-some-bytes. This better represents what the
2648 io-buffer is actually doing.
2650 +------------------------------------------------------------------------------+
2651 | |
2652 ;; We make an io-buffer, which takes care of reading from the
2653 ;; socket and echoing the information it read back onto the
2654 ;; socket. The buffer takes care of this with two internal
2655 ;; handlers, a read handler and a write handler.
2656 (let ((io-buffer
2657 (make-ex8-io-buffer client who port
2658 (make-ex8-server-disconnector client))))
2660 ;; set up an unsigned byte echo function for the
2661 ;; client socket. The internals of the buffer will
2662 ;; perform the appropriate registration/unregistration of
2663 ;; the required handlers at the right time depending upon
2664 ;; data availability.
2666 (set-io-handler *ex8-event-base*
2667 (socket-os-fd client)
2668 :read
2669 (funcall io-buffer :read-some-bytes))
2671 (set-io-handler *ex8-event-base*
2672 (socket-os-fd client)
2673 :write
2674 (funcall io-buffer :write-some-bytes))))))))
2675 | |
2676 +------------------------------------------------------------------------------+
2678 The rest of the server is extremely similar to ex7-server.
2680 Now, we'll show the io-buffer specific to ex8-server.
2682 0. The internal state of the io-buffer closure:
2684 The binding echo-buf is an unsigned-byte array of size max-bytes. This is
2685 where data from the client is stored before it is written back to the
2686 client.
2688 The binding read-index keeps track of the beginning of the empty space in
2689 the echo-buf buffer where more data could be stored during a read.
2691 The binding write-index keeps track of how much data has been written to
2692 the client. It moves towards read-index, and when it has the same value as
2693 read-index it means that there is no data left to write to the client.
2695 The bindings read-handler-registered and write-handler-registered allow the
2696 io-buffer to know when it has registered a handler for reading and writing
2697 data.
2699 The binding eof-seen marks when the client has closed its write connection
2700 to the server. The server will push out all data to the client, then close
2701 socket to the client.
2703 +------------------------------------------------------------------------------+
2704 | |
2705 (defun make-ex8-io-buffer (socket who port disconnector &key (max-bytes 16384))
2706 (let ((echo-buf (make-array max-bytes :element-type 'unsigned-byte))
2707 (read-index 0)
2708 (write-index 0)
2709 (read-handler-registered nil)
2710 (write-handler-registered nil)
2711 (eof-seen nil))
2712 | |
2713 +------------------------------------------------------------------------------+
2715 1. Reading bytes form the client:
2717 In this function, we will convert the return value 0 of RECEIVE-FROM on the
2718 read of a closed socket into a signaled END-OF-FILE condition to keep the
2719 structure of our code similar to what has transpired before. Once we read
2720 some bytes, we increment the read-index pointer and ensure to register a
2721 write handler to write the data back out. We optimize the writing process a
2722 little bit and try to write the data out immediately without checking to
2723 see if the socket is ready. Then if there is no more room in the echo-buf
2724 array, we unregister ourselves so we don't try and read more data from the
2725 client until we are ready to accept it (by having written all of the data
2726 back to the client). We mark the END-OF-FILE flag and unregister the read
2727 handler if we see the client has closed its connection. We optimize the
2728 knowledge that if we have no more data to write we just close the
2729 connection to the client.
2731 +------------------------------------------------------------------------------+
2732 | |
2733 (labels
2734 ;; This is the function responsible for reading bytes from the client.
2735 ((read-some-bytes (fd event exception)
2736 (handler-case
2737 (progn
2738 ;; Read however much we are able.
2739 (multiple-value-bind (buf bytes-read)
2740 (receive-from socket
2741 :buffer echo-buf
2742 :start read-index
2743 :end max-bytes)
2745 ;; Unlike read-ing from a stream, receive-from
2746 ;; returns zero on an end-of-file read, so we turn
2747 ;; around and signal that condition so our
2748 ;; handler-case can deal with it properly like our
2749 ;; other examples.
2750 (when (zerop bytes-read)
2751 (error 'end-of-file))
2753 (format t "Read ~A bytes from ~A:~A~%" bytes-read who port)
2754 (incf read-index bytes-read))
2756 ;; Register the write handler if there is data to
2757 ;; write.
2758 ;;
2759 ;; Then, try to write some data to the socket right
2760 ;; away even though it might not be ready simply to
2761 ;; avoid another go around. The write-some-bytes
2762 ;; function must be able to catch econnreset because
2763 ;; this connection may be closed at the time of this
2764 ;; call. Normally, if the multiplexer has told me I
2765 ;; could write then it'd be ok, but since this write
2766 ;; is outside of the multiplexer and an optimization,
2767 ;; it needs to check.
2768 (when (/= write-index read-index)
2769 (unless write-handler-registered
2770 (set-io-handler *ex8-event-base*
2771 (socket-os-fd socket)
2772 :write
2773 #'write-some-bytes)
2774 (setf write-handler-registered t))
2776 ;; See if I can write it right away!
2777 (write-some-bytes fd :write nil))
2779 ;; If I'm out of room to store more data then remove
2780 ;; myself from the io handler. When the write handler
2781 ;; notices that it has finished writing everything,
2782 ;; all indicies get set back to zero and the write
2783 ;; handler removes itself. If write-some-bytes in
2784 ;; the call above worked, then read-index might not
2785 ;; equal max-bytes when this line of code gets
2786 ;; executed.
2787 (when (= read-index max-bytes)
2788 (funcall disconnector who port :read)
2789 (setf read-handler-registered nil)))
2791 (socket-connection-reset-error ()
2792 ;; Handle the client sending a reset.
2793 (format t "Client ~A:~A: connection reset by peer.~%" who port)
2794 (funcall disconnector who port :close))
2796 (end-of-file ()
2797 ;; When we get an end of file, that doesn't necessarily
2798 ;; mean the client went away, it could just mean that
2799 ;; the client performed a shutdown on the write end of
2800 ;; its socket and it is expecting the data stored in
2801 ;; the server to be written to it. However, if there
2802 ;; is nothing left to write and our read end is closed,
2803 ;; we shall consider it that the client went away and
2804 ;; close the connection.
2805 (format t "Client ~A:~A produced end-of-file on a read.~%"
2806 who port)
2807 (if (= read-index write-index)
2808 (funcall disconnector who port :close)
2809 (progn
2810 (funcall disconnector who port :read)
2811 (setf read-handler-registered nil)
2812 (setf eof-seen t))))))
2813 | |
2814 +------------------------------------------------------------------------------+
2816 2. Writing bytes to the client:
2818 While there are more bytes to write, we write them, keeping track of how
2819 much we wrote. Once we are out of data to write, we unregister the write
2820 handler, since we don't want to be called unnecessarily--usually the client
2821 socket is always ready to write. If we've seen the eof marker and are out
2822 of data, we close the client connection and are done. If we haven't seen
2823 it, then we determine if we are at the end of the buffer, if so, we reset
2824 the indices to the beginning. Either way, we re-register the read handler
2825 to acquire more data.
2827 We handle some new conditions here: isys:ewouldblock is needed because
2828 sometimes the underlying OS will mark an fd as ready to write when in fact
2829 it isn't when we get around to writing it. We might also see this condition
2830 when we tried to optimize the write of the data in the read handler since
2831 we did it outside of the multiplexer--this is idiomatic and saves a trip
2832 through the multiplexer more often than not. Seeing isys:ewouldblock simply
2833 aborts the write and we'll try again later. Under some conditions, send-to
2834 will signal an isys:epipe error, which means the client closed its
2835 connection. It is similar to a HANGUP condition in a format call with the
2836 stream API. We treat it similarly to a HANGUP.
2838 +------------------------------------------------------------------------------+
2839 | |
2840 ;; This is the function responsible for writing bytes to the client.
2841 (write-some-bytes (fd event exception)
2842 (handler-case
2843 (progn
2844 ;; If there is data to be written, write it. NOTE:
2845 ;; There is often no indication of failure to write
2846 ;; with send-to. If I'm writing to a closed (by the
2847 ;; client) socket, it could be that send-to tells me
2848 ;; nothing is wrong and returns the number of bytes
2849 ;; wrotten. In this case, nothing was written but we
2850 ;; have no way of knowing. Usually in this case, the
2851 ;; read handler will get a 0 bytes read on the socket
2852 ;; and we can know the connection is broken.
2853 (when (> read-index write-index)
2854 (let ((wrote-bytes (send-to socket echo-buf
2855 :start write-index
2856 :end read-index)))
2857 (format t "Wrote ~A bytes to ~A:~A~%" wrote-bytes who port)
2858 (incf write-index wrote-bytes)))
2860 ;; If we see we're out of data to write and we saw an eof,
2861 ;; then close the connection, we're done. If we didn't see an
2862 ;; eof, then unregister the write handler and reregister the
2863 ;; read handler to get more data. If the buffer indices
2864 ;; are at the very end, reset them to the beginning.
2865 (when (= read-index write-index)
2866 (if eof-seen
2867 (funcall disconnector who port :close)
2868 (progn
2870 ;; nothing more to write, so unregister writer
2871 (funcall disconnector who port :write)
2872 (setf write-handler-registered nil)
2874 ;; If we're at the end of the buffer, move to the
2875 ;; beginning so there is more room for data.
2876 (when (= read-index write-index max-bytes)
2877 (setf read-index 0
2878 write-index 0))
2880 ;; Reregister the read handler to get more data
2881 (unless read-handler-registered
2882 (set-io-handler *ex8-event-base*
2883 (socket-os-fd socket)
2884 :read
2885 #'read-some-bytes)
2886 (setf read-handler-registered t))))))
2888 (socket-connection-reset-error ()
2889 ;; If for somer eaon the client reset the network connection,
2890 ;; we'll get this signal.
2891 (format t "Client ~A:~A: connection reset by peer.~%" who port)
2892 (funcall disconnector who port :close))
2894 (isys:ewouldblock ()
2895 ;; Sometimes this happens on a write even though it
2896 ;; might have been marked as ready. Also we might have
2897 ;; asked to write on an unknown status socket. Ignore
2898 ;; it and we will try again later.
2899 (format t "write-some-bytes: ewouldblock~%")
2900 nil)
2902 (isys:epipe ()
2903 ;; In this server, if the client doesn't accept data,
2904 ;; it also means it will never send us data again. So
2905 ;; close the connection for good.
2906 (format t "Client ~A:~A got hangup on write.~%" who port)
2907 (funcall disconnector who port :close)))))
2908 | |
2909 +------------------------------------------------------------------------------+
2911 3. The returned closure of the io-buffer:
2913 Much like make-ex7-io-buffer, we return one of the internal closures which
2914 are appropriate for reading or writing by the multiplexer.
2916 +------------------------------------------------------------------------------+
2917 | |
2918 ;; This is the function returned from make-ex8-io-buffer which
2919 ;; allows us access to the read/writer in the scope of the
2920 ;; closure. We will ask for the correct functions when setting
2921 ;; up the io handlers. NOTE: By simply asking for the handler,
2922 ;; I've assumed it is to be immediately put into an iolib event
2923 ;; handler. This is why they are considered registered at this
2924 ;; point.
2925 (lambda (msg)
2926 (cond
2927 ((equalp msg :read-some-bytes)
2928 (setf read-handler-registered t)
2929 #'read-some-bytes)
2930 ((equalp msg :write-some-bytes)
2931 (setf write-handler-registered t)
2932 #'write-some-bytes)
2933 (t
2934 (error "make-ex8-buffer: Please supply :read-some-bytes or :write-some-bytes~%")))))))
2936 | |
2937 +------------------------------------------------------------------------------+
2940 Future Directions
2941 -----------------
2943 Of course, more information should go into this tutorial, such as non-blocking
2944 connects/accepts, urgent TCP data, UDP examples, and IPV6. As time permits or
2945 contributions come in, these will be added.
2947 Appendix A
2948 ----------
2950 This holds a rough approximation between the sources in this tutorial and the
2951 original sources in the network programming book by Stevens mentioned in the
2952 beginning of the tutorial. Aspects about the implementation of each client or
2953 server are summarized here.
2955 The Clients
2956 -----------
2958 Figure 1.5, page 6
2959 - ex1-client: Blocking I/O, daytime client, C Style
2961 Figure 1.5, page 6
2962 - ex2-client: Blocking I/O, daytime client, Lisp Style
2964 Figure 1.5, page 6
2965 - ex3-client: ex2-client, but with much more error handling
2967 Figure 5.4, 5.5, page 114, 115
2968 - ex4-client: Blocking I/O, line oriented
2970 Figure 6.9, page 157
2971 - ex5a-client: I/O multiplexing with iolib, line oriented, blocking I/O
2972 - note: since this is still blocking I/O, I'm using *standard-input*
2973 and friends. Also note, with batch input, it will close the socket with
2974 in-flight data still present which is incorrect.
2976 Figure 6.13, page 162
2977 - ex5b-client: Same as ex5a-client EXCEPT shutdown is called when the input
2978 reaches end-of-file as to prevent in flight data from being destroyed
2979 on the way to the server.
2981 The servers
2982 -----------
2984 Figure 4.11, page 101
2985 - ex1-server: Iterative, blocking I/O daytime server, C Style, no
2986 error handling, one shot, line oriented
2988 Figure 4.11, page 101
2989 - ex2-server: Iterative, blocking I/O daytime server, Lisp Style,
2990 no error handling, loop forever, line oriented
2992 Figure 4.11, page 101
2993 - ex3-server: daytime server, ex2-server, but with error handling, line oriented
2995 Figure 4.13, page 105
2996 - ex4-server: daytime server, concurrent, blocking I/O, line oriented
2998 Figure 5.2, 5.3, page 113, 114
2999 - ex5-server: Concurrent, blocking I/O, echo server, line oriented
3001 Figure 6.21,6.22 page 165,166
3002 - ex6-server: I/O multiplexing of clients with iolib, line oriented,
3003 blocking I/O
3005 Figure 6.21, 6.22 page 165,166
3006 - ex7-server, ex7-buffer: individual I/O handlers for read/write,
3007 I/O multiplexing of clients with iolib, line oriented, blocking I/O,
3008 has problem with denial of service, page 167.
3010 Figure 15.3, 15.4, 15.5 page 400-403
3011 - ex8-server, ex8-buffer: nonblocking I/O, event-dispatch, send-to, receive-from