Doc/lib/libsocksvr.tex

   1 \section{\module{SocketServer} ---
   2          A framework for network servers}
   3
   4 \declaremodule{standard}{SocketServer}
   5 \modulesynopsis{A framework for network servers.}
   6
   7
   8 The \module{SocketServer} module simplifies the task of writing network
   9 servers.
  10
  11 There are four basic server classes: \class{TCPServer} uses the
  12 Internet TCP protocol, which provides for continuous streams of data
  13 between the client and server.  \class{UDPServer} uses datagrams, which
  14 are discrete packets of information that may arrive out of order or be
  15 lost while in transit.  The more infrequently used
  16 \class{UnixStreamServer} and \class{UnixDatagramServer} classes are
  17 similar, but use \UNIX{} domain sockets; they're not available on
  18 non-\UNIX{} platforms.  For more details on network programming, consult
  19 a book such as W. Richard Steven's \citetitle{UNIX Network Programming}
  20 or Ralph Davis's \citetitle{Win32 Network Programming}.
  21
  22 These four classes process requests \dfn{synchronously}; each request
  23 must be completed before the next request can be started.  This isn't
  24 suitable if each request takes a long time to complete, because it
  25 requires a lot of computation, or because it returns a lot of data
  26 which the client is slow to process.  The solution is to create a
  27 separate process or thread to handle each request; the
  28 \class{ForkingMixIn} and \class{ThreadingMixIn} mix-in classes can be
  29 used to support asynchronous behaviour.
  30
  31 Creating a server requires several steps.  First, you must create a
  32 request handler class by subclassing the \class{BaseRequestHandler}
  33 class and overriding its \method{handle()} method; this method will
  34 process incoming requests.  Second, you must instantiate one of the
  35 server classes, passing it the server's address and the request
  36 handler class.  Finally, call the \method{handle_request()} or
  37 \method{serve_forever()} method of the server object to process one or
  38 many requests.
  39
  40 When inheriting from \class{ThreadingMixIn} for threaded connection
  41 behavior, you should explicitly declare how you want your threads
  42 to behave on an abrupt shutdown. The \class{ThreadingMixIn} class
  43 defines an attribute \var{daemon_threads}, which indicates whether
  44 or not the server should wait for thread termination. You should
  45 set the flag explicitly if you would like threads to behave
  46 autonomously; the default is \constant{False}, meaning that Python
  47 will not exit until all threads created by \class{ThreadingMixIn} have
  48 exited.
  49
  50 Server classes have the same external methods and attributes, no
  51 matter what network protocol they use:
  52
  53 \setindexsubitem{(SocketServer protocol)}
  54
  55 \subsection{Server Creation Notes}
  56
  57 There are five classes in an inheritance diagram, four of which represent
  58 synchronous servers of four types:
  59
  60 \begin{verbatim}
  61         +------------+
  62         | BaseServer |
  63         +------------+
  64               |
  65               v
  66         +-----------+        +------------------+
  67         | TCPServer |------->| UnixStreamServer |
  68         +-----------+        +------------------+
  69               |
  70               v
  71         +-----------+        +--------------------+
  72         | UDPServer |------->| UnixDatagramServer |
  73         +-----------+        +--------------------+
  74 \end{verbatim}
  75
  76 Note that \class{UnixDatagramServer} derives from \class{UDPServer}, not
  77 from \class{UnixStreamServer} --- the only difference between an IP and a
  78 \UNIX{} stream server is the address family, which is simply repeated in both
  79 \UNIX{} server classes.
  80
  81 Forking and threading versions of each type of server can be created using
  82 the \class{ForkingMixIn} and \class{ThreadingMixIn} mix-in classes.  For
  83 instance, a threading UDP server class is created as follows:
  84
  85 \begin{verbatim}
  86     class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass
  87 \end{verbatim}
  88
  89 The mix-in class must come first, since it overrides a method defined in
  90 \class{UDPServer}.  Setting the various member variables also changes the
  91 behavior of the underlying server mechanism.
  92
  93 To implement a service, you must derive a class from
  94 \class{BaseRequestHandler} and redefine its \method{handle()} method.  You
  95 can then run various versions of the service by combining one of the server
  96 classes with your request handler class.  The request handler class must be
  97 different for datagram or stream services.  This can be hidden by using the
  98 handler subclasses \class{StreamRequestHandler} or \class{DatagramRequestHandler}.
  99
 100 Of course, you still have to use your head!  For instance, it makes no sense
 101 to use a forking server if the service contains state in memory that can be
 102 modified by different requests, since the modifications in the child process
 103 would never reach the initial state kept in the parent process and passed to
 104 each child.  In this case, you can use a threading server, but you will
 105 probably have to use locks to protect the integrity of the shared data.
 106
 107 On the other hand, if you are building an HTTP server where all data is
 108 stored externally (for instance, in the file system), a synchronous class
 109 will essentially render the service "deaf" while one request is being
 110 handled -- which may be for a very long time if a client is slow to receive
 111 all the data it has requested.  Here a threading or forking server is
 112 appropriate.
 113
 114 In some cases, it may be appropriate to process part of a request
 115 synchronously, but to finish processing in a forked child depending on the
 116 request data.  This can be implemented by using a synchronous server and
 117 doing an explicit fork in the request handler class \method{handle()}
 118 method.
 119
 120 Another approach to handling multiple simultaneous requests in an
 121 environment that supports neither threads nor \function{fork()} (or where
 122 these are too expensive or inappropriate for the service) is to maintain an
 123 explicit table of partially finished requests and to use \function{select()}
 124 to decide which request to work on next (or whether to handle a new incoming
 125 request).  This is particularly important for stream services where each
 126 client can potentially be connected for a long time (if threads or
 127 subprocesses cannot be used).
 128
 129 %XXX should data and methods be intermingled, or separate?
 130 % how should the distinction between class and instance variables be
 131 % drawn?
 132
 133 \subsection{Server Objects}
 134
 135 \begin{funcdesc}{fileno}{}
 136 Return an integer file descriptor for the socket on which the server
 137 is listening.  This function is most commonly passed to
 138 \function{select.select()}, to allow monitoring multiple servers in the
 139 same process.
 140 \end{funcdesc}
 141
 142 \begin{funcdesc}{handle_request}{}
 143 Process a single request.  This function calls the following methods
 144 in order: \method{get_request()}, \method{verify_request()}, and
 145 \method{process_request()}.  If the user-provided \method{handle()}
 146 method of the handler class raises an exception, the server's
 147 \method{handle_error()} method will be called.
 148 \end{funcdesc}
 149
 150 \begin{funcdesc}{serve_forever}{}
 151 Handle an infinite number of requests.  This simply calls
 152 \method{handle_request()} inside an infinite loop.
 153 \end{funcdesc}
 154
 155 \begin{datadesc}{address_family}
 156 The family of protocols to which the server's socket belongs.
 157 \constant{socket.AF_INET} and \constant{socket.AF_UNIX} are two
 158 possible values.
 159 \end{datadesc}
 160
 161 \begin{datadesc}{RequestHandlerClass}
 162 The user-provided request handler class; an instance of this class is
 163 created for each request.
 164 \end{datadesc}
 165
 166 \begin{datadesc}{server_address}
 167 The address on which the server is listening.  The format of addresses
 168 varies depending on the protocol family; see the documentation for the
 169 socket module for details.  For Internet protocols, this is a tuple
 170 containing a string giving the address, and an integer port number:
 171 \code{('127.0.0.1', 80)}, for example.
 172 \end{datadesc}
 173
 174 \begin{datadesc}{socket}
 175 The socket object on which the server will listen for incoming requests.
 176 \end{datadesc}
 177
 178 % XXX should class variables be covered before instance variables, or
 179 % vice versa?
 180
 181 The server classes support the following class variables:
 182
 183 \begin{datadesc}{allow_reuse_address}
 184 Whether the server will allow the reuse of an address. This defaults
 185 to \constant{False}, and can be set in subclasses to change the policy.
 186 \end{datadesc}
 187
 188 \begin{datadesc}{request_queue_size}
 189 The size of the request queue.  If it takes a long time to process a
 190 single request, any requests that arrive while the server is busy are
 191 placed into a queue, up to \member{request_queue_size} requests.  Once
 192 the queue is full, further requests from clients will get a
 193 ``Connection denied'' error.  The default value is usually 5, but this
 194 can be overridden by subclasses.
 195 \end{datadesc}
 196
 197 \begin{datadesc}{socket_type}
 198 The type of socket used by the server; \constant{socket.SOCK_STREAM}
 199 and \constant{socket.SOCK_DGRAM} are two possible values.
 200 \end{datadesc}
 201
 202 There are various server methods that can be overridden by subclasses
 203 of base server classes like \class{TCPServer}; these methods aren't
 204 useful to external users of the server object.
 205
 206 % should the default implementations of these be documented, or should
 207 % it be assumed that the user will look at SocketServer.py?
 208
 209 \begin{funcdesc}{finish_request}{}
 210 Actually processes the request by instantiating
 211 \member{RequestHandlerClass} and calling its \method{handle()} method.
 212 \end{funcdesc}
 213
 214 \begin{funcdesc}{get_request}{}
 215 Must accept a request from the socket, and return a 2-tuple containing
 216 the \emph{new} socket object to be used to communicate with the
 217 client, and the client's address.
 218 \end{funcdesc}
 219
 220 \begin{funcdesc}{handle_error}{request, client_address}
 221 This function is called if the \member{RequestHandlerClass}'s
 222 \method{handle()} method raises an exception.  The default action is
 223 to print the traceback to standard output and continue handling
 224 further requests.
 225 \end{funcdesc}
 226
 227 \begin{funcdesc}{process_request}{request, client_address}
 228 Calls \method{finish_request()} to create an instance of the
 229 \member{RequestHandlerClass}.  If desired, this function can create a
 230 new process or thread to handle the request; the \class{ForkingMixIn}
 231 and \class{ThreadingMixIn} classes do this.
 232 \end{funcdesc}
 233
 234 % Is there any point in documenting the following two functions?
 235 % What would the purpose of overriding them be: initializing server
 236 % instance variables, adding new network families?
 237
 238 \begin{funcdesc}{server_activate}{}
 239 Called by the server's constructor to activate the server.  The default
 240 behavior just \method{listen}s to the server's socket.
 241 May be overridden.
 242 \end{funcdesc}
 243
 244 \begin{funcdesc}{server_bind}{}
 245 Called by the server's constructor to bind the socket to the desired
 246 address.  May be overridden.
 247 \end{funcdesc}
 248
 249 \begin{funcdesc}{verify_request}{request, client_address}
 250 Must return a Boolean value; if the value is \constant{True}, the request will be
 251 processed, and if it's \constant{False}, the request will be denied.
 252 This function can be overridden to implement access controls for a server.
 253 The default implementation always returns \constant{True}.
 254 \end{funcdesc}
 255
 256 \subsection{RequestHandler Objects}
 257
 258 The request handler class must define a new \method{handle()} method,
 259 and can override any of the following methods.  A new instance is
 260 created for each request.
 261
 262 \begin{funcdesc}{finish}{}
 263 Called after the \method{handle()} method to perform any clean-up
 264 actions required.  The default implementation does nothing.  If
 265 \method{setup()} or \method{handle()} raise an exception, this
 266 function will not be called.
 267 \end{funcdesc}
 268
 269 \begin{funcdesc}{handle}{}
 270 This function must do all the work required to service a request.
 271 The default implementation does nothing.
 272 Several instance attributes are available to it; the request is
 273 available as \member{self.request}; the client address as
 274 \member{self.client_address}; and the server instance as
 275 \member{self.server}, in case it needs access to per-server
 276 information.
 277
 278 The type of \member{self.request} is different for datagram or stream
 279 services.  For stream services, \member{self.request} is a socket
 280 object; for datagram services, \member{self.request} is a string.
 281 However, this can be hidden by using the  request handler subclasses
 282 \class{StreamRequestHandler} or \class{DatagramRequestHandler}, which
 283 override the \method{setup()} and \method{finish()} methods, and
 284 provide \member{self.rfile} and \member{self.wfile} attributes.
 285 \member{self.rfile} and \member{self.wfile} can be read or written,
 286 respectively, to get the request data or return data to the client.
 287 \end{funcdesc}
 288
 289 \begin{funcdesc}{setup}{}
 290 Called before the \method{handle()} method to perform any
 291 initialization actions required.  The default implementation does
 292 nothing.
 293 \end{funcdesc}