0001 """Generic socket server classes. 0002 0003 This module tries to capture the various aspects of defining a server: 0004 0005 For socket-based servers: 0006 0007 - address family: 0008 - AF_INET{,6}: IP (Internet Protocol) sockets (default) 0009 - AF_UNIX: Unix domain sockets 0010 - others, e.g. AF_DECNET are conceivable (see <socket.h> 0011 - socket type: 0012 - SOCK_STREAM (reliable stream, e.g. TCP) 0013 - SOCK_DGRAM (datagrams, e.g. UDP) 0014 0015 For request-based servers (including socket-based): 0016 0017 - client address verification before further looking at the request 0018 (This is actually a hook for any processing that needs to look 0019 at the request before anything else, e.g. logging) 0020 - how to handle multiple requests: 0021 - synchronous (one request is handled at a time) 0022 - forking (each request is handled by a new process) 0023 - threading (each request is handled by a new thread) 0024 0025 The classes in this module favor the server type that is simplest to 0026 write: a synchronous TCP/IP server. This is bad class design, but 0027 save some typing. (There's also the issue that a deep class hierarchy 0028 slows down method lookups.) 0029 0030 There are five classes in an inheritance diagram, four of which represent 0031 synchronous servers of four types: 0032 0033 +------------+ 0034 | BaseServer | 0035 +------------+ 0036 | 0037 v 0038 +-----------+ +------------------+ 0039 | TCPServer |------->| UnixStreamServer | 0040 +-----------+ +------------------+ 0041 | 0042 v 0043 +-----------+ +--------------------+ 0044 | UDPServer |------->| UnixDatagramServer | 0045 +-----------+ +--------------------+ 0046 0047 Note that UnixDatagramServer derives from UDPServer, not from 0048 UnixStreamServer -- the only difference between an IP and a Unix 0049 stream server is the address family, which is simply repeated in both 0050 unix server classes. 0051 0052 Forking and threading versions of each type of server can be created 0053 using the ForkingServer and ThreadingServer mix-in classes. For 0054 instance, a threading UDP server class is created as follows: 0055 0056 class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass 0057 0058 The Mix-in class must come first, since it overrides a method defined 0059 in UDPServer! Setting the various member variables also changes 0060 the behavior of the underlying server mechanism. 0061 0062 To implement a service, you must derive a class from 0063 BaseRequestHandler and redefine its handle() method. You can then run 0064 various versions of the service by combining one of the server classes 0065 with your request handler class. 0066 0067 The request handler class must be different for datagram or stream 0068 services. This can be hidden by using the mix-in request handler 0069 classes StreamRequestHandler or DatagramRequestHandler. 0070 0071 Of course, you still have to use your head! 0072 0073 For instance, it makes no sense to use a forking server if the service 0074 contains state in memory that can be modified by requests (since the 0075 modifications in the child process would never reach the initial state 0076 kept in the parent process and passed to each child). In this case, 0077 you can use a threading server, but you will probably have to use 0078 locks to avoid two requests that come in nearly simultaneous to apply 0079 conflicting changes to the server state. 0080 0081 On the other hand, if you are building e.g. an HTTP server, where all 0082 data is stored externally (e.g. in the file system), a synchronous 0083 class will essentially render the service "deaf" while one request is 0084 being handled -- which may be for a very long time if a client is slow 0085 to reqd all the data it has requested. Here a threading or forking 0086 server is appropriate. 0087 0088 In some cases, it may be appropriate to process part of a request 0089 synchronously, but to finish processing in a forked child depending on 0090 the request data. This can be implemented by using a synchronous 0091 server and doing an explicit fork in the request handler class 0092 handle() method. 0093 0094 Another approach to handling multiple simultaneous requests in an 0095 environment that supports neither threads nor fork (or where these are 0096 too expensive or inappropriate for the service) is to maintain an 0097 explicit table of partially finished requests and to use select() to 0098 decide which request to work on next (or whether to handle a new 0099 incoming request). This is particularly important for stream services 0100 where each client can potentially be connected for a long time (if 0101 threads or subprocesses cannot be used). 0102 0103 Future work: 0104 - Standard classes for Sun RPC (which uses either UDP or TCP) 0105 - Standard mix-in classes to implement various authentication 0106 and encryption schemes 0107 - Standard framework for select-based multiplexing 0108 0109 XXX Open problems: 0110 - What to do with out-of-band data? 0111 0112 BaseServer: 0113 - split generic "request" functionality out into BaseServer class. 0114 Copyright (C) 2000 Luke Kenneth Casson Leighton <lkcl@samba.org> 0115 0116 example: read entries from a SQL database (requires overriding 0117 get_request() to return a table entry from the database). 0118 entry is processed by a RequestHandlerClass. 0119 0120 """ 0121 0122 # Author of the BaseServer patch: Luke Kenneth Casson Leighton 0123 0124 # XXX Warning! 0125 # There is a test suite for this module, but it cannot be run by the 0126 # standard regression test. 0127 # To run it manually, run Lib/test/test_socketserver.py. 0128 0129 __version__ = "0.4" 0130 0131 0132 import socket 0133 import sys 0134 import os 0135 0136 __all__ = ["TCPServer","UDPServer","ForkingUDPServer","ForkingTCPServer", 0137 "ThreadingUDPServer","ThreadingTCPServer","BaseRequestHandler", 0138 "StreamRequestHandler","DatagramRequestHandler", 0139 "ThreadingMixIn", "ForkingMixIn"] 0140 if hasattr(socket, "AF_UNIX"): 0141 __all__.extend(["UnixStreamServer","UnixDatagramServer", 0142 "ThreadingUnixStreamServer", 0143 "ThreadingUnixDatagramServer"]) 0144 0145 class BaseServer: 0146 0147 """Base class for server classes. 0148 0149 Methods for the caller: 0150 0151 - __init__(server_address, RequestHandlerClass) 0152 - serve_forever() 0153 - handle_request() # if you do not use serve_forever() 0154 - fileno() -> int # for select() 0155 0156 Methods that may be overridden: 0157 0158 - server_bind() 0159 - server_activate() 0160 - get_request() -> request, client_address 0161 - verify_request(request, client_address) 0162 - server_close() 0163 - process_request(request, client_address) 0164 - close_request(request) 0165 - handle_error() 0166 0167 Methods for derived classes: 0168 0169 - finish_request(request, client_address) 0170 0171 Class variables that may be overridden by derived classes or 0172 instances: 0173 0174 - address_family 0175 - socket_type 0176 - allow_reuse_address 0177 0178 Instance variables: 0179 0180 - RequestHandlerClass 0181 - socket 0182 0183 """ 0184 0185 def __init__(self, server_address, RequestHandlerClass): 0186 """Constructor. May be extended, do not override.""" 0187 self.server_address = server_address 0188 self.RequestHandlerClass = RequestHandlerClass 0189 0190 def server_activate(self): 0191 """Called by constructor to activate the server. 0192 0193 May be overridden. 0194 0195 """ 0196 pass 0197 0198 def serve_forever(self): 0199 """Handle one request at a time until doomsday.""" 0200 while 1: 0201 self.handle_request() 0202 0203 # The distinction between handling, getting, processing and 0204 # finishing a request is fairly arbitrary. Remember: 0205 # 0206 # - handle_request() is the top-level call. It calls 0207 # get_request(), verify_request() and process_request() 0208 # - get_request() is different for stream or datagram sockets 0209 # - process_request() is the place that may fork a new process 0210 # or create a new thread to finish the request 0211 # - finish_request() instantiates the request handler class; 0212 # this constructor will handle the request all by itself 0213 0214 def handle_request(self): 0215 """Handle one request, possibly blocking.""" 0216 try: 0217 request, client_address = self.get_request() 0218 except socket.error: 0219 return 0220 if self.verify_request(request, client_address): 0221 try: 0222 self.process_request(request, client_address) 0223 except: 0224 self.handle_error(request, client_address) 0225 self.close_request(request) 0226 0227 def verify_request(self, request, client_address): 0228 """Verify the request. May be overridden. 0229 0230 Return True if we should proceed with this request. 0231 0232 """ 0233 return True 0234 0235 def process_request(self, request, client_address): 0236 """Call finish_request. 0237 0238 Overridden by ForkingMixIn and ThreadingMixIn. 0239 0240 """ 0241 self.finish_request(request, client_address) 0242 self.close_request(request) 0243 0244 def server_close(self): 0245 """Called to clean-up the server. 0246 0247 May be overridden. 0248 0249 """ 0250 pass 0251 0252 def finish_request(self, request, client_address): 0253 """Finish one request by instantiating RequestHandlerClass.""" 0254 self.RequestHandlerClass(request, client_address, self) 0255 0256 def close_request(self, request): 0257 """Called to clean up an individual request.""" 0258 pass 0259 0260 def handle_error(self, request, client_address): 0261 """Handle an error gracefully. May be overridden. 0262 0263 The default is to print a traceback and continue. 0264 0265 """ 0266 print '-'*40 0267 print 'Exception happened during processing of request from', 0268 print client_address 0269 import traceback 0270 traceback.print_exc() # XXX But this goes to stderr! 0271 print '-'*40 0272 0273 0274 class TCPServer(BaseServer): 0275 0276 """Base class for various socket-based server classes. 0277 0278 Defaults to synchronous IP stream (i.e., TCP). 0279 0280 Methods for the caller: 0281 0282 - __init__(server_address, RequestHandlerClass) 0283 - serve_forever() 0284 - handle_request() # if you don't use serve_forever() 0285 - fileno() -> int # for select() 0286 0287 Methods that may be overridden: 0288 0289 - server_bind() 0290 - server_activate() 0291 - get_request() -> request, client_address 0292 - verify_request(request, client_address) 0293 - process_request(request, client_address) 0294 - close_request(request) 0295 - handle_error() 0296 0297 Methods for derived classes: 0298 0299 - finish_request(request, client_address) 0300 0301 Class variables that may be overridden by derived classes or 0302 instances: 0303 0304 - address_family 0305 - socket_type 0306 - request_queue_size (only for stream sockets) 0307 - allow_reuse_address 0308 0309 Instance variables: 0310 0311 - server_address 0312 - RequestHandlerClass 0313 - socket 0314 0315 """ 0316 0317 address_family = socket.AF_INET 0318 0319 socket_type = socket.SOCK_STREAM 0320 0321 request_queue_size = 5 0322 0323 allow_reuse_address = False 0324 0325 def __init__(self, server_address, RequestHandlerClass): 0326 """Constructor. May be extended, do not override.""" 0327 BaseServer.__init__(self, server_address, RequestHandlerClass) 0328 self.socket = socket.socket(self.address_family, 0329 self.socket_type) 0330 self.server_bind() 0331 self.server_activate() 0332 0333 def server_bind(self): 0334 """Called by constructor to bind the socket. 0335 0336 May be overridden. 0337 0338 """ 0339 if self.allow_reuse_address: 0340 self.socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1) 0341 self.socket.bind(self.server_address) 0342 0343 def server_activate(self): 0344 """Called by constructor to activate the server. 0345 0346 May be overridden. 0347 0348 """ 0349 self.socket.listen(self.request_queue_size) 0350 0351 def server_close(self): 0352 """Called to clean-up the server. 0353 0354 May be overridden. 0355 0356 """ 0357 self.socket.close() 0358 0359 def fileno(self): 0360 """Return socket file number. 0361 0362 Interface required by select(). 0363 0364 """ 0365 return self.socket.fileno() 0366 0367 def get_request(self): 0368 """Get the request and client address from the socket. 0369 0370 May be overridden. 0371 0372 """ 0373 return self.socket.accept() 0374 0375 def close_request(self, request): 0376 """Called to clean up an individual request.""" 0377 request.close() 0378 0379 0380 class UDPServer(TCPServer): 0381 0382 """UDP server class.""" 0383 0384 allow_reuse_address = False 0385 0386 socket_type = socket.SOCK_DGRAM 0387 0388 max_packet_size = 8192 0389 0390 def get_request(self): 0391 data, client_addr = self.socket.recvfrom(self.max_packet_size) 0392 return (data, self.socket), client_addr 0393 0394 def server_activate(self): 0395 # No need to call listen() for UDP. 0396 pass 0397 0398 def close_request(self, request): 0399 # No need to close anything. 0400 pass 0401 0402 class ForkingMixIn: 0403 0404 """Mix-in class to handle each request in a new process.""" 0405 0406 active_children = None 0407 max_children = 40 0408 0409 def collect_children(self): 0410 """Internal routine to wait for died children.""" 0411 while self.active_children: 0412 if len(self.active_children) < self.max_children: 0413 options = os.WNOHANG 0414 else: 0415 # If the maximum number of children are already 0416 # running, block while waiting for a child to exit 0417 options = 0 0418 try: 0419 pid, status = os.waitpid(0, options) 0420 except os.error: 0421 pid = None 0422 if not pid: break 0423 self.active_children.remove(pid) 0424 0425 def process_request(self, request, client_address): 0426 """Fork a new subprocess to process the request.""" 0427 self.collect_children() 0428 pid = os.fork() 0429 if pid: 0430 # Parent process 0431 if self.active_children is None: 0432 self.active_children = [] 0433 self.active_children.append(pid) 0434 self.close_request(request) 0435 return 0436 else: 0437 # Child process. 0438 # This must never return, hence os._exit()! 0439 try: 0440 self.finish_request(request, client_address) 0441 os._exit(0) 0442 except: 0443 try: 0444 self.handle_error(request, client_address) 0445 finally: 0446 os._exit(1) 0447 0448 0449 class ThreadingMixIn: 0450 """Mix-in class to handle each request in a new thread.""" 0451 0452 # Decides how threads will act upon termination of the 0453 # main process 0454 daemon_threads = False 0455 0456 def process_request_thread(self, request, client_address): 0457 """Same as in BaseServer but as a thread. 0458 0459 In addition, exception handling is done here. 0460 0461 """ 0462 try: 0463 self.finish_request(request, client_address) 0464 self.close_request(request) 0465 except: 0466 self.handle_error(request, client_address) 0467 self.close_request(request) 0468 0469 def process_request(self, request, client_address): 0470 """Start a new thread to process the request.""" 0471 import threading 0472 t = threading.Thread(target = self.process_request_thread, 0473 args = (request, client_address)) 0474 if self.daemon_threads: 0475 t.setDaemon (1) 0476 t.start() 0477 0478 0479 class ForkingUDPServer(ForkingMixIn, UDPServer): pass 0480 class ForkingTCPServer(ForkingMixIn, TCPServer): pass 0481 0482 class ThreadingUDPServer(ThreadingMixIn, UDPServer): pass 0483 class ThreadingTCPServer(ThreadingMixIn, TCPServer): pass 0484 0485 if hasattr(socket, 'AF_UNIX'): 0486 0487 class UnixStreamServer(TCPServer): 0488 address_family = socket.AF_UNIX 0489 0490 class UnixDatagramServer(UDPServer): 0491 address_family = socket.AF_UNIX 0492 0493 class ThreadingUnixStreamServer(ThreadingMixIn, UnixStreamServer): pass 0494 0495 class ThreadingUnixDatagramServer(ThreadingMixIn, UnixDatagramServer): pass 0496 0497 class BaseRequestHandler: 0498 0499 """Base class for request handler classes. 0500 0501 This class is instantiated for each request to be handled. The 0502 constructor sets the instance variables request, client_address 0503 and server, and then calls the handle() method. To implement a 0504 specific service, all you need to do is to derive a class which 0505 defines a handle() method. 0506 0507 The handle() method can find the request as self.request, the 0508 client address as self.client_address, and the server (in case it 0509 needs access to per-server information) as self.server. Since a 0510 separate instance is created for each request, the handle() method 0511 can define arbitrary other instance variariables. 0512 0513 """ 0514 0515 def __init__(self, request, client_address, server): 0516 self.request = request 0517 self.client_address = client_address 0518 self.server = server 0519 try: 0520 self.setup() 0521 self.handle() 0522 self.finish() 0523 finally: 0524 sys.exc_traceback = None # Help garbage collection 0525 0526 def setup(self): 0527 pass 0528 0529 def handle(self): 0530 pass 0531 0532 def finish(self): 0533 pass 0534 0535 0536 # The following two classes make it possible to use the same service 0537 # class for stream or datagram servers. 0538 # Each class sets up these instance variables: 0539 # - rfile: a file object from which receives the request is read 0540 # - wfile: a file object to which the reply is written 0541 # When the handle() method returns, wfile is flushed properly 0542 0543 0544 class StreamRequestHandler(BaseRequestHandler): 0545 0546 """Define self.rfile and self.wfile for stream sockets.""" 0547 0548 # Default buffer sizes for rfile, wfile. 0549 # We default rfile to buffered because otherwise it could be 0550 # really slow for large data (a getc() call per byte); we make 0551 # wfile unbuffered because (a) often after a write() we want to 0552 # read and we need to flush the line; (b) big writes to unbuffered 0553 # files are typically optimized by stdio even when big reads 0554 # aren't. 0555 rbufsize = -1 0556 wbufsize = 0 0557 0558 def setup(self): 0559 self.connection = self.request 0560 self.rfile = self.connection.makefile('rb', self.rbufsize) 0561 self.wfile = self.connection.makefile('wb', self.wbufsize) 0562 0563 def finish(self): 0564 if not self.wfile.closed: 0565 self.wfile.flush() 0566 self.wfile.close() 0567 self.rfile.close() 0568 0569 0570 class DatagramRequestHandler(BaseRequestHandler): 0571 0572 # XXX Regrettably, I cannot get this working on Linux; 0573 # s.recvfrom() doesn't return a meaningful client address. 0574 0575 """Define self.rfile and self.wfile for datagram sockets.""" 0576 0577 def setup(self): 0578 import StringIO 0579 self.packet, self.socket = self.request 0580 self.rfile = StringIO.StringIO(self.packet) 0581 self.wfile = StringIO.StringIO() 0582 0583 def finish(self): 0584 self.socket.sendto(self.wfile.getvalue(), self.client_address) 0585
Generated by PyXR 0.9.4