1:tocdepth: 2 2 3========================= 4Library and Extension FAQ 5========================= 6 7.. only:: html 8 9 .. contents:: 10 11General Library Questions 12========================= 13 14How do I find a module or application to perform task X? 15-------------------------------------------------------- 16 17Check :ref:`the Library Reference <library-index>` to see if there's a relevant 18standard library module. (Eventually you'll learn what's in the standard 19library and will be able to skip this step.) 20 21For third-party packages, search the `Python Package Index 22<https://pypi.python.org/pypi>`_ or try `Google <https://www.google.com>`_ or 23another Web search engine. Searching for "Python" plus a keyword or two for 24your topic of interest will usually find something helpful. 25 26 27Where is the math.py (socket.py, regex.py, etc.) source file? 28------------------------------------------------------------- 29 30If you can't find a source file for a module it may be a built-in or 31dynamically loaded module implemented in C, C++ or other compiled language. 32In this case you may not have the source file or it may be something like 33:file:`mathmodule.c`, somewhere in a C source directory (not on the Python Path). 34 35There are (at least) three kinds of modules in Python: 36 371) modules written in Python (.py); 382) modules written in C and dynamically loaded (.dll, .pyd, .so, .sl, etc); 393) modules written in C and linked with the interpreter; to get a list of these, 40 type:: 41 42 import sys 43 print(sys.builtin_module_names) 44 45 46How do I make a Python script executable on Unix? 47------------------------------------------------- 48 49You need to do two things: the script file's mode must be executable and the 50first line must begin with ``#!`` followed by the path of the Python 51interpreter. 52 53The first is done by executing ``chmod +x scriptfile`` or perhaps ``chmod 755 54scriptfile``. 55 56The second can be done in a number of ways. The most straightforward way is to 57write :: 58 59 #!/usr/local/bin/python 60 61as the very first line of your file, using the pathname for where the Python 62interpreter is installed on your platform. 63 64If you would like the script to be independent of where the Python interpreter 65lives, you can use the :program:`env` program. Almost all Unix variants support 66the following, assuming the Python interpreter is in a directory on the user's 67:envvar:`PATH`:: 68 69 #!/usr/bin/env python 70 71*Don't* do this for CGI scripts. The :envvar:`PATH` variable for CGI scripts is 72often very minimal, so you need to use the actual absolute pathname of the 73interpreter. 74 75Occasionally, a user's environment is so full that the :program:`/usr/bin/env` 76program fails; or there's no env program at all. In that case, you can try the 77following hack (due to Alex Rezinsky):: 78 79 #! /bin/sh 80 """:" 81 exec python $0 ${1+"$@"} 82 """ 83 84The minor disadvantage is that this defines the script's __doc__ string. 85However, you can fix that by adding :: 86 87 __doc__ = """...Whatever...""" 88 89 90 91Is there a curses/termcap package for Python? 92--------------------------------------------- 93 94.. XXX curses *is* built by default, isn't it? 95 96For Unix variants: The standard Python source distribution comes with a curses 97module in the :source:`Modules` subdirectory, though it's not compiled by default. 98(Note that this is not available in the Windows distribution -- there is no 99curses module for Windows.) 100 101The :mod:`curses` module supports basic curses features as well as many additional 102functions from ncurses and SYSV curses such as colour, alternative character set 103support, pads, and mouse support. This means the module isn't compatible with 104operating systems that only have BSD curses, but there don't seem to be any 105currently maintained OSes that fall into this category. 106 107For Windows: use `the consolelib module 108<http://effbot.org/zone/console-index.htm>`_. 109 110 111Is there an equivalent to C's onexit() in Python? 112------------------------------------------------- 113 114The :mod:`atexit` module provides a register function that is similar to C's 115:c:func:`onexit`. 116 117 118Why don't my signal handlers work? 119---------------------------------- 120 121The most common problem is that the signal handler is declared with the wrong 122argument list. It is called as :: 123 124 handler(signum, frame) 125 126so it should be declared with two arguments:: 127 128 def handler(signum, frame): 129 ... 130 131 132Common tasks 133============ 134 135How do I test a Python program or component? 136-------------------------------------------- 137 138Python comes with two testing frameworks. The :mod:`doctest` module finds 139examples in the docstrings for a module and runs them, comparing the output with 140the expected output given in the docstring. 141 142The :mod:`unittest` module is a fancier testing framework modelled on Java and 143Smalltalk testing frameworks. 144 145To make testing easier, you should use good modular design in your program. 146Your program should have almost all functionality 147encapsulated in either functions or class methods -- and this sometimes has the 148surprising and delightful effect of making the program run faster (because local 149variable accesses are faster than global accesses). Furthermore the program 150should avoid depending on mutating global variables, since this makes testing 151much more difficult to do. 152 153The "global main logic" of your program may be as simple as :: 154 155 if __name__ == "__main__": 156 main_logic() 157 158at the bottom of the main module of your program. 159 160Once your program is organized as a tractable collection of functions and class 161behaviours you should write test functions that exercise the behaviours. A test 162suite that automates a sequence of tests can be associated with each module. 163This sounds like a lot of work, but since Python is so terse and flexible it's 164surprisingly easy. You can make coding much more pleasant and fun by writing 165your test functions in parallel with the "production code", since this makes it 166easy to find bugs and even design flaws earlier. 167 168"Support modules" that are not intended to be the main module of a program may 169include a self-test of the module. :: 170 171 if __name__ == "__main__": 172 self_test() 173 174Even programs that interact with complex external interfaces may be tested when 175the external interfaces are unavailable by using "fake" interfaces implemented 176in Python. 177 178 179How do I create documentation from doc strings? 180----------------------------------------------- 181 182The :mod:`pydoc` module can create HTML from the doc strings in your Python 183source code. An alternative for creating API documentation purely from 184docstrings is `epydoc <http://epydoc.sourceforge.net/>`_. `Sphinx 185<http://sphinx-doc.org>`_ can also include docstring content. 186 187 188How do I get a single keypress at a time? 189----------------------------------------- 190 191For Unix variants there are several solutions. It's straightforward to do this 192using curses, but curses is a fairly large module to learn. 193 194.. XXX this doesn't work out of the box, some IO expert needs to check why 195 196 Here's a solution without curses:: 197 198 import termios, fcntl, sys, os 199 fd = sys.stdin.fileno() 200 201 oldterm = termios.tcgetattr(fd) 202 newattr = termios.tcgetattr(fd) 203 newattr[3] = newattr[3] & ~termios.ICANON & ~termios.ECHO 204 termios.tcsetattr(fd, termios.TCSANOW, newattr) 205 206 oldflags = fcntl.fcntl(fd, fcntl.F_GETFL) 207 fcntl.fcntl(fd, fcntl.F_SETFL, oldflags | os.O_NONBLOCK) 208 209 try: 210 while True: 211 try: 212 c = sys.stdin.read(1) 213 print("Got character", repr(c)) 214 except OSError: 215 pass 216 finally: 217 termios.tcsetattr(fd, termios.TCSAFLUSH, oldterm) 218 fcntl.fcntl(fd, fcntl.F_SETFL, oldflags) 219 220 You need the :mod:`termios` and the :mod:`fcntl` module for any of this to 221 work, and I've only tried it on Linux, though it should work elsewhere. In 222 this code, characters are read and printed one at a time. 223 224 :func:`termios.tcsetattr` turns off stdin's echoing and disables canonical 225 mode. :func:`fcntl.fnctl` is used to obtain stdin's file descriptor flags 226 and modify them for non-blocking mode. Since reading stdin when it is empty 227 results in an :exc:`OSError`, this error is caught and ignored. 228 229 .. versionchanged:: 3.3 230 *sys.stdin.read* used to raise :exc:`IOError`. Starting from Python 3.3 231 :exc:`IOError` is alias for :exc:`OSError`. 232 233 234Threads 235======= 236 237How do I program using threads? 238------------------------------- 239 240Be sure to use the :mod:`threading` module and not the :mod:`_thread` module. 241The :mod:`threading` module builds convenient abstractions on top of the 242low-level primitives provided by the :mod:`_thread` module. 243 244Aahz has a set of slides from his threading tutorial that are helpful; see 245http://www.pythoncraft.com/OSCON2001/. 246 247 248None of my threads seem to run: why? 249------------------------------------ 250 251As soon as the main thread exits, all threads are killed. Your main thread is 252running too quickly, giving the threads no time to do any work. 253 254A simple fix is to add a sleep to the end of the program that's long enough for 255all the threads to finish:: 256 257 import threading, time 258 259 def thread_task(name, n): 260 for i in range(n): 261 print(name, i) 262 263 for i in range(10): 264 T = threading.Thread(target=thread_task, args=(str(i), i)) 265 T.start() 266 267 time.sleep(10) # <---------------------------! 268 269But now (on many platforms) the threads don't run in parallel, but appear to run 270sequentially, one at a time! The reason is that the OS thread scheduler doesn't 271start a new thread until the previous thread is blocked. 272 273A simple fix is to add a tiny sleep to the start of the run function:: 274 275 def thread_task(name, n): 276 time.sleep(0.001) # <--------------------! 277 for i in range(n): 278 print(name, i) 279 280 for i in range(10): 281 T = threading.Thread(target=thread_task, args=(str(i), i)) 282 T.start() 283 284 time.sleep(10) 285 286Instead of trying to guess a good delay value for :func:`time.sleep`, 287it's better to use some kind of semaphore mechanism. One idea is to use the 288:mod:`queue` module to create a queue object, let each thread append a token to 289the queue when it finishes, and let the main thread read as many tokens from the 290queue as there are threads. 291 292 293How do I parcel out work among a bunch of worker threads? 294--------------------------------------------------------- 295 296The easiest way is to use the new :mod:`concurrent.futures` module, 297especially the :mod:`~concurrent.futures.ThreadPoolExecutor` class. 298 299Or, if you want fine control over the dispatching algorithm, you can write 300your own logic manually. Use the :mod:`queue` module to create a queue 301containing a list of jobs. The :class:`~queue.Queue` class maintains a 302list of objects and has a ``.put(obj)`` method that adds items to the queue and 303a ``.get()`` method to return them. The class will take care of the locking 304necessary to ensure that each job is handed out exactly once. 305 306Here's a trivial example:: 307 308 import threading, queue, time 309 310 # The worker thread gets jobs off the queue. When the queue is empty, it 311 # assumes there will be no more work and exits. 312 # (Realistically workers will run until terminated.) 313 def worker(): 314 print('Running worker') 315 time.sleep(0.1) 316 while True: 317 try: 318 arg = q.get(block=False) 319 except queue.Empty: 320 print('Worker', threading.currentThread(), end=' ') 321 print('queue empty') 322 break 323 else: 324 print('Worker', threading.currentThread(), end=' ') 325 print('running with argument', arg) 326 time.sleep(0.5) 327 328 # Create queue 329 q = queue.Queue() 330 331 # Start a pool of 5 workers 332 for i in range(5): 333 t = threading.Thread(target=worker, name='worker %i' % (i+1)) 334 t.start() 335 336 # Begin adding work to the queue 337 for i in range(50): 338 q.put(i) 339 340 # Give threads time to run 341 print('Main thread sleeping') 342 time.sleep(5) 343 344When run, this will produce the following output: 345 346.. code-block:: none 347 348 Running worker 349 Running worker 350 Running worker 351 Running worker 352 Running worker 353 Main thread sleeping 354 Worker <Thread(worker 1, started 130283832797456)> running with argument 0 355 Worker <Thread(worker 2, started 130283824404752)> running with argument 1 356 Worker <Thread(worker 3, started 130283816012048)> running with argument 2 357 Worker <Thread(worker 4, started 130283807619344)> running with argument 3 358 Worker <Thread(worker 5, started 130283799226640)> running with argument 4 359 Worker <Thread(worker 1, started 130283832797456)> running with argument 5 360 ... 361 362Consult the module's documentation for more details; the :class:`~queue.Queue` 363class provides a featureful interface. 364 365 366What kinds of global value mutation are thread-safe? 367---------------------------------------------------- 368 369A :term:`global interpreter lock` (GIL) is used internally to ensure that only one 370thread runs in the Python VM at a time. In general, Python offers to switch 371among threads only between bytecode instructions; how frequently it switches can 372be set via :func:`sys.setswitchinterval`. Each bytecode instruction and 373therefore all the C implementation code reached from each instruction is 374therefore atomic from the point of view of a Python program. 375 376In theory, this means an exact accounting requires an exact understanding of the 377PVM bytecode implementation. In practice, it means that operations on shared 378variables of built-in data types (ints, lists, dicts, etc) that "look atomic" 379really are. 380 381For example, the following operations are all atomic (L, L1, L2 are lists, D, 382D1, D2 are dicts, x, y are objects, i, j are ints):: 383 384 L.append(x) 385 L1.extend(L2) 386 x = L[i] 387 x = L.pop() 388 L1[i:j] = L2 389 L.sort() 390 x = y 391 x.field = y 392 D[x] = y 393 D1.update(D2) 394 D.keys() 395 396These aren't:: 397 398 i = i+1 399 L.append(L[-1]) 400 L[i] = L[j] 401 D[x] = D[x] + 1 402 403Operations that replace other objects may invoke those other objects' 404:meth:`__del__` method when their reference count reaches zero, and that can 405affect things. This is especially true for the mass updates to dictionaries and 406lists. When in doubt, use a mutex! 407 408 409Can't we get rid of the Global Interpreter Lock? 410------------------------------------------------ 411 412.. XXX link to dbeazley's talk about GIL? 413 414The :term:`global interpreter lock` (GIL) is often seen as a hindrance to Python's 415deployment on high-end multiprocessor server machines, because a multi-threaded 416Python program effectively only uses one CPU, due to the insistence that 417(almost) all Python code can only run while the GIL is held. 418 419Back in the days of Python 1.5, Greg Stein actually implemented a comprehensive 420patch set (the "free threading" patches) that removed the GIL and replaced it 421with fine-grained locking. Adam Olsen recently did a similar experiment 422in his `python-safethread <http://code.google.com/p/python-safethread/>`_ 423project. Unfortunately, both experiments exhibited a sharp drop in single-thread 424performance (at least 30% slower), due to the amount of fine-grained locking 425necessary to compensate for the removal of the GIL. 426 427This doesn't mean that you can't make good use of Python on multi-CPU machines! 428You just have to be creative with dividing the work up between multiple 429*processes* rather than multiple *threads*. The 430:class:`~concurrent.futures.ProcessPoolExecutor` class in the new 431:mod:`concurrent.futures` module provides an easy way of doing so; the 432:mod:`multiprocessing` module provides a lower-level API in case you want 433more control over dispatching of tasks. 434 435Judicious use of C extensions will also help; if you use a C extension to 436perform a time-consuming task, the extension can release the GIL while the 437thread of execution is in the C code and allow other threads to get some work 438done. Some standard library modules such as :mod:`zlib` and :mod:`hashlib` 439already do this. 440 441It has been suggested that the GIL should be a per-interpreter-state lock rather 442than truly global; interpreters then wouldn't be able to share objects. 443Unfortunately, this isn't likely to happen either. It would be a tremendous 444amount of work, because many object implementations currently have global state. 445For example, small integers and short strings are cached; these caches would 446have to be moved to the interpreter state. Other object types have their own 447free list; these free lists would have to be moved to the interpreter state. 448And so on. 449 450And I doubt that it can even be done in finite time, because the same problem 451exists for 3rd party extensions. It is likely that 3rd party extensions are 452being written at a faster rate than you can convert them to store all their 453global state in the interpreter state. 454 455And finally, once you have multiple interpreters not sharing any state, what 456have you gained over running each interpreter in a separate process? 457 458 459Input and Output 460================ 461 462How do I delete a file? (And other file questions...) 463----------------------------------------------------- 464 465Use ``os.remove(filename)`` or ``os.unlink(filename)``; for documentation, see 466the :mod:`os` module. The two functions are identical; :func:`~os.unlink` is simply 467the name of the Unix system call for this function. 468 469To remove a directory, use :func:`os.rmdir`; use :func:`os.mkdir` to create one. 470``os.makedirs(path)`` will create any intermediate directories in ``path`` that 471don't exist. ``os.removedirs(path)`` will remove intermediate directories as 472long as they're empty; if you want to delete an entire directory tree and its 473contents, use :func:`shutil.rmtree`. 474 475To rename a file, use ``os.rename(old_path, new_path)``. 476 477To truncate a file, open it using ``f = open(filename, "rb+")``, and use 478``f.truncate(offset)``; offset defaults to the current seek position. There's 479also ``os.ftruncate(fd, offset)`` for files opened with :func:`os.open`, where 480*fd* is the file descriptor (a small integer). 481 482The :mod:`shutil` module also contains a number of functions to work on files 483including :func:`~shutil.copyfile`, :func:`~shutil.copytree`, and 484:func:`~shutil.rmtree`. 485 486 487How do I copy a file? 488--------------------- 489 490The :mod:`shutil` module contains a :func:`~shutil.copyfile` function. Note 491that on MacOS 9 it doesn't copy the resource fork and Finder info. 492 493 494How do I read (or write) binary data? 495------------------------------------- 496 497To read or write complex binary data formats, it's best to use the :mod:`struct` 498module. It allows you to take a string containing binary data (usually numbers) 499and convert it to Python objects; and vice versa. 500 501For example, the following code reads two 2-byte integers and one 4-byte integer 502in big-endian format from a file:: 503 504 import struct 505 506 with open(filename, "rb") as f: 507 s = f.read(8) 508 x, y, z = struct.unpack(">hhl", s) 509 510The '>' in the format string forces big-endian data; the letter 'h' reads one 511"short integer" (2 bytes), and 'l' reads one "long integer" (4 bytes) from the 512string. 513 514For data that is more regular (e.g. a homogeneous list of ints or floats), 515you can also use the :mod:`array` module. 516 517.. note:: 518 519 To read and write binary data, it is mandatory to open the file in 520 binary mode (here, passing ``"rb"`` to :func:`open`). If you use 521 ``"r"`` instead (the default), the file will be open in text mode 522 and ``f.read()`` will return :class:`str` objects rather than 523 :class:`bytes` objects. 524 525 526I can't seem to use os.read() on a pipe created with os.popen(); why? 527--------------------------------------------------------------------- 528 529:func:`os.read` is a low-level function which takes a file descriptor, a small 530integer representing the opened file. :func:`os.popen` creates a high-level 531file object, the same type returned by the built-in :func:`open` function. 532Thus, to read *n* bytes from a pipe *p* created with :func:`os.popen`, you need to 533use ``p.read(n)``. 534 535 536.. XXX update to use subprocess. See the :ref:`subprocess-replacements` section. 537 538 How do I run a subprocess with pipes connected to both input and output? 539 ------------------------------------------------------------------------ 540 541 Use the :mod:`popen2` module. For example:: 542 543 import popen2 544 fromchild, tochild = popen2.popen2("command") 545 tochild.write("input\n") 546 tochild.flush() 547 output = fromchild.readline() 548 549 Warning: in general it is unwise to do this because you can easily cause a 550 deadlock where your process is blocked waiting for output from the child 551 while the child is blocked waiting for input from you. This can be caused 552 by the parent expecting the child to output more text than it does or 553 by data being stuck in stdio buffers due to lack of flushing. 554 The Python parent can of course explicitly flush the data it sends to the 555 child before it reads any output, but if the child is a naive C program it 556 may have been written to never explicitly flush its output, even if it is 557 interactive, since flushing is normally automatic. 558 559 Note that a deadlock is also possible if you use :func:`popen3` to read 560 stdout and stderr. If one of the two is too large for the internal buffer 561 (increasing the buffer size does not help) and you ``read()`` the other one 562 first, there is a deadlock, too. 563 564 Note on a bug in popen2: unless your program calls ``wait()`` or 565 ``waitpid()``, finished child processes are never removed, and eventually 566 calls to popen2 will fail because of a limit on the number of child 567 processes. Calling :func:`os.waitpid` with the :data:`os.WNOHANG` option can 568 prevent this; a good place to insert such a call would be before calling 569 ``popen2`` again. 570 571 In many cases, all you really need is to run some data through a command and 572 get the result back. Unless the amount of data is very large, the easiest 573 way to do this is to write it to a temporary file and run the command with 574 that temporary file as input. The standard module :mod:`tempfile` exports a 575 :func:`~tempfile.mktemp` function to generate unique temporary file names. :: 576 577 import tempfile 578 import os 579 580 class Popen3: 581 """ 582 This is a deadlock-safe version of popen that returns 583 an object with errorlevel, out (a string) and err (a string). 584 (capturestderr may not work under windows.) 585 Example: print(Popen3('grep spam','\n\nhere spam\n\n').out) 586 """ 587 def __init__(self,command,input=None,capturestderr=None): 588 outfile=tempfile.mktemp() 589 command="( %s ) > %s" % (command,outfile) 590 if input: 591 infile=tempfile.mktemp() 592 open(infile,"w").write(input) 593 command=command+" <"+infile 594 if capturestderr: 595 errfile=tempfile.mktemp() 596 command=command+" 2>"+errfile 597 self.errorlevel=os.system(command) >> 8 598 self.out=open(outfile,"r").read() 599 os.remove(outfile) 600 if input: 601 os.remove(infile) 602 if capturestderr: 603 self.err=open(errfile,"r").read() 604 os.remove(errfile) 605 606 Note that many interactive programs (e.g. vi) don't work well with pipes 607 substituted for standard input and output. You will have to use pseudo ttys 608 ("ptys") instead of pipes. Or you can use a Python interface to Don Libes' 609 "expect" library. A Python extension that interfaces to expect is called 610 "expy" and available from http://expectpy.sourceforge.net. A pure Python 611 solution that works like expect is `pexpect 612 <https://pypi.python.org/pypi/pexpect/>`_. 613 614 615How do I access the serial (RS232) port? 616---------------------------------------- 617 618For Win32, POSIX (Linux, BSD, etc.), Jython: 619 620 http://pyserial.sourceforge.net 621 622For Unix, see a Usenet post by Mitch Chapman: 623 624 https://groups.google.com/groups?selm=34A04430.CF9@ohioee.com 625 626 627Why doesn't closing sys.stdout (stdin, stderr) really close it? 628--------------------------------------------------------------- 629 630Python :term:`file objects <file object>` are a high-level layer of 631abstraction on low-level C file descriptors. 632 633For most file objects you create in Python via the built-in :func:`open` 634function, ``f.close()`` marks the Python file object as being closed from 635Python's point of view, and also arranges to close the underlying C file 636descriptor. This also happens automatically in ``f``'s destructor, when 637``f`` becomes garbage. 638 639But stdin, stdout and stderr are treated specially by Python, because of the 640special status also given to them by C. Running ``sys.stdout.close()`` marks 641the Python-level file object as being closed, but does *not* close the 642associated C file descriptor. 643 644To close the underlying C file descriptor for one of these three, you should 645first be sure that's what you really want to do (e.g., you may confuse 646extension modules trying to do I/O). If it is, use :func:`os.close`:: 647 648 os.close(stdin.fileno()) 649 os.close(stdout.fileno()) 650 os.close(stderr.fileno()) 651 652Or you can use the numeric constants 0, 1 and 2, respectively. 653 654 655Network/Internet Programming 656============================ 657 658What WWW tools are there for Python? 659------------------------------------ 660 661See the chapters titled :ref:`internet` and :ref:`netdata` in the Library 662Reference Manual. Python has many modules that will help you build server-side 663and client-side web systems. 664 665.. XXX check if wiki page is still up to date 666 667A summary of available frameworks is maintained by Paul Boddie at 668https://wiki.python.org/moin/WebProgramming\ . 669 670Cameron Laird maintains a useful set of pages about Python web technologies at 671http://phaseit.net/claird/comp.lang.python/web_python. 672 673 674How can I mimic CGI form submission (METHOD=POST)? 675-------------------------------------------------- 676 677I would like to retrieve web pages that are the result of POSTing a form. Is 678there existing code that would let me do this easily? 679 680Yes. Here's a simple example that uses urllib.request:: 681 682 #!/usr/local/bin/python 683 684 import urllib.request 685 686 # build the query string 687 qs = "First=Josephine&MI=Q&Last=Public" 688 689 # connect and send the server a path 690 req = urllib.request.urlopen('http://www.some-server.out-there' 691 '/cgi-bin/some-cgi-script', data=qs) 692 with req: 693 msg, hdrs = req.read(), req.info() 694 695Note that in general for percent-encoded POST operations, query strings must be 696quoted using :func:`urllib.parse.urlencode`. For example, to send 697``name=Guy Steele, Jr.``:: 698 699 >>> import urllib.parse 700 >>> urllib.parse.urlencode({'name': 'Guy Steele, Jr.'}) 701 'name=Guy+Steele%2C+Jr.' 702 703.. seealso:: :ref:`urllib-howto` for extensive examples. 704 705 706What module should I use to help with generating HTML? 707------------------------------------------------------ 708 709.. XXX add modern template languages 710 711You can find a collection of useful links on the `Web Programming wiki page 712<https://wiki.python.org/moin/WebProgramming>`_. 713 714 715How do I send mail from a Python script? 716---------------------------------------- 717 718Use the standard library module :mod:`smtplib`. 719 720Here's a very simple interactive mail sender that uses it. This method will 721work on any host that supports an SMTP listener. :: 722 723 import sys, smtplib 724 725 fromaddr = input("From: ") 726 toaddrs = input("To: ").split(',') 727 print("Enter message, end with ^D:") 728 msg = '' 729 while True: 730 line = sys.stdin.readline() 731 if not line: 732 break 733 msg += line 734 735 # The actual mail send 736 server = smtplib.SMTP('localhost') 737 server.sendmail(fromaddr, toaddrs, msg) 738 server.quit() 739 740A Unix-only alternative uses sendmail. The location of the sendmail program 741varies between systems; sometimes it is ``/usr/lib/sendmail``, sometimes 742``/usr/sbin/sendmail``. The sendmail manual page will help you out. Here's 743some sample code:: 744 745 import os 746 747 SENDMAIL = "/usr/sbin/sendmail" # sendmail location 748 p = os.popen("%s -t -i" % SENDMAIL, "w") 749 p.write("To: receiver@example.com\n") 750 p.write("Subject: test\n") 751 p.write("\n") # blank line separating headers from body 752 p.write("Some text\n") 753 p.write("some more text\n") 754 sts = p.close() 755 if sts != 0: 756 print("Sendmail exit status", sts) 757 758 759How do I avoid blocking in the connect() method of a socket? 760------------------------------------------------------------ 761 762The :mod:`select` module is commonly used to help with asynchronous I/O on 763sockets. 764 765To prevent the TCP connect from blocking, you can set the socket to non-blocking 766mode. Then when you do the ``connect()``, you will either connect immediately 767(unlikely) or get an exception that contains the error number as ``.errno``. 768``errno.EINPROGRESS`` indicates that the connection is in progress, but hasn't 769finished yet. Different OSes will return different values, so you're going to 770have to check what's returned on your system. 771 772You can use the ``connect_ex()`` method to avoid creating an exception. It will 773just return the errno value. To poll, you can call ``connect_ex()`` again later 774-- ``0`` or ``errno.EISCONN`` indicate that you're connected -- or you can pass this 775socket to select to check if it's writable. 776 777.. note:: 778 The :mod:`asyncore` module presents a framework-like approach to the problem 779 of writing non-blocking networking code. 780 The third-party `Twisted <https://twistedmatrix.com/trac/>`_ library is 781 a popular and feature-rich alternative. 782 783 784Databases 785========= 786 787Are there any interfaces to database packages in Python? 788-------------------------------------------------------- 789 790Yes. 791 792Interfaces to disk-based hashes such as :mod:`DBM <dbm.ndbm>` and :mod:`GDBM 793<dbm.gnu>` are also included with standard Python. There is also the 794:mod:`sqlite3` module, which provides a lightweight disk-based relational 795database. 796 797Support for most relational databases is available. See the 798`DatabaseProgramming wiki page 799<https://wiki.python.org/moin/DatabaseProgramming>`_ for details. 800 801 802How do you implement persistent objects in Python? 803-------------------------------------------------- 804 805The :mod:`pickle` library module solves this in a very general way (though you 806still can't store things like open files, sockets or windows), and the 807:mod:`shelve` library module uses pickle and (g)dbm to create persistent 808mappings containing arbitrary Python objects. 809 810 811Mathematics and Numerics 812======================== 813 814How do I generate random numbers in Python? 815------------------------------------------- 816 817The standard module :mod:`random` implements a random number generator. Usage 818is simple:: 819 820 import random 821 random.random() 822 823This returns a random floating point number in the range [0, 1). 824 825There are also many other specialized generators in this module, such as: 826 827* ``randrange(a, b)`` chooses an integer in the range [a, b). 828* ``uniform(a, b)`` chooses a floating point number in the range [a, b). 829* ``normalvariate(mean, sdev)`` samples the normal (Gaussian) distribution. 830 831Some higher-level functions operate on sequences directly, such as: 832 833* ``choice(S)`` chooses random element from a given sequence 834* ``shuffle(L)`` shuffles a list in-place, i.e. permutes it randomly 835 836There's also a ``Random`` class you can instantiate to create independent 837multiple random number generators. 838