1#! /usr/bin/env python 2 3""" 4Usage: 5 6python -m test.regrtest [options] [test_name1 [test_name2 ...]] 7python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]] 8 9 10If no arguments or options are provided, finds all files matching 11the pattern "test_*" in the Lib/test subdirectory and runs 12them in alphabetical order (but see -M and -u, below, for exceptions). 13 14For more rigorous testing, it is useful to use the following 15command line: 16 17python -E -tt -Wd -3 -m test.regrtest [options] [test_name1 ...] 18 19 20Options: 21 22-h/--help -- print this text and exit 23 24Verbosity 25 26-v/--verbose -- run tests in verbose mode with output to stdout 27-w/--verbose2 -- re-run failed tests in verbose mode 28-W/--verbose3 -- re-run failed tests in verbose mode immediately 29-q/--quiet -- no output unless one or more tests fail 30-S/--slow -- print the slowest 10 tests 31 --header -- print header with interpreter info 32 33Selecting tests 34 35-r/--randomize -- randomize test execution order (see below) 36 --randseed -- pass a random seed to reproduce a previous random run 37-f/--fromfile -- read names of tests to run from a file (see below) 38-x/--exclude -- arguments are tests to *exclude* 39-s/--single -- single step through a set of tests (see below) 40-u/--use RES1,RES2,... 41 -- specify which special resource intensive tests to run 42-M/--memlimit LIMIT 43 -- run very large memory-consuming tests 44 45Special runs 46 47-l/--findleaks -- if GC is available detect tests that leak memory 48-L/--runleaks -- run the leaks(1) command just before exit 49-R/--huntrleaks RUNCOUNTS 50 -- search for reference leaks (needs debug build, v. slow) 51-j/--multiprocess PROCESSES 52 -- run PROCESSES processes at once 53-T/--coverage -- turn on code coverage tracing using the trace module 54-D/--coverdir DIRECTORY 55 -- Directory where coverage files are put 56-N/--nocoverdir -- Put coverage files alongside modules 57-t/--threshold THRESHOLD 58 -- call gc.set_threshold(THRESHOLD) 59-F/--forever -- run the specified tests in a loop, until an error happens 60-P/--pgo -- enable Profile Guided Optimization training 61 62 63Additional Option Details: 64 65-r randomizes test execution order. You can use --randseed=int to provide an 66int seed value for the randomizer; this is useful for reproducing troublesome 67test orders. 68 69-s On the first invocation of regrtest using -s, the first test file found 70or the first test file given on the command line is run, and the name of 71the next test is recorded in a file named pynexttest. If run from the 72Python build directory, pynexttest is located in the 'build' subdirectory, 73otherwise it is located in tempfile.gettempdir(). On subsequent runs, 74the test in pynexttest is run, and the next test is written to pynexttest. 75When the last test has been run, pynexttest is deleted. In this way it 76is possible to single step through the test files. This is useful when 77doing memory analysis on the Python interpreter, which process tends to 78consume too many resources to run the full regression test non-stop. 79 80-f reads the names of tests from the file given as f's argument, one 81or more test names per line. Whitespace is ignored. Blank lines and 82lines beginning with '#' are ignored. This is especially useful for 83whittling down failures involving interactions among tests. 84 85-L causes the leaks(1) command to be run just before exit if it exists. 86leaks(1) is available on Mac OS X and presumably on some other 87FreeBSD-derived systems. 88 89-R runs each test several times and examines sys.gettotalrefcount() to 90see if the test appears to be leaking references. The argument should 91be of the form stab:run:fname where 'stab' is the number of times the 92test is run to let gettotalrefcount settle down, 'run' is the number 93of times further it is run and 'fname' is the name of the file the 94reports are written to. These parameters all have defaults (5, 4 and 95"reflog.txt" respectively), and the minimal invocation is '-R :'. 96 97-M runs tests that require an exorbitant amount of memory. These tests 98typically try to ascertain containers keep working when containing more than 992 billion objects, which only works on 64-bit systems. There are also some 100tests that try to exhaust the address space of the process, which only makes 101sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit, 102which is a string in the form of '2.5Gb', determines howmuch memory the 103tests will limit themselves to (but they may go slightly over.) The number 104shouldn't be more memory than the machine has (including swap memory). You 105should also keep in mind that swap memory is generally much, much slower 106than RAM, and setting memlimit to all available RAM or higher will heavily 107tax the machine. On the other hand, it is no use running these tests with a 108limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect 109to use more than memlimit memory will be skipped. The big-memory tests 110generally run very, very long. 111 112-u is used to specify which special resource intensive tests to run, 113such as those requiring large file support or network connectivity. 114The argument is a comma-separated list of words indicating the 115resources to test. Currently only the following are defined: 116 117 all - Enable all special resources. 118 119 audio - Tests that use the audio device. (There are known 120 cases of broken audio drivers that can crash Python or 121 even the Linux kernel.) 122 123 curses - Tests that use curses and will modify the terminal's 124 state and output modes. 125 126 largefile - It is okay to run some test that may create huge 127 files. These tests can take a long time and may 128 consume >2GB of disk space temporarily. 129 130 network - It is okay to run tests that use external network 131 resource, e.g. testing SSL support for sockets. 132 133 bsddb - It is okay to run the bsddb testsuite, which takes 134 a long time to complete. 135 136 decimal - Test the decimal module against a large suite that 137 verifies compliance with standards. 138 139 cpu - Used for certain CPU-heavy tests. 140 141 subprocess Run all tests for the subprocess module. 142 143 urlfetch - It is okay to download files required on testing. 144 145 gui - Run tests that require a running GUI. 146 147 xpickle - Test pickle and cPickle against Python 2.4, 2.5 and 2.6 to 148 test backwards compatibility. These tests take a long time 149 to run. 150 151To enable all resources except one, use '-uall,-<resource>'. For 152example, to run all the tests except for the bsddb tests, give the 153option '-uall,-bsddb'. 154""" 155 156import StringIO 157import getopt 158import json 159import os 160import random 161import re 162import shutil 163import sys 164import time 165import traceback 166import warnings 167import unittest 168import tempfile 169import imp 170import platform 171import sysconfig 172 173 174# Some times __path__ and __file__ are not absolute (e.g. while running from 175# Lib/) and, if we change the CWD to run the tests in a temporary dir, some 176# imports might fail. This affects only the modules imported before os.chdir(). 177# These modules are searched first in sys.path[0] (so '' -- the CWD) and if 178# they are found in the CWD their __file__ and __path__ will be relative (this 179# happens before the chdir). All the modules imported after the chdir, are 180# not found in the CWD, and since the other paths in sys.path[1:] are absolute 181# (site.py absolutize them), the __file__ and __path__ will be absolute too. 182# Therefore it is necessary to absolutize manually the __file__ and __path__ of 183# the packages to prevent later imports to fail when the CWD is different. 184for module in sys.modules.itervalues(): 185 if hasattr(module, '__path__'): 186 module.__path__ = [os.path.abspath(path) for path in module.__path__] 187 if hasattr(module, '__file__'): 188 module.__file__ = os.path.abspath(module.__file__) 189 190 191# MacOSX (a.k.a. Darwin) has a default stack size that is too small 192# for deeply recursive regular expressions. We see this as crashes in 193# the Python test suite when running test_re.py and test_sre.py. The 194# fix is to set the stack limit to 2048. 195# This approach may also be useful for other Unixy platforms that 196# suffer from small default stack limits. 197if sys.platform == 'darwin': 198 try: 199 import resource 200 except ImportError: 201 pass 202 else: 203 soft, hard = resource.getrlimit(resource.RLIMIT_STACK) 204 newsoft = min(hard, max(soft, 1024*2048)) 205 resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard)) 206 207# Windows, Tkinter, and resetting the environment after each test don't 208# mix well. To alleviate test failures due to Tcl/Tk not being able to 209# find its library, get the necessary environment massage done once early. 210if sys.platform == 'win32': 211 try: 212 import FixTk 213 except Exception: 214 pass 215 216# Test result constants. 217PASSED = 1 218FAILED = 0 219ENV_CHANGED = -1 220SKIPPED = -2 221RESOURCE_DENIED = -3 222INTERRUPTED = -4 223 224from test import test_support 225 226RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb', 227 'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui', 228 'xpickle') 229 230TEMPDIR = os.path.abspath(tempfile.gettempdir()) 231 232 233def usage(code, msg=''): 234 print __doc__ 235 if msg: print msg 236 sys.exit(code) 237 238 239def main(tests=None, testdir=None, verbose=0, quiet=False, 240 exclude=False, single=False, randomize=False, fromfile=None, 241 findleaks=False, use_resources=None, trace=False, coverdir='coverage', 242 runleaks=False, huntrleaks=False, verbose2=False, print_slow=False, 243 random_seed=None, use_mp=None, verbose3=False, forever=False, 244 header=False, pgo=False): 245 """Execute a test suite. 246 247 This also parses command-line options and modifies its behavior 248 accordingly. 249 250 tests -- a list of strings containing test names (optional) 251 testdir -- the directory in which to look for tests (optional) 252 253 Users other than the Python test suite will certainly want to 254 specify testdir; if it's omitted, the directory containing the 255 Python test suite is searched for. 256 257 If the tests argument is omitted, the tests listed on the 258 command-line will be used. If that's empty, too, then all *.py 259 files beginning with test_ will be used. 260 261 The other default arguments (verbose, quiet, exclude, 262 single, randomize, findleaks, use_resources, trace, coverdir, 263 print_slow, and random_seed) allow programmers calling main() 264 directly to set the values that would normally be set by flags 265 on the command line. 266 """ 267 268 test_support.record_original_stdout(sys.stdout) 269 try: 270 opts, args = getopt.getopt(sys.argv[1:], 'hvqxsSrf:lu:t:TD:NLR:FwWM:j:P', 271 ['help', 'verbose', 'verbose2', 'verbose3', 'quiet', 272 'exclude', 'single', 'slow', 'randomize', 'fromfile=', 'findleaks', 273 'use=', 'threshold=', 'trace', 'coverdir=', 'nocoverdir', 274 'runleaks', 'huntrleaks=', 'memlimit=', 'randseed=', 275 'multiprocess=', 'slaveargs=', 'forever', 'header', 'pgo']) 276 except getopt.error, msg: 277 usage(2, msg) 278 279 # Defaults 280 if random_seed is None: 281 random_seed = random.randrange(10000000) 282 if use_resources is None: 283 use_resources = [] 284 for o, a in opts: 285 if o in ('-h', '--help'): 286 usage(0) 287 elif o in ('-v', '--verbose'): 288 verbose += 1 289 elif o in ('-w', '--verbose2'): 290 verbose2 = True 291 elif o in ('-W', '--verbose3'): 292 verbose3 = True 293 elif o in ('-q', '--quiet'): 294 quiet = True; 295 verbose = 0 296 elif o in ('-x', '--exclude'): 297 exclude = True 298 elif o in ('-s', '--single'): 299 single = True 300 elif o in ('-S', '--slow'): 301 print_slow = True 302 elif o in ('-r', '--randomize'): 303 randomize = True 304 elif o == '--randseed': 305 random_seed = int(a) 306 elif o in ('-f', '--fromfile'): 307 fromfile = a 308 elif o in ('-l', '--findleaks'): 309 findleaks = True 310 elif o in ('-L', '--runleaks'): 311 runleaks = True 312 elif o in ('-t', '--threshold'): 313 import gc 314 gc.set_threshold(int(a)) 315 elif o in ('-T', '--coverage'): 316 trace = True 317 elif o in ('-D', '--coverdir'): 318 coverdir = os.path.join(os.getcwd(), a) 319 elif o in ('-N', '--nocoverdir'): 320 coverdir = None 321 elif o in ('-R', '--huntrleaks'): 322 huntrleaks = a.split(':') 323 if len(huntrleaks) not in (2, 3): 324 print a, huntrleaks 325 usage(2, '-R takes 2 or 3 colon-separated arguments') 326 if not huntrleaks[0]: 327 huntrleaks[0] = 5 328 else: 329 huntrleaks[0] = int(huntrleaks[0]) 330 if not huntrleaks[1]: 331 huntrleaks[1] = 4 332 else: 333 huntrleaks[1] = int(huntrleaks[1]) 334 if len(huntrleaks) == 2 or not huntrleaks[2]: 335 huntrleaks[2:] = ["reflog.txt"] 336 elif o in ('-M', '--memlimit'): 337 test_support.set_memlimit(a) 338 elif o in ('-u', '--use'): 339 u = [x.lower() for x in a.split(',')] 340 for r in u: 341 if r == 'all': 342 use_resources[:] = RESOURCE_NAMES 343 continue 344 remove = False 345 if r[0] == '-': 346 remove = True 347 r = r[1:] 348 if r not in RESOURCE_NAMES: 349 usage(1, 'Invalid -u/--use option: ' + a) 350 if remove: 351 if r in use_resources: 352 use_resources.remove(r) 353 elif r not in use_resources: 354 use_resources.append(r) 355 elif o in ('-F', '--forever'): 356 forever = True 357 elif o in ('-j', '--multiprocess'): 358 use_mp = int(a) 359 elif o == '--header': 360 header = True 361 elif o == '--slaveargs': 362 args, kwargs = json.loads(a) 363 try: 364 result = runtest(*args, **kwargs) 365 except BaseException, e: 366 result = INTERRUPTED, e.__class__.__name__ 367 print # Force a newline (just in case) 368 print json.dumps(result) 369 sys.exit(0) 370 elif o in ('-P', '--pgo'): 371 pgo = True 372 else: 373 print >>sys.stderr, ("No handler for option {}. Please " 374 "report this as a bug at http://bugs.python.org.").format(o) 375 sys.exit(1) 376 if single and fromfile: 377 usage(2, "-s and -f don't go together!") 378 if use_mp and trace: 379 usage(2, "-T and -j don't go together!") 380 if use_mp and findleaks: 381 usage(2, "-l and -j don't go together!") 382 383 good = [] 384 bad = [] 385 skipped = [] 386 resource_denieds = [] 387 environment_changed = [] 388 interrupted = False 389 390 if findleaks: 391 try: 392 import gc 393 except ImportError: 394 print 'No GC available, disabling findleaks.' 395 findleaks = False 396 else: 397 # Uncomment the line below to report garbage that is not 398 # freeable by reference counting alone. By default only 399 # garbage that is not collectable by the GC is reported. 400 #gc.set_debug(gc.DEBUG_SAVEALL) 401 found_garbage = [] 402 403 if single: 404 filename = os.path.join(TEMPDIR, 'pynexttest') 405 try: 406 fp = open(filename, 'r') 407 next_test = fp.read().strip() 408 tests = [next_test] 409 fp.close() 410 except IOError: 411 pass 412 413 if fromfile: 414 tests = [] 415 fp = open(os.path.join(test_support.SAVEDCWD, fromfile)) 416 for line in fp: 417 guts = line.split() # assuming no test has whitespace in its name 418 if guts and not guts[0].startswith('#'): 419 tests.extend(guts) 420 fp.close() 421 422 # Strip .py extensions. 423 removepy(args) 424 removepy(tests) 425 426 stdtests = STDTESTS[:] 427 nottests = NOTTESTS.copy() 428 if exclude: 429 for arg in args: 430 if arg in stdtests: 431 stdtests.remove(arg) 432 nottests.add(arg) 433 args = [] 434 435 # For a partial run, we do not need to clutter the output. 436 if verbose or header or not (quiet or single or tests or args): 437 if not pgo: 438 # Print basic platform information 439 print "==", platform.python_implementation(), \ 440 " ".join(sys.version.split()) 441 print "== ", platform.platform(aliased=True), \ 442 "%s-endian" % sys.byteorder 443 print "== ", os.getcwd() 444 print "Testing with flags:", sys.flags 445 446 alltests = findtests(testdir, stdtests, nottests) 447 selected = tests or args or alltests 448 if single: 449 selected = selected[:1] 450 try: 451 next_single_test = alltests[alltests.index(selected[0])+1] 452 except IndexError: 453 next_single_test = None 454 if randomize: 455 random.seed(random_seed) 456 print "Using random seed", random_seed 457 random.shuffle(selected) 458 if trace: 459 import trace 460 tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix], 461 trace=False, count=True) 462 463 test_times = [] 464 test_support.use_resources = use_resources 465 save_modules = set(sys.modules) 466 467 def accumulate_result(test, result): 468 ok, test_time = result 469 test_times.append((test_time, test)) 470 if ok == PASSED: 471 good.append(test) 472 elif ok == FAILED: 473 bad.append(test) 474 elif ok == ENV_CHANGED: 475 environment_changed.append(test) 476 elif ok == SKIPPED: 477 skipped.append(test) 478 elif ok == RESOURCE_DENIED: 479 skipped.append(test) 480 resource_denieds.append(test) 481 482 if forever: 483 def test_forever(tests=list(selected)): 484 while True: 485 for test in tests: 486 yield test 487 if bad: 488 return 489 tests = test_forever() 490 test_count = '' 491 test_count_width = 3 492 else: 493 tests = iter(selected) 494 test_count = '/{}'.format(len(selected)) 495 test_count_width = len(test_count) - 1 496 497 if use_mp: 498 try: 499 from threading import Thread 500 except ImportError: 501 print "Multiprocess option requires thread support" 502 sys.exit(2) 503 from Queue import Queue 504 from subprocess import Popen, PIPE 505 debug_output_pat = re.compile(r"\[\d+ refs\]$") 506 output = Queue() 507 def tests_and_args(): 508 for test in tests: 509 args_tuple = ( 510 (test, verbose, quiet), 511 dict(huntrleaks=huntrleaks, use_resources=use_resources, 512 pgo=pgo) 513 ) 514 yield (test, args_tuple) 515 pending = tests_and_args() 516 opt_args = test_support.args_from_interpreter_flags() 517 base_cmd = [sys.executable] + opt_args + ['-m', 'test.regrtest'] 518 # required to spawn a new process with PGO flag on/off 519 if pgo: 520 base_cmd = base_cmd + ['--pgo'] 521 def work(): 522 # A worker thread. 523 try: 524 while True: 525 try: 526 test, args_tuple = next(pending) 527 except StopIteration: 528 output.put((None, None, None, None)) 529 return 530 # -E is needed by some tests, e.g. test_import 531 popen = Popen(base_cmd + ['--slaveargs', json.dumps(args_tuple)], 532 stdout=PIPE, stderr=PIPE, 533 universal_newlines=True, 534 close_fds=(os.name != 'nt')) 535 stdout, stderr = popen.communicate() 536 # Strip last refcount output line if it exists, since it 537 # comes from the shutdown of the interpreter in the subcommand. 538 stderr = debug_output_pat.sub("", stderr) 539 stdout, _, result = stdout.strip().rpartition("\n") 540 if not result: 541 output.put((None, None, None, None)) 542 return 543 result = json.loads(result) 544 output.put((test, stdout.rstrip(), stderr.rstrip(), result)) 545 except BaseException: 546 output.put((None, None, None, None)) 547 raise 548 workers = [Thread(target=work) for i in range(use_mp)] 549 for worker in workers: 550 worker.start() 551 finished = 0 552 test_index = 1 553 try: 554 while finished < use_mp: 555 test, stdout, stderr, result = output.get() 556 if test is None: 557 finished += 1 558 continue 559 if stdout: 560 print stdout 561 if stderr and not pgo: 562 print >>sys.stderr, stderr 563 sys.stdout.flush() 564 sys.stderr.flush() 565 if result[0] == INTERRUPTED: 566 assert result[1] == 'KeyboardInterrupt' 567 raise KeyboardInterrupt # What else? 568 accumulate_result(test, result) 569 if not quiet: 570 fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" 571 print(fmt.format( 572 test_count_width, test_index, test_count, 573 len(bad), test)) 574 test_index += 1 575 except KeyboardInterrupt: 576 interrupted = True 577 pending.close() 578 for worker in workers: 579 worker.join() 580 else: 581 for test_index, test in enumerate(tests, 1): 582 if not quiet: 583 fmt = "[{1:{0}}{2}/{3}] {4}" if bad else "[{1:{0}}{2}] {4}" 584 print(fmt.format( 585 test_count_width, test_index, test_count, len(bad), test)) 586 sys.stdout.flush() 587 if trace: 588 # If we're tracing code coverage, then we don't exit with status 589 # if on a false return value from main. 590 tracer.runctx('runtest(test, verbose, quiet)', 591 globals=globals(), locals=vars()) 592 else: 593 try: 594 result = runtest(test, verbose, quiet, huntrleaks, None, pgo) 595 accumulate_result(test, result) 596 if verbose3 and result[0] == FAILED: 597 if not pgo: 598 print "Re-running test %r in verbose mode" % test 599 runtest(test, True, quiet, huntrleaks, None, pgo) 600 except KeyboardInterrupt: 601 interrupted = True 602 break 603 except: 604 raise 605 if findleaks: 606 gc.collect() 607 if gc.garbage: 608 print "Warning: test created", len(gc.garbage), 609 print "uncollectable object(s)." 610 # move the uncollectable objects somewhere so we don't see 611 # them again 612 found_garbage.extend(gc.garbage) 613 del gc.garbage[:] 614 # Unload the newly imported modules (best effort finalization) 615 for module in sys.modules.keys(): 616 if module not in save_modules and module.startswith("test."): 617 test_support.unload(module) 618 619 if interrupted and not pgo: 620 # print a newline after ^C 621 print 622 print "Test suite interrupted by signal SIGINT." 623 omitted = set(selected) - set(good) - set(bad) - set(skipped) 624 print count(len(omitted), "test"), "omitted:" 625 printlist(omitted) 626 if good and not quiet and not pgo: 627 if not bad and not skipped and not interrupted and len(good) > 1: 628 print "All", 629 print count(len(good), "test"), "OK." 630 if print_slow: 631 test_times.sort(reverse=True) 632 print "10 slowest tests:" 633 for time, test in test_times[:10]: 634 print "%s: %.1fs" % (test, time) 635 if bad and not pgo: 636 print count(len(bad), "test"), "failed:" 637 printlist(bad) 638 if environment_changed and not pgo: 639 print "{} altered the execution environment:".format( 640 count(len(environment_changed), "test")) 641 printlist(environment_changed) 642 if skipped and not quiet and not pgo: 643 print count(len(skipped), "test"), "skipped:" 644 printlist(skipped) 645 646 e = _ExpectedSkips() 647 plat = sys.platform 648 if e.isvalid(): 649 surprise = set(skipped) - e.getexpected() - set(resource_denieds) 650 if surprise: 651 print count(len(surprise), "skip"), \ 652 "unexpected on", plat + ":" 653 printlist(surprise) 654 else: 655 print "Those skips are all expected on", plat + "." 656 else: 657 print "Ask someone to teach regrtest.py about which tests are" 658 print "expected to get skipped on", plat + "." 659 660 if verbose2 and bad: 661 print "Re-running failed tests in verbose mode" 662 for test in bad[:]: 663 print "Re-running test %r in verbose mode" % test 664 sys.stdout.flush() 665 try: 666 test_support.verbose = True 667 ok = runtest(test, True, quiet, huntrleaks, None, pgo) 668 except KeyboardInterrupt: 669 # print a newline separate from the ^C 670 print 671 break 672 else: 673 if ok[0] in {PASSED, ENV_CHANGED, SKIPPED, RESOURCE_DENIED}: 674 bad.remove(test) 675 else: 676 if bad: 677 print count(len(bad), "test"), "failed again:" 678 printlist(bad) 679 680 if single: 681 if next_single_test: 682 with open(filename, 'w') as fp: 683 fp.write(next_single_test + '\n') 684 else: 685 os.unlink(filename) 686 687 if trace: 688 r = tracer.results() 689 r.write_results(show_missing=True, summary=True, coverdir=coverdir) 690 691 if runleaks: 692 os.system("leaks %d" % os.getpid()) 693 694 sys.exit(len(bad) > 0 or interrupted) 695 696 697STDTESTS = [ 698 'test_grammar', 699 'test_opcodes', 700 'test_dict', 701 'test_builtin', 702 'test_exceptions', 703 'test_types', 704 'test_unittest', 705 'test_doctest', 706 'test_doctest2', 707] 708 709NOTTESTS = { 710 'test_support', 711 'test_future1', 712 'test_future2', 713} 714 715def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS): 716 """Return a list of all applicable test modules.""" 717 testdir = findtestdir(testdir) 718 names = os.listdir(testdir) 719 tests = [] 720 others = set(stdtests) | nottests 721 for name in names: 722 modname, ext = os.path.splitext(name) 723 if modname[:5] == "test_" and ext == ".py" and modname not in others: 724 tests.append(modname) 725 return stdtests + sorted(tests) 726 727def runtest(test, verbose, quiet, 728 huntrleaks=False, use_resources=None, pgo=False): 729 """Run a single test. 730 731 test -- the name of the test 732 verbose -- if true, print more messages 733 quiet -- if true, don't print 'skipped' messages (probably redundant) 734 test_times -- a list of (time, test_name) pairs 735 huntrleaks -- run multiple times to test for leaks; requires a debug 736 build; a triple corresponding to -R's three arguments 737 pgo -- if true, do not print unnecessary info when running the test 738 for Profile Guided Optimization build 739 740 Returns one of the test result constants: 741 INTERRUPTED KeyboardInterrupt when run under -j 742 RESOURCE_DENIED test skipped because resource denied 743 SKIPPED test skipped for some other reason 744 ENV_CHANGED test failed because it changed the execution environment 745 FAILED test failed 746 PASSED test passed 747 """ 748 749 test_support.verbose = verbose # Tell tests to be moderately quiet 750 if use_resources is not None: 751 test_support.use_resources = use_resources 752 try: 753 return runtest_inner(test, verbose, quiet, huntrleaks, pgo) 754 finally: 755 cleanup_test_droppings(test, verbose) 756 757 758# Unit tests are supposed to leave the execution environment unchanged 759# once they complete. But sometimes tests have bugs, especially when 760# tests fail, and the changes to environment go on to mess up other 761# tests. This can cause issues with buildbot stability, since tests 762# are run in random order and so problems may appear to come and go. 763# There are a few things we can save and restore to mitigate this, and 764# the following context manager handles this task. 765 766class saved_test_environment: 767 """Save bits of the test environment and restore them at block exit. 768 769 with saved_test_environment(testname, verbose, quiet): 770 #stuff 771 772 Unless quiet is True, a warning is printed to stderr if any of 773 the saved items was changed by the test. The attribute 'changed' 774 is initially False, but is set to True if a change is detected. 775 776 If verbose is more than 1, the before and after state of changed 777 items is also printed. 778 """ 779 780 changed = False 781 782 def __init__(self, testname, verbose=0, quiet=False, pgo=False): 783 self.testname = testname 784 self.verbose = verbose 785 self.quiet = quiet 786 self.pgo = pgo 787 788 # To add things to save and restore, add a name XXX to the resources list 789 # and add corresponding get_XXX/restore_XXX functions. get_XXX should 790 # return the value to be saved and compared against a second call to the 791 # get function when test execution completes. restore_XXX should accept 792 # the saved value and restore the resource using it. It will be called if 793 # and only if a change in the value is detected. 794 # 795 # Note: XXX will have any '.' replaced with '_' characters when determining 796 # the corresponding method names. 797 798 resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr', 799 'os.environ', 'sys.path', 'asyncore.socket_map', 800 'files', 801 ) 802 803 def get_sys_argv(self): 804 return id(sys.argv), sys.argv, sys.argv[:] 805 def restore_sys_argv(self, saved_argv): 806 sys.argv = saved_argv[1] 807 sys.argv[:] = saved_argv[2] 808 809 def get_cwd(self): 810 return os.getcwd() 811 def restore_cwd(self, saved_cwd): 812 os.chdir(saved_cwd) 813 814 def get_sys_stdout(self): 815 return sys.stdout 816 def restore_sys_stdout(self, saved_stdout): 817 sys.stdout = saved_stdout 818 819 def get_sys_stderr(self): 820 return sys.stderr 821 def restore_sys_stderr(self, saved_stderr): 822 sys.stderr = saved_stderr 823 824 def get_sys_stdin(self): 825 return sys.stdin 826 def restore_sys_stdin(self, saved_stdin): 827 sys.stdin = saved_stdin 828 829 def get_os_environ(self): 830 return id(os.environ), os.environ, dict(os.environ) 831 def restore_os_environ(self, saved_environ): 832 os.environ = saved_environ[1] 833 os.environ.clear() 834 os.environ.update(saved_environ[2]) 835 836 def get_sys_path(self): 837 return id(sys.path), sys.path, sys.path[:] 838 def restore_sys_path(self, saved_path): 839 sys.path = saved_path[1] 840 sys.path[:] = saved_path[2] 841 842 def get_asyncore_socket_map(self): 843 asyncore = sys.modules.get('asyncore') 844 # XXX Making a copy keeps objects alive until __exit__ gets called. 845 return asyncore and asyncore.socket_map.copy() or {} 846 def restore_asyncore_socket_map(self, saved_map): 847 asyncore = sys.modules.get('asyncore') 848 if asyncore is not None: 849 asyncore.close_all(ignore_all=True) 850 asyncore.socket_map.update(saved_map) 851 852 def get_test_support_TESTFN(self): 853 if os.path.isfile(test_support.TESTFN): 854 result = 'f' 855 elif os.path.isdir(test_support.TESTFN): 856 result = 'd' 857 else: 858 result = None 859 return result 860 def restore_test_support_TESTFN(self, saved_value): 861 if saved_value is None: 862 if os.path.isfile(test_support.TESTFN): 863 os.unlink(test_support.TESTFN) 864 elif os.path.isdir(test_support.TESTFN): 865 shutil.rmtree(test_support.TESTFN) 866 867 def get_files(self): 868 return sorted(fn + ('/' if os.path.isdir(fn) else '') 869 for fn in os.listdir(os.curdir)) 870 def restore_files(self, saved_value): 871 fn = test_support.TESTFN 872 if fn not in saved_value and (fn + '/') not in saved_value: 873 if os.path.isfile(fn): 874 test_support.unlink(fn) 875 elif os.path.isdir(fn): 876 test_support.rmtree(fn) 877 878 def resource_info(self): 879 for name in self.resources: 880 method_suffix = name.replace('.', '_') 881 get_name = 'get_' + method_suffix 882 restore_name = 'restore_' + method_suffix 883 yield name, getattr(self, get_name), getattr(self, restore_name) 884 885 def __enter__(self): 886 self.saved_values = dict((name, get()) for name, get, restore 887 in self.resource_info()) 888 return self 889 890 def __exit__(self, exc_type, exc_val, exc_tb): 891 saved_values = self.saved_values 892 del self.saved_values 893 for name, get, restore in self.resource_info(): 894 current = get() 895 original = saved_values.pop(name) 896 # Check for changes to the resource's value 897 if current != original: 898 self.changed = True 899 restore(original) 900 if not self.quiet and not self.pgo: 901 print >>sys.stderr, ( 902 "Warning -- {} was modified by {}".format( 903 name, self.testname)) 904 if self.verbose > 1 and not self.pgo: 905 print >>sys.stderr, ( 906 " Before: {}\n After: {} ".format( 907 original, current)) 908 # XXX (ncoghlan): for most resources (e.g. sys.path) identity 909 # matters at least as much as value. For others (e.g. cwd), 910 # identity is irrelevant. Should we add a mechanism to check 911 # for substitution in the cases where it matters? 912 return False 913 914 915def runtest_inner(test, verbose, quiet, huntrleaks=False, pgo=False): 916 test_support.unload(test) 917 if verbose: 918 capture_stdout = None 919 else: 920 capture_stdout = StringIO.StringIO() 921 922 test_time = 0.0 923 refleak = False # True if the test leaked references. 924 try: 925 save_stdout = sys.stdout 926 try: 927 if capture_stdout: 928 sys.stdout = capture_stdout 929 if test.startswith('test.'): 930 abstest = test 931 else: 932 # Always import it from the test package 933 abstest = 'test.' + test 934 clear_caches() 935 with saved_test_environment(test, verbose, quiet, pgo) as environment: 936 start_time = time.time() 937 the_package = __import__(abstest, globals(), locals(), []) 938 the_module = getattr(the_package, test) 939 # Old tests run to completion simply as a side-effect of 940 # being imported. For tests based on unittest or doctest, 941 # explicitly invoke their test_main() function (if it exists). 942 indirect_test = getattr(the_module, "test_main", None) 943 if indirect_test is not None: 944 indirect_test() 945 if huntrleaks: 946 refleak = dash_R(the_module, test, indirect_test, 947 huntrleaks) 948 test_time = time.time() - start_time 949 finally: 950 sys.stdout = save_stdout 951 except test_support.ResourceDenied, msg: 952 if not quiet and not pgo: 953 print test, "skipped --", msg 954 sys.stdout.flush() 955 return RESOURCE_DENIED, test_time 956 except unittest.SkipTest, msg: 957 if not quiet and not pgo: 958 print test, "skipped --", msg 959 sys.stdout.flush() 960 return SKIPPED, test_time 961 except KeyboardInterrupt: 962 raise 963 except test_support.TestFailed, msg: 964 if not pgo: 965 print >>sys.stderr, "test", test, "failed --", msg 966 sys.stderr.flush() 967 return FAILED, test_time 968 except: 969 type, value = sys.exc_info()[:2] 970 if not pgo: 971 print >>sys.stderr, "test", test, "crashed --", str(type) + ":", value 972 sys.stderr.flush() 973 if verbose and not pgo: 974 traceback.print_exc(file=sys.stderr) 975 sys.stderr.flush() 976 return FAILED, test_time 977 else: 978 if refleak: 979 return FAILED, test_time 980 if environment.changed: 981 return ENV_CHANGED, test_time 982 # Except in verbose mode, tests should not print anything 983 if verbose or huntrleaks: 984 return PASSED, test_time 985 output = capture_stdout.getvalue() 986 if not output: 987 return PASSED, test_time 988 print "test", test, "produced unexpected output:" 989 print "*" * 70 990 print output 991 print "*" * 70 992 sys.stdout.flush() 993 return FAILED, test_time 994 995def cleanup_test_droppings(testname, verbose): 996 import stat 997 import gc 998 999 # First kill any dangling references to open files etc. 1000 gc.collect() 1001 1002 # Try to clean up junk commonly left behind. While tests shouldn't leave 1003 # any files or directories behind, when a test fails that can be tedious 1004 # for it to arrange. The consequences can be especially nasty on Windows, 1005 # since if a test leaves a file open, it cannot be deleted by name (while 1006 # there's nothing we can do about that here either, we can display the 1007 # name of the offending test, which is a real help). 1008 for name in (test_support.TESTFN, 1009 "db_home", 1010 ): 1011 if not os.path.exists(name): 1012 continue 1013 1014 if os.path.isdir(name): 1015 kind, nuker = "directory", shutil.rmtree 1016 elif os.path.isfile(name): 1017 kind, nuker = "file", os.unlink 1018 else: 1019 raise SystemError("os.path says %r exists but is neither " 1020 "directory nor file" % name) 1021 1022 if verbose: 1023 print "%r left behind %s %r" % (testname, kind, name) 1024 try: 1025 # if we have chmod, fix possible permissions problems 1026 # that might prevent cleanup 1027 if (hasattr(os, 'chmod')): 1028 os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO) 1029 nuker(name) 1030 except Exception, msg: 1031 print >> sys.stderr, ("%r left behind %s %r and it couldn't be " 1032 "removed: %s" % (testname, kind, name, msg)) 1033 1034def dash_R(the_module, test, indirect_test, huntrleaks): 1035 """Run a test multiple times, looking for reference leaks. 1036 1037 Returns: 1038 False if the test didn't leak references; True if we detected refleaks. 1039 """ 1040 # This code is hackish and inelegant, but it seems to do the job. 1041 import copy_reg, _abcoll, _pyio 1042 1043 if not hasattr(sys, 'gettotalrefcount'): 1044 raise Exception("Tracking reference leaks requires a debug build " 1045 "of Python") 1046 1047 # Save current values for dash_R_cleanup() to restore. 1048 fs = warnings.filters[:] 1049 ps = copy_reg.dispatch_table.copy() 1050 pic = sys.path_importer_cache.copy() 1051 try: 1052 import zipimport 1053 except ImportError: 1054 zdc = None # Run unmodified on platforms without zipimport support 1055 else: 1056 zdc = zipimport._zip_directory_cache.copy() 1057 abcs = {} 1058 modules = _abcoll, _pyio 1059 for abc in [getattr(mod, a) for mod in modules for a in mod.__all__]: 1060 # XXX isinstance(abc, ABCMeta) leads to infinite recursion 1061 if not hasattr(abc, '_abc_registry'): 1062 continue 1063 for obj in abc.__subclasses__() + [abc]: 1064 abcs[obj] = obj._abc_registry.copy() 1065 1066 if indirect_test: 1067 def run_the_test(): 1068 indirect_test() 1069 else: 1070 def run_the_test(): 1071 imp.reload(the_module) 1072 1073 deltas = [] 1074 nwarmup, ntracked, fname = huntrleaks 1075 fname = os.path.join(test_support.SAVEDCWD, fname) 1076 repcount = nwarmup + ntracked 1077 print >> sys.stderr, "beginning", repcount, "repetitions" 1078 print >> sys.stderr, ("1234567890"*(repcount//10 + 1))[:repcount] 1079 dash_R_cleanup(fs, ps, pic, zdc, abcs) 1080 for i in range(repcount): 1081 rc_before = sys.gettotalrefcount() 1082 run_the_test() 1083 sys.stderr.write('.') 1084 dash_R_cleanup(fs, ps, pic, zdc, abcs) 1085 rc_after = sys.gettotalrefcount() 1086 if i >= nwarmup: 1087 deltas.append(rc_after - rc_before) 1088 print >> sys.stderr 1089 if any(deltas): 1090 msg = '%s leaked %s references, sum=%s' % (test, deltas, sum(deltas)) 1091 print >> sys.stderr, msg 1092 with open(fname, "a") as refrep: 1093 print >> refrep, msg 1094 refrep.flush() 1095 return True 1096 return False 1097 1098def dash_R_cleanup(fs, ps, pic, zdc, abcs): 1099 import gc, copy_reg 1100 1101 # Restore some original values. 1102 warnings.filters[:] = fs 1103 copy_reg.dispatch_table.clear() 1104 copy_reg.dispatch_table.update(ps) 1105 sys.path_importer_cache.clear() 1106 sys.path_importer_cache.update(pic) 1107 try: 1108 import zipimport 1109 except ImportError: 1110 pass # Run unmodified on platforms without zipimport support 1111 else: 1112 zipimport._zip_directory_cache.clear() 1113 zipimport._zip_directory_cache.update(zdc) 1114 1115 # clear type cache 1116 sys._clear_type_cache() 1117 1118 # Clear ABC registries, restoring previously saved ABC registries. 1119 for abc, registry in abcs.items(): 1120 abc._abc_registry = registry.copy() 1121 abc._abc_cache.clear() 1122 abc._abc_negative_cache.clear() 1123 1124 clear_caches() 1125 1126def clear_caches(): 1127 import gc 1128 1129 # Clear the warnings registry, so they can be displayed again 1130 for mod in sys.modules.values(): 1131 if hasattr(mod, '__warningregistry__'): 1132 del mod.__warningregistry__ 1133 1134 # Clear assorted module caches. 1135 # Don't worry about resetting the cache if the module is not loaded 1136 try: 1137 distutils_dir_util = sys.modules['distutils.dir_util'] 1138 except KeyError: 1139 pass 1140 else: 1141 distutils_dir_util._path_created.clear() 1142 1143 re.purge() 1144 1145 try: 1146 _strptime = sys.modules['_strptime'] 1147 except KeyError: 1148 pass 1149 else: 1150 _strptime._regex_cache.clear() 1151 1152 try: 1153 urlparse = sys.modules['urlparse'] 1154 except KeyError: 1155 pass 1156 else: 1157 urlparse.clear_cache() 1158 1159 try: 1160 urllib = sys.modules['urllib'] 1161 except KeyError: 1162 pass 1163 else: 1164 urllib.urlcleanup() 1165 1166 try: 1167 urllib2 = sys.modules['urllib2'] 1168 except KeyError: 1169 pass 1170 else: 1171 urllib2.install_opener(None) 1172 1173 try: 1174 dircache = sys.modules['dircache'] 1175 except KeyError: 1176 pass 1177 else: 1178 dircache.reset() 1179 1180 try: 1181 linecache = sys.modules['linecache'] 1182 except KeyError: 1183 pass 1184 else: 1185 linecache.clearcache() 1186 1187 try: 1188 mimetypes = sys.modules['mimetypes'] 1189 except KeyError: 1190 pass 1191 else: 1192 mimetypes._default_mime_types() 1193 1194 try: 1195 filecmp = sys.modules['filecmp'] 1196 except KeyError: 1197 pass 1198 else: 1199 filecmp._cache.clear() 1200 1201 try: 1202 struct = sys.modules['struct'] 1203 except KeyError: 1204 pass 1205 else: 1206 struct._clearcache() 1207 1208 try: 1209 doctest = sys.modules['doctest'] 1210 except KeyError: 1211 pass 1212 else: 1213 doctest.master = None 1214 1215 try: 1216 ctypes = sys.modules['ctypes'] 1217 except KeyError: 1218 pass 1219 else: 1220 ctypes._reset_cache() 1221 1222 # Collect cyclic trash. 1223 gc.collect() 1224 1225def findtestdir(path=None): 1226 return path or os.path.dirname(__file__) or os.curdir 1227 1228def removepy(names): 1229 if not names: 1230 return 1231 for idx, name in enumerate(names): 1232 basename, ext = os.path.splitext(name) 1233 if ext == '.py': 1234 names[idx] = basename 1235 1236def count(n, word): 1237 if n == 1: 1238 return "%d %s" % (n, word) 1239 else: 1240 return "%d %ss" % (n, word) 1241 1242def printlist(x, width=70, indent=4): 1243 """Print the elements of iterable x to stdout. 1244 1245 Optional arg width (default 70) is the maximum line length. 1246 Optional arg indent (default 4) is the number of blanks with which to 1247 begin each line. 1248 """ 1249 1250 from textwrap import fill 1251 blanks = ' ' * indent 1252 # Print the sorted list: 'x' may be a '--random' list or a set() 1253 print fill(' '.join(str(elt) for elt in sorted(x)), width, 1254 initial_indent=blanks, subsequent_indent=blanks) 1255 1256# Map sys.platform to a string containing the basenames of tests 1257# expected to be skipped on that platform. 1258# 1259# Special cases: 1260# test_pep277 1261# The _ExpectedSkips constructor adds this to the set of expected 1262# skips if not os.path.supports_unicode_filenames. 1263# test_timeout 1264# Controlled by test_timeout.skip_expected. Requires the network 1265# resource and a socket module. 1266# 1267# Tests that are expected to be skipped everywhere except on one platform 1268# are also handled separately. 1269 1270_expectations = { 1271 'win32': 1272 """ 1273 test__locale 1274 test_bsddb185 1275 test_bsddb3 1276 test_commands 1277 test_crypt 1278 test_curses 1279 test_dbm 1280 test_dl 1281 test_fcntl 1282 test_fork1 1283 test_epoll 1284 test_gdbm 1285 test_grp 1286 test_ioctl 1287 test_largefile 1288 test_kqueue 1289 test_mhlib 1290 test_openpty 1291 test_ossaudiodev 1292 test_pipes 1293 test_poll 1294 test_posix 1295 test_pty 1296 test_pwd 1297 test_resource 1298 test_signal 1299 test_spwd 1300 test_threadsignals 1301 test_timing 1302 test_wait3 1303 test_wait4 1304 """, 1305 'linux2': 1306 """ 1307 test_bsddb185 1308 test_curses 1309 test_dl 1310 test_largefile 1311 test_kqueue 1312 test_ossaudiodev 1313 """, 1314 'unixware7': 1315 """ 1316 test_bsddb 1317 test_bsddb185 1318 test_dl 1319 test_epoll 1320 test_largefile 1321 test_kqueue 1322 test_minidom 1323 test_openpty 1324 test_pyexpat 1325 test_sax 1326 test_sundry 1327 """, 1328 'openunix8': 1329 """ 1330 test_bsddb 1331 test_bsddb185 1332 test_dl 1333 test_epoll 1334 test_largefile 1335 test_kqueue 1336 test_minidom 1337 test_openpty 1338 test_pyexpat 1339 test_sax 1340 test_sundry 1341 """, 1342 'sco_sv3': 1343 """ 1344 test_asynchat 1345 test_bsddb 1346 test_bsddb185 1347 test_dl 1348 test_fork1 1349 test_epoll 1350 test_gettext 1351 test_largefile 1352 test_locale 1353 test_kqueue 1354 test_minidom 1355 test_openpty 1356 test_pyexpat 1357 test_queue 1358 test_sax 1359 test_sundry 1360 test_thread 1361 test_threaded_import 1362 test_threadedtempfile 1363 test_threading 1364 """, 1365 'riscos': 1366 """ 1367 test_asynchat 1368 test_atexit 1369 test_bsddb 1370 test_bsddb185 1371 test_bsddb3 1372 test_commands 1373 test_crypt 1374 test_dbm 1375 test_dl 1376 test_fcntl 1377 test_fork1 1378 test_epoll 1379 test_gdbm 1380 test_grp 1381 test_largefile 1382 test_locale 1383 test_kqueue 1384 test_mmap 1385 test_openpty 1386 test_poll 1387 test_popen2 1388 test_pty 1389 test_pwd 1390 test_strop 1391 test_sundry 1392 test_thread 1393 test_threaded_import 1394 test_threadedtempfile 1395 test_threading 1396 test_timing 1397 """, 1398 'darwin': 1399 """ 1400 test__locale 1401 test_bsddb 1402 test_bsddb3 1403 test_curses 1404 test_epoll 1405 test_gdb 1406 test_gdbm 1407 test_largefile 1408 test_locale 1409 test_kqueue 1410 test_minidom 1411 test_ossaudiodev 1412 test_poll 1413 """, 1414 'sunos5': 1415 """ 1416 test_bsddb 1417 test_bsddb185 1418 test_curses 1419 test_dbm 1420 test_epoll 1421 test_kqueue 1422 test_gdbm 1423 test_gzip 1424 test_openpty 1425 test_zipfile 1426 test_zlib 1427 """, 1428 'hp-ux11': 1429 """ 1430 test_bsddb 1431 test_bsddb185 1432 test_curses 1433 test_dl 1434 test_epoll 1435 test_gdbm 1436 test_gzip 1437 test_largefile 1438 test_locale 1439 test_kqueue 1440 test_minidom 1441 test_openpty 1442 test_pyexpat 1443 test_sax 1444 test_zipfile 1445 test_zlib 1446 """, 1447 'atheos': 1448 """ 1449 test_bsddb185 1450 test_curses 1451 test_dl 1452 test_gdbm 1453 test_epoll 1454 test_largefile 1455 test_locale 1456 test_kqueue 1457 test_mhlib 1458 test_mmap 1459 test_poll 1460 test_popen2 1461 test_resource 1462 """, 1463 'cygwin': 1464 """ 1465 test_bsddb185 1466 test_bsddb3 1467 test_curses 1468 test_dbm 1469 test_epoll 1470 test_ioctl 1471 test_kqueue 1472 test_largefile 1473 test_locale 1474 test_ossaudiodev 1475 test_socketserver 1476 """, 1477 'os2emx': 1478 """ 1479 test_audioop 1480 test_bsddb185 1481 test_bsddb3 1482 test_commands 1483 test_curses 1484 test_dl 1485 test_epoll 1486 test_kqueue 1487 test_largefile 1488 test_mhlib 1489 test_mmap 1490 test_openpty 1491 test_ossaudiodev 1492 test_pty 1493 test_resource 1494 test_signal 1495 """, 1496 'freebsd4': 1497 """ 1498 test_bsddb 1499 test_bsddb3 1500 test_epoll 1501 test_gdbm 1502 test_locale 1503 test_ossaudiodev 1504 test_pep277 1505 test_pty 1506 test_socketserver 1507 test_tcl 1508 test_tk 1509 test_ttk_guionly 1510 test_ttk_textonly 1511 test_timeout 1512 test_urllibnet 1513 test_multiprocessing 1514 """, 1515 'aix5': 1516 """ 1517 test_bsddb 1518 test_bsddb185 1519 test_bsddb3 1520 test_bz2 1521 test_dl 1522 test_epoll 1523 test_gdbm 1524 test_gzip 1525 test_kqueue 1526 test_ossaudiodev 1527 test_tcl 1528 test_tk 1529 test_ttk_guionly 1530 test_ttk_textonly 1531 test_zipimport 1532 test_zlib 1533 """, 1534 'openbsd3': 1535 """ 1536 test_ascii_formatd 1537 test_bsddb 1538 test_bsddb3 1539 test_ctypes 1540 test_dl 1541 test_epoll 1542 test_gdbm 1543 test_locale 1544 test_normalization 1545 test_ossaudiodev 1546 test_pep277 1547 test_tcl 1548 test_tk 1549 test_ttk_guionly 1550 test_ttk_textonly 1551 test_multiprocessing 1552 """, 1553 'netbsd3': 1554 """ 1555 test_ascii_formatd 1556 test_bsddb 1557 test_bsddb185 1558 test_bsddb3 1559 test_ctypes 1560 test_curses 1561 test_dl 1562 test_epoll 1563 test_gdbm 1564 test_locale 1565 test_ossaudiodev 1566 test_pep277 1567 test_tcl 1568 test_tk 1569 test_ttk_guionly 1570 test_ttk_textonly 1571 test_multiprocessing 1572 """, 1573} 1574_expectations['freebsd5'] = _expectations['freebsd4'] 1575_expectations['freebsd6'] = _expectations['freebsd4'] 1576_expectations['freebsd7'] = _expectations['freebsd4'] 1577_expectations['freebsd8'] = _expectations['freebsd4'] 1578 1579class _ExpectedSkips: 1580 def __init__(self): 1581 import os.path 1582 from test import test_timeout 1583 1584 self.valid = False 1585 if sys.platform in _expectations: 1586 s = _expectations[sys.platform] 1587 self.expected = set(s.split()) 1588 1589 # expected to be skipped on every platform, even Linux 1590 self.expected.add('test_linuxaudiodev') 1591 1592 if not os.path.supports_unicode_filenames: 1593 self.expected.add('test_pep277') 1594 1595 if test_timeout.skip_expected: 1596 self.expected.add('test_timeout') 1597 1598 if sys.maxint == 9223372036854775807L: 1599 self.expected.add('test_imageop') 1600 1601 if sys.platform != "darwin": 1602 MAC_ONLY = ["test_macos", "test_macostools", "test_aepack", 1603 "test_plistlib", "test_scriptpackages", 1604 "test_applesingle"] 1605 for skip in MAC_ONLY: 1606 self.expected.add(skip) 1607 elif len(u'\0'.encode('unicode-internal')) == 4: 1608 self.expected.add("test_macostools") 1609 1610 1611 if sys.platform != "win32": 1612 # test_sqlite is only reliable on Windows where the library 1613 # is distributed with Python 1614 WIN_ONLY = ["test_unicode_file", "test_winreg", 1615 "test_winsound", "test_startfile", 1616 "test_sqlite", "test_msilib"] 1617 for skip in WIN_ONLY: 1618 self.expected.add(skip) 1619 1620 if sys.platform != 'irix': 1621 IRIX_ONLY = ["test_imageop", "test_al", "test_cd", "test_cl", 1622 "test_gl", "test_imgfile"] 1623 for skip in IRIX_ONLY: 1624 self.expected.add(skip) 1625 1626 if sys.platform != 'sunos5': 1627 self.expected.add('test_sunaudiodev') 1628 self.expected.add('test_nis') 1629 1630 if not sys.py3kwarning: 1631 self.expected.add('test_py3kwarn') 1632 1633 self.valid = True 1634 1635 def isvalid(self): 1636 "Return true iff _ExpectedSkips knows about the current platform." 1637 return self.valid 1638 1639 def getexpected(self): 1640 """Return set of test names we expect to skip on current platform. 1641 1642 self.isvalid() must be true. 1643 """ 1644 1645 assert self.isvalid() 1646 return self.expected 1647 1648if __name__ == '__main__': 1649 # findtestdir() gets the dirname out of __file__, so we have to make it 1650 # absolute before changing the working directory. 1651 # For example __file__ may be relative when running trace or profile. 1652 # See issue #9323. 1653 __file__ = os.path.abspath(__file__) 1654 1655 # sanity check 1656 assert __file__ == os.path.abspath(sys.argv[0]) 1657 1658 # When tests are run from the Python build directory, it is best practice 1659 # to keep the test files in a subfolder. It eases the cleanup of leftover 1660 # files using command "make distclean". 1661 if sysconfig.is_python_build(): 1662 TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build') 1663 TEMPDIR = os.path.abspath(TEMPDIR) 1664 if not os.path.exists(TEMPDIR): 1665 os.mkdir(TEMPDIR) 1666 1667 # Define a writable temp dir that will be used as cwd while running 1668 # the tests. The name of the dir includes the pid to allow parallel 1669 # testing (see the -j option). 1670 TESTCWD = 'test_python_{}'.format(os.getpid()) 1671 1672 TESTCWD = os.path.join(TEMPDIR, TESTCWD) 1673 1674 # Run the tests in a context manager that temporary changes the CWD to a 1675 # temporary and writable directory. If it's not possible to create or 1676 # change the CWD, the original CWD will be used. The original CWD is 1677 # available from test_support.SAVEDCWD. 1678 with test_support.temp_cwd(TESTCWD, quiet=True): 1679 main() 1680