• Home
  • Line#
  • Scopes#
  • Navigate#
  • Raw
  • Download
1****************************
2  What's New in Python 2.2
3****************************
4
5:Author: A.M. Kuchling
6
7.. |release| replace:: 1.02
8
9.. $Id: whatsnew22.tex 37315 2004-09-10 19:33:00Z akuchling $
10
11
12Introduction
13============
14
15This article explains the new features in Python 2.2.2, released on October 14,
162002.  Python 2.2.2 is a bugfix release of Python 2.2, originally released on
17December 21, 2001.
18
19Python 2.2 can be thought of as the "cleanup release".  There are some features
20such as generators and iterators that are completely new, but most of the
21changes, significant and far-reaching though they may be, are aimed at cleaning
22up irregularities and dark corners of the language design.
23
24This article doesn't attempt to provide a complete specification of the new
25features, but instead provides a convenient overview.  For full details, you
26should refer to the documentation for Python 2.2, such as the `Python Library
27Reference <https://docs.python.org/2.2/lib/lib.html>`_ and the `Python
28Reference Manual <https://docs.python.org/2.2/ref/ref.html>`_.  If you want to
29understand the complete implementation and design rationale for a change, refer
30to the PEP for a particular new feature.
31
32
33.. see also, now defunct
34
35   http://www.unixreview.com/documents/s=1356/urm0109h/0109h.htm
36      "What's So Special About Python 2.2?" is also about the new 2.2 features, and
37      was written by Cameron Laird and Kathryn Soraiz.
38
39.. ======================================================================
40
41
42PEPs 252 and 253: Type and Class Changes
43========================================
44
45The largest and most far-reaching changes in Python 2.2 are to Python's model of
46objects and classes.  The changes should be backward compatible, so it's likely
47that your code will continue to run unchanged, but the changes provide some
48amazing new capabilities. Before beginning this, the longest and most
49complicated section of this article, I'll provide an overview of the changes and
50offer some comments.
51
52A long time ago I wrote a web page listing flaws in Python's design.  One of the
53most significant flaws was that it's impossible to subclass Python types
54implemented in C.  In particular, it's not possible to subclass built-in types,
55so you can't just subclass, say, lists in order to add a single useful method to
56them. The :mod:`UserList` module provides a class that supports all of the
57methods of lists and that can be subclassed further, but there's lots of C code
58that expects a regular Python list and won't accept a :class:`UserList`
59instance.
60
61Python 2.2 fixes this, and in the process adds some exciting new capabilities.
62A brief summary:
63
64* You can subclass built-in types such as lists and even integers, and your
65  subclasses should work in every place that requires the original type.
66
67* It's now possible to define static and class methods, in addition to the
68  instance methods available in previous versions of Python.
69
70* It's also possible to automatically call methods on accessing or setting an
71  instance attribute by using a new mechanism called :dfn:`properties`.  Many uses
72  of :meth:`__getattr__` can be rewritten to use properties instead, making the
73  resulting code simpler and faster.  As a small side benefit, attributes can now
74  have docstrings, too.
75
76* The list of legal attributes for an instance can be limited to a particular
77  set using :dfn:`slots`, making it possible to safeguard against typos and
78  perhaps make more optimizations possible in future versions of Python.
79
80Some users have voiced concern about all these changes.  Sure, they say, the new
81features are neat and lend themselves to all sorts of tricks that weren't
82possible in previous versions of Python, but they also make the language more
83complicated.  Some people have said that they've always recommended Python for
84its simplicity, and feel that its simplicity is being lost.
85
86Personally, I think there's no need to worry.  Many of the new features are
87quite esoteric, and you can write a lot of Python code without ever needed to be
88aware of them.  Writing a simple class is no more difficult than it ever was, so
89you don't need to bother learning or teaching them unless they're actually
90needed.  Some very complicated tasks that were previously only possible from C
91will now be possible in pure Python, and to my mind that's all for the better.
92
93I'm not going to attempt to cover every single corner case and small change that
94were required to make the new features work.  Instead this section will paint
95only the broad strokes.  See section :ref:`sect-rellinks`, "Related Links", for
96further sources of information about Python 2.2's new object model.
97
98
99Old and New Classes
100-------------------
101
102First, you should know that Python 2.2 really has two kinds of classes: classic
103or old-style classes, and new-style classes.  The old-style class model is
104exactly the same as the class model in earlier versions of Python.  All the new
105features described in this section apply only to new-style classes. This
106divergence isn't intended to last forever; eventually old-style classes will be
107dropped, possibly in Python 3.0.
108
109So how do you define a new-style class?  You do it by subclassing an existing
110new-style class.  Most of Python's built-in types, such as integers, lists,
111dictionaries, and even files, are new-style classes now.  A new-style class
112named :class:`object`, the base class for all built-in types, has also been
113added so if no built-in type is suitable, you can just subclass
114:class:`object`::
115
116   class C(object):
117       def __init__ (self):
118           ...
119       ...
120
121This means that :keyword:`class` statements that don't have any base classes are
122always classic classes in Python 2.2.  (Actually you can also change this by
123setting a module-level variable named :attr:`__metaclass__` --- see :pep:`253`
124for the details --- but it's easier to just subclass :class:`object`.)
125
126The type objects for the built-in types are available as built-ins, named using
127a clever trick.  Python has always had built-in functions named :func:`int`,
128:func:`float`, and :func:`str`.  In 2.2, they aren't functions any more, but
129type objects that behave as factories when called. ::
130
131   >>> int
132   <type 'int'>
133   >>> int('123')
134   123
135
136To make the set of types complete, new type objects such as :func:`dict` and
137:func:`file` have been added.  Here's a more interesting example, adding a
138:meth:`lock` method to file objects::
139
140   class LockableFile(file):
141       def lock (self, operation, length=0, start=0, whence=0):
142           import fcntl
143           return fcntl.lockf(self.fileno(), operation,
144                              length, start, whence)
145
146The now-obsolete :mod:`posixfile` module contained a class that emulated all of
147a file object's methods and also added a :meth:`lock` method, but this class
148couldn't be passed to internal functions that expected a built-in file,
149something which is possible with our new :class:`LockableFile`.
150
151
152Descriptors
153-----------
154
155In previous versions of Python, there was no consistent way to discover what
156attributes and methods were supported by an object. There were some informal
157conventions, such as defining :attr:`__members__` and :attr:`__methods__`
158attributes that were lists of names, but often the author of an extension type
159or a class wouldn't bother to define them.  You could fall back on inspecting
160the :attr:`~object.__dict__` of an object, but when class inheritance or an arbitrary
161:meth:`__getattr__` hook were in use this could still be inaccurate.
162
163The one big idea underlying the new class model is that an API for describing
164the attributes of an object using :dfn:`descriptors` has been formalized.
165Descriptors specify the value of an attribute, stating whether it's a method or
166a field.  With the descriptor API, static methods and class methods become
167possible, as well as more exotic constructs.
168
169Attribute descriptors are objects that live inside class objects, and have a few
170attributes of their own:
171
172* :attr:`~definition.__name__` is the attribute's name.
173
174* :attr:`__doc__` is the attribute's docstring.
175
176* ``__get__(object)`` is a method that retrieves the attribute value from
177  *object*.
178
179* ``__set__(object, value)`` sets the attribute on *object* to *value*.
180
181* ``__delete__(object, value)`` deletes the *value*  attribute of *object*.
182
183For example, when you write ``obj.x``, the steps that Python actually performs
184are::
185
186   descriptor = obj.__class__.x
187   descriptor.__get__(obj)
188
189For methods, :meth:`descriptor.__get__` returns a temporary object that's
190callable, and wraps up the instance and the method to be called on it. This is
191also why static methods and class methods are now possible; they have
192descriptors that wrap up just the method, or the method and the class.  As a
193brief explanation of these new kinds of methods, static methods aren't passed
194the instance, and therefore resemble regular functions.  Class methods are
195passed the class of the object, but not the object itself.  Static and class
196methods are defined like this::
197
198   class C(object):
199       def f(arg1, arg2):
200           ...
201       f = staticmethod(f)
202
203       def g(cls, arg1, arg2):
204           ...
205       g = classmethod(g)
206
207The :func:`staticmethod` function takes the function :func:`f`, and returns it
208wrapped up in a descriptor so it can be stored in the class object.  You might
209expect there to be special syntax for creating such methods (``def static f``,
210``defstatic f()``, or something like that) but no such syntax has been defined
211yet; that's been left for future versions of Python.
212
213More new features, such as slots and properties, are also implemented as new
214kinds of descriptors, and it's not difficult to write a descriptor class that
215does something novel.  For example, it would be possible to write a descriptor
216class that made it possible to write Eiffel-style preconditions and
217postconditions for a method.  A class that used this feature might be defined
218like this::
219
220   from eiffel import eiffelmethod
221
222   class C(object):
223       def f(self, arg1, arg2):
224           # The actual function
225           ...
226       def pre_f(self):
227           # Check preconditions
228           ...
229       def post_f(self):
230           # Check postconditions
231           ...
232
233       f = eiffelmethod(f, pre_f, post_f)
234
235Note that a person using the new :func:`eiffelmethod` doesn't have to understand
236anything about descriptors.  This is why I think the new features don't increase
237the basic complexity of the language. There will be a few wizards who need to
238know about it in order to write :func:`eiffelmethod` or the ZODB or whatever,
239but most users will just write code on top of the resulting libraries and ignore
240the implementation details.
241
242
243Multiple Inheritance: The Diamond Rule
244--------------------------------------
245
246Multiple inheritance has also been made more useful through changing the rules
247under which names are resolved.  Consider this set of classes (diagram taken
248from :pep:`253` by Guido van Rossum)::
249
250         class A:
251           ^ ^  def save(self): ...
252          /   \
253         /     \
254        /       \
255       /         \
256   class B     class C:
257       ^         ^  def save(self): ...
258        \       /
259         \     /
260          \   /
261           \ /
262         class D
263
264The lookup rule for classic classes is simple but not very smart; the base
265classes are searched depth-first, going from left to right.  A reference to
266:meth:`D.save` will search the classes :class:`D`, :class:`B`, and then
267:class:`A`, where :meth:`save` would be found and returned.  :meth:`C.save`
268would never be found at all.  This is bad, because if :class:`C`'s :meth:`save`
269method is saving some internal state specific to :class:`C`, not calling it will
270result in that state never getting saved.
271
272New-style classes follow a different algorithm that's a bit more complicated to
273explain, but does the right thing in this situation. (Note that Python 2.3
274changes this algorithm to one that produces the same results in most cases, but
275produces more useful results for really complicated inheritance graphs.)
276
277#. List all the base classes, following the classic lookup rule and include a
278   class multiple times if it's visited repeatedly.  In the above example, the list
279   of visited classes is [:class:`D`, :class:`B`, :class:`A`, :class:`C`,
280   :class:`A`].
281
282#. Scan the list for duplicated classes.  If any are found, remove all but one
283   occurrence, leaving the *last* one in the list.  In the above example, the list
284   becomes [:class:`D`, :class:`B`, :class:`C`, :class:`A`] after dropping
285   duplicates.
286
287Following this rule, referring to :meth:`D.save` will return :meth:`C.save`,
288which is the behaviour we're after.  This lookup rule is the same as the one
289followed by Common Lisp.  A new built-in function, :func:`super`, provides a way
290to get at a class's superclasses without having to reimplement Python's
291algorithm. The most commonly used form will be  ``super(class, obj)``, which
292returns  a bound superclass object (not the actual class object).  This form
293will be used in methods to call a method in the superclass; for example,
294:class:`D`'s :meth:`save` method would look like this::
295
296   class D (B,C):
297       def save (self):
298           # Call superclass .save()
299           super(D, self).save()
300           # Save D's private information here
301           ...
302
303:func:`super` can also return unbound superclass objects when called as
304``super(class)`` or ``super(class1, class2)``, but this probably won't
305often be useful.
306
307
308Attribute Access
309----------------
310
311A fair number of sophisticated Python classes define hooks for attribute access
312using :meth:`__getattr__`; most commonly this is done for convenience, to make
313code more readable by automatically mapping an attribute access such as
314``obj.parent`` into a method call such as ``obj.get_parent``.  Python 2.2 adds
315some new ways of controlling attribute access.
316
317First, ``__getattr__(attr_name)`` is still supported by new-style classes,
318and nothing about it has changed.  As before, it will be called when an attempt
319is made to access ``obj.foo`` and no attribute named ``foo`` is found in the
320instance's dictionary.
321
322New-style classes also support a new method,
323``__getattribute__(attr_name)``.  The difference between the two methods is
324that :meth:`__getattribute__` is *always* called whenever any attribute is
325accessed, while the old :meth:`__getattr__` is only called if ``foo`` isn't
326found in the instance's dictionary.
327
328However, Python 2.2's support for :dfn:`properties` will often be a simpler way
329to trap attribute references.  Writing a :meth:`__getattr__` method is
330complicated because to avoid recursion you can't use regular attribute accesses
331inside them, and instead have to mess around with the contents of
332:attr:`~object.__dict__`. :meth:`__getattr__` methods also end up being called by Python
333when it checks for other methods such as :meth:`__repr__` or :meth:`__coerce__`,
334and so have to be written with this in mind. Finally, calling a function on
335every attribute access results in a sizable performance loss.
336
337:class:`property` is a new built-in type that packages up three functions that
338get, set, or delete an attribute, and a docstring.  For example, if you want to
339define a :attr:`size` attribute that's computed, but also settable, you could
340write::
341
342   class C(object):
343       def get_size (self):
344           result = ... computation ...
345           return result
346       def set_size (self, size):
347           ... compute something based on the size
348           and set internal state appropriately ...
349
350       # Define a property.  The 'delete this attribute'
351       # method is defined as None, so the attribute
352       # can't be deleted.
353       size = property(get_size, set_size,
354                       None,
355                       "Storage size of this instance")
356
357That is certainly clearer and easier to write than a pair of
358:meth:`__getattr__`/:meth:`__setattr__` methods that check for the :attr:`size`
359attribute and handle it specially while retrieving all other attributes from the
360instance's :attr:`~object.__dict__`.  Accesses to :attr:`size` are also the only ones
361which have to perform the work of calling a function, so references to other
362attributes run at their usual speed.
363
364Finally, it's possible to constrain the list of attributes that can be
365referenced on an object using the new :attr:`~object.__slots__` class attribute. Python
366objects are usually very dynamic; at any time it's possible to define a new
367attribute on an instance by just doing ``obj.new_attr=1``.   A new-style class
368can define a class attribute named :attr:`~object.__slots__` to limit the legal
369attributes  to a particular set of names.  An example will make this clear::
370
371   >>> class C(object):
372   ...     __slots__ = ('template', 'name')
373   ...
374   >>> obj = C()
375   >>> print obj.template
376   None
377   >>> obj.template = 'Test'
378   >>> print obj.template
379   Test
380   >>> obj.newattr = None
381   Traceback (most recent call last):
382     File "<stdin>", line 1, in ?
383   AttributeError: 'C' object has no attribute 'newattr'
384
385Note how you get an :exc:`AttributeError` on the attempt to assign to an
386attribute not listed in :attr:`~object.__slots__`.
387
388
389.. _sect-rellinks:
390
391Related Links
392-------------
393
394This section has just been a quick overview of the new features, giving enough
395of an explanation to start you programming, but many details have been
396simplified or ignored.  Where should you go to get a more complete picture?
397
398https://docs.python.org/dev/howto/descriptor.html is a lengthy tutorial introduction to
399the descriptor features, written by Guido van Rossum. If my description has
400whetted your appetite, go read this tutorial next, because it goes into much
401more detail about the new features while still remaining quite easy to read.
402
403Next, there are two relevant PEPs, :pep:`252` and :pep:`253`.  :pep:`252` is
404titled "Making Types Look More Like Classes", and covers the descriptor API.
405:pep:`253` is titled "Subtyping Built-in Types", and describes the changes to
406type objects that make it possible to subtype built-in objects.  :pep:`253` is
407the more complicated PEP of the two, and at a few points the necessary
408explanations of types and meta-types may cause your head to explode.  Both PEPs
409were written and implemented by Guido van Rossum, with substantial assistance
410from the rest of the Zope Corp. team.
411
412Finally, there's the ultimate authority: the source code.  Most of the machinery
413for the type handling is in :file:`Objects/typeobject.c`, but you should only
414resort to it after all other avenues have been exhausted, including posting a
415question to python-list or python-dev.
416
417.. ======================================================================
418
419
420PEP 234: Iterators
421==================
422
423Another significant addition to 2.2 is an iteration interface at both the C and
424Python levels.  Objects can define how they can be looped over by callers.
425
426In Python versions up to 2.1, the usual way to make ``for item in obj`` work is
427to define a :meth:`__getitem__` method that looks something like this::
428
429   def __getitem__(self, index):
430       return <next item>
431
432:meth:`__getitem__` is more properly used to define an indexing operation on an
433object so that you can write ``obj[5]`` to retrieve the sixth element.  It's a
434bit misleading when you're using this only to support :keyword:`for` loops.
435Consider some file-like object that wants to be looped over; the *index*
436parameter is essentially meaningless, as the class probably assumes that a
437series of :meth:`__getitem__` calls will be made with *index* incrementing by
438one each time.  In other words, the presence of the :meth:`__getitem__` method
439doesn't mean that using ``file[5]``  to randomly access the sixth element will
440work, though it really should.
441
442In Python 2.2, iteration can be implemented separately, and :meth:`__getitem__`
443methods can be limited to classes that really do support random access.  The
444basic idea of iterators is  simple.  A new built-in function, ``iter(obj)``
445or ``iter(C, sentinel)``, is used to get an iterator. ``iter(obj)`` returns
446an iterator for the object *obj*, while ``iter(C, sentinel)`` returns an
447iterator that will invoke the callable object *C* until it returns *sentinel* to
448signal that the iterator is done.
449
450Python classes can define an :meth:`__iter__` method, which should create and
451return a new iterator for the object; if the object is its own iterator, this
452method can just return ``self``.  In particular, iterators will usually be their
453own iterators.  Extension types implemented in C can implement a :c:member:`~PyTypeObject.tp_iter`
454function in order to return an iterator, and extension types that want to behave
455as iterators can define a :c:member:`~PyTypeObject.tp_iternext` function.
456
457So, after all this, what do iterators actually do?  They have one required
458method, :meth:`next`, which takes no arguments and returns the next value.  When
459there are no more values to be returned, calling :meth:`next` should raise the
460:exc:`StopIteration` exception. ::
461
462   >>> L = [1,2,3]
463   >>> i = iter(L)
464   >>> print i
465   <iterator object at 0x8116870>
466   >>> i.next()
467   1
468   >>> i.next()
469   2
470   >>> i.next()
471   3
472   >>> i.next()
473   Traceback (most recent call last):
474     File "<stdin>", line 1, in ?
475   StopIteration
476   >>>
477
478In 2.2, Python's :keyword:`for` statement no longer expects a sequence; it
479expects something for which :func:`iter` will return an iterator. For backward
480compatibility and convenience, an iterator is automatically constructed for
481sequences that don't implement :meth:`__iter__` or a :c:member:`~PyTypeObject.tp_iter` slot, so
482``for i in [1,2,3]`` will still work.  Wherever the Python interpreter loops
483over a sequence, it's been changed to use the iterator protocol.  This means you
484can do things like this::
485
486   >>> L = [1,2,3]
487   >>> i = iter(L)
488   >>> a,b,c = i
489   >>> a,b,c
490   (1, 2, 3)
491
492Iterator support has been added to some of Python's basic types.   Calling
493:func:`iter` on a dictionary will return an iterator which loops over its keys::
494
495   >>> m = {'Jan': 1, 'Feb': 2, 'Mar': 3, 'Apr': 4, 'May': 5, 'Jun': 6,
496   ...      'Jul': 7, 'Aug': 8, 'Sep': 9, 'Oct': 10, 'Nov': 11, 'Dec': 12}
497   >>> for key in m: print key, m[key]
498   ...
499   Mar 3
500   Feb 2
501   Aug 8
502   Sep 9
503   May 5
504   Jun 6
505   Jul 7
506   Jan 1
507   Apr 4
508   Nov 11
509   Dec 12
510   Oct 10
511
512That's just the default behaviour.  If you want to iterate over keys, values, or
513key/value pairs, you can explicitly call the :meth:`iterkeys`,
514:meth:`itervalues`, or :meth:`iteritems` methods to get an appropriate iterator.
515In a minor related change, the :keyword:`in` operator now works on dictionaries,
516so ``key in dict`` is now equivalent to ``dict.has_key(key)``.
517
518Files also provide an iterator, which calls the :meth:`readline` method until
519there are no more lines in the file.  This means you can now read each line of a
520file using code like this::
521
522   for line in file:
523       # do something for each line
524       ...
525
526Note that you can only go forward in an iterator; there's no way to get the
527previous element, reset the iterator, or make a copy of it. An iterator object
528could provide such additional capabilities, but the iterator protocol only
529requires a :meth:`next` method.
530
531
532.. seealso::
533
534   :pep:`234` - Iterators
535      Written by Ka-Ping Yee and GvR; implemented  by the Python Labs crew, mostly by
536      GvR and Tim Peters.
537
538.. ======================================================================
539
540
541PEP 255: Simple Generators
542==========================
543
544Generators are another new feature, one that interacts with the introduction of
545iterators.
546
547You're doubtless familiar with how function calls work in Python or C.  When you
548call a function, it gets a private namespace where its local variables are
549created.  When the function reaches a :keyword:`return` statement, the local
550variables are destroyed and the resulting value is returned to the caller.  A
551later call to the same function will get a fresh new set of local variables.
552But, what if the local variables weren't thrown away on exiting a function?
553What if you could later resume the function where it left off?  This is what
554generators provide; they can be thought of as resumable functions.
555
556Here's the simplest example of a generator function::
557
558   def generate_ints(N):
559       for i in range(N):
560           yield i
561
562A new keyword, :keyword:`yield`, was introduced for generators.  Any function
563containing a :keyword:`!yield` statement is a generator function; this is
564detected by Python's bytecode compiler which compiles the function specially as
565a result.  Because a new keyword was introduced, generators must be explicitly
566enabled in a module by including a ``from __future__ import generators``
567statement near the top of the module's source code.  In Python 2.3 this
568statement will become unnecessary.
569
570When you call a generator function, it doesn't return a single value; instead it
571returns a generator object that supports the iterator protocol.  On executing
572the :keyword:`yield` statement, the generator outputs the value of ``i``,
573similar to a :keyword:`return` statement.  The big difference between
574:keyword:`!yield` and a :keyword:`!return` statement is that on reaching a
575:keyword:`!yield` the generator's state of execution is suspended and local
576variables are preserved.  On the next call to the generator's ``next()`` method,
577the function will resume executing immediately after the :keyword:`!yield`
578statement.  (For complicated reasons, the :keyword:`!yield` statement isn't
579allowed inside the :keyword:`!try` block of a
580:keyword:`try`...\ :keyword:`finally` statement; read :pep:`255` for a full
581explanation of the interaction between :keyword:`!yield` and exceptions.)
582
583Here's a sample usage of the :func:`generate_ints` generator::
584
585   >>> gen = generate_ints(3)
586   >>> gen
587   <generator object at 0x8117f90>
588   >>> gen.next()
589   0
590   >>> gen.next()
591   1
592   >>> gen.next()
593   2
594   >>> gen.next()
595   Traceback (most recent call last):
596     File "<stdin>", line 1, in ?
597     File "<stdin>", line 2, in generate_ints
598   StopIteration
599
600You could equally write ``for i in generate_ints(5)``, or ``a,b,c =
601generate_ints(3)``.
602
603Inside a generator function, the :keyword:`return` statement can only be used
604without a value, and signals the end of the procession of values; afterwards the
605generator cannot return any further values. :keyword:`!return` with a value, such
606as ``return 5``, is a syntax error inside a generator function.  The end of the
607generator's results can also be indicated by raising :exc:`StopIteration`
608manually, or by just letting the flow of execution fall off the bottom of the
609function.
610
611You could achieve the effect of generators manually by writing your own class
612and storing all the local variables of the generator as instance variables.  For
613example, returning a list of integers could be done by setting ``self.count`` to
6140, and having the :meth:`next` method increment ``self.count`` and return it.
615However, for a moderately complicated generator, writing a corresponding class
616would be much messier. :file:`Lib/test/test_generators.py` contains a number of
617more interesting examples.  The simplest one implements an in-order traversal of
618a tree using generators recursively. ::
619
620   # A recursive generator that generates Tree leaves in in-order.
621   def inorder(t):
622       if t:
623           for x in inorder(t.left):
624               yield x
625           yield t.label
626           for x in inorder(t.right):
627               yield x
628
629Two other examples in :file:`Lib/test/test_generators.py` produce solutions for
630the N-Queens problem (placing $N$ queens on an $NxN$ chess board so that no
631queen threatens another) and the Knight's Tour (a route that takes a knight to
632every square of an $NxN$ chessboard without visiting any square twice).
633
634The idea of generators comes from other programming languages, especially Icon
635(https://www.cs.arizona.edu/icon/), where the idea of generators is central.  In
636Icon, every expression and function call behaves like a generator.  One example
637from "An Overview of the Icon Programming Language" at
638https://www.cs.arizona.edu/icon/docs/ipd266.htm gives an idea of what this looks
639like::
640
641   sentence := "Store it in the neighboring harbor"
642   if (i := find("or", sentence)) > 5 then write(i)
643
644In Icon the :func:`find` function returns the indexes at which the substring
645"or" is found: 3, 23, 33.  In the :keyword:`if` statement, ``i`` is first
646assigned a value of 3, but 3 is less than 5, so the comparison fails, and Icon
647retries it with the second value of 23.  23 is greater than 5, so the comparison
648now succeeds, and the code prints the value 23 to the screen.
649
650Python doesn't go nearly as far as Icon in adopting generators as a central
651concept.  Generators are considered a new part of the core Python language, but
652learning or using them isn't compulsory; if they don't solve any problems that
653you have, feel free to ignore them. One novel feature of Python's interface as
654compared to Icon's is that a generator's state is represented as a concrete
655object (the iterator) that can be passed around to other functions or stored in
656a data structure.
657
658
659.. seealso::
660
661   :pep:`255` - Simple Generators
662      Written by Neil Schemenauer, Tim Peters, Magnus Lie Hetland.  Implemented mostly
663      by Neil Schemenauer and Tim Peters, with other fixes from the Python Labs crew.
664
665.. ======================================================================
666
667
668PEP 237: Unifying Long Integers and Integers
669============================================
670
671In recent versions, the distinction between regular integers, which are 32-bit
672values on most machines, and long integers, which can be of arbitrary size, was
673becoming an annoyance.  For example, on platforms that support files larger than
674``2**32`` bytes, the :meth:`tell` method of file objects has to return a long
675integer. However, there were various bits of Python that expected plain integers
676and would raise an error if a long integer was provided instead.  For example,
677in Python 1.5, only regular integers could be used as a slice index, and
678``'abc'[1L:]`` would raise a :exc:`TypeError` exception with the message 'slice
679index must be int'.
680
681Python 2.2 will shift values from short to long integers as required. The 'L'
682suffix is no longer needed to indicate a long integer literal, as now the
683compiler will choose the appropriate type.  (Using the 'L' suffix will be
684discouraged in future 2.x versions of Python, triggering a warning in Python
6852.4, and probably dropped in Python 3.0.)  Many operations that used to raise an
686:exc:`OverflowError` will now return a long integer as their result.  For
687example::
688
689   >>> 1234567890123
690   1234567890123L
691   >>> 2 ** 64
692   18446744073709551616L
693
694In most cases, integers and long integers will now be treated identically.  You
695can still distinguish them with the :func:`type` built-in function, but that's
696rarely needed.
697
698
699.. seealso::
700
701   :pep:`237` - Unifying Long Integers and Integers
702      Written by Moshe Zadka and Guido van Rossum.  Implemented mostly by Guido van
703      Rossum.
704
705.. ======================================================================
706
707
708PEP 238: Changing the Division Operator
709=======================================
710
711The most controversial change in Python 2.2 heralds the start of an effort to
712fix an old design flaw that's been in Python from the beginning. Currently
713Python's division operator, ``/``, behaves like C's division operator when
714presented with two integer arguments: it returns an integer result that's
715truncated down when there would be a fractional part.  For example, ``3/2`` is
7161, not 1.5, and ``(-1)/2`` is -1, not -0.5.  This means that the results of
717division can vary unexpectedly depending on the type of the two operands and
718because Python is dynamically typed, it can be difficult to determine the
719possible types of the operands.
720
721(The controversy is over whether this is *really* a design flaw, and whether
722it's worth breaking existing code to fix this.  It's caused endless discussions
723on python-dev, and in July 2001 erupted into a storm of acidly sarcastic
724postings on :newsgroup:`comp.lang.python`. I won't argue for either side here
725and will stick to describing what's  implemented in 2.2.  Read :pep:`238` for a
726summary of arguments and counter-arguments.)
727
728Because this change might break code, it's being introduced very gradually.
729Python 2.2 begins the transition, but the switch won't be complete until Python
7303.0.
731
732First, I'll borrow some terminology from :pep:`238`.  "True division" is the
733division that most non-programmers are familiar with: 3/2 is 1.5, 1/4 is 0.25,
734and so forth.  "Floor division" is what Python's ``/`` operator currently does
735when given integer operands; the result is the floor of the value returned by
736true division.  "Classic division" is the current mixed behaviour of ``/``; it
737returns the result of floor division when the operands are integers, and returns
738the result of true division when one of the operands is a floating-point number.
739
740Here are the changes 2.2 introduces:
741
742* A new operator, ``//``, is the floor division operator. (Yes, we know it looks
743  like C++'s comment symbol.)  ``//`` *always* performs floor division no matter
744  what the types of its operands are, so ``1 // 2`` is 0 and ``1.0 // 2.0`` is
745  also 0.0.
746
747  ``//`` is always available in Python 2.2; you don't need to enable it using a
748  ``__future__`` statement.
749
750* By including a ``from __future__ import division`` in a module, the ``/``
751  operator will be changed to return the result of true division, so ``1/2`` is
752  0.5.  Without the ``__future__`` statement, ``/`` still means classic division.
753  The default meaning of ``/`` will not change until Python 3.0.
754
755* Classes can define methods called :meth:`__truediv__` and :meth:`__floordiv__`
756  to overload the two division operators.  At the C level, there are also slots in
757  the :c:type:`PyNumberMethods` structure so extension types can define the two
758  operators.
759
760* Python 2.2 supports some command-line arguments for testing whether code will
761  work with the changed division semantics.  Running python with :option:`!-Q
762  warn` will cause a warning to be issued whenever division is applied to two
763  integers.  You can use this to find code that's affected by the change and fix
764  it.  By default, Python 2.2 will simply perform classic division without a
765  warning; the warning will be turned on by default in Python 2.3.
766
767
768.. seealso::
769
770   :pep:`238` - Changing the Division Operator
771      Written by Moshe Zadka and  Guido van Rossum.  Implemented by Guido van Rossum..
772
773.. ======================================================================
774
775
776Unicode Changes
777===============
778
779Python's Unicode support has been enhanced a bit in 2.2.  Unicode strings are
780usually stored as UCS-2, as 16-bit unsigned integers. Python 2.2 can also be
781compiled to use UCS-4, 32-bit unsigned integers, as its internal encoding by
782supplying :option:`!--enable-unicode=ucs4` to the configure script.   (It's also
783possible to specify :option:`!--disable-unicode` to completely disable Unicode
784support.)
785
786When built to use UCS-4 (a "wide Python"), the interpreter can natively handle
787Unicode characters from U+000000 to U+110000, so the range of legal values for
788the :func:`unichr` function is expanded accordingly.  Using an interpreter
789compiled to use UCS-2 (a "narrow Python"), values greater than 65535 will still
790cause :func:`unichr` to raise a :exc:`ValueError` exception. This is all
791described in :pep:`261`, "Support for 'wide' Unicode characters"; consult it for
792further details.
793
794Another change is simpler to explain. Since their introduction, Unicode strings
795have supported an :meth:`encode` method to convert the string to a selected
796encoding such as UTF-8 or Latin-1.  A symmetric ``decode([*encoding*])``
797method has been added to 8-bit strings (though not to Unicode strings) in 2.2.
798:meth:`decode` assumes that the string is in the specified encoding and decodes
799it, returning whatever is returned by the codec.
800
801Using this new feature, codecs have been added for tasks not directly related to
802Unicode.  For example, codecs have been added for uu-encoding, MIME's base64
803encoding, and compression with the :mod:`zlib` module::
804
805   >>> s = """Here is a lengthy piece of redundant, overly verbose,
806   ... and repetitive text.
807   ... """
808   >>> data = s.encode('zlib')
809   >>> data
810   'x\x9c\r\xc9\xc1\r\x80 \x10\x04\xc0?Ul...'
811   >>> data.decode('zlib')
812   'Here is a lengthy piece of redundant, overly verbose,\nand repetitive text.\n'
813   >>> print s.encode('uu')
814   begin 666 <data>
815   M2&5R92!I<R!A(&QE;F=T:'D@<&EE8V4@;V8@<F5D=6YD86YT+"!O=F5R;'D@
816   >=F5R8F]S92P*86YD(')E<&5T:71I=F4@=&5X="X*
817
818   end
819   >>> "sheesh".encode('rot-13')
820   'furrfu'
821
822To convert a class instance to Unicode, a :meth:`__unicode__` method can be
823defined by a class, analogous to :meth:`__str__`.
824
825:meth:`encode`, :meth:`decode`, and :meth:`__unicode__` were implemented by
826Marc-André Lemburg.  The changes to support using UCS-4 internally were
827implemented by Fredrik Lundh and Martin von Löwis.
828
829
830.. seealso::
831
832   :pep:`261` - Support for 'wide' Unicode characters
833      Written by Paul Prescod.
834
835.. ======================================================================
836
837
838PEP 227: Nested Scopes
839======================
840
841In Python 2.1, statically nested scopes were added as an optional feature, to be
842enabled by a ``from __future__ import nested_scopes`` directive.  In 2.2 nested
843scopes no longer need to be specially enabled, and are now always present.  The
844rest of this section is a copy of the description of nested scopes from my
845"What's New in Python 2.1" document; if you read it when 2.1 came out, you can
846skip the rest of this section.
847
848The largest change introduced in Python 2.1, and made complete in 2.2, is to
849Python's scoping rules.  In Python 2.0, at any given time there are at most
850three namespaces used to look up variable names: local, module-level, and the
851built-in namespace.  This often surprised people because it didn't match their
852intuitive expectations.  For example, a nested recursive function definition
853doesn't work::
854
855   def f():
856       ...
857       def g(value):
858           ...
859           return g(value-1) + 1
860       ...
861
862The function :func:`g` will always raise a :exc:`NameError` exception, because
863the binding of the name ``g`` isn't in either its local namespace or in the
864module-level namespace.  This isn't much of a problem in practice (how often do
865you recursively define interior functions like this?), but this also made using
866the :keyword:`lambda` expression clumsier, and this was a problem in practice.
867In code which uses :keyword:`!lambda` you can often find local variables being
868copied by passing them as the default values of arguments. ::
869
870   def find(self, name):
871       "Return list of any entries equal to 'name'"
872       L = filter(lambda x, name=name: x == name,
873                  self.list_attribute)
874       return L
875
876The readability of Python code written in a strongly functional style suffers
877greatly as a result.
878
879The most significant change to Python 2.2 is that static scoping has been added
880to the language to fix this problem.  As a first effect, the ``name=name``
881default argument is now unnecessary in the above example.  Put simply, when a
882given variable name is not assigned a value within a function (by an assignment,
883or the :keyword:`def`, :keyword:`class`, or :keyword:`import` statements),
884references to the variable will be looked up in the local namespace of the
885enclosing scope.  A more detailed explanation of the rules, and a dissection of
886the implementation, can be found in the PEP.
887
888This change may cause some compatibility problems for code where the same
889variable name is used both at the module level and as a local variable within a
890function that contains further function definitions. This seems rather unlikely
891though, since such code would have been pretty confusing to read in the first
892place.
893
894One side effect of the change is that the ``from module import *`` and
895``exec`` statements have been made illegal inside a function scope under
896certain conditions.  The Python reference manual has said all along that ``from
897module import *`` is only legal at the top level of a module, but the CPython
898interpreter has never enforced this before.  As part of the implementation of
899nested scopes, the compiler which turns Python source into bytecodes has to
900generate different code to access variables in a containing scope.  ``from
901module import *`` and ``exec`` make it impossible for the compiler to
902figure this out, because they add names to the local namespace that are
903unknowable at compile time. Therefore, if a function contains function
904definitions or :keyword:`lambda` expressions with free variables, the compiler
905will flag this by raising a :exc:`SyntaxError` exception.
906
907To make the preceding explanation a bit clearer, here's an example::
908
909   x = 1
910   def f():
911       # The next line is a syntax error
912       exec 'x=2'
913       def g():
914           return x
915
916Line 4 containing the ``exec`` statement is a syntax error, since
917``exec`` would define a new local variable named ``x`` whose value should
918be accessed by :func:`g`.
919
920This shouldn't be much of a limitation, since ``exec`` is rarely used in
921most Python code (and when it is used, it's often a sign of a poor design
922anyway).
923
924
925.. seealso::
926
927   :pep:`227` - Statically Nested Scopes
928      Written and implemented by Jeremy Hylton.
929
930.. ======================================================================
931
932
933New and Improved Modules
934========================
935
936* The :mod:`xmlrpclib` module was contributed to the standard library by Fredrik
937  Lundh, providing support for writing XML-RPC clients.  XML-RPC is a simple
938  remote procedure call protocol built on top of HTTP and XML. For example, the
939  following snippet retrieves a list of RSS channels from the O'Reilly Network,
940  and then  lists the recent headlines for one channel::
941
942     import xmlrpclib
943     s = xmlrpclib.Server(
944           'http://www.oreillynet.com/meerkat/xml-rpc/server.php')
945     channels = s.meerkat.getChannels()
946     # channels is a list of dictionaries, like this:
947     # [{'id': 4, 'title': 'Freshmeat Daily News'}
948     #  {'id': 190, 'title': '32Bits Online'},
949     #  {'id': 4549, 'title': '3DGamers'}, ... ]
950
951     # Get the items for one channel
952     items = s.meerkat.getItems( {'channel': 4} )
953
954     # 'items' is another list of dictionaries, like this:
955     # [{'link': 'http://freshmeat.net/releases/52719/',
956     #   'description': 'A utility which converts HTML to XSL FO.',
957     #   'title': 'html2fo 0.3 (Default)'}, ... ]
958
959  The :mod:`SimpleXMLRPCServer` module makes it easy to create straightforward
960  XML-RPC servers.  See http://xmlrpc.scripting.com/ for more information about XML-RPC.
961
962* The new :mod:`hmac` module implements the HMAC algorithm described by
963  :rfc:`2104`. (Contributed by Gerhard Häring.)
964
965* Several functions that originally returned lengthy tuples now return
966  pseudo-sequences that still behave like tuples but also have mnemonic attributes such
967  as memberst_mtime or :attr:`tm_year`. The enhanced functions include
968  :func:`stat`, :func:`fstat`, :func:`statvfs`, and :func:`fstatvfs` in the
969  :mod:`os` module, and :func:`localtime`, :func:`gmtime`, and :func:`strptime` in
970  the :mod:`time` module.
971
972  For example, to obtain a file's size using the old tuples, you'd end up writing
973  something like ``file_size = os.stat(filename)[stat.ST_SIZE]``, but now this can
974  be written more clearly as ``file_size = os.stat(filename).st_size``.
975
976  The original patch for this feature was contributed by Nick Mathewson.
977
978* The Python profiler has been extensively reworked and various errors in its
979  output have been corrected.  (Contributed by Fred L. Drake, Jr. and Tim Peters.)
980
981* The :mod:`socket` module can be compiled to support IPv6; specify the
982  :option:`!--enable-ipv6` option to Python's configure script.  (Contributed by
983  Jun-ichiro "itojun" Hagino.)
984
985* Two new format characters were added to the :mod:`struct` module for 64-bit
986  integers on platforms that support the C :c:type:`long long` type.  ``q`` is for
987  a signed 64-bit integer, and ``Q`` is for an unsigned one.  The value is
988  returned in Python's long integer type.  (Contributed by Tim Peters.)
989
990* In the interpreter's interactive mode, there's a new built-in function
991  :func:`help` that uses the :mod:`pydoc` module introduced in Python 2.1 to
992  provide interactive help. ``help(object)`` displays any available help text
993  about *object*.  :func:`help` with no argument puts you in an online help
994  utility, where you can enter the names of functions, classes, or modules to read
995  their help text. (Contributed by Guido van Rossum, using Ka-Ping Yee's
996  :mod:`pydoc` module.)
997
998* Various bugfixes and performance improvements have been made to the SRE engine
999  underlying the :mod:`re` module.  For example, the :func:`re.sub` and
1000  :func:`re.split` functions have been rewritten in C.  Another contributed patch
1001  speeds up certain Unicode character ranges by a factor of two, and a new
1002  :meth:`finditer`  method that returns an iterator over all the non-overlapping
1003  matches in  a given string.  (SRE is maintained by Fredrik Lundh.  The
1004  BIGCHARSET patch was contributed by Martin von Löwis.)
1005
1006* The :mod:`smtplib` module now supports :rfc:`2487`, "Secure SMTP over TLS", so
1007  it's now possible to encrypt the SMTP traffic between a Python program and the
1008  mail transport agent being handed a message.  :mod:`smtplib` also supports SMTP
1009  authentication.  (Contributed by Gerhard Häring.)
1010
1011* The :mod:`imaplib` module, maintained by Piers Lauder, has support for several
1012  new extensions: the NAMESPACE extension defined in :rfc:`2342`, SORT, GETACL and
1013  SETACL.  (Contributed by Anthony Baxter and Michel Pelletier.)
1014
1015* The :mod:`rfc822` module's parsing of email addresses is now compliant with
1016  :rfc:`2822`, an update to :rfc:`822`.  (The module's name is *not* going to be
1017  changed to ``rfc2822``.)  A new package, :mod:`email`, has also been added for
1018  parsing and generating e-mail messages.  (Contributed by Barry Warsaw, and
1019  arising out of his work on Mailman.)
1020
1021* The :mod:`difflib` module now contains a new :class:`Differ` class for
1022  producing human-readable lists of changes (a "delta") between two sequences of
1023  lines of text.  There are also two generator functions, :func:`ndiff` and
1024  :func:`restore`, which respectively return a delta from two sequences, or one of
1025  the original sequences from a delta. (Grunt work contributed by David Goodger,
1026  from ndiff.py code by Tim Peters who then did the generatorization.)
1027
1028* New constants :const:`ascii_letters`, :const:`ascii_lowercase`, and
1029  :const:`ascii_uppercase` were added to the :mod:`string` module.  There were
1030  several modules in the standard library that used :const:`string.letters` to
1031  mean the ranges A-Za-z, but that assumption is incorrect when locales are in
1032  use, because :const:`string.letters` varies depending on the set of legal
1033  characters defined by the current locale.  The buggy modules have all been fixed
1034  to use :const:`ascii_letters` instead. (Reported by an unknown person; fixed by
1035  Fred L. Drake, Jr.)
1036
1037* The :mod:`mimetypes` module now makes it easier to use alternative MIME-type
1038  databases by the addition of a :class:`MimeTypes` class, which takes a list of
1039  filenames to be parsed.  (Contributed by Fred L. Drake, Jr.)
1040
1041* A :class:`Timer` class was added to the :mod:`threading` module that allows
1042  scheduling an activity to happen at some future time.  (Contributed by Itamar
1043  Shtull-Trauring.)
1044
1045.. ======================================================================
1046
1047
1048Interpreter Changes and Fixes
1049=============================
1050
1051Some of the changes only affect people who deal with the Python interpreter at
1052the C level because they're writing Python extension modules, embedding the
1053interpreter, or just hacking on the interpreter itself. If you only write Python
1054code, none of the changes described here will affect you very much.
1055
1056* Profiling and tracing functions can now be implemented in C, which can operate
1057  at much higher speeds than Python-based functions and should reduce the overhead
1058  of profiling and tracing.  This  will be of interest to authors of development
1059  environments for Python.  Two new C functions were added to Python's API,
1060  :c:func:`PyEval_SetProfile` and :c:func:`PyEval_SetTrace`. The existing
1061  :func:`sys.setprofile` and :func:`sys.settrace` functions still exist, and have
1062  simply been changed to use the new C-level interface.  (Contributed by Fred L.
1063  Drake, Jr.)
1064
1065* Another low-level API, primarily of interest to implementors of Python
1066  debuggers and development tools, was added. :c:func:`PyInterpreterState_Head` and
1067  :c:func:`PyInterpreterState_Next` let a caller walk through all the existing
1068  interpreter objects; :c:func:`PyInterpreterState_ThreadHead` and
1069  :c:func:`PyThreadState_Next` allow looping over all the thread states for a given
1070  interpreter.  (Contributed by David Beazley.)
1071
1072* The C-level interface to the garbage collector has been changed to make it
1073  easier to write extension types that support garbage collection and to debug
1074  misuses of the functions. Various functions have slightly different semantics,
1075  so a bunch of functions had to be renamed.  Extensions that use the old API will
1076  still compile but will *not* participate in garbage collection, so updating them
1077  for 2.2 should be considered fairly high priority.
1078
1079  To upgrade an extension module to the new API, perform the following steps:
1080
1081* Rename :c:func:`Py_TPFLAGS_GC` to :c:func:`PyTPFLAGS_HAVE_GC`.
1082
1083* Use :c:func:`PyObject_GC_New` or :c:func:`PyObject_GC_NewVar` to allocate
1084    objects, and :c:func:`PyObject_GC_Del` to deallocate them.
1085
1086* Rename :c:func:`PyObject_GC_Init` to :c:func:`PyObject_GC_Track` and
1087    :c:func:`PyObject_GC_Fini` to :c:func:`PyObject_GC_UnTrack`.
1088
1089* Remove :c:func:`PyGC_HEAD_SIZE` from object size calculations.
1090
1091* Remove calls to :c:func:`PyObject_AS_GC` and :c:func:`PyObject_FROM_GC`.
1092
1093* A new ``et`` format sequence was added to :c:func:`PyArg_ParseTuple`; ``et``
1094  takes both a parameter and an encoding name, and converts the parameter to the
1095  given encoding if the parameter turns out to be a Unicode string, or leaves it
1096  alone if it's an 8-bit string, assuming it to already be in the desired
1097  encoding.  This differs from the ``es`` format character, which assumes that
1098  8-bit strings are in Python's default ASCII encoding and converts them to the
1099  specified new encoding. (Contributed by M.-A. Lemburg, and used for the MBCS
1100  support on Windows described in the following section.)
1101
1102* A different argument parsing function, :c:func:`PyArg_UnpackTuple`, has been
1103  added that's simpler and presumably faster.  Instead of specifying a format
1104  string, the caller simply gives the minimum and maximum number of arguments
1105  expected, and a set of pointers to :c:type:`PyObject\*` variables that will be
1106  filled in with argument values.
1107
1108* Two new flags :const:`METH_NOARGS` and :const:`METH_O` are available in method
1109  definition tables to simplify implementation of methods with no arguments or a
1110  single untyped argument. Calling such methods is more efficient than calling a
1111  corresponding method that uses :const:`METH_VARARGS`.  Also, the old
1112  :const:`METH_OLDARGS` style of writing C methods is  now officially deprecated.
1113
1114* Two new wrapper functions, :c:func:`PyOS_snprintf` and :c:func:`PyOS_vsnprintf`
1115  were added to provide  cross-platform implementations for the relatively new
1116  :c:func:`snprintf` and :c:func:`vsnprintf` C lib APIs. In contrast to the standard
1117  :c:func:`sprintf` and :c:func:`vsprintf` functions, the Python versions check the
1118  bounds of the buffer used to protect against buffer overruns. (Contributed by
1119  M.-A. Lemburg.)
1120
1121* The :c:func:`_PyTuple_Resize` function has lost an unused parameter, so now it
1122  takes 2 parameters instead of 3.  The third argument was never used, and can
1123  simply be discarded when porting code from earlier versions to Python 2.2.
1124
1125.. ======================================================================
1126
1127
1128Other Changes and Fixes
1129=======================
1130
1131As usual there were a bunch of other improvements and bugfixes scattered
1132throughout the source tree.  A search through the CVS change logs finds there
1133were 527 patches applied and 683 bugs fixed between Python 2.1 and 2.2; 2.2.1
1134applied 139 patches and fixed 143 bugs; 2.2.2 applied 106 patches and fixed 82
1135bugs.  These figures are likely to be underestimates.
1136
1137Some of the more notable changes are:
1138
1139* The code for the MacOS port for Python, maintained by Jack Jansen, is now kept
1140  in the main Python CVS tree, and many changes have been made to support MacOS X.
1141
1142  The most significant change is the ability to build Python as a framework,
1143  enabled by supplying the :option:`!--enable-framework` option to the configure
1144  script when compiling Python.  According to Jack Jansen, "This installs a
1145  self-contained Python installation plus the OS X framework "glue" into
1146  :file:`/Library/Frameworks/Python.framework` (or another location of choice).
1147  For now there is little immediate added benefit to this (actually, there is the
1148  disadvantage that you have to change your PATH to be able to find Python), but
1149  it is the basis for creating a full-blown Python application, porting the
1150  MacPython IDE, possibly using Python as a standard OSA scripting language and
1151  much more."
1152
1153  Most of the MacPython toolbox modules, which interface to MacOS APIs such as
1154  windowing, QuickTime, scripting, etc. have been ported to OS X, but they've been
1155  left commented out in :file:`setup.py`.  People who want to experiment with
1156  these modules can uncomment them manually.
1157
1158  .. Jack's original comments:
1159     The main change is the possibility to build Python as a
1160     framework. This installs a self-contained Python installation plus the
1161     OSX framework "glue" into /Library/Frameworks/Python.framework (or
1162     another location of choice). For now there is little immediate added
1163     benefit to this (actually, there is the disadvantage that you have to
1164     change your PATH to be able to find Python), but it is the basis for
1165     creating a fullblown Python application, porting the MacPython IDE,
1166     possibly using Python as a standard OSA scripting language and much
1167     more. You enable this with "configure --enable-framework".
1168     The other change is that most MacPython toolbox modules, which
1169     interface to all the MacOS APIs such as windowing, quicktime,
1170     scripting, etc. have been ported. Again, most of these are not of
1171     immediate use, as they need a full application to be really useful, so
1172     they have been commented out in setup.py. People wanting to experiment
1173     can uncomment them. Gestalt and Internet Config modules are enabled by
1174     default.
1175
1176* Keyword arguments passed to built-in functions that don't take them now cause a
1177  :exc:`TypeError` exception to be raised, with the message "*function* takes no
1178  keyword arguments".
1179
1180* Weak references, added in Python 2.1 as an extension module, are now part of
1181  the core because they're used in the implementation of new-style classes.  The
1182  :exc:`ReferenceError` exception has therefore moved from the :mod:`weakref`
1183  module to become a built-in exception.
1184
1185* A new script, :file:`Tools/scripts/cleanfuture.py` by Tim Peters,
1186  automatically removes obsolete ``__future__`` statements from Python source
1187  code.
1188
1189* An additional *flags* argument has been added to the built-in function
1190  :func:`compile`, so the behaviour of ``__future__`` statements can now be
1191  correctly observed in simulated shells, such as those presented by IDLE and
1192  other development environments.  This is described in :pep:`264`. (Contributed
1193  by Michael Hudson.)
1194
1195* The new license introduced with Python 1.6 wasn't GPL-compatible.  This is
1196  fixed by some minor textual changes to the 2.2 license, so it's now legal to
1197  embed Python inside a GPLed program again.  Note that Python itself is not
1198  GPLed, but instead is under a license that's essentially equivalent to the BSD
1199  license, same as it always was.  The license changes were also applied to the
1200  Python 2.0.1 and 2.1.1 releases.
1201
1202* When presented with a Unicode filename on Windows, Python will now convert it
1203  to an MBCS encoded string, as used by the Microsoft file APIs.  As MBCS is
1204  explicitly used by the file APIs, Python's choice of ASCII as the default
1205  encoding turns out to be an annoyance.  On Unix, the locale's character set is
1206  used if ``locale.nl_langinfo(CODESET)`` is available.  (Windows support was
1207  contributed by Mark Hammond with assistance from Marc-André Lemburg. Unix
1208  support was added by Martin von Löwis.)
1209
1210* Large file support is now enabled on Windows.  (Contributed by Tim Peters.)
1211
1212* The :file:`Tools/scripts/ftpmirror.py` script now parses a :file:`.netrc`
1213  file, if you have one. (Contributed by Mike Romberg.)
1214
1215* Some features of the object returned by the :func:`xrange` function are now
1216  deprecated, and trigger warnings when they're accessed; they'll disappear in
1217  Python 2.3. :class:`xrange` objects tried to pretend they were full sequence
1218  types by supporting slicing, sequence multiplication, and the :keyword:`in`
1219  operator, but these features were rarely used and therefore buggy.  The
1220  :meth:`tolist` method and the :attr:`start`, :attr:`stop`, and :attr:`step`
1221  attributes are also being deprecated.  At the C level, the fourth argument to
1222  the :c:func:`PyRange_New` function, ``repeat``, has also been deprecated.
1223
1224* There were a bunch of patches to the dictionary implementation, mostly to fix
1225  potential core dumps if a dictionary contains objects that sneakily changed
1226  their hash value, or mutated the dictionary they were contained in. For a while
1227  python-dev fell into a gentle rhythm of Michael Hudson finding a case that
1228  dumped core, Tim Peters fixing the bug, Michael finding another case, and round
1229  and round it went.
1230
1231* On Windows, Python can now be compiled with Borland C thanks to a number of
1232  patches contributed by Stephen Hansen, though the result isn't fully functional
1233  yet.  (But this *is* progress...)
1234
1235* Another Windows enhancement: Wise Solutions generously offered PythonLabs use
1236  of their InstallerMaster 8.1 system.  Earlier PythonLabs Windows installers used
1237  Wise 5.0a, which was beginning to show its age.  (Packaged up by Tim Peters.)
1238
1239* Files ending in ``.pyw`` can now be imported on Windows. ``.pyw`` is a
1240  Windows-only thing, used to indicate that a script needs to be run using
1241  PYTHONW.EXE instead of PYTHON.EXE in order to prevent a DOS console from popping
1242  up to display the output.  This patch makes it possible to import such scripts,
1243  in case they're also usable as modules.  (Implemented by David Bolen.)
1244
1245* On platforms where Python uses the C :c:func:`dlopen` function  to load
1246  extension modules, it's now possible to set the flags used  by :c:func:`dlopen`
1247  using the :func:`sys.getdlopenflags` and :func:`sys.setdlopenflags` functions.
1248  (Contributed by Bram Stolk.)
1249
1250* The :func:`pow` built-in function no longer supports 3 arguments when
1251  floating-point numbers are supplied. ``pow(x, y, z)`` returns ``(x**y) % z``,
1252  but this is never useful for floating point numbers, and the final result varies
1253  unpredictably depending on the platform.  A call such as ``pow(2.0, 8.0, 7.0)``
1254  will now raise a :exc:`TypeError` exception.
1255
1256.. ======================================================================
1257
1258
1259Acknowledgements
1260================
1261
1262The author would like to thank the following people for offering suggestions,
1263corrections and assistance with various drafts of this article: Fred Bremmer,
1264Keith Briggs, Andrew Dalke, Fred L. Drake, Jr., Carel Fellinger, David Goodger,
1265Mark Hammond, Stephen Hansen, Michael Hudson, Jack Jansen, Marc-André Lemburg,
1266Martin von Löwis, Fredrik Lundh, Michael McLay, Nick Mathewson, Paul Moore,
1267Gustavo Niemeyer, Don O'Donnell, Joonas Paalasma, Tim Peters, Jens Quade, Tom
1268Reinhardt, Neil Schemenauer, Guido van Rossum, Greg Ward, Edward Welbourne.
1269
1270