Lines Matching +full:is +full:- +full:generator +full:- +full:function
11 :term:`iterator`\s and :term:`generator`\s and relevant library modules such as
30 perform the computation efficiently. SQL is the declarative language you're
35 * **Object-oriented** programs manipulate collections of objects. Objects have
37 some way. Smalltalk and Java are object-oriented languages. C++ and Python
38 are languages that support object-oriented programming, but don't force the
39 use of object-oriented features.
43 internal state that affects the output produced for a given input. Well-known
49 use a different approach. Other languages are multi-paradigm languages that
51 multi-paradigm; you can write programs or libraries that are largely
52 procedural, object-oriented, or functional in all of these languages. In a
54 the GUI might be object-oriented while the processing logic is procedural or
57 In a functional program, input flows through a set of functions. Each function
60 that aren't visible in the function's return value. Functions that have no side
62 not using data structures that get updated as a program runs; every function's
75 functional-appearing interface but will use non-functional features internally.
76 For example, the implementation of a function will still use assignments to
79 Functional programming can be considered the opposite of object-oriented
85 that take and return instances representing objects in your application (e-mail
99 ------------------
101 A theoretical benefit is that it's easier to construct a mathematical proof that
102 a functional program is correct.
105 mathematically prove programs correct. This is different from testing a program
106 on numerous inputs and concluding that its output is usually correct, or reading
107 a program's source code and concluding that the code looks right; the goal is
111 The technique used to prove programs correct is to write down **invariants**,
114 **before** the line is executed, the slightly different invariants X' and Y' are
115 true **after** the line is executed. This continues until you reach the end of
124 Unfortunately, proving programs correct is largely impractical and not relevant
135 ----------
137 A more practical benefit of functional programming is that it forces you to
139 result. It's easier to specify and write a small function that does one thing
140 than a large function that performs a complicated transformation. Small
145 -----------------------------
147 Testing and debugging a functional-style program is easier.
149 Debugging is simplified because functions are generally small and clearly
150 specified. When a program doesn't work, each function is an interface point
152 inputs and outputs to quickly isolate the function that's responsible for a bug.
154 Testing is easier because each function is a potential subject for a unit test.
161 -------------
163 As you work on a functional-style program, you'll write a number of functions
166 variety of programs. For example, a function that takes a directory path and
167 returns all the XML files in the directory, or a function that takes a filename
179 foundation for writing functional-style programs: iterators.
181 An iterator is an object representing a stream of data; this object returns the
189 The built-in :func:`iter` function takes an arbitrary object and tries to return
192 built-in data types support iteration, the most common being lists and
193 dictionaries. An object is called an **iterable** object if you can get an
235 N elements, you can unpack them into an N-tuple:
243 Built-in functions such as :func:`max` and :func:`min` can take a single
245 and ``"not in"`` operators also support iterators: ``X in iterator`` is true if
246 X is found in the stream returned by the iterator. You'll run into obvious
247 problems if the iterator is infinite; ``max()``, ``min()``
261 ---------------------------------
291 Note that the order is essentially random, because it's based on the hash
323 Generator expressions and list comprehensions
332 List comprehensions and generator expressions (short form: "listcomps" and
339 # Generator expression -- returns iterator
342 # List comprehension -- returns list
350 With a list comprehension, you get back a Python list; ``stripped_list`` is a
351 list containing the resulting lines, not an iterator. Generator expressions
355 large amount of data. Generator expressions are preferable in these situations.
357 Generator expressions are surrounded by parentheses ("()") and list
358 comprehensions are surrounded by square brackets ("[]"). Generator expressions
375 is only evaluated and added to the result when ``condition`` is true.
377 Generator expressions always have to be written inside parentheses, but the
378 parentheses signalling a function call also count. If you want to create an
379 iterator that will be immediately passed to a function you can write::
386 ``sequence2`` is looped over from the beginning. ``sequence3`` is then looped
389 To put it another way, a list comprehension or generator expression is
409 list is 9 elements long:
421 To avoid introducing an ambiguity into Python's grammar, if ``expression`` is
423 comprehension below is a syntax error, while the second one is correct::
438 You're doubtless familiar with how regular function calls work in Python or C.
439 When you call a function, it gets a private namespace where its local variables
440 are created. When the function reaches a ``return`` statement, the local
441 variables are destroyed and the value is returned to the caller. A later call
442 to the same function creates a new private namespace and a fresh set of local
444 function? What if you could later resume the function where it left off? This
445 is what generators provide; they can be thought of as resumable functions.
447 Here's the simplest example of a generator function:
455 Any function containing a ``yield`` keyword is a generator function; this is
456 detected by Python's :term:`bytecode` compiler which compiles the function
459 When you call a generator function, it doesn't return a single value; instead it
460 returns a generator object that supports the iterator protocol. On executing
461 the ``yield`` expression, the generator outputs the value of ``i``, similar to a
463 statement is that on reaching a ``yield`` the generator's state of execution is
465 generator's ``.next()`` method, the function will resume executing.
467 Here's a sample usage of the ``generate_ints()`` generator:
471 <generator object generate_ints at ...>
487 Inside a generator function, the ``return`` statement can only be used without a
489 ``return`` the generator cannot return any further values. ``return`` with a
490 value, such as ``return 5``, is a syntax error inside a generator function. The
491 end of the generator's results can also be indicated by raising
493 the bottom of the function.
496 and storing all the local variables of the generator as instance variables. For
499 However, for a moderately complicated generator, writing a corresponding class
503 a number of more interesting examples. Here's one generator that implements an
504 in-order traversal of a tree using generators recursively. ::
506 # A recursive generator that generates Tree leaves in in-order.
517 Two other examples in ``test_generators.py`` produce solutions for the N-Queens
524 Passing values into a generator
525 -------------------------------
527 In Python 2.4 and earlier, generators only produced output. Once a generator's
529 information into the function when its execution is resumed. You could hack
530 together this ability by making the generator look at a global variable or by
534 In Python 2.5 there's a simple way to pass values into a generator.
545 (PEP 342 explains the exact rules, which are that a ``yield``-expression must
546 always be parenthesized except when it occurs at the top-level expression on the
547 right-hand side of an assignment. This means you can write ``val = yield i``
551 Values are sent into a generator by calling its ``send(value)`` method. This
552 method resumes the generator's code and the ``yield`` expression returns the
553 specified value. If the regular ``next()`` method is called, the ``yield``
566 if val is not None:
590 ``send()`` method will be the only method used to resume your generator
591 function.
595 * ``throw(type, value=None, traceback=None)`` is used to raise an exception
596 inside the generator; the exception is raised by the ``yield`` expression
597 where the generator's execution is paused.
599 * ``close()`` raises a :exc:`GeneratorExit` exception inside the generator to
600 terminate the iteration. On receiving this exception, the generator's code
602 exception and doing anything else is illegal and will trigger a
604 collector when the generator is garbage-collected.
609 The cumulative effect of these changes is to turn generators from one-way
614 function, and a ``return`` statement), but coroutines can be entered, exited,
618 Built-in functions
621 Let's look in more detail at built-in functions often used with iterators.
623 Two of Python's built-in functions, :func:`map` and :func:`filter`, are somewhat
640 :func:`itertools.imap` function does the same thing but can handle infinite
644 elements that meet a certain condition, and is similarly duplicated by list
645 comprehensions. A **predicate** is a function that returns the truth value of
667 ``func`` must be a function that takes two elements and returns a single value.
671 and continues until the iterable is exhausted. If the iterable returns no
672 values at all, a :exc:`TypeError` exception is raised. If the initial value is
673 supplied, it's used as a starting point and ``func(initial_value, A)`` is the
689 elements of the iterable. This case is so common that there's a special
690 built-in called :func:`sum` to compute it:
711 ``enumerate(iter)`` counts off the elements in the iterable, returning 2-tuples
720 :func:`enumerate` is often used when looping through a list and recording the
743 (For a more detailed discussion of sorting, see the Sorting mini-HOWTO in the
746 The ``any(iter)`` and ``all(iter)`` built-ins look at the truth values of an
747 iterable's contents. :func:`any` returns ``True`` if any element in the iterable is
768 When writing functional-style programs, you'll often need little functions that
771 If there's a Python built-in or a module function that's suitable, you don't
772 need to define a new function at all::
777 If the function you need doesn't exist, you need to write it. One way to write
778 small functions is to use the ``lambda`` statement. ``lambda`` takes a number
780 function that returns the value of the expression::
788 An alternative is to just use the ``def`` statement and define a function in the
800 Which alternative is preferable? That's a style question; my usual course is to
803 One reason for my preference is that ``lambda`` is quite limited in the
829 Or the :func:`sum` built-in and a generator expression::
838 1) Write a lambda function.
846 about whether this lambda-free style is better.
852 The :mod:`itertools` module contains a number of commonly-used iterators as well
859 * Functions for treating an iterator's elements as function arguments.
861 * A function for grouping an iterator's output.
864 ----------------------
882 returns the element endlessly if ``n`` is not provided. ::
902 It's similar to the built-in :func:`zip` function, but doesn't construct an
903 in-memory list and exhaust all the input iterators before returning; instead
905 term for this behaviour is `lazy evaluation
908 This iterator is intended to be used with iterables that are all of the same
922 ``stop-start`` elements, and if you supply a value for ``step``, elements will
935 If you don't supply a value for ``n``, the default is 2. Replicating iterators
937 significant memory if the iterator is large and one of the new iterators is
943 where iterA ->
946 and iterB ->
951 -----------------------------
978 ------------------
992 ``itertools.ifilterfalse(predicate, iter)`` is the opposite, returning all
1026 -----------------
1028 The last function I'll discuss, ``itertools.groupby(iter, key_func=None)``, is
1029 the most complicated. ``key_func(elem)`` is a function that can compute a key
1031 function, the key is simply each element itself.
1034 that have the same key value, and returns a stream of 2-tuples containing a key
1049 ('AL', iterator-1),
1050 ('AK', iterator-2),
1051 ('AZ', iterator-3), ...
1054 iterator-1 =>
1056 iterator-2 =>
1058 iterator-3 =>
1063 underlying iterable, so you have to consume the results of iterator-1 before
1064 requesting iterator-2 and its corresponding key.
1070 The :mod:`functools` module in Python 2.5 contains some higher-order functions.
1071 A **higher-order function** takes one or more functions as input and returns a
1072 new function. The most useful tool in this module is the
1073 :func:`functools.partial` function.
1077 Consider a Python function ``f(a, b, c)``; you may wish to create a new function
1079 one of ``f()``'s parameters. This is called "partial function application".
1081 The constructor for ``partial`` takes the arguments ``(function, arg1, arg2,
1082 ... kwarg1=value1, kwarg2=value2)``. The resulting object is callable, so you
1083 can just call it to invoke ``function`` with the filled-in arguments.
1099 -------------------
1103 in functional-style code because they save you from writing trivial functions
1143 -------
1150 design approaches described in these chapters are applicable to functional-style
1163 Python-specific
1164 ---------------
1167 :title-reference:`Text Processing in Python` discusses functional programming
1168 for text processing, in the section titled "Utilizing Higher-Order Functions in
1171 Mertz also wrote a 3-part series of articles on functional programming
1174 `part 1 <https://www.ibm.com/developerworks/linux/library/l-prog/index.html>`__,
1175 `part 2 <https://www.ibm.com/developerworks/linux/library/l-prog2/index.html>`__, and
1176 `part 3 <https://www.ibm.com/developerworks/linux/library/l-prog3/index.html>`__,
1180 --------------------
1186 :pep:`289`: "Generator Expressions"
1188 :pep:`342`: "Coroutines via Enhanced Generators" describes the new generator
1194 -----------------------------
1209 Functions are strictly input-output, no internal state
1214 Assignment is difficult to reason about
1232 Built-in functions
1239 Handy little function for printing part of an iterator -- used
1245 for elem in slice[:-1]:
1248 print elem[-1]