Advanced

1. Generators

Generators are very easy to implement, but a bit difficult to understand.

Generators are used to create iterators, but with a different approach. Generators are simple functions which return an iterable set of items, one at a time, in a special way.

When an iteration over a set of item starts using the for statement, the generator is run. Once the generator's function code reaches a "yield" statement, the generator yields its execution back to the for loop, returning a new value from the set. The generator function can generate as many values (possibly infinite) as it wants, yielding each one in its turn.

Here is a simple example of a generator function which returns 7 random integers:

import random

def lottery():

# returns 6 numbers between 1 and 40

for i in xrange(6):

yield random.randint(1, 40)

# returns a 7th number between 1 and 15

yield random.randint(1,15)

for random_number in lottery():

print "And the next number is... %d!" % random_number

This function decides how to generate the random numbers on its own, and executes the yield statements one at a time, pausing in between to yield execution back to the main for loop.

# fill in this function

def fib():

a, b = 1, 1

while 1:

yield a

a, b = b, a + b

# testing code

import types

if type(fib()) == types.GeneratorType:

print "Good, The fib function is a generator."

counter = 0

for n in fib():

print n

counter += 1

if counter == 10:

break

Good, The fib function is a generator.

2. List Comprehensions

List Comprehensions is a very powerful tool, which creates a new list based on another list, in a single, readable line.

For example, let's say we need to create a list of integers which specify the length of each word in a certain sentence, but only if the word is not the word "the".

sentence = "the quick brown fox jumps over the lazy dog"

words = sentence.split()

word_lengths = []

for word in words:

if word != "the":

word_lengths.append(len(word))

Using a list comprehension, we could simplify this process to this notation:

sentence = "the quick brown fox jumps over the lazy dog"

words = sentence.split()

word_lengths = [len(word) for word in words if word != "the"]

3. Multiple Function Arguments

Every function in Python receives a predefined number of arguments, if declared normally, like this:

def myfunction(first, second, third):

# do something with the 3 variables

...

It is possible to declare functions which receive a variable number of arguments, using the following syntax:

def foo(first, second, third, *therest):

print "First: %s" % first

print "Second: %s" % second

print "Third: %s" % third

print "And all the rest... %s" % list(therest)

The "therest" variable is a list of variables, which receives all arguments which were given to the "foo" function after the first 3 arguments. So calling foo(1,2,3,4,5) will print out:

First: 1

Second: 2

Third: 3

And all the rest... [4, 5]

It is also possible to send functions arguments by keyword, so that the order of the argument does not matter, using the following syntax:

def bar(first, second, third, **options):

if options.get("action") == "sum":

print "The sum is: %d" % (first + second + third)

if options.get("number") == "first":

return first

result = bar(1, 2, 3, action = "sum", number = "first")

print "Result: %d" % result

The following code yields the following output:

The sum is: 6

Result: 1

The "bar" function receives 3 arguments. If an additional "action" argument is received, and it instructs on summing up the numbers, then the sum is printed out. Alternatively, the function also knows it must return the first argument, if a "return" argument is received which instructs it.

4. Regular Expressions

Regular Expressions (sometimes shortened to regexp, regex, or re) are a tool for matching patterns in text. In Python, we have the re module. The applications for regular expressions are wide-spread, but they are fairly complex, so when contemplating using a regex for a certain task, think about alternatives, and come to regexes as a last resort.

An example regex is r"^(From|To|Cc).*?python-list@python.org" Now for an explanation: the caret ^ matches text at the beginning of a line. The following group, the part with (From|To|Cc) means that the line has to start with one of the words that are separated by the pipe |. That is called the OR operator, and the regex will match if the line starts with any of the words in the group. The .*? means to un-greedily match any number of characters, except the newline \n character. The un-greedy part means to match as few repetitions as possible. The . character means any non-newline character, the * means to repeat 0 or more times, and the ? character makes it un-greedy.

So, the following lines would be matched by that regex: From: python-list@python.org To: !asp]<,. python-list@python.org

A complete reference for the re syntax is available at the python docs.

5. Exception Handling

When programming, errors happen. It's just a fact of life. Perhaps the user gave bad input. Maybe a network resource was unavailable. Maybe the program ran out of memory. Or the programmer may have even made a mistake!

Python's solution to errors are exceptions. You might have seen an exception before.

>>> print a

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

NameError: name 'a' is not defined

Oops! Forgot to assign a value to the 'a' variable.

But sometimes you don't want exceptions to completely stop the program. You might want to do something special when an exception is raised. This is done in a try/except block.

Here's a trivial example: Suppose you're iterating over a list. You need to iterate over 20 numbers, but the list is made from user input, and might not have 20 numbers in it. After you reach the end of the list, you just want the rest of the numbers to be interpreted as a 0. Here's how you could do that:

def do_stuff_with_number(n):

print n

the_list = (1, 2, 3, 4, 5)

for i in range(20):

try:

do_stuff_with_number(the_list[i])

except IndexError: # Raised when accessing a non-existing index of a list

do_stuff_with_number(0)

6. Sets

Sets are lists with no duplicate entries. Let's say you want to collect a list of words used in a paragraph:

print set("my name is Eric and Eric is my name".split())

This will print out a list containing "my", "name", "is", "Eric", and finally "and". Since the rest of the sentence uses words which are already in the set, they are not inserted twice.

Sets are a powerful tool in Python since they have the ability to calculate differences and intersections between other sets. For example, say you have a list of participants in events A and B:

a = set(["Jake", "John", "Eric"])

b = set(["John", "Jill"])

To find out which members attended both events, you may use the "intersection" method:

>>> a.intersection(b)

set(['John'])

>>> b.intersection(a)

set(['John'])

To find out which members attended only one of the events, use the "symmetric_difference" method:

>>> a.symmetric_difference(b)

set(['Jill', 'Jake', 'Eric'])

>>> b.symmetric_difference(a)

set(['Jill', 'Jake', 'Eric'])

To find out which members attended only one event and not the other, use the "difference" method:

>>> a.difference(b)

set(['Jake', 'Eric'])

>>> b.difference(a)

set(['Jill'])

To receive a list of all participants, use the "union" method:

>>> a.union(b)

set(['Jill', 'Jake', 'John', 'Eric'])

7. Serialization

Python provides built-in JSON libraries to encode and decode JSON.

In Python 2.5, the simplejson module is used, whereas in Python 2.7, the json module is used. Since this interpreter uses Python 2.7, we'll be using json.

In order to use the json module, it must first be imported:

import json

There are two basic formats for JSON data. Either in a string or the object datastructure. The object datastructure, in Python, consists of lists and dictionaries nested inside each other. The object datastructure allows one to use python methods (for lists and dictionaries) to add, list, search and remove elements from the datastructure. The String format is mainly used to pass the data into another program or load into a datastructure.

To load JSON back to a data structure, use the "loads" method. This method takes a string and turns it back into the json object datastructure:

print json.loads(json_string)

To encode a data structure to JSON, use the "dumps" method. This method takes an object and returns a String:

json_string = json.dumps([1, 2, 3, "a", "b", "c"])

Python supports a Python proprietary data serialization method called pickle (and a faster alternative called cPickle).

You can use it exactly the same way.

import cPickle

pickled_string = cPickle.dumps([1, 2, 3, "a", "b", "c"])

print cPickle.loads(pickled_string)

8. Partial functions

You can create partial functions in python by using the partial function from the functools library.

Partial functions allow one to derive a function with x parameters to a function with fewer parameters and fixed values set for the more limited function.

Import required:

from functools import partial

Example: from functools import partial def multiply(x,y): return x * y

# create a new function that multiplies by 2

dbl = partial(multiply,2)

print dbl(4)

This code will return 8.

An important note: the default values will start replacing variables from the left. The 2 will replace x. y will equal 4 when dbl(4) is called. It does not make a difference in this example, but it does in the example below.

from functools import partial

def func(u,v,w,x):

return u*4 + v*3 + w*2 + x

p = partial(func,5,6,7)

print p(8) # 60

9. Code Introspection

Code introspection is the ability to examine classes, functions and keywords to know what they are, what they do and what they know.

Python provides several functions and utilities for code introspection.

help()

dir()

hasattr()

id()

type()

repr()

callable()

issubclass()

isinstance()

__doc__

__name__

##define the Vehicle class

class Vehicle:

name = ""

kind = "car"

color = ""

value = 100.00

def description(self):

desc_str = "%s is a %s %s worth $%.2f." % (self.name, self.color, self.kind, self.value)

return desc_str

print dir(Vehicle) # ['__doc__', '__module__', 'color', 'description', 'kind', 'name', 'value']