Homework Questions?
Solutions to the dict/set lab, and some others in the class repo in: Solutions
A few tidbits:
The dict isn’t sorted, so what if you want to do something in a sorted way?
The “old” way:
keys = d.keys()
keys.sort()
for key in keys:
...
collections.OrderedDict
sorted()
(demo)
When defining a function, you can specify only what you need – in any order
In [150]: from __future__ import print_function
In [151]: def fun(x, y=0, z=0):
.....: print(x, y, z, end=" ")
.....:
In [152]: fun(1, 2, 3)
1 2 3
In [153]: fun(1, z=3)
1 0 3
In [154]: fun(1, z=3, y=2)
1 2 3
A Common Idiom:
def fun(x, y=None):
if y is None:
do_something_different
go_on_here
Can set defaults to variables
In [156]: y = 4
In [157]: def fun(x=y):
print(u"x is: %s" % x)
.....:
In [158]: fun()
x is: 4
Defaults are evaluated when the function is defined
In [156]: y = 4
In [157]: def fun(x=y):
print(u"x is: %s" % x)
.....:
In [158]: fun()
x is: 4
In [159]: y = 6
In [160]: fun()
x is: 4
function arguments are really just:
In [1]: def f(x, y, w=0, h=0):
...: msg = u"position: %s, %s -- shape: %s, %s"
...: print(msg % (x, y, w, h))
...:
In [2]: position = (3, 4)
In [3]: size = {'h': 10, 'w': 20}
In [4]: f(*position, **size)
position: 3, 4 -- shape: 20, 10
You can also pull the parameters out in the function as a tuple and a dict:
In [10]: def f(*args, **kwargs):
....: print(u"the positional arguments are: %s" % unicode(args))
....: print(u"the optional arguments are: %s" % unicode(kwargs))
....:
In [11]: f(2, 3, this=5, that=7)
the positional arguments are: (2, 3)
the optional arguments are: {'this': 5, 'that': 7}
Now that you know that keyword args are really a dict, you can do this nifty trick:
The format method takes keyword arguments:
In [24]: u"My name is {first} {last}".format(last=u"Ewing", first=u"Cris")
Out[24]: u'My name is Cris Ewing'
Build a dict of the keys and values:
In [25]: d = {u"last": u"Ewing", u"first": u"Cris"}
And pass to format()``with ``**
In [26]: u"My name is {first} {last}".format(**d)
Out[26]: u'My name is Cris Ewing'
Let’s do this right now:
keyword arguments
We’ve talked about this: mutable objects can have their contents changed in place.
Immutable objects can not.
This has implications when you have a container with mutable objects in it:
In [28]: list1 = [ [1,2,3], ['a','b'] ]
one way to make a copy of a list:
In [29]: list2 = list1[:]
In [30]: list2 is list1
Out[30]: False
they are different lists.
What if we set an element to a new value?
In [31]: list1[0] = [5,6,7]
In [32]: list1
Out[32]: [[5, 6, 7], ['a', 'b']]
In [33]: list2
Out[33]: [[1, 2, 3], ['a', 'b']]
So they are independent.
But what if we mutate an element?
In [34]: list1[1].append('c')
In [35]: list1
Out[35]: [[5, 6, 7], ['a', 'b', 'c']]
In [36]: list2
Out[36]: [[1, 2, 3], ['a', 'b', 'c']]
uh oh! mutating an element in one list mutated the one in the other list.
Why is that?
In [38]: list1[1] is list2[1]
Out[38]: True
The elements are the same object!
This is known as a “shallow” copy – Python doesn’t want to copy more than it needs to, so in this case, it makes a new list, but does not make copies of the contents.
Same for dicts (and any container type)
If the elements are immutable, it doesn’t really make a differnce – but be very careful with mutable elements.
most objects have a way to make copies (dict.copy() for instance).
but if not, you can use the copy module to make a copy:
In [39]: import copy
In [40]: list3 = copy.copy(list2)
In [41]: list3
Out[41]: [[1, 2, 3], ['a', 'b', 'c']]
This is also a shallow copy.
But there is another option:
In [3]: list1
Out[3]: [[1, 2, 3], ['a', 'b', 'c']]
In [4]: list2 = copy.deepcopy(list1)
In [5]: list1[0].append(4)
In [6]: list1
Out[6]: [[1, 2, 3, 4], ['a', 'b', 'c']]
In [7]: list2
Out[7]: [[1, 2, 3], ['a', 'b', 'c']]
deepcopy recurses through the object, making copies of everything as it goes.
I happened on this thread on stack overflow:
http://stackoverflow.com/questions/3975376/understanding-dict-copy-shallow-or-deep
The OP is pretty confused – can you sort it out?
Make sure you understand the difference between a reference, a shallow copy, and a deep copy.
Another “gotcha” is using mutables as default arguments:
In [11]: def fun(x, a=[]):
....: a.append(x)
....: print(a)
....:
This makes sense: maybe you’d pass in a list, but the default is an empty list.
But:
In [12]: fun(3)
[3]
In [13]: fun(4)
[3, 4]
Huh?!
Remember:
The standard practice for such a mutable default argument:
In [15]: def fun(x, a=None):
....: if a is None:
....: a = []
....: a.append(x)
....: print(a)
In [16]: fun(3)
[3]
In [17]: fun(4)
[4]
You get a new list every time the function is called
A bit of functional programming
consider this common for loop structure:
new_list = []
for variable in a_list:
new_list.append(expression)
This can be expressed with a single line using a “list comprehension”
new_list = [expression for variable in a_list]
What about nested for loops?
new_list = []
for var in a_list:
for var2 in a_list2:
new_list.append(expression)
Can also be expressed in one line:
new_list = [exp for var in a_list for var2 in a_list2]
You get the “outer product”, i.e. all combinations.
(demo)
But usually you at least have a conditional in the loop:
new_list = []
for variable in a_list:
if something_is_true:
new_list.append(expression)
You can add a conditional to the comprehension:
new_list = [expr for var in a_list if something_is_true]
(demo)
Examples:
In [341]: [x ** 2 for x in range(3)]
Out[341]: [0, 1, 4]
In [342]: [x + y for x in range(3) for y in range(5, 7)]
Out[342]: [5, 6, 6, 7, 7, 8]
In [343]: [x * 2 for x in range(6) if not x % 2]
Out[343]: [0, 4, 8]
Remember this from last week?
[name for name in dir(__builtin__) if "Error" in name]
['ArithmeticError',
'AssertionError',
'AttributeError',
....
You can do it with sets, too:
new_set = {value for value in a_sequence}
the same as this for loop:
new_set = set()
for value in a_sequence:
new_set.add(value)
Example: finding all the vowels in a string...
In [19]: s = "a not very long string"
In [20]: vowels = set('aeiou')
In [21]: { let for let in s if let in vowels }
Out[21]: {'a', 'e', 'i', 'o'}
Side note: why did I do set('aeiou') rather than just aeiou?
Also with dictionaries
new_dict = { key:value for key, value in a_sequence}
the same as this for loop:
new_dict = {}
for key, value in a_sequence:
new_dict[key] = value
Example
In [22]: {i: "this_%i" % i for i in range(5)}
Out[22]: {0: 'this_0', 1: 'this_1', 2: 'this_2',
3: 'this_3', 4: 'this_4'}
Can you do the same thing with the dict() constructor?
λ
In [171]: f = lambda x, y: x+y
In [172]: f(2,3)
Out[172]: 5
Content can only be an expression – not a statement
Anyone remember what the difference is?
Called “Anonymous”: it doesn’t need a name.
It’s a python object, it can be stored in a list or other container
In [6]: l = [lambda x, y: x + y]
In [7]: l
Out[7]: [<function __main__.<lambda>>]
In [8]: type(l[0])
Out[8]: function
And you can call it:
In [9]: l[0](3,4)
Out[9]: 7
You can do that with “regular” functions too:
In [12]: def fun(x,y):
....: return x + y
....:
In [13]: l = [fun]
In [14]: type(l[0])
Out[14]: function
In [15]: l[0](3, 4)
Out[15]: 7
map: “maps” a function onto a sequence of objects – It applies the function to each item in the list, returning another list
In [23]: l = [2, 5, 7, 12, 6, 4]
In [24]: def fun(x):
return x * 2 + 10
In [25]: map(fun, l)
Out[25]: [14, 20, 24, 34, 22, 18]
But if it’s a small function, and you only need it once:
In [26]: map(lambda x: x * 2 + 10, l)
Out[26]: [14, 20, 24, 34, 22, 18]
filter: “filters” a sequence of objects with a boolean function – It keeps only those for which the function is True
To get only the even numbers:
In [27]: l = [2, 5, 7, 12, 6, 4]
In [28]: filter(lambda x: not x % 2, l)
Out[28]: [2, 12, 6, 4]
reduce: “reduces” a sequence of objects to a single object with a function that combines two arguments
To get the sum:
In [30]: l = [2, 5, 7, 12, 6, 4]
In [31]: reduce(lambda x, y: x + y, l)
Out[31]: 36
To get the product:
In [32]: reduce(lambda x,y: x*y, l)
Out[32]: 20160
Couldn’t you do all this with comprehensions?
Yes:
In [33]: [x + 2 + 10 for x in l]
Out[33]: [14, 17, 19, 24, 18, 16]
In [34]: [x for x in l if not x % 2]
Out[34]: [2, 12, 6, 4]
(Except Reduce)
But Guido thinks almost all uses of reduce are really sum()
Comprehensions and map, filter, reduce are all “functional programming” approaches}
map, filter and reduce pre-date comprehensions in Python’s history
Some people like that syntax better
And “map-reduce” is a big concept these days for parallel processing of “Big Data” in NoSQL databases.
(Hadoop, EMR, MongoDB, etc.)
Can also use keyword arguments
In [186]: l = []
In [187]: for i in range(3):
.....: l.append(lambda x, e=i: x**e)
.....:
In [189]: for f in l:
.....: print(f(3))
1
3
9
Note when the keyword argument is evaluated
This turns out to be very handy!
Note: this is a bit of a “backwards” exercise – given some code, you figure out what it does.
In canvas, you’ll take a quiz where each of these questions is worth 1 point.
You can take the quiz repeatedly if you have trouble.
>>> feast = ['lambs', 'sloths', 'orangutans', 'breakfast cereals', 'fruit bats']
>>> comprehension = [delicacy.capitalize() for delicacy in feast]
What is the output of:
>>> comprehension[0]
???
>>> comprehension[2]
???
(figure it out before you try it)
>>> feast = ['spam', 'sloths', 'orangutans', 'breakfast cereals', 'fruit bats']
>>> comprehension = [delicacy for delicacy in feast if len(delicacy) > 6]
What is the output of:
>>> len(feast)
???
>>> len(comprehension)
???
(figure it out first!)
>>> list_of_tuples = [(1, 'lumberjack'), (2, 'inquisition'), (4, 'spam')]
>>> comprehension = [ skit * number for number, skit in list_of_tuples ]
What is the output of:
>>> comprehension[0]
???
>>> len(comprehension[2])
???
(figure it out first!)
>>> list_of_eggs = ['poached egg', 'fried egg']
>>> list_of_meats = ['lite spam', 'ham spam', 'fried spam']
>>> comprehension = ['{0} and {1}'.format(egg, meat)
for egg in list_of_eggs
for meat in list_of_meats]
What is the output of:
>>> len(comprehension)
???
>>> comprehension[0]
???
>>> comprehension = {x for x in 'aabbbcccc'}
What is the output of:
>>> comprehension
???
>>> dict_of_weapons = {'first': 'fear',
'second': 'surprise',
'third':'ruthless efficiency',
'forth':'fanatical devotion',
'fifth': None}
>>> dict_comprehension = \
{k.upper(): weapon for k, weapon in dict_of_weapons.iteritems() if weapon}
What is the output of:
>>> 'first' in dict_comprehension
???
>>> 'FIRST' in dict_comprehension
???
>>> len(dict_of_weapons)
???
>>> len(dict_comprehension)
???
See also:
https://github.com/gregmalcolm/python_koans
https://github.com/gregmalcolm/python_koans/blob/master/python2/koans/about_comprehension.py
(submit this one to gitHub for credit on this assignment)
This is from CodingBat “count_evens” (http://codingbat.com/prob/p189616)
Using a list comprehension, return the number of even ints in the given array.
Note: the % “mod” operator computes the remainder, e.g. 5 % 2 is 1.
count_evens([2, 1, 2, 3, 4]) == 3
count_evens([2, 2, 0]) == 3
count_evens([1, 3, 5]) == 0
def count_evens(nums):
one_line_comprehension_here
Let’s revisiting the dict/set lab – see how much you can do with comprehensions instead.
Specifically, look at these:
First a slightly bigger, more interesting (or at least bigger..) dict:
food_prefs = {"name": u"Cris",
u"city": u"Seattle",
u"cake": u"lemon",
u"fruit": u"pomegranate",
u"salad": u"chop",
u"pasta": u"lasagna"}
(make a dictionary that includes your answers, not mine)
Print the dict by passing it to a string format method, so that you get something like:
"Cris is from Seattle, and he likes lemon cake, pomegranate fruit,
chop salad, and lasagna pasta"
Using a list comprehension, build a dictionary of numbers from zero to fifteen and the hexadecimal equivalent (string is fine).
Do the previous entirely with a dict comprehension – should be a one-liner
Using the dictionary from item 1: Make a dictionary using the same keys but with the number of ‘a’s in each value. You can do this either by editing the dict in place, or making a new one. If you edit in place, make a copy first!
Write a function that returns a list of n functions, such that each one, when called, will return the input value, incremented by an increasing number.
Use a for loop, a lambda, and a keyword argument
( Extra credit ):
Do it with a list comprehension, instead of a for loop
Not clear? here’s what you should get:
In [96]: the_list = function_builder(4)
### so the_list should contain n functions (callables)
In [97]: the_list[0](2)
Out[97]: 2
## the zeroth element of the list: a function that adds 0 to the input
In [98]: the_list[1](2)
Out[98]: 3
## the 1st element of the list: a function that adds 1 to the input
In [100]: for f in the_list:
.....: print(f(5), end=" ")
.....:
5
6
7
8
Write a program that takes a filename and “cleans” the file be removing all the leading and trailing whitespace from each line.
Read in the original file and write out a new one, either creating a new file or overwriting the existing one.
Give your user the option of which to perform.
Use map() to do the work.
Write a second version using a comprehension.
sys.argv hold the command line arguments the user typed in. If the user types:
$ python the_script a_file_name
Then:
import sys
filename = sys.argv[1]
will get filename == "a_file_name"
http://learnpythonthehardway.org/book/