Homework Questions?
Solutions to the dict/set lab, and some others in the class repo in: Solutions
A few tidbits:
The dict isn’t sorted, so what if you want to do something in a sorted way?
The “old” way:
keys = d.keys()
keys.sort()
for key in keys:
...
collections.OrderedDict
sorted()
(demo)
When defining a function, you can specify only what you need – in any order
In [150]: from __future__ import print_function
In [151]: def fun(x, y=0, z=0):
.....: print(x, y, z, end=" ")
.....:
In [152]: fun(1, 2, 3)
1 2 3
In [153]: fun(1, z=3)
1 0 3
In [154]: fun(1, z=3, y=2)
1 2 3
A Common Idiom:
def fun(x, y=None):
if y is None:
do_something_different
go_on_here
Can set defaults to variables
In [156]: y = 4
In [157]: def fun(x=y):
print(u"x is: %s" % x)
.....:
In [158]: fun()
x is: 4
Defaults are evaluated when the function is defined
In [156]: y = 4
In [157]: def fun(x=y):
print(u"x is: %s" % x)
.....:
In [158]: fun()
x is: 4
In [159]: y = 6
In [160]: fun()
x is: 4
function arguments are really just:
In [1]: def f(x, y, w=0, h=0):
...: msg = u"position: %s, %s -- shape: %s, %s"
...: print(msg % (x, y, w, h))
...:
In [2]: position = (3, 4)
In [3]: size = {'h': 10, 'w': 20}
In [4]: f(*position, **size)
position: 3, 4 -- shape: 20, 10
You can also pull the parameters out in the function as a tuple and a dict:
In [10]: def f(*args, **kwargs):
....: print(u"the positional arguments are: %s" % unicode(args))
....: print(u"the optional arguments are: %s" % unicode(kwargs))
....:
In [11]: f(2, 3, this=5, that=7)
the positional arguments are: (2, 3)
the optional arguments are: {'this': 5, 'that': 7}
Now that you know that keyword args are really a dict, you can do this nifty trick:
The format method takes keyword arguments:
In [24]: u"My name is {first} {last}".format(last=u"Ewing", first=u"Cris")
Out[24]: u'My name is Cris Ewing'
Build a dict of the keys and values:
In [25]: d = {u"last": u"Ewing", u"first": u"Cris"}
And pass to format()``with ``**
In [26]: u"My name is {first} {last}".format(**d)
Out[26]: u'My name is Cris Ewing'
Let’s do this right now:
keyword arguments
We’ve talked about this: mutable objects can have their contents changed in place.
Immutable objects can not.
This has implications when you have a container with mutable objects in it:
In [28]: list1 = [ [1,2,3], ['a','b'] ]
one way to make a copy of a list:
In [29]: list2 = list1[:]
In [30]: list2 is list1
Out[30]: False
they are different lists.
What if we set an element to a new value?
In [31]: list1[0] = [5,6,7]
In [32]: list1
Out[32]: [[5, 6, 7], ['a', 'b']]
In [33]: list2
Out[33]: [[1, 2, 3], ['a', 'b']]
So they are independent.
But what if we mutate an element?
In [34]: list1[1].append('c')
In [35]: list1
Out[35]: [[5, 6, 7], ['a', 'b', 'c']]
In [36]: list2
Out[36]: [[1, 2, 3], ['a', 'b', 'c']]
uh oh! mutating an element in one list mutated the one in the other list.
Why is that?
In [38]: list1[1] is list2[1]
Out[38]: True
The elements are the same object!
This is known as a “shallow” copy – Python doesn’t want to copy more than it needs to, so in this case, it makes a new list, but does not make copies of the contents.
Same for dicts (and any container type)
If the elements are immutable, it doesn’t really make a differnce – but be very careful with mutable elements.
most objects have a way to make copies (dict.copy() for instance).
but if not, you can use the copy module to make a copy:
In [39]: import copy
In [40]: list3 = copy.copy(list2)
In [41]: list3
Out[41]: [[1, 2, 3], ['a', 'b', 'c']]
This is also a shallow copy.
But there is another option:
In [3]: list1
Out[3]: [[1, 2, 3], ['a', 'b', 'c']]
In [4]: list2 = copy.deepcopy(list1)
In [5]: list1[0].append(4)
In [6]: list1
Out[6]: [[1, 2, 3, 4], ['a', 'b', 'c']]
In [7]: list2
Out[7]: [[1, 2, 3], ['a', 'b', 'c']]
deepcopy recurses through the object, making copies of everything as it goes.
I happened on this thread on stack overflow:
http://stackoverflow.com/questions/3975376/understanding-dict-copy-shallow-or-deep
The OP is pretty confused – can you sort it out?
Make sure you understand the difference between a reference, a shallow copy, and a deep copy.
Another “gotcha” is using mutables as default arguments:
In [11]: def fun(x, a=[]):
....: a.append(x)
....: print(a)
....:
This makes sense: maybe you’d pass in a list, but the default is an empty list.
But:
In [12]: fun(3)
[3]
In [13]: fun(4)
[3, 4]
Huh?!
Remember:
The standard practice for such a mutable default argument:
In [15]: def fun(x, a=None):
....: if a is None:
....: a = []
....: a.append(x)
....: print(a)
In [16]: fun(3)
[3]
In [17]: fun(4)
[4]
You get a new list every time the function is called
A bit of functional programming
consider this common for loop structure:
new_list = []
for variable in a_list:
new_list.append(expression)
This can be expressed with a single line using a “list comprehension”
new_list = [expression for variable in a_list]
What about nested for loops?
new_list = []
for var in a_list:
for var2 in a_list2:
new_list.append(expression)
Can also be expressed in one line:
new_list = [exp for var in a_list for var2 in a_list2]
You get the “outer product”, i.e. all combinations.
(demo)
But usually you at least have a conditional in the loop:
new_list = []
for variable in a_list:
if something_is_true:
new_list.append(expression)
You can add a conditional to the comprehension:
new_list = [expr for var in a_list if something_is_true]
(demo)
Examples:
In [341]: [x ** 2 for x in range(3)]
Out[341]: [0, 1, 4]
In [342]: [x + y for x in range(3) for y in range(5, 7)]
Out[342]: [5, 6, 6, 7, 7, 8]
In [343]: [x * 2 for x in range(6) if not x % 2]
Out[343]: [0, 4, 8]
Remember this from last week?
[name for name in dir(__builtin__) if "Error" in name]
['ArithmeticError',
'AssertionError',
'AttributeError',
....
You can do it with sets, too:
new_set = {value for value in a_sequence}
the same as this for loop:
new_set = set()
for value in a_sequence:
new_set.add(value)
Example: finding all the vowels in a string...
In [19]: s = "a not very long string"
In [20]: vowels = set('aeiou')
In [21]: { let for let in s if let in vowels }
Out[21]: {'a', 'e', 'i', 'o'}
Side note: why did I do set('aeiou') rather than just aeiou?
Also with dictionaries
new_dict = { key:value for key, value in a_sequence}
the same as this for loop:
new_dict = {}
for key, value in a_sequence:
new_dict[key] = value
Example
In [22]: {i: "this_%i" % i for i in range(5)}
Out[22]: {0: 'this_0', 1: 'this_1', 2: 'this_2',
3: 'this_3', 4: 'this_4'}
Can you do the same thing with the dict() constructor?
λ
In [171]: f = lambda x, y: x+y
In [172]: f(2,3)
Out[172]: 5
Content can only be an expression – not a statement
Anyone remember what the difference is?
Called “Anonymous”: it doesn’t need a name.
It’s a python object, it can be stored in a list or other container
In [6]: l = [lambda x, y: x + y]
In [7]: l
Out[7]: [<function __main__.<lambda>>]
In [8]: type(l[0])
Out[8]: function
And you can call it:
In [9]: l[0](3,4)
Out[9]: 7
You can do that with “regular” functions too:
In [12]: def fun(x,y):
....: return x + y
....:
In [13]: l = [fun]
In [14]: type(l[0])
Out[14]: function
In [15]: l[0](3, 4)
Out[15]: 7
map: “maps” a function onto a sequence of objects – It applies the function to each item in the list, returning another list
In [23]: l = [2, 5, 7, 12, 6, 4]
In [24]: def fun(x):
return x * 2 + 10
In [25]: map(fun, l)
Out[25]: [14, 20, 24, 34, 22, 18]
But if it’s a small function, and you only need it once:
In [26]: map(lambda x: x * 2 + 10, l)
Out[26]: [14, 20, 24, 34, 22, 18]
filter: “filters” a sequence of objects with a boolean function – It keeps only those for which the function is True
To get only the even numbers:
In [27]: l = [2, 5, 7, 12, 6, 4]
In [28]: filter(lambda x: not x % 2, l)
Out[28]: [2, 12, 6, 4]
reduce: “reduces” a sequence of objects to a single object with a function that combines two arguments
To get the sum:
In [30]: l = [2, 5, 7, 12, 6, 4]
In [31]: reduce(lambda x, y: x + y, l)
Out[31]: 36
To get the product:
In [32]: reduce(lambda x,y: x*y, l)
Out[32]: 20160
Couldn’t you do all this with comprehensions?
Yes:
In [33]: [x + 2 + 10 for x in l]
Out[33]: [14, 17, 19, 24, 18, 16]
In [34]: [x for x in l if not x % 2]
Out[34]: [2, 12, 6, 4]
(Except Reduce)
But Guido thinks almost all uses of reduce are really sum()
Comprehensions and map, filter, reduce are all “functional programming” approaches}
map, filter and reduce pre-date comprehensions in Python’s history
Some people like that syntax better
And “map-reduce” is a big concept these days for parallel processing of “Big Data” in NoSQL databases.
(Hadoop, EMR, MongoDB, etc.)
Can also use keyword arguments
In [186]: l = []
In [187]: for i in range(3):
.....: l.append(lambda x, e=i: x**e)
.....:
In [189]: for f in l:
.....: print(f(3))
1
3
9
Note when the keyword argument is evaluated
This turns out to be very handy!
In your student folder, create a subdirectory called session05. Create a new branch called task13 and switch to it (git checkout -b task13).
Within the session05 subdirectory, create a new file called list_comp.py.
Add the file to your clone of the repository and commit changes frequently while working on the following tasks. When you are done, push your changes to GitHub and create a pull request titled Task 13 pull request from Your Name where you should substitute your name for Your Name.
Note: this is a bit of a “backwards” exercise – given some code, you figure out what it does.
In canvas, you’ll take a quiz where each of these questions is worth 1 point.
You can take the quiz repeatedly if you have trouble.
>>> feast = ['lambs', 'sloths', 'orangutans', 'breakfast cereals', 'fruit bats']
>>> comprehension = [delicacy.capitalize() for delicacy in feast]
What is the output of:
>>> comprehension[0]
???
>>> comprehension[2]
???
(figure it out before you try it)
>>> feast = ['spam', 'sloths', 'orangutans', 'breakfast cereals', 'fruit bats']
>>> comprehension = [delicacy for delicacy in feast if len(delicacy) > 6]
What is the output of:
>>> len(feast)
???
>>> len(comprehension)
???
(figure it out first!)
>>> list_of_tuples = [(1, 'lumberjack'), (2, 'inquisition'), (4, 'spam')]
>>> comprehension = [ skit * number for number, skit in list_of_tuples ]
What is the output of:
>>> comprehension[0]
???
>>> len(comprehension[2])
???
(figure it out first!)
>>> list_of_eggs = ['poached egg', 'fried egg']
>>> list_of_meats = ['lite spam', 'ham spam', 'fried spam']
>>> comprehension = ['{0} and {1}'.format(egg, meat)
for egg in list_of_eggs
for meat in list_of_meats]
What is the output of:
>>> len(comprehension)
???
>>> comprehension[0]
???
>>> comprehension = {x for x in 'aabbbcccc'}
What is the output of:
>>> comprehension
???
>>> dict_of_weapons = {'first': 'fear',
'second': 'surprise',
'third':'ruthless efficiency',
'forth':'fanatical devotion',
'fifth': None}
>>> dict_comprehension = \
{k.upper(): weapon for k, weapon in dict_of_weapons.iteritems() if weapon}
What is the output of:
>>> 'first' in dict_comprehension
???
>>> 'FIRST' in dict_comprehension
???
>>> len(dict_of_weapons)
???
>>> len(dict_comprehension)
???
See also:
https://github.com/gregmalcolm/python_koans
https://github.com/gregmalcolm/python_koans/blob/master/python2/koans/about_comprehension.py
(submit this one to gitHub for credit on this assignment)
This is from CodingBat “count_evens” (http://codingbat.com/prob/p189616)
Using a list comprehension, return the number of even ints in the given array.
Note: the % “mod” operator computes the remainder, e.g. 5 % 2 is 1.
count_evens([2, 1, 2, 3, 4]) == 3
count_evens([2, 2, 0]) == 3
count_evens([1, 3, 5]) == 0
def count_evens(nums):
one_line_comprehension_here
In your session05 directory, write the following code into a new file called dict_comp.py.
Add the file to your local working repository and commit changes frequently while working on the following tasks. When you are done, push your changes to GitHub and create a pull request titled Task 14 pull request from Your Name where you should substitute your name for Your Name.
Let’s revisiting the dict/set lab – see how much you can do with comprehensions instead.
Specifically, look at these:
First a slightly bigger, more interesting (or at least bigger..) dict:
food_prefs = {"name": u"Cris",
u"city": u"Seattle",
u"cake": u"lemon",
u"fruit": u"pomegranate",
u"salad": u"chop",
u"pasta": u"lasagna"}
(make a dictionary that includes your answers, not mine)
Print the dict by passing it to a string format method, so that you get something like:
"Cris is from Seattle, and he likes lemon cake, pomegranate fruit,
chop salad, and lasagna pasta"
Using a list comprehension, build a dictionary of numbers from zero to fifteen and the hexadecimal equivalent (string is fine).
Do the previous entirely with a dict comprehension – should be a one-liner
Using the dictionary from item 1: Make a dictionary using the same keys but with the number of ‘a’s in each value. You can do this either by editing the dict in place, or making a new one. If you edit in place, make a copy first!
In your session05 directory, write the following code into a new file called lambda.py.
Add the file to your local working repository and commit changes frequently while working on the following tasks. When you are done, push your changes to GitHub and create a pull request titled Task 15 pull request from Your Name where you should substitute your name for Your Name.
Write a function that returns a list of n functions, such that each one, when called, will return the input value, incremented by an increasing number.
Use a for loop, a lambda, and a keyword argument
( Extra credit ):
Do it with a list comprehension, instead of a for loop
Not clear? here’s what you should get:
In [96]: the_list = function_builder(4)
### so the_list should contain n functions (callables)
In [97]: the_list[0](2)
Out[97]: 2
## the zeroth element of the list: a function that adds 0 to the input
In [98]: the_list[1](2)
Out[98]: 3
## the 1st element of the list: a function that adds 1 to the input
In [100]: for f in the_list:
.....: print(f(5), end=" ")
.....:
5
6
7
8
Read through the Session 6 slides.
http://codefellows.github.io/sea-c34-python/session06.html
There are four sections. For each one, come up with the following numbers of questions.
Write some Python code to answer these questions, one function per question.
For each function, write a good docstring describing what question you are trying to answer.
Put the functions in four separate modules (files) called oop.py, classes.py, and subclasses.py in the session05 subdirectory of your student directory.
That is, you should have seven questions, and seven functions, total, spread out across three files.
Use everything you’ve learned so far as needed (including lists, tuples, slicing, iteration, functions, booleans, printing, modules, assertions, dictionaries, sets, exceptions, file reading/writing, and paths).
Create a branch in your local repo called task16 and switch to it (git checkout task16).
Add your files to that branch, commit and push, then create a pull request to the main class repo, titled Task 16 pull request from Your Name where you should substitute your name for Your Name.
Finally, submit your assignment in Canvas by giving the URL of the pull request.
http://learnpythonthehardway.org/book/