Mocking, Monkey Patching, and Faking Functionality¶

Sometimes while testing you need some fake data.

Maybe you’re doing a third-party API call that can be expensive in execution, or has some limit that you don’t want to risk reaching. Or, maybe you need to create a whole object on the fly just for one or two pieces of functionality, but that object can be difficult to construct from scratch. Or maybe your needs are simpler. Maybe you just want to track some metrics about function/method calls in your codebase, and you have no simple way to do it outside of logging every function call with print() statements or the Python logger.

For these reasons and more, there exists the mock library. Several languages have their own ways and means for mocking behavior, but mock is a specific, pip installable library in Python 2. It was so useful that it was built into Python 3.3+’s unittest library.

We’re currently using pytest, so we don’t have to worry about accessing mock from the unittest library; we can just use pytest-mock. If this package is installed, then when we run py.test we’ll have access to the mocker fixture, which has attached all of the functionality of the unittest.mock library.

This won’t be an exhaustive explanation of what can be done with the pytest-mock library, but it will cover a couple of the more common use cases.

Monkey Patching¶

monkeypatch is a part of the pytest-mock library that allows you to intercept what a function would normally do, substituting its full execution with a return value of your own specification. Note that monkey patching a function call does not count as actually testing that function call! You ARE NOT actually using the function that you’ve monkey patched; you are rejecting its default behavior and subsituting it with new behavior.

Let’s take as an example the following functions for getting and parsing data from GitHub

# called users.py

import requests
import json

def get_user_followers(username):
    """Grab the JSON object from a given user's followers."""
    response = requests.get('https://api.github.com/users/{}/followers'.format(username))
    return response.content

def get_follower_names(username):
    """Given a username of a GitHub user, return a list of follower usernames."""
    json_out = get_user_followers(username)
    as_dict = json.loads(json_out)
    return list(map(lambda x: x["login"], as_dict))

GitHub sets a limit on the rate at which you can access its data. However, as we test get_follower_names and other code that may call get_user_followers, we’ll have to call this function over and over again.

# in test_users.py

def test_get_follower_names_returns_name_list():
    from users import get_follower_names
    assert 'jradavenport' in get_follower_names('nhuntwalker')

Before long, we’ll reach our rate limit (even with an API token). Any test we would run with this function after this point would automatically fail.

So what to do?

In our test file, we can “monkey patch” the call to GitHub’s API. we can do this using the monkeypatch fixture provided by pytest-mock. You don’t have to import it into the file. All you have to do is have pip installed pytest-mock.

# in test_users.py

def substitute_func(username):
    return '[{"login": "aishapectyo"},{"login": "jradavenport"},{"login": "kridicule"}]'

def test_get_follower_names_returns_name_list(monkeypatch):
    import users
    monkeypatch.setattr(users, 'get_user_followers', substitute_func)
    assert 'jradavenport' in users.get_follower_names('nhuntwalker')

Notice the change in imports. monkeypatch is an object unto itself with a variety of methods for faking attributes of other objects or whole namespaces. In the example above, we use the .setattr method to swap out our real users.get_user_followers function with some other substitute function, substitute_func. Treating the users module as an object, monkeypatch changes the behavior of the get_user_followers function inside the module when called for this test.

The substitute function in turn simply returns whatever we tell it to for the purposes of the test(s). In the example above we hardcode a string that is a proper JSON object, just like users.get_follower_names is expecting. The substitute function otherwise does no work that’s not specified in the function definition.

The end result is that, FOR THIS TEST, whenever we would make the the full HTTP request to GitHub for its data we instead get back the return value of substitute_func().

Outside of this test, unless we use monkeypatch again, users.get_user_followers will work the way that it’s supposed to.

As with most testing problems, if we want to have the same behavior occur across a variety of tests, we can always set up a fixture. Remember that whenever you include a fixture in your test function, the code inside of the fixture is run in its entirety before the test itself is run. We can use that to our advantage.

# in test_users.py
import pytest

def substitute_func(username):
    return '[{"login": "aishapectyo"},{"login": "jradavenport"},{"login": "kridicule"}]'

@pytest.fixture
def gh_patched(monkeypatch):
    import users
    monkeypatch.setattr(users, 'get_user_followers', substitute_func)

def test_get_follower_names_returns_name_list(gh_patched):
    from users import get_follower_names
    assert 'jradavenport' in get_follower_names('nhuntwalker')

If we want it so that across every test this behavior is patched without us having to think about it, we can set the autouse keyword argument of pytest.fixture to True.

# in test_users.py
import pytest

def substitute_func(username):
    return '[{"login": "aishapectyo"},{"login": "jradavenport"},{"login": "kridicule"}]'

@pytest.fixture(autouse=True)
def gh_patched(monkeypatch):
    import users
    monkeypatch.setattr(users, 'get_user_followers', substitute_func)

def test_get_follower_names_returns_name_list():
    from users import get_follower_names
    assert 'jradavenport' in get_follower_names('nhuntwalker')

Note how the test no longer includes our fixture in the parameter list.

Beware of changing behavior universally. If we monkey patch the behavior of get_user_followers automatically for every test, we risk not being able to actually test the function after all.

MagicMock and Faking Objects¶

Sometimes it’s not enough to patch over a single function; sometimes you need an instance of a whole object, but constructing that object is a non-trivial endeavor. Consider the following example:

def some_view(request):
    if request.method == "GET":
        return {}
    if request.method == "POST":
        new_entry = Entry(
            title = request.POST['title'],
            body = request.POST['body']
        )
        request.dbsession.add(new_entry)
        return HTTPFound(request.route_url('entry_list'))

Here we have a view function that handles both GET and POST requests. It’s expecting as an argument some sort of request object, but typically we can only build request objects from a real HTTP request. So we must either have some sort of test client set up that can send requests, or receive real requests to test our view.

(Ignore the fact that this is based on Pyramid’s pattern for building requests and that Pyramid has its own built-in DummyRequest object)

With the MagicMock object, we can build an object that can act like a request without having to actually be a REAL instance of any Request class. On that object we can define any methods or attributes that might be useful for the test. This way, we worry less about the configuration that goes into testing a function and focus instead only on giving the function what it needs to work.

To get access to the MagicMock object from pytest-mock, we have to first include the mocker fixture provided to us by pytest-mock. The MagicMock object is an attribute of that fixture, and can be used as you please from there.

In our test for some_view, we might write

def test_some_view_get_req_returns_dict(mocker):
    from views import some_view
    req = mocker.MagicMock()
    req.method = 'GET'
    assert some_view(req) == {}

In this way we test that the some_view function given an object with a method of GET returns the dictionary that we expect. Similarly, we can write more tests assuming that whatever mocked object we pass through is the real object.

def test_some_view_post_returns_redirect(mocker):
    from views import some_view
    req = mocker.MagicMock()
    req.method = 'POST'
    req.POST = {'title': 'some title', 'body': 'some body text'}
    req.dbsession.add = lambda arg: None
    assert isinstance(some_view(req), HTTPFound)

Here we’ve made an object with a method attribute that has a value of 'POST', a POST attribute that is a dictionary containing some values, and a dbsession attribute that has its own fake method, add(). None of these are required to actually work in order to make our test pass. We bypass the overhead of having to set up that functionality just in order to run these tests. Instead, we make sure that our fake object has all the attributes we need to make the function work. Then we pass it to the function with the function being none the wiser.

As we can see, the MagicMock object is pretty much sculpting clay, taking on whatever form and functionality that we need for the moment. It’s good for unit tests of functions that don’t require that we also check for side effects. Depending on the side effect we’re expecting, we may even be able to mock that by providing the side_effect keyword argument on initialization of MagicMock, but if we’re getting to the point of testing interconnected functionality we may want to choose a different testing method.

`mocker.spy` for Tracking Your Methods¶

Sometimes you don’t want to completely hijack a function. Sometimes you just want to keep track of a function’s usage in your application or codebase. For that, there’s mocker.spy.

mocker.spy will allow your object or function to act exactly as it normally would in all cases. The benefit is that you can use some of the features of a MagicMock object alongside your function or object’s regular operations.

For example, consider the following massively-inefficient object

class Numbers(object):
    def __init__(self, iterable):
        self._container = iterable

    def make_unique(self):
        i = 0
        visited = []
        while i < len(self._container):
            if self._container[i] in visited:
                self._drop_val(i)
                i = 0
                continue
            visited.append(self._container[i])
            i += 1

    def _drop_val(self, idx):
        self._container.pop(idx)

The method _drop_val should be called whenever there is a value to be removed from the container list. Perhaps for a given list of values (i.e. [1,2,1,2,1,2]) I want to make sure that _drop_val is called a specific number of times (i.e. 4). I can bake that into my tests by spying on _drop_val and checking the number of times this method was called.

# test code
def test_values_are_dropped_if_already_seen(mocker):
    nums = Numbers([1,2,1,2,1,2])
    mocker.spy(nums, '_drop_val')
    nums.make_unique()
    assert nums._make_unique.call_count == 4

And now, every time that _drop_val was called, that tally was kept and can be verified later on.

In addition to spying on the call count, spied-on methods have a handful of other useful functionality:

assert_called_with(*args, **kwargs)
assert_any_call(*args, **kwargs)
called
mock_calls

The above is not an exhaustive list. Check the Mock Class documentation for the full details.

You can use this in a number of ways, like finding bottlenecks in your code execution by searching for methods that get called a bunch of times, troubleshooting function execution by verifying what it’s being called with, finding out what functions/methods are calling the one you’re interested in, etc.

Mocking, Monkey Patching, and Faking Functionality¶

Monkey Patching¶

MagicMock and Faking Objects¶

`mocker.spy` for Tracking Your Methods¶

Table Of Contents

Previous topic

Next topic

Mocking, Monkey Patching, and Faking Functionality¶

Monkey Patching¶

MagicMock and Faking Objects¶

mocker.spy for Tracking Your Methods¶

`mocker.spy` for Tracking Your Methods¶