Functional testing in an environment of Flask micro-services

You can find the source code written in this article in the flask-boilerplate that we use at Theodo.

Functionally Testing an API consists in calling its routes and checking that the response status and the content of the response are what you expect.

It is quite simple to set up functional tests for your app. Writing a bunch of HTTP requests in the language of you choice that call your app (that has previously been launched) will do the job.

However, with this approach, your app is a blackbox that can only be accessed from its doors (i.e. its URLs). Although the goal of functional tests is actually to handle the app as a blackbox, it is still convenient while testing an API to have access to its database and to be able to mock calls to other services (especially in a context of micro-services environment).

Moreover it is important that the tests remain independent from each other. In other words, if a resource is added into the database during a test, the next test should not have to deal with it. This is not easy to handle unless the whole app is relaunched before each test. Even if it is done, some tests require different fixtures. It would be tricky to handle.

With this first approach, our functional tests were getting more complex than the code they were testing. I would like to share how we improved our tests using the flask test client class.

You don’t need to know about flask/python to understand the following snippets.

The API allows to post and get users. First we can write a route to get a user given its id:

# src/route/user.py
from model import User

# When requesting the URL /user/5, the get_user_by_id will be executed with id=5
@app.route('/user/<int:id>', methods=['GET'])
def get_user_by_id(self, id):
    user = User.query.get(id)
    return user.json  # user.json is a dictionary with user data such as its email

This route can be tested with the flask test client class:

# test/route/test_user.py
import unittest
import json

from server import app
from model import db, User

class TestUser(unittest.TestCase):

    # this method is run before each test
    def setUp(self):
        self.client = app.test_client()  # we instantiate a flask test client

        db.create_all()  # create the database objects
        # add some fixtures to the database
        self.user = User(
            email='joe@theodo.fr',
            password='super-secret-password'
        )
        db.session.add(self.user)
        db.session.commit()

    # this method is run after each test
    def tearDown(self):
        db.session.remove()
        db.drop_all()

    def test_get_user(self):
        # the test client can request a route
        response = self.client.get(
            '/user/%d' % self.user.id,
        )

        self.assertEqual(response.status_code, 200)
        user = json.loads(response.data.decode('utf-8'))
        self.assertEqual(user['email'], 'joe@theodo.fr')

if __name__ == '__main__':
    unittest.main()

In these tests:

• all tests are independent: the database objects are rebuilt and fixtures are inserted before each test.
• we have access to the database via the db object during the tests. So if you test a ‘POST’ route, you can check that a resource has been successfuly added into the database.

Another benefit is that you can easily mock a call to another API. Let’s improve our API: the get_user_by_id function will call an external API to check if the user is a superhero:

# src/client/superhero.py
import requests

def is_superhero(email):
    """Call the superhero API to find out if this email belongs to a superhero."""
    response = requests.get('http://127.0.0.1:5001/superhero/%s' % email)
    return response.status_code == 200

from client import superhero
# ...
@app.route('/user/<int:id>', methods=['GET'])
def get_user_by_id(self, id):
    user = User.query.get(id)
    user_json = user.json
    user_json['is_superhero'] = superhero.is_superhero(user.email)
    return user_json

To prevent the tests from depending on this external API, we can mock the client in our tests:

# test/route/test_user.py
from mock import patch
#...

@patch('client.superhero.is_superhero')  # we mock the function is_superhero
def test_get_user(self, is_superhero_mock):
    # when is_superhero is called, it returns true instead of calling the API
    is_superhero_mock.return_value = True

    response = self.client.get(
        '/user/%d' % self.user.id,
    )
    self.assertEqual(response.status_code, 200)
    user = json.loads(response.data.decode('utf-8'))
    self.assertEqual(user['email'], 'joe@theodo.fr')
    self.assertEqual(user['is_superhero'], True)

To use this mock for all tests, the mock can be instantiated in the setUp method:

# test/route/test_user.py
def setUp(self):
    #...
    self.patcher = patch('client.superhero.is_superhero')
    is_superhero_mock.return_value = True
    is_superhero_mock.start()
    #...

def tearDown(self):
    #...
    is_superhero_mock.stop()
    #...

Conclusion

With the Flask test client, you can write functional tests, keep control over the database and mock external calls. Here is a flask boilerplate to help you get started with a flask API.