Nose2: Unit Testing for Data Science with Python
Catch expensive mistakes early with nose2 and parameterized tests
You’ve deployed a new machine learning model at work. You can finally enjoy the weekend, you thought to yourself. Little did you know that an imminent storm of errors is about to tear down your model and ruin your weekend.
Why does that happen? Insufficient error checking. Data scientists are taught to perform data exploration and modeling, but we’re not taught to perform unit tests, particularly on edge cases.
In this post, I will introduce a skill that I found immensely useful in my day-to-day: unit testing using nose2. In particular, I’ll share
- What is unit testing?
- Why should data scientists perform unit tests?
- What is nose2?
- How to perform a simple unit test?
- What are parameterized unit tests?
- How to do a simple parameterized test?
- How to do a parameterized test that checks for errors?
- How to do a parameterized test with pandas dataframe?
What is Unit Testing?
Unit testing is one of the most powerful skills a data scientist can master, it’s the Griffin of programming. It’s a test that checks a single component of code, usually modularized as a function, and ensures that it performs as expected
Ideally, we want our tests to be small. The smaller the better. This is because smaller tests aren’t only more efficient from a practical standpoint — since testing smaller units will allow your tests to run faster — but also conceptually, as it will provide you with a more detailed view of how the granular code is behaving.
Why Perform Unit Testing?
There are many! Here’s a quick runthrough.
- You can find bugs easily in the development cycle since the functions/classes are modularised/isolated so code is tested one part at a time, this leads to increased efficiency, reduced downtime, and reduced costs that would otherwise arise as a result of the whole design process stalling.
- You can refactor code easier when you test each component of the software individually. Problems that are detected early on can be nipped in the bud.
- You can use them as a form of documentation when done well.
Let us look at a simple example using unittest which is built into the standard python library since version 2.1.
Creating test cases is accomplished by subclassing unittest.TestCase. An example is below.
import unittestdef add(x): return x + 1class MyTest(unittest.TestCase): def test(self): self.assertEqual(add(3), 4)
You can also find other assert methods here.
While unittest is great for simple tests, it can quickly become a little cumbersome when dealing with more complex code. Hence, nose2 was developed for extending unittest to ease the testing process.
What is nose2?
In comparison to unittest, Nose2 provides a better plugin API and simplifies internal interfaces and processes. There are many plugins that built-in the Nose2 module, and these plugins are loaded by default. Some of the major plugins that are loaded by default aid in the parameterization of tests, organizing test fixtures into layers, capturing log messages, providing test coverage reporting, and more.
Here are the different unit testing frameworks available in python.
How to perform a simple unit test?
Before we get started you need to install the Nose2 framework in your systems
pip install nose2==0.9.2
pip install parameterizedWe will start with a simple example. Consider the case where we have a list of four scores from an exam. The score should range from 0 to 100.
+-------+
| score |
+-------+
| 44 |
| 64 |
| -5 |
| 101 |
+-------+Let’s write a test that will help us catch the two values (-5 and 101) that do not fit the criteria and tell python to run the test.
"""
Filename: test.py
"""
import unittest
import pandas as pd
grade_list = [44, 64, -5, 101]
"""
The TestPassed class iterates through the list of grades.
For each grade, it checks if it is between 0 and 100.
If not, it fails the test.
"""
class TestPassed(unittest.TestCase):
def test_grade_between_0_and_100(self):
# Iterate through grade_list
for grade in grade_list:
# subTest divides one unit test into smaller tests.
with self.subTest(i=grade):
# checks if grades is between 0 and 100 (inclusive)
self.assertEqual(grade >= 0 and grade <= 100,
True)
""" The main function that tells python to run the test"""
if __name__ == '__main__':
import nose2
nose2.main() Now that we have created test.py, we have to run it. To do so, you can open up your terminal, and run the following command. (Please replace “/directory/to/your/test.py” with the directory at which your test.py is stored.)
python3 /directory/to/your/test.pyUpon running this command, you should see the following.
As expected, our test has failed! Hooray.
What are parameterized unit tests?
Developers tend to write one test per case. Some related cases are then grouped into “suites”. Consider the following case.
class CodeTestSuite(TestCase):
def test_for_int_input(self):
assert correct_output()
def test_for_float_input(self):
assert correct_output()
def test_for_string_input(self):
assert correct_input() As you can imagine, most of these cases are closely related. This could result in dedundancies, large testing codebases, and potential technical debt to be repaid when changes are made.
Enter parameterized testing, which generates one test for with multiple parameters for easy testing. You can pass your parameters in through a decorator on your method in your test suite.
Some of the codes below are built on Richard D Jones’ “You Should be Doing Parameterized Testing”. Do check it out!
Parameterised Tests with Decorators
Let’s say that we want to test this simple function `compute`.
def compute(a, b):
return (a + b) / (a * b)In the example below, we use perform two parameterized tests on the compute function. In particular:
- test_int tests if the output of compute(a=0,b=1) is 1. If so, the test is passed.
- test_float tests if the output of compute(a=1,b=1) is 1. If so, the test is passed.
The code snippet is as follows.
class TestSuite(unittest.TestCase):
@parameterized.expand([
# each tuple contains
# (name , a , b , expected_output
("test_int" , 1 , 1 , 2),
("test_float", 1. , 1. , 2.)
])
def test_compute(self, name, a, b, expected_output):
assert compute(a, b) == expected_outputNotice how the `@parameterized.expand` decorator takes in a list of tuples. Each of the tuples is a test case whose output will be changed.
The full code block is below.
"""
Filename: test_param.py
"""
import unittest
from nose2.tools import params
import pandas as pd
from parameterized import parameterized
def compute(a, b):
return (a + b) / (a * b)
class TestSuite(unittest.TestCase):
@parameterized.expand(
[
#each tuple contains (name, name, a, b, expected_output)
("test_int", 1, 1, 2),
("test_float", 1., 1., 2.)
]
)
def test_compute(self, name, a, b, expected_output):
assert compute(a, b) == expected_output
if __name__ == '__main__':
import nose2
nose2.main() You can run the above test using the following command. All the tests should pass.
python3 /directory/to/your/test_param.pyHow to do a parameterized test that checks for errors?
In some cases, we also want to perform unit testing to ensure that error is raised correctly. To do that, we can specify the error to be raised.
The code snippet is as follows.
def test_compute(self, name, a, b, expected_output, expected_error=None):
# If no error is expected from the test case,
# check if the actual output matches the expected output
if expected_error is None:
assert compute(a, b) == expected_output
# If an error is expected from the test case,
# check if the actual error matches the expected error
else:
with self.assertRaises(expected_error):
compute(a, b)The below example `test_divisionbyzero` tests if compute(0,0) correctly raise a ZeroDivisionError.
"""
Filename: test_param_with_error.py
"""
import unittest
from nose2.tools import params
import pandas as pd
from parameterized import parameterized
def compute(a, b):
return (a + b) / (a * b)
"""
The TestSuite class checks treats each test case in the
decorator @params as an individual test case.
For test case, it checks if the actual output matches the expected output.
If not, the test fails.
"""
class TestSuite(unittest.TestCase):
@parameterized.expand(
[
#each tuple contains (name, a, b, expected_output, expected_error)
("test_divisionbyzero", 0, 0, None, ZeroDivisionError),
("test_int", 1, 1, 2),
("test_float", 1., 1., 2.)
]
)
def test_compute(self, name, a, b, expected_output, expected_error=None):
# If no error is expected from the test case,
# check if the actual output matches the expected output
if expected_error is None:
assert compute(a, b) == expected_output
# If an error is expected from the test case,
# check if the actual error matches the expected error
else:
with self.assertRaises(expected_error):
compute(a, b)
""" The main function that tells python to run the test"""
if __name__ == '__main__':
import nose2
nose2.main() You can run the above test using the following command. All the tests should pass.
python3 /directory/to/your/test_params_with_error.pyHow to do a parameterized test with pandas dataframe?
What if you want to pass in a pandas dataframe for your test? No problem.
We can use the following code snippet, where load_test_case() is a function that takes in the data.
"""
Filename: test_param_with_dataframe.py
"""
import unittest
from nose2.tools import params
import pandas as pd
from parameterized import parameterized
def compute(a, b):
return (a + b) / (a * b)
def load_test_cases():
""" Create test cases in the format of a dataframe. """
df = pd.DataFrame.from_dict({
# Each row contains
# [name , a , b , expected_output , expected_error ]
'test_1': ['negative_int_test' , -2 , 2 , 0 , None ],
'test_2': ['positive_int_test' , 2 , 2 , 1 , None ],
'test_3': ['decimal_test' , .5 , .4 , 4.5 , None ],
'test_4': ['none_type_test' , None , 2 , None , TypeError ],
'test_5': ['string_type_test' , '10' , 1 , None , TypeError ],
'test_6': ['zero_division_test', 0 , 2 , None , ZeroDivisionError ]
},
orient='index'
)
df.columns = ['name','a','b','expected_output', 'expected_error']
return list(df.itertuples(index=False, name=None)) # return dataframe as a list of tuples.
class TestSuite(unittest.TestCase):
@parameterized.expand(
#each tuple contains (name, a, b, expected_output, expected_error)
load_test_cases()
)
def test_compute(self, name, a, b, expected_output, expected_error=None):
# If no error is expected from the test case,
# check if the actual output matches the expected output
if expected_error is None:
assert compute(a, b) == expected_output
# If an error is expected from the test case,
# check if the actual error matches the expected error
else:
with self.assertRaises(expected_error):
compute(a, b)
if __name__ == '__main__':
import nose2
nose2.main() You can run the above test using the following command. All the tests should pass.
python3 /directory/to/your/test_params_with_dataframe.pyUnit testing is key to a stress-free weekend
The unit test module can be complemented by the nose2 module to create powerful unit tests. In particular, parameterized tests offers a convenient way to organize multiple unit tests.
I will share more unit test strategies in the upcoming week. Stay tuned!
I write blogs like this.
If you’d like more to learn more about data analytics, data science, and machine learning, consider following me on Medium and on LinkedIn!
