26  Testing

import sandbox_widget

This chapter is about how you figure out if your code solves the problem the way you intended. You will be surprised how often that is not the case, even for seasoned programmers. There are many good reasons to to test your code. The three most important ones are:

Makes you think: Testing forces you to slow down and think about exactly what the code you want to write is supposed to do. Writing tests before you start writing the actual code, helps you anticipate errors and corner cases that need to be addressed in the way you implement your solution.

Gives you peace of mind: Testing allows you to trust that a function you have written actually works as it is supposed to. That frees up your mind to focus on about how to use it as a component in solving the larger, more complicated problem.

Makes you trust code from an AI: When it comes to code you did not write yourself, the only way to really know if you can trust it, is to write your own tests for the code and/or to read any tests the AI wrote.

assert means “make sure”

Testing can be done and organized in many different ways, but always boils down to to “making sure” that something is as you expect. In Python, the keyword for “making sure” is called assert. E.g., to make sure your function variable x is larger than zero, you simply write assert x > 0. Literally, an assertion is a claim that something is true, so using assert you claim that the expression following it is true. Let’s take as an example the function below that splits an open reading frame into a list of codons:

%%sandbox
x = -5
assert x > 0

If you want, you can add a string after the test expression to explain the problem the test exposed:

%%sandbox
x = -5
assert x > 0, "x is not larger than zero"

Testing your functions

Functions divide you code into smaller parts that each solve a small problem. If function is implemented correctly, you can go on with your project using the function in other functions, without having to remember how the function works or worry about whether it works. Testing your functions buys you that peace of mind. Separating code into functions also makes it easier to understand what is wrong if your code does crash or do something unexpected. Run the cell below to define a function that splits an open reading frame (ORF) into a list of codons:

def split_codons(orf):
    codons = []
    for i in range(0, len(orf)-3+1, 3):
        codon = orf[i:i+3]
        codons.append(codon)
    return codons

split_codons('ATGCCATAATGA')

Now you can write assert statements to “make sure”/test that the function returns what it should given the argument passed to it. If you run the cell below, no errors are produced because the function passes all the tests:

assert split_codons('ATGCCATAATGA') == ['ATG', 'CCA', 'TAA', 'TGA']
assert split_codons('ATG') == ['ATG']

Exercise 26-1

SOLO

Try to add as many tests you can think off to the cell above. How about lower case characters? How about different numbers of codons? What if the ORF does not divide evenly into codons? What if the ORF is an empty string? Be creative and see how many corner cases you can cover.

It makes sense to group the tests for each function in test functions. A test function a function runs the assert statements for a function in your project. The convention, which you should follow, is to name each test function after the project function it tests but with a test_ prefix. So in the example below, test_split_orf is the function that we run to test that split_orf works as it should. All you have to do to test split_orf is to run test_split_orf.

def split_codons(orf):
    codons = []
    for i in range(0, len(orf)-3+1, 3):
        codon = orf[i:i+3]
        codons.append(codon)
    return codons

def test_split_codons():
    assert split_codons('ATGCCATAATGA') == ['ATG', 'CCA', 'TAA', 'TGA']
    assert split_codons('ATG') == ['ATG']

However, as you can probably imagine, in project with many functions and as many test functions, finding and calling all your test functions becomes quite a job. If only there was a way to automate that. Enter pytest. pytest is a python package and command line tool that finds the files you have written your tests in and then runs all of them, producing a nice overview of how many tests passed and how many failed. If you put the code in the above cell in a file called orfs.py then running pytest -v orfs.py in the terminal will run the tests in that file and produce an output like this:

========================== test session starts ===========================
platform darwin -- Python 3.13.15, pytest-9.1.1, pluggy-1.6.0
rootdir: /Users/kmt/instructing-machines/week5
plugins: anyio-4.14.2, im-pytest-0.1.33
collected 1 item                                                                                     

orfs.py::test_split_codons PASSED                                      [100%]

=========================== 1 passed in 0.01s ============================

That is the way professional programmers test their code. To ease the learning curve a bit, I have made a %%test cell magic that lets you do the same from within a notebook. Run the cell below to import it.

import im_pytest

Now run this cell:

%%test --raw
def split_codons(orf):
    codons = []
    for i in range(0, len(orf)-3+1, 3):
        codon = orf[i:i+3]
        codons.append(codon)
    return codons

def test_split_codons():
    assert split_codons('ATGCCATAATGA') == ['ATG', 'CCA', 'TAA', 'TGA']
    assert split_codons('ATG') == ['ATG']

Uisng %%test without the --raw flag produces a simpler and more useful output. Among other things, it stops when it runs a test function that fails. So if you test your functions in the same order as they are used in your project, It helps you find any problems in the order that makes most sense: Say you have written a function a that is called inside another function b. A bug in function a will then break both the a and the b function. With test functions in the right order, %%test will stop testing if it finds a bug in function a and let you fix that first.

%%test 
def split_codons(orf):
    codons = []
    for i in range(0, len(orf)-3+1, 3):
        codon = orf[i:i+3]
        codons.append(codon)
    return codons

def test_split_codons():
    assert split_codons('ATGCCATAATGA') == ['ATG', 'CCA', 'TAA', 'TGA']
    assert split_codons('ATG') == ['ATG']

In the example above, the test functions were in the same file as the project functions they test, but is it more common to put the tests in a separate file to keep the file with the project functions nice and clean without noise and distration. This is also the way we organize tests for the programming projects you will do in this course. More about that in Chapter 32.

assert anywhere

You can add an assert statement inside the function for each thing you want to “make sure” of. Be creative, think of as many things to check as you can. Here are some:

def split_orf(orf):    
    codons = []
    for i in range(0, len(orf)-3+1, 3):
        assert i < len(orf)
        codon = orf[i:i+3]
        assert len(codon) == 3
        codons.append(codon)
    assert len(orf) == 3 * len(codons) # check codon list has the right length
    return codons

split_orf('ATGCCATAATGA')

Exercise 26-2

AI: Explainer

Now try to add a nucleotide to the end of the ORF passed to the function and rerun the cell to see what happens. If you get an error message, read it to see what caused the error and what kind of error it is. Then try to hand the error to the assistant and see if the its explanation matches your own. If not, make sure you understand its explanation well enough to explain it to someone else.