FlowTest documentation

The flowtest package provides an integration testing framework for Python applications.

Testing is done by implementing test steps. Each step can continue where a previous step stopped. Therefore, each step can define it's predecessor and the test steps are thus organized in a tree or more general a directed graph.

More formally, testing is defined as a test graph, where each node in the graph corresponds to a test step (and a step is not necessarily a single test) and each edge denotes a test continuation.

Nodes without incoming edges are called start nodes, and start nodes cannot have incoming edges. The flow graph is used to create a set of test flows as follows: each possible path starting from a start node corresponds to a single test flow.

User guide

Installation

Install using pip:

pip install flowtest

Examples

Usage examples can be found in the examples directory. The complete basics.py example is shown below:

import operator
from functools import reduce

import sys
from flowtest import FlowGraph

graph = FlowGraph()


@graph.step_decorator()
def start():
    return [1, 2, 3]


@graph.step_decorator(start)
def verify_sum(data):
    assert sum(data) == 6


@graph.step_decorator(start)
def compute_product(data):
    return reduce(operator.mul, data, 1)


@graph.step_decorator(compute_product)
def verify_product(product):
    assert product == 6


if not graph.run_all_flows():
    sys.exit(1)

Basic usage

Integration tests are written as Python modules.

  1. Instantiate a flowtest.FlowGraph object
  2. Add test steps to the FlowGraph object using the FlowGraph.step_decorator decorator function.
  3. Call FlowGraph.run_all_flows() to run all possible flows.

Adding a start step (e.g. the first step for one or more flows) can be done as follows:

@graph.step_decorator()
def start():
    return 1

Adding a followup node can be done as follows:

@graph.step_decorator(start)
def double_it(number):
    return number * 2

Now, the double_it step will be called with the output from the start step.

The FlowGraph.step_decorator should always be called in your code, e.g. the following won't work:

@graph.step_decorator
def foo():
    pass

It is possible to define step functions that are part of multiple flows. For example we might want to apply double_it to multiple numbers. This can be done as follows (assuming nodes number_one and number_two already exist):

@graph.step_decorator(number_one, number_two)
def double_it(number):
    return number * 2

Simple test steps

Often there are steps that don't alter any state but only verify some aspects of the input. For example, in the following graph:

@graph.step_decorator()
def data():
    return 42

@graph.step_decorator(data)
def verify_A(data):
    return data / 2 == 21

@graph.step_decorator(data)
def verify_B(data):
    return data * 2 == 84

Here, we would get two chains: data -> verify_A and data -> verify_B. However, both the verify steps only read the passed data. By passing simple=True there will be only one chain with just the data step, and verify_A and verify_B are called as 'test steps' on the result of data:

@graph.step_decorator()
def data():
    return 42

@graph.step_decorator(data, simple=True)
def verify_A(data):
    return data / 2 == 21

@graph.step_decorator(data, simple=True)
def verify_B(data):
    return data * 2 == 84

Resources

If a step allocates some resource that should be closed after the flow is finished, or requires some other cleanup action, the cleanup argument can be used:

@graph.step_decorator(cleanup=lambda fh: fh.close())
def allocate_file():
    return TemporaryFile()

The cleanup function will be called with the result of the step, but only after all other steps in this flow have finished.