PyModel concepts and vocabulary

PyModel is an open-source model-based testing framework in Python.


In model-based testing, you code a model that generates and checks as many test cases as desired.

A model can generate behaviors in the form of traces: sequences of actions. Controllable actions represent function calls that the tester pmt can make to cause the implementation under test do something (for example, send a message). Observable actions represent events that the tester can detect in the implementation (for example, receive a message). A trace can be used as a test case: the tester uses the trace to execute and monitor the implementation during a test run. The actions in the trace contain all the information (return values, message contents) that the tester needs to check whether the implementation passes or fails the test case. A test suite is a collection of related test cases (traces).

Each trace should end in an accepting state where no work is left unfinished (for example, no open connections). A model can identify cleanup actions that make progress toward an accepting state (for example, by closing connections).

The tester connects to the implementation through a test harness called a stepper.

In offline testing, test suites are generated from the model in advance, then executed later by the tester. In on-the-fly testing, test cases are generated by the tester while the test run is executing. On-the-fly testing can execute indefinitely long nonrepeating test runs. An offline test generator or on-the-fly test runner can use an optional strategy to select among possible traces in order to improve test coverage (the default strategy is random selection).

PyModel supports three ways to write models: model programs, finite state machines (FSMs), and test suites (test suites can also act as models). PyModel can use composition to combine models into a new model called the product. A typical use of composition is to combine a model program with an FSM that represents a scenario in order to generate a test suite.

Before generating tests from a model, it is helpful to use an analyzer to visualize the model's behaviors. PyModel provides an analyzer tool pma that generates an FSM from a model, using a procedure called exploration. Each transition in the FSM represents an action, and each path through the FSM represents a trace.

The same concepts are used by NModel (but PyModel is not a translation or re-implementation of NModel). The concepts are explained in the book, Model-based Software Testing and Analysis with C#.


Revised January 2010