Introduction

This guideline shows how to identify test ideas from statecharts and other design structures that consist mainly of nodes connected by arcs and that show something of the possible control flows of a program. The main goal of this testing is to traverse every arc in some test. If you've never exercised an arc, why do you think it will work when a customer does?

Testing the Implementation

Consider this statechart:

Fig1: HVAC Statechart

Here's a first list of test ideas:

• Idle state receives Too Hot event
• Idle state receives Too Cool event
• Cooling/Startup state receives Compressor Running event
• Cooling/Ready state receives Fan Running event
• Cooling/Running state receives OK event
• Cooling/Running state receives Failure event
• Failure state receives Failure Cleared event
• Heating state receives OK event
• Heating state receives Failure event

These test ideas could all be exercised in a single test, or you could create several tests that each exercise a few. As with all test design, strive for a balance between the ease of implementation of many simple tests and the additional defect-finding power of complex tests. (See "test design using the list" in the Concept: Test Ideas List page.) If you have use case scenarios that describe certain paths through the statechart, you should favor tests that take those paths.

In any case, the tests should check that all actions required by the statechart actually take place. For example, is the alarm started on entry to the Failure state, then stopped upon exit?

The test should also check that the transition leads to the correct next state. That can be a difficult problem when the states are invisible from the outside. The only way to detect an incorrect state is to inject some sequence of events that leads to incorrect output. More precisely, you would need to construct a follow-on sequence of events whose externally-visible results for the correct state differ from those that the same sequence would provoke from each possible incorrect state.

In the example above, how would you know that the Failure Cleared event in the Failure state correctly led to the Idle state, instead of staying in the Failure state? You might trust that the stopping of the Alarm meant that transition had been made, but it might be better to check by lowering the temperature enough to make the heater start or raising it enough to turn on cooling. If something happens, you're more confident that the transition was correct. If nothing happens, it's likely the device stayed in the Failure state.

At the very least, determining whether the resulting state is correct complicates test design. It is often better to make the state machine explicit and make its states visible to the tests.

Other statechart constructs

Statecharts consist of more than arcs and arrows. Here is a list of statechart constructs and the effect they have on the test idea list.

Event actions, entry actions, and exit actions

These do not generate test ideas per se. Rather, the tests should check that the actions behave as specified. If the actions represent substantial programs, those programs must be tested. The test ideas for the programs might be combined with test ideas from the statechart, but it's probably more manageable to separate them. Make the decision based on the effort involved and on your suspicion that there might be interactions between events. That is, if a particular action on one arc cannot possibly share data with an action on another arc, there is no reason to exercise the two actions in the same test (as you would if they were part of the same path through a statechart test).

Guard conditions

Guard conditions are boolean expressions. The test ideas for guard conditions are derived as described in Guideline: Test Ideas for Booleans and Boundaries.

In the example above, the Too Cool transition from the Idle state is guarded with [restart time >= 5 mins]. That leads to two separate test ideas:

• Idle state receives Too Cool event when restart time is five minutes (transition taken)
• Idle state receives Too Cool event when restart time is just less than five minutes (transition blocked)

In both cases, any test that uses the test idea should check that the correct state is reached.

Internal transitions

An internal transition adds the same sort of ideas to a test idea list as an external transition does. It's merely that the next state is the same as the original state. It would be prudent to set up the test such that the state's entry and exit actions would cause an observable effect if they were incorrectly triggered.

Nested states

When constructing tests, set them up such that entry and exit events of the composite state have observable effects. You want to notice if they're skipped.

Concurrent substates

Testing of concurrency falls outside of the scope of developer testing.

Deferred events

If you suspect an event might be handled differently depending on whether it was deferred and queued rather than generated while the program was actually in the receiving state, you might test those two cases.

If the event in the receiving state has a guard condition, consider the ramifications of changes to the condition's variables between the time the event is generated and the time it is received.

If more than one state can handle a deferred event, consider testing deferral to each of the possible receiving states. Perhaps the implementation assumes that the "obvious" state will handle the event.

History states

Here is an example of a history state:

Fig2: History State Example

The transition into the history state represents three real transitions, and thus three test ideas:

• BackupUp event in Command state leads to Collecting state
• BackupUp event in Command state leads to Copying state
• BackupUp event in Command state leads to CleaningUp state

Chain states

Chain states do not seem to have any implications for test design, except that they introduce more actions that need to be checked.

Testing the Design

The preceding discussion focuses on checking whether the implementation matches the design. But the design might also be wrong. While examining the design to find test ideas, also check for two types of problems:

Missing events. The statechart shows a state's response to events that the designer anticipated could arrive in that state. It's not unknown for designers to overlook events. For example, in this statechart (repeated from the top of the page), perhaps the designer forgot that a failure can occur in the Ready substate of Cooling, not just when the fan is Running.

Fig3: HVAC Statechart

For this reason, it's wise to ask, for each state, whether any of the events that apply to other states might apply to this one. If you discover that one does, correct your design.

Incomplete or missing guard conditions. Similarly, perhaps guard conditions on one transition will suggest guard conditions on others. For example, the above statechart takes care not to restart the heater too often, but there is no such restriction on the cooling system. Should there be?

It is also possible that variables used on one guard condition will suggest that other guard conditions are too simple.

Testing Interactions

Testing each arc in a graph is by no means complete testing. For example, suppose the start state initializes a variable to 0, state Setter sets it to 5, and state Divider divides it into 100 (100/variable). If there's a path from the start state to Divider that does not pass through Setter, you have a divide-by-zero exception. If the statechart has many states, simply exercising each arc might miss that path.

Except for very simple statecharts, testing every path is infeasible. In practice, tests that are complex and correspond to use case scenarios are often sufficient. If you desire stronger tests, consider requiring a path from each state where a datum is given a value to each state that uses it.