Basic Unity ExampleHierarchical State Machine Example

This example is loosely based on Matt Chernosky's↑ video blog post "Test-driving with mocks instead of hardware"↑. In this video blog, Matt presents a technique called the mock object↑, which, in the traditional unit testing approach, is needed to verify complex, multi-stage interactions between the CUT and the collaborator impersonated by the mock object.

Note: To better understand this QUTest example, it is highly recommended to watch Chernosky's video blog↑.

The primary purpose of this example is to show the traditional approach (with the mock object) and contrast it with the much more straightforward solution enabled by the QUTest approach. Of course, to make comparisons meaningful, both approaches test the same CUT.

Note: Because of the different design philosophy of QUTest, which is not based on xUnit, the classic "mock object" is actually not needed and can be replaced with a simpler "spy" test double↑. To dynamically alter the behavior of this "spy" test double, this example applies QUTest Test Probes. Test Probes can be changed interactively from the test script.

Remarks: Since the CUT is in C, the Mock Unity example is only available in the QP/C framework↑ (and is not available in the QP/C++ framework↑).

This simple example runs QUTest tests on the host (Windows, Linux, or macOS) for both Unity and QUTest. The QUTest version also runs on embedded boards (STM32 NUCLEO-U545RE, NUCLEO-L053R8 from ST, EFM32 Pearl-Gecko board from Silicon Labs, TivaC LaunchPad from Texas Instruments, and many others). The instructions for building and running the code on the embedded boards are located at the end of this lesson.

Code Under Test (CUT)

The CUT in this example implements a simple bar of 5 LEDs that displays a percentage, as shown in the animation below. The "LedBar device" interacts with the discrete LEDs by turning them on and off to achieve the desired effect. The main objective of this example is to demonstrate how to verify the collaboration with the discrete LEDs, even without the actual hardware.

Bar of 5 LEDs displaying the percentage

The LedBar CUT is located in the file LedBar.c in the directory qpc\examples\qutest\unity_ledbar\src. The CUT consists of just one function LedBar_setPercent(), which turns the individual LEDs on and off such that they display the desired percent parameter. The function LedBar_setPercent() returns the total power consumption (in microwatts) of all LEDs that are turned on.

#include <stdint.h>
#include "LedBar.h"
#include "Led.h"
 
#include "qassert.h" /* for embedded systems-friendly assertions */
 
Q_DEFINE_THIS_MODULE("LedBar")
 
/*
 Example sequence diagram for the LedBar_setPercent() implementation:
 
 +----------+       +-----+
 |  LedBar  |       | LED |
 +----------+       +-----+
       |               |
      +-+ Led_on(0)    |
      | |------------->|
      | |              |
      | | Led_on(1)    |
      | |------------->|
      | |              |
      | | Led_on(2)    |
      | |------------->|
      | |              |
      | | Led_off(3)   |
      | |------------->|
      | |              |
      | | Led_off(4)   |
      | |------------->|
      | |              |
      +-+              |
       |               |
*/
uint32_t LedBar_setPercent(uint8_t const percent) {
    uint8_t const n_on = (uint8_t)((percent * MAX_LED) / 100U);
 
    /*! @pre percent must not exceed 100 */
    Q_REQUIRE_ID(100, n_on <= MAX_LED);
 
    uint32_t p = 0U; /* power draw in [uW] */
    uint8_t i;
    for (i = 0U; i < n_on; ++i) {
        p += Led_on(i);
    }
    for (i = n_on; i < MAX_LED; ++i) {
        Led_off(i);
    }
    return p;
}

file LedBar.c

The low-level interface to the LEDs is defined below (Led.h). An index parameter selects the individual LEDs. Each LED can be turned on by means of the LED_on() function. The LED_on() function returns a value, which corresponds to the power consumption of this LED while on. The LED can be turned off by the LED_off() function, which returns void.

#ifndef LED_H
#define LED_H
 
enum { MAX_LED = 5 };
 
/* turns a given LED on and returns the power drawn by it in uW */
uint32_t Led_on(uint8_t index);
 
/* turns a given LED off */
void Led_off(uint8_t index);
 
#endif /* LED_H */

file Led.h

The diagram below shows the relationships between the CUT and the test code: Unity case on the left and QUTest case on the right. In both cases, the LedBar CUT interacts with the hardware through the Led.h interface. The explanation section below the diagram clarifies the interesting elements of the diagram.

Components of the Unity Mock test: Unity left and QUTest right

Center: Code Under Test

1: The LedBar.c file contains the Code Under Test (CUT).
2: The CUT interacts with the hardware through the Led.h interface. This interface abstracts the LEDs and is all that the CUT needs to "know" about the hardware.

Left: Unity testing framework (traditional approach)

3: The TestLedBar.c file implements the Unity test fixture, which calls the CUT and verifies that it performs as expected.

NOTE: A Unity test fixture for a mock requires a special "test runner", which initializes and cleans up after the mock. This "test runner" must typically be generated automatically, as it is too complex to code manually.
4: The Unity test fixture uses the MockLed.c implementation of the Led.h interface. The mock object implemented in MockLed.c needs to be "programmed" for each expected scenario before calling the CUT. Thus, the structure of conventional tests with a mock is "backwards", meaning that you first specify the expectations and the test ends with a call to the CUT.

NOTE: This mock-object must typically be generated automatically, as it is too complex to code manually. Here, the mock-object was generated by means of the CMock tool↑, which is included with Unity, but requires installation of the ruby programming language↑.

Right: QUTest testing framework (simplified approach)

5: The test_LedBar.c file implements the QUTest test fixture, which calls the CUT, but it does NOT check whether it performs as expected.
6: The QUTest test fixture uses the spy_Led.c implementation of the Led.h interface. This spy test double is much simpler than the mock-object used by Unity, so it can easily be coded manually.
7: The QUTest test is driven by the test_LedBar.py test script. This test script sends commands to the test_LedBar.c fixture and verifies the generated output against the expectations of the test.

Running the Test with Unity

The complete code for the mock Unity example is provided in the QP/C framework directory C:\qp\qpc\examples\qutest\unity_ledbar\test_unity. To run the mock Unity test (on Windows), open a command prompt and type:

cd C:\qp\qpc\examples\qutest\unity_ledbar\test_unity
make

Note: The provided Makefile will also work on Linux and macOS. The only difference from Windows is that you open a terminal window and change the directory to ~/qp/qpc/examples/qutest/unity_ledbar/test.

This will build the test fixture as a host executable, and then it will run it. The screenshot below shows the output produced by the make command.

Unity mock test build and run with Unity

Running the Test with QUTest

The complete code for the mock Unity example is provided in the QP/C framework, directory C:\qp\qpc\examples\qutest\unity_ledbar\test. To run the mock test (on Windows), open a command prompt and type:

qspy

This will start the QSPY host utility with the TCP/IP connection to the Target.

Next, open a second command prompt window and type:

cd C:\qp\qpc\examples\qutest\unity_ledbar\test
make

Note: The provided Makefile will also work on Linux and macOS. The only difference from Windows is that you open a terminal window and change the directory to ~/qp/qpc/examples/qutest/unity_ledbar/qutest.

This will build the test fixture as a host executable, and then it will run the test script (in Python). The screen shot below shows the output produced in these two command-prompt windows.

Unity mock test build and run with QUTest (right) and QSPY output (left)

Spy Object (Mock Replacement)

The following listing shows the "spy-object" (file spy_led.c) test-double for the low-level LED module. The explanation section following the listing clarifies the interesting lines of code (lines starting with [xx] labels).

 [1] #include "qpc.h"  /* QUTest interface */
 [2] #include "Led.h"  /* original interface */
 
     //Q_DEFINE_THIS_FILE
 
     enum {
 [3]     LED_MOD = QS_USER1 /* QS app-specific record for the LED module */
     };
 
 [4] static uint32_t led_power[MAX_LED] = {
         10, 20, 10, 20, 10
     };
 
     /*--------------------------------------------------------------------------*/
 [5] void Led_DICTIONARY(void) {
         QS_FUN_DICTIONARY(&Led_on);
         QS_FUN_DICTIONARY(&Led_off);
         QS_USR_DICTIONARY(LED_MOD);
         QS_OBJ_DICTIONARY(&led_power);
     }
 
     /*--------------------------------------------------------------------------*/
     /* turns a given LED off */
 [6] void Led_off(uint8_t index) {
         QS_BEGIN_ID(LED_MOD, (void *)0) /* user-specific record */
             QS_FUN(&Led_off); /* function called */
             QS_U8 (0, index); /* parameter */
         QS_END()
     }
 
     /* turns a given LED on and returns the power drawn by it in uW */
 [7] uint32_t Led_on(uint8_t index) {
 [8]     uint32_t ret = led_power[index]; /* assume default power */
 [9]     QS_TEST_PROBE_DEF(&Led_on)
 
         /* tweak the returned power draw from the test probe */
[10]     QS_TEST_PROBE(
[11]         ret = (uint32_t)qs_tp_;
         )
 
[12]     QS_BEGIN_ID(LED_MOD, (void *)0) /* user-specific record */
             QS_FUN(&Led_on);  /* function called */
             QS_U32(0, ret);   /* value returned */
             QS_U8 (0, index); /* parameter */
         QS_END()
 
[13]     return ret;
     }

1

The "qpc.h" header file includes the QP/C framework, which contains the QUTest interface. Typically, you need to include this header file in QUTest test doubles.

2

The "Led.h" header file specifies the interface to the code being impersonated by this "spy-object". The whole purpose of the spy-object is to implement this interface such that the CUT can call it as its collaborator.

NOTE: The CUT does not "know" that it is collaborating with a test double (the spy-object in this case).

3

This enumeration lists the application-specific QS trace records that will be produced by the fake LED operations. The enumeration starts with #QS_USER1 offset, which is the second group of user-specific records. The first group (#QS_USER0 offset) is used by the main test fixture.

4

The led_power[] array contains the default power ratings for the individual LEDs in the LED-bar. These values will be returned from the fake Led_on() implementation (if not overridden by the "Test Probe").

5

The Led_DICTIONARY() function produces the dictionaries for the spy-object. This function is not part of the real LED interface (see Led.h in the src directory), but is needed only for testing.

6

This is a fake Led_off() implementation, which generates an application-specific QS trace record to report the call and the parameter to QSPY.

7

This is a fake Led_on() implementation.

8

The ret variable will be the power rating for this LED returned from this function. The variable is initialized from the led_power[] array, which contains the default power rating for the LEDs.

9

The macro QS_TEST_PROBE_DEF() defines a Test Probe for this function (notice the &Led_on function-pointer parameter). This Test Probe retrieves a value set for this function from the test script. (If no value has been set, the QS_TEST_PROBE_DEF() retrieves value 0).

10

The QS_TEST_PROBE() macro executes the enclosed snippet of code only if the Test Probe (defined at label [9]) is not zero.

11

If the Test Probe is not zero (it has been set from the test script), the ret value is updated from the value of the Test Probe qs_tp_.

NOTE: This demonstrates how to program a return value in a spy-object. This corresponds directly to the same capability of traditional mock-objects.

NOTE: Test Probe is just one way in which a pre-programmed value can be returned from a fake function. This option is illustrated in test script [25]. Another way in QUTest is to use the poke() test command to alter the values in the led_power[] array. This option is illustrated in test script [21].

12: An application-specific QS trace record is generated to report the call, the parameter, and the return value to QSPY.
13: The ret value is returned to the caller (CUT).

Test Fixture

The following listing shows the test fixture for the LedBar tests (file test_ledbar.c). The explanation section following the listing clarifies the interesting lines of code (lines starting with [xx] labels).

 [1] #include "qpc.h"  /* QUTest interface */
 [2] #include "LedBar.h" /* CUT interface */
 
     Q_DEFINE_THIS_FILE
 
     /*--------------------------------------------------------------------------*/
 [3] void Led_DICTIONARY(void); /* dictionaries for the Led "spy " test double */
 
     /*--------------------------------------------------------------------------*/
     int main(int argc, char *argv[]) {
 
 [4]     QF_init();  /* initialize the framework */
 
         /* initialize the QS software tracing */
 [5]     Q_ALLEGE(QS_INIT(argc > 1 ? argv[1] : (void *)0));
 
         /* dictionaries... */
 [6]     Led_DICTIONARY();
 [7]     QS_FUN_DICTIONARY(&LedBar_setPercent);
 
         /* filter setup... */
 [8]     QS_FILTER_ON(QS_ALL_RECORDS);
 
 [9]     return QF_run(); /* run the tests */
     }
 
     /*--------------------------------------------------------------------------*/
     void QS_onTestSetup(void) {
     }
     /*..........................................................................*/
     void QS_onTestTeardown(void) {
     }
 
     /*..........................................................................*/
     void QS_onCommand(uint8_t cmdId,
                       uint32_t param1, uint32_t param2, uint32_t param3)
     {
         switch (cmdId) {
[10]         case 0: { /* call the CUT: LedBar_setPercent */
[11]             uint32_t ret = LedBar_setPercent((uint8_t)param1);
[12]             QS_BEGIN_ID(QS_USER + cmdId, (void *)0) /* user-specific record */
[13]                 QS_FUN(&LedBar_setPercent); /* function called */
[14]                 QS_U32(0, ret); /* value returned */
[15]                 QS_U8 (0, (uint8_t)param1); /* parameter */
                 QS_END()
                 break;
             }
             default:
                 break;
         }
 
         /* unused parameters... */
         //(void)param1;
         (void)param2;
         (void)param3;
     }
     /*..........................................................................*/
     /* host callback function to "massage" the event, if necessary */
     void QS_onTestEvt(QEvt *e) {
         (void)e;
     #ifdef Q_HOST  /* is this test compiled for a desktop Host computer? */
     #else /* this test is compiled for an embedded Target system */
     #endif
     }
     /*..........................................................................*/
     /*! callback function to output the posted QP events (not used here) */
     void QS_onTestPost(void const *sender, QActive *recipient,
                        QEvt const *e, bool status)
     {
         (void)sender;
         (void)recipient;
         (void)e;
         (void)status;
     }

1: The "qpc.h" header file includes the QP/C framework, which contains the QUTest interface. Typically, you need to include this header file in QUTest test doubles.

NOTE: for test fixtures based on the QP/C++ framework, you need to include the "qpcpp.h" header file.
2: The "LedBar.h" header file specifies the interface to the CUT.
3: The Led_DICTIONARY() function prototype is declared directly in the test fixture. This is the only extension from the "Led.h" interface implemented in the spy-object.
4: As usual, the main() function of the test fixture initializes the QF framework.
5: As usual, the main() function of the test fixture initializes the QS software tracing.
6: The call to the Led_DICTIONARY() function outputs the QS dictionaries for the spy-object.
7: The QS dictionary for the LedBar_setPercentn() CUT is produced as well.
8: The QS global filter is set up to output all QS records.
9: The QF_run() function enters the event loop to wait for commands from the test script.

Test Script

The following listing shows the test script for the LedBar tests (file test_LedBar.py). The explanation section following the listing clarifies the interesting lines of code (lines starting with [xx] labels).

Note: The following test script performs the same tests as the TestLedBar.c Unity test fixture. Compared to the mock-object approach, however, the QUTest test script has a more intuitive structure, in which a command triggering CUT is followed by the expectations. In contrast, the traditional Unity tests with mock-object were structured "backwards" (the expectations preceded the call to CUT).

 [1] # preamble...
 
     # tests...
 [2] test("LedBar 0% all off")
 [3] command(0, 0)
 [4] expect("@timestamp LED_MOD Led_off 0")
     expect("@timestamp LED_MOD Led_off 1")
     expect("@timestamp LED_MOD Led_off 2")
     expect("@timestamp LED_MOD Led_off 3")
     expect("@timestamp LED_MOD Led_off 4")
 [5] expect("@timestamp USER+000 LedBar_setPercent 0 0")
 [6] expect("@timestamp Trg-Done QS_RX_COMMAND")
 
 [7] test("LedBar 100% all on", NORESET)
 [8] command(0, 100)
 [9] expect("@timestamp LED_MOD Led_on 10 0")
     expect("@timestamp LED_MOD Led_on 20 1")
     expect("@timestamp LED_MOD Led_on 10 2")
     expect("@timestamp LED_MOD Led_on 20 3")
     expect("@timestamp LED_MOD Led_on 10 4")
[10] expect("@timestamp USER+000 LedBar_setPercent 70 100")
[11] expect("@timestamp Trg-Done QS_RX_COMMAND")
 
[12] test("LedBar 19% all off", NORESET)
[13] command(0, 19)
[14] expect("@timestamp LED_MOD Led_off 0")
     expect("@timestamp LED_MOD Led_off 1")
     expect("@timestamp LED_MOD Led_off 2")
     expect("@timestamp LED_MOD Led_off 3")
     expect("@timestamp LED_MOD Led_off 4")
[15] expect("@timestamp USER+000 LedBar_setPercent 0 19")
     expect("@timestamp Trg-Done QS_RX_COMMAND")
 
[16] test("LedBar 20% one on", NORESET)
[17] command(0, 20)
     expect("@timestamp LED_MOD Led_on 10 0")
     expect("@timestamp LED_MOD Led_off 1")
     expect("@timestamp LED_MOD Led_off 2")
     expect("@timestamp LED_MOD Led_off 3")
     expect("@timestamp LED_MOD Led_off 4")
[18] expect("@timestamp USER+000 LedBar_setPercent 10 20")
     expect("@timestamp Trg-Done QS_RX_COMMAND")
 
[19] test("LedBar 50% two on", NORESET)
[20] current_obj(OBJ_AP, "led_power")
[21] poke(0, 4, pack("<LL", 25, 15))
[22] command(0, 50)
     expect("@timestamp LED_MOD Led_on 25 0")
     expect("@timestamp LED_MOD Led_on 15 1")
     expect("@timestamp LED_MOD Led_off 2")
     expect("@timestamp LED_MOD Led_off 3")
     expect("@timestamp LED_MOD Led_off 4")
[23] expect("@timestamp USER+000 LedBar_setPercent 40 50")
     expect("@timestamp Trg-Done QS_RX_COMMAND")
 
[24] test("LedBar 99% four on", NORESET)
[25] probe("Led_on", 17)
[26] probe("Led_on", 13)
[27] command(0, 99)
[28] expect("@timestamp TstProbe Fun=Led_on,Data=17")
[29] expect("@timestamp LED_MOD Led_on 17 0")
[30] expect("@timestamp TstProbe Fun=Led_on,Data=13")
[31] expect("@timestamp LED_MOD Led_on 13 1")
     expect("@timestamp LED_MOD Led_on 10 2")
     expect("@timestamp LED_MOD Led_on 20 3")
     expect("@timestamp LED_MOD Led_off 4")
[32] expect("@timestamp USER+000 LedBar_setPercent 60 99")
     expect("@timestamp Trg-Done QS_RX_COMMAND")

1: The preamble is empty, because no special actions are needed on reset/setup/teardown.

Test: "LedBar 0% all off" checks that all LEDs are turned off to display 0%

2: The test() command starts the test.
3: The command(0, 0) calls the LedBar_setPercent() CUT with the 0 percent argument.
4: This expect() command verifies the output produced by calling the LED_off() function from the spy-object.
5: This expect() command verifies the output produced by calling the LedBar_setPercent() function from the CUT. Note that the returned total power consumption is zero.
6: This expect() command verifies that all output from the original command(0, 0) has been produced.

Test: "LedBar 100% all on" checks that all LEDs are turned on to display 100%

8: The test() command starts the test.
9: The command(0, 100) calls the LedBar_setPercent() CUT with the 100 percent argument.
10: This expect() command verifies the output produced by calling the LedBar_setPercent() function from the CUT. Note that the returned total power consumption is 70.
11: This expect() command verifies that all output from the original command(0, 100) has been produced.

Test: "LedBar 19% all off" checks that all LEDs are turned off to display 19%

12: The test() command starts the test (NOTE: this is a NORESET test).
13: The command(0, 19) calls the LedBar_setPercent() CUT with the 19 percent argument.
14: This expect() command verifies the output produced by calling the LED_off() function from the spy-object.
15: This expect() command verifies the output produced by calling the LedBar_setPercent() function from the CUT. Note that the returned total power consumption is 0.

Test: "LedBar 20% one on" checks that one LED is turned on to display 20%

16: The test() command starts the test (NOTE: this is a NORESET test).
17: The command(0, 20) calls the LedBar_setPercent() CUT with the 20 percent argument.
18: This expect() command verifies the output produced by calling the LedBar_setPercent() function from the CUT. Note that the returned total power consumption is 10.

Test: "LedBar 50% two on" checks that two LEDs are turned on to display 50%

19: The test() command starts the test (NOTE: this is a NORESET test).
20: The current_obj(OBJ_AP, led_power")" command sets the "current application object" to "led_power" (see spy-object[4]).
21: The poke(0, 4, pack(<LL", 25, 15))" command pokes the given data into the "current application object". Specifically, this poke() command pokes the data starting from offset 0, data-size 4, binary-data pack("<LL", 25, 15).
22: The command(0, 50) calls the LedBar_setPercent() CUT with the 50 percent argument.
23: This expect() command verifies the output produced by calling the LedBar_setPercent() function from the CUT. Note that the returned total power consumption is 40, which is due to poking the data into the led_power array.

Test: "LedBar 99% two on" checks that two LEDs are turned on to display 99%

24: The test() command starts the test (NOTE: this is a NORESET test).
25: The probe(Led_on", 17)" command sends the test-probe value 17 for function Led_on
26: The probe(Led_on", 13)" command sends the test-probe another value 13 for function Led_on
27: The command(0, 99) calls the LedBar_setPercent() CUT with the 99 percent argument.
28: This expect() command verifies the test-probe sent at step [25] has been retrieved.
29: This expect() command verifies the function Led_on has been called, and that it returned 17.
30: This expect() command verifies the test-probe sent at step [26] has been retrieved.
31: This expect() command verifies the function Led_on has been called, and that it returned 13.
32: This expect() command verifies the output produced by calling the LedBar_setPercent() function from the CUT. Note that the returned total power consumption is 60, which is due to using test-probes to alter the return values returned from Led_on().

Basic Unity ExampleHierarchical State Machine Example