Basic Unity Example

QUTest™ TutorialUnity Mock Example

This example is based on the simple example_1 that ships with the Unity unit testing framework. This simplest example of Unity has nothing to do with the QP frameworks. The purpose is to illustrate that QUTest™ can be used with generic C code and to compare QUTest™ with Unity.

Remarks

This simple example runs QUTest tests on the host (Windows, Linux, or macOS) for both Unity and QUTest. The QUTest version also runs on embedded boards (NUCLEO-L053R8 from ST, EFM32 Pearl-Gecko board from Silicon Labs and TivaC LaunchPad from Texas Instruments and ). The instructions for building and running the code on the embedded boards are located at the end of this lesson.

Code Under Test (CUT)

Note

Because it uses pure C code, the Basic Unity example is only available in the QP/C framework, and is not available in the QP/C++ framework.

The CUT in this example is the file my_strlen.c located in the directory C:\qp\qpc\examples\qutest\unity_strlen\src:

#include "my_strlen.h"
 
uint16_t my_strlen(char const *str) {
    uint16_t len = 0U;
    for (; *str != '\0'; ++str) {
        ++len;
    }
    return len;
}

Running the Test with Unity

The complete code for the basic Unity example is provided in the QP/C framework directory C:\qp\qpc\examples\qutest\unity_strlen\test. To run the basic Unity test (on Windows), open a command-prompt and type:

cd C:\qp\qpc\examples\qutest\unity_strlen\test
make

Note

The provided Makefile will also work on Linux and macOS. The only difference from Windows is that you open a terminal window and change the directory to ~/qp/qpc/examples/qutest/unity_strlen/test.

This will build the test fixture as a host executable and then it will run it. The screen shot below shows the output produced from the make command.

Unity example_1 test build and run with Unity

Running the Test with QUTest

The complete code for the basic Unity example is provided in the QP/C framework directory C:\qp\qpc\examples\qutest\unity_strlen\qutest. To run the basic test (on Windows), open a command prompt and type:

qspy

This will start the QSPY host utility with the TCP/IP connection to the Target.

Next, open a second command prompt window and type:

cd C:\qp\qpc\examples\qutest\unity_strlen\qutest
make

Note

The provided Makefile will also work on Linux and MacOS. The only difference from Windows is that you open a terminal window and change the directory to ~/qp/qpc/examples/qutest/unity_strlen/qutest.

This will build the test fixture as a host executable and then it will run the test script (in Python). The screen shot below shows the output produced in these two command-prompt windows.

Unity example_1 test build and run with QUTest (right) and QSPY output (left)

Test Fixture

The job of a test fixture is to exercise the CUT (my_strlen.c in this case) and report the results back to the QSPY host application. Note that a test fixture in QUTest™ is not supposed to perform any checks whether the CUT operates "correctly". Instead, your test fixture should only provide facilities to thoroughly exercise the CUT remotely from the test script(s). A properly written test fixture can typically be used for many tests (implemented in multiple test scripts).

Remarks

Coming up with a "good" test fixture requires some practice, but when you study the examples in this Tutorial, you will see some instances of flexible test fixtures that allow you to to run a wide variety of tests on them.

The following listing shows the complete QUTest test fixture for the basic tests (file test_strlen.c in the directory C:\qp\qpc\examples\qutest\unity_strlen\qutest). The explanation section following the listing clarifies the interesting lines of code (lines starting with [xx] labels).

 [1] #include "qpc.h"  /* QUTest interface */
 [2] #include "my_strlen.h" /* CUT interface */
 
 [3] Q_DEFINE_THIS_FILE
 
     /*--------------------------------------------------------------------------*/
 [4]   static char string[128];
 
     /*--------------------------------------------------------------------------*/
 [5] int main(int argc, char *argv[]) {
 
 [6]     QF_init();  /* initialize the framework */
 
         /* initialize the QS software tracing */
 [7]     Q_ALLEGE(QS_INIT(argc > 1 ? argv[1] : (void *)0));
 
         /* dictionaries... */
 [8]     QS_FUN_DICTIONARY(&my_strlen);
 [9]     QS_OBJ_DICTIONARY(string);
 
         /* filter setup... */
[10]     QS_GLB_FILTER(QS_ALL_RECORDS);
[11]     QS_setCurrObj(AP_OBJ, string);
 
[12]     return QF_run(); /* run the tests */
     }
 
     /*--------------------------------------------------------------------------*/
[13] void QS_onTestSetup(void) {
     }
     /*..........................................................................*/
[14] void QS_onTestTeardown(void) {
     }
 
     /*..........................................................................*/
[15] void QS_onCommand(uint8_t cmdId,
                       uint32_t param1, uint32_t param2, uint32_t param3)
     {
         switch (cmdId) {
             case 0: { /* call the CUT: my_strlen(string) */
[16]             uint16_t ret = my_strlen(string);
[17]             QS_BEGIN_ID(QS_USER + cmdId, 0U) /* app-specific record */
[18]                 QS_FUN(&my_strlen); /* function called */
[19]                 QS_U16(0, ret); /* value returned */
[20]                 QS_STR(string); /* the current string */
[21]             QS_END()
                 break;
             }
             default:
                 break;
         }
 
         /* unused parameters... */
         (void)param1;
         (void)param2;
         (void)param3;
     }
     /*..........................................................................*/
     /* host callback function to "massage" the event, if necessary */
[22] void QS_onTestEvt(QEvt *e) {
         (void)e;
     #ifdef Q_HOST  /* is this test compiled for a desktop Host computer? */
     #else /* this test is compiled for an embedded Target system */
     #endif
     }
     /*..........................................................................*/
     /*! callback function to output the posted QP events (not used here) */
[23] void QS_onTestPost(void const *sender, QActive *recipient,
                        QEvt const *e, bool status)
     {
         (void)sender;
         (void)recipient;
         (void)e;
         (void)status;
     }

1

The "qpc.h" header file contains the QP/C framework API, which includes the QUTest interface. Typically, you need to include this header file in QUTest test doubles.

‍NOTE: for test fixtures based on the QP/C++ framework, you need to include the "qpcpp.h" header file.

2

You also need to include the interface to the CUT, which is my_strlen.h in this case.

3

The macro Q_DEFINE_THIS_FILE is needed for "DbC assertions" (they have nothing to do with "test assertions"). Later in this file, a "DbC assertion" is used to guard against failure in the initialization of the QS target-resident component (see step [7]).

4

The variable string is used to control the parameter passed into the CUT.

5

A QUTest test fixture code needs the main() function, which has the usual structure of a QP/C application (and in fact in the more advanced tests it can be the same function as used by the actual QP/C application). But here, it contains the bare minimum function calls, as described below.

6

The main() function must start with calling QF_init() to initialize the QP framework.

7

Next, you need to initialize the QS target-resident component (QS_INIT()). This macro is wrapped with the Q_ALLEGE() assertion, which will fire if the QS initialization fails (in which case continuation of the test makes no sense).

8-9

Next, you produce QS dictionaries for all functions you wish to test as well as objects you might need to inspect. NOTE: you need to do this so that the test scripts can refer to the functions and objects by the same symbolic names as the CUT/test-fixture.

10

The QS_GLB_FILTER() macro sets the global filter in the Target. Here all output is enabled (QS_ALL_RECORDS).

11

The QS_setCurrObj(AP_OBJ) function sets the "current Application Object" in the Target. Subsequently, commands sent to the Target from the test script will pertain to that "current Application Object".

12

Finally, at the end of main() you need to call QF_run() to run the tests.

13

The callback function QS_onTestSetup() allows you to include code that will be run at the beginning of each test. Here this simple CUT does not need any setup, but you still need to provide (an empty) implementation to satisfy the linker.

14

The callback function QS_onTestTeardown() allows you to include code that will be run at the end of each test. Here this simple CUT does not need any teardown, but you still need to provide (an empty) implementation to satisfy the linker.

15

The callback function QS_onCommand() allows you to remotely execute commands inside the Target. Here is where you execute the CUT and report results back to QSPY.

16

The command with cmdId==0 will be used to call the my_strlen() CUT. NOTE: You can use other cmdIds to call other pieces of CUT or to provide different variants of calling the same CUT, as you see fit. Much of the art of writing test fixtures lies in constructing flexible remote commands that exercise your CUT.

17

The QS_BEGIN() macro starts the application-specific trace record that will report results of calling the my_strlen() CUT to the test script.

18

The QS_FUN() macro sends the address of the function to the test script. This address will be converted to the name of the function, because the dictionary for this function has been generated in step 8.

19

The QS_U16() macro sends a 16-bit unsigned integer (uint16_t) to the test script. Here you output the return value from my_strlen().

20

The QS_STR() macro sends a zero-terminated string to the test script. Here you output the argument passed to my_strlen().

21

The QS_END() macro ends the application-specific trace record.

22

The QS_onTestEvt() callback function is not used in this test, but needs to be provided to satisfy the linker.

23

The QS_onTestPost() callback function is not used in this test, but needs to be provided to satisfy the linker.

Test Script

A test script contains a group of related tests (a "test group"). The basic job of these tests is to send inputs to the test fixture running in the Target and to compare the QSPY textual output produced with the expectations of the test. The following listing shows the test script (Python) for the Unity basic tests (file test_strlen.py). The explanation section following the listing clarifies the interesting lines of code (lines starting with [xx] labels).

Remarks

Even though a test script uses Python under the hood, you don't need to be a Python expert to write effective test scripts. The actual set of commands that you use and need to know about forms a small Domain Specific Language (DSL) for unit testing, which just happens to be implemented with Python as a command interpreter.

Note

The following test script performs the same tests as the Unity test fixture test_strlen.c in the directory qpc\examples\qutest\unity_strlen\qutest.

 [1] # test-script for QUTest unit testing harness
     # see https://www.state-machine.com/qtools/qutest.html
 
     # preambe...
 [2] #def on_reset():
 [3] #    expect_run()
 [4] #    current_obj(OBJ_AP,"string")
 
     # tests...
 [5] test("my_strlen 0 for_empty_string")
 [6] poke(0,1,b"\0")
 [7] command(0)
 [8] expect("@timestamp USER+000 my_strlen 0 ''")
 [9] expect("@timestamp Trg-Done QS_RX_COMMAND")
 
 
[10] test("my_strlen n for various strings", NORESET)
     poke(0,1,b"Foo\0")
     command(0)
     expect("@timestamp USER+000 my_strlen 3 Foo")
     expect("@timestamp Trg-Done QS_RX_COMMAND")
     poke(0,1,b" Foo\0")
     command(0)
     expect("@timestamp USER+000 my_strlen 4  Foo")
     expect("@timestamp Trg-Done QS_RX_COMMAND")
 
 
     test("my_strlen n for unprintable chars", NORESET)
     poke(0,1,b"\tFoo\0")
     command(0)
     expect("@timestamp USER+000 my_strlen 4 \tFoo")
     expect("@timestamp Trg-Done QS_RX_COMMAND")
     poke(0,1,b"\a\b\n\0")
     command(0)
     expect("@timestamp USER+000 my_strlen 3 \a\b\n")
     expect("@timestamp Trg-Done QS_RX_COMMAND")

1

Lines starting with a pound sign ('#') or empty lines are comments which are ignored by QUTest.

"preamble" defines the on-reset code common to all tests in the group:

2

The function on_reset() is executed after Target reset.

3

The expect_run() directive consumes the QF-RUN trace record produced after Target reset.

4

The current_obj() command sets the "current object" of the application-specific kind (OBJ_AP) in the Target. Subsequent commands (such as poke() in the next step) will act on this "current object".

"tests" performs various tests:

5

The test() directive starts a test and gives it a name (in double quotes). The name of the test will be displayed as the test is executed and should be a quick reminder about the objective of this test. This test command also resets the Target, which brings the Target into a well-defined initial state and produces the QS dictionary records (see test-fixture[12]).

‍NOTE: The ability to perform the full Target reset is a unique feature of QUTest. Other unit testing frameworks, including Unity and CppUTest, don't reset the Target. They merely call the test setup() / tearDown() functions at the beginning and end of the test, respectively. QUTest also calls onReset() / onSetup() / onTeardown(), but obviously the full Target reset is a much better guarantee that the Target starts in exactly the same state.

6

The poke() directive pokes the specified "Application Current Object" starting with the specified offset from the beginning of the object in memory (which here is 0) with the data elements of size 4 (the second argument) with the data provided in the third argument pack() "pack("<L", 0x5A5A)" into the previously established "Application Current Object".

7

The command() directive causes the invocation of the QS_onCommand() callback inside the Target. The argument 0 is the cmdId parameter (see test-fixture[16]). NOTE: The first parameter of command() "command" here is just a number (0), but it is also possible to use a symbolic name for the first parameter. This symbolic name will be looked up in the user dictionary (QS_USR_DICTIONARY()).

8

The expect() command represents an expectation of this "test assertion". This is the expected output generated by the command(0) command from the previous step. You need to consult the test fixture to determine what you should expect in this case. NOTE: the expected string starts with a number 0000000001, which is the Target Time-Stamp. In QUTest, the "timestamp" simply counts all the QS trace records produced, so that you know that no entries have been lost. In the later tests you will see how you can count the steps automatically with the @timestamp placeholder.

8

The test finishes with the expectation for the Trg-Done QS_RX_COMMAND trace record, which means that all output generated by command() has been generated.

9

This expectation consumes the "Trg-Done QS_RX_COMMAND" trace record, which ensures that all output for this command has been produced.

Attention

The QSPY host application limits the QS dictionaries to the first 63 characters. This means that longer names of functions or objects will be truncated to this limit. Please keep this limit in mind when choosing various names in your application.

Running the Test on Embedded Targets

As mentioned in the initial description of this example, the directory C:\qp\qpc\examples\qutest\unity_strlen\qutest contains makefiles to build the code and run the tests on the embedded boards (TivaC LaunchPad from Texas Instruments and EFM32 Pearl-Gecko board from Silicon Labs). Both these boards open a virtual COM port on the machine they are attached to via the USB cable. This virtual COM port provides an ideal connection for the QS communication with the QSPY host utility.

Targets for running QUTests. From the left: host computer, TivaC LaunchPad and EFM32 Pearl-Gecko

For example, to test the EFM32 Pearl-Gecko board (ARM Cortex-M4), open a command prompt (on Windows) and type:

qspy -c COM6

This will start the QSPY host utility with com-port connection to the embedded board. Of course, you need to adjust the serial port number to the actual number of the virtual COM port on your machine.

Next, open a second command prompt window and type:

cd C:\qp\qpc\examples\qutest\unity_strlen\qutest
make -f make_efm32

The rest of the test is then exactly the same as with the host executable described above, except that the test fixture runs on the embedded board.

To run the tests on the TivaC LaunchPad (TM4C123 board), you use the make_tm4c123 in the last step.

QUTest™ TutorialUnity Mock Example