If you haven’t put your hands on the recent James Grenning’s book “Test-Driven Development for Embedded C” yet, I highly recommend you do. Here is why.
TDD Is NOT About Testing!
First, you need to realize that TDD is not really about testing–it is about software development (Test-Driven Development). Testing is just a means to an end. The central idea behind TDD is that software, as any complex system in nature, has to evolve gradually and has to keep working throughout all the development stages.
Universal Origin of Complex Systems
This idea is of course not new and goes back all the way to the Darwin’s “On the Origin of Species by Means of Natural Selection, or the Preservation of Favoured Races in the Struggle for Life”.
More recently, in his 1977 book “Systemantics: How Systems Work and Especially How They Fail” John Gall wrote:
“A complex system that works is invariably found to have evolved from a simple system that worked…. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over, beginning with a working simple system”.
Code Needs to Struggle for Existence
The key point of TDD is to subject the software to constant “struggle for existence” to actually see if it indeed is still working and in the process weed out any undesired “mutations”. We do this by constantly subjecting the software to the tests.
Shortening Evolutionary Time
Of course, in developing software we don’t have the deep evolutionary time, so we need to accelerate the pace of software evolution. We do this by automating the testing.
Avoid DOH!
For embedded development this means avoiding the target system bottleneck (James calls it DOH!-Development On Hardware). The embedded TDD strategy is to develop embedded software on the desktop and only occasionally check it on the real embedded hardware. This means that the C/C++ compilers and tools for the desktop (such as Visual C++, MinGW, or Cygwin for Windows and GCC for Linux and macOS) are important for us.
TDD on Windows
The TDD book comes with testing frameworks (Unity and CppUTest) and plenty of example code. The code works right of the box on Linux, but I had some issues running it on Windows. In the process of learning the tools, I’ve prepared a small template for Visual C++ 2008, which is available for download from:
https://www.state-machine.com/attachments/blinky_tdd.zip
This demo assumes that you download and install the CppUTest framework (https://sourceforge.net/projects/cpputest/) and that you define the environment variable CPP_U_TEST to point to the directory where you installed CppUTest. The Visual Studio solution AllTest.sln is located in the blinky\tests directory.
I’d love to hear about your experiences with TDD in embedded programming. I’m sure I will blog more about it in the future.
7 Responses
Hi Miro!
Thanks for the book tip! I bought it from
http://pragprog.com/book/jgade/test-driven-development-for-embedded-c
22$ for a DRM free E-book, in what ever format (pdf, epub and mobi (for kindel))!!!
I have just started to read, but I have a general question for you:
According to the book you should start with a super simple solution, that just implements the solution for the last added test.
Do you use the QP from the beginning or is it after some re-factoring?
It would have been really interesting to see how TDD works on a QP program like for instance the “Fly ‘n’ Shoot” game. And when using QM having in mind not to code to much, just enough for the last test
Kind regards
Anders
In the active object computing model supported by QP active objects are the natural units for testing. So yes, I would recommend using QP from the beginning of test-driving a new QP application. I would assume that QP itself is tested and trustworthy, just as James Grenning assumes in his book that an RTOS is tested and trustworthy (see Appendix E in his book).
The first, simplest unit test would be for a completely empty active object which has only the top-most initial transition and one state that does nothing. The test setup executes the top-most initial transition (QHsm_init()). All other tests take it from there, inject events to the active object and verify that the appropriate actions are executed and that the state machine transitions to the correct state.
For most unit tests of active objects you would actually use just the QEP (event processor) component, which means that most unit tests would call the QHsm_dispatch() function. This fits very nicely in the QP design, because active objects inherit from QHsm, so calling QHsm_dispatch() directly on an active object is perfectly legal.
Also, many unit tests would call the action functions inside the state machine under test, so you would test at even lower-level than state machine.
This means that you don’t test the entire application, you test one active object at a time. Testing entire application (bu running QF_run()) is subject of integration testing and is beyond the scope of unit testing.
An example of TDD approach to test-drive development of QP application should be available some time this year. I’m still debating which unit test framework to use.
–Miro
I’ve been using TDD for my embedded work since the start of this year. I’ve found it’s made a tremendous impact on my confidence in my code and the amount of time I have to start up a debugger. I really can’t say enough about it. In addition to James’ book, I’d recommend a short monograph by Mark VanderVoord on Unity, CMock and CException. Those two books are my constant companions.
So far, I’ve used TDD only for “greenfield” work, though I want to apply it to legacy code as well. Another interesting field is BDD (Behavior Driven Development).
The one gap I find that remains is when I am still not 100% sure how the hardware actually works (usually because documentation is poor). Some folks (at Atomic Embedded) have done some work on “hardware learning” via TDD.
Matt
As you know TDD, tells you the code does what you think, not that it is right. Once you get it to do what you think, you can find out if you were right. Much of the time you will be. When you are not, then change the tests to make reality match the tests and the production code.
Learning tests can be good for that, for hardware and other things you must learn.
Glad its working for you!
James
While most software unfortunately needs a lot of evolutionary testing and fixing, and semi-automated Test-Driven Development certainly helps to speed up the process, there’s another approach to development which I like to keep pushing, because it’s getting increasingly sidelined. Unfortunately, both the alternative abbreviations I had in mind have been “taken”:
TDD = Thought-Driven Development
BDD = Brain-Driven Development
Better designs => less fixing.
However, I’m sure that thorough testing will remain a necessity for the foreseeable future!
Brian,
Thinking is a good idea. I recommend that people doing TDD do it all the time.
Miro, thanks for the review.
James
Hi Miro!
Have you tested also the Unity framwork?
in VC++ and some other embedded compiler?
I have problems with the macro’s in the unity_fixture.h
The problem is the macro construction that generates a function declaration followed by the call to it, see code snippet below, strangely it compiles in GCC
TEST_GROUP_RUNNER(LedDriver)
{
// RUN_TEST_CASE(LedDriver, LedsOffAfterCreate);
void TEST_LedDriver_LedsOffAfterCreate_run();
TEST_LedDriver_LedsOffAfterCreate_run();
// RUN_TEST_CASE(LedDriver, TurnOnLedOne);
void TEST_LedDriver_TurnOnLedOne_run(); // this is row 30 that the compiler reports an error for
TEST_LedDriver_TurnOnLedOne_run();
}
—-—– compile error in VC++ 2008 express –—-
code/unity/LedDriver/LedDriverTestRunner.c(30) : error C2143: syntax error : missing ’;’ before ‘type’
-—-—-—-—-—-—-—-—-—-—-—-