Hello and welcome to the Modern Embedded Systems Programming course. I'm Miro Samek, and in this lesson, I continue exploring the different ways to execute event-driven Active Objects. Today's focus is on extending modern event-driven programming--with Active Objects and hierarchical state machines--to Linux, and more generally, POSIX operating systems, which include any OS with a POSIX-compatibility layer (Integrity, QNX, macOS, and many others)

Just like the previous lessons in this segment, this lesson also brings together the major modern embedded-programming themes of the course: event-driven programming, object-oriented design, and the Active Object model of computation. This time, however, we apply all of these concepts to general-purpose operating systems such as Linux.

These systems provide even more reasons to apply the event-driven Active Object model and advanced state machines. First, embedded Linux is a very popular embedded target in its own right.

But in addition, desktop Linux or macOS are also the host operating systems used to develop other embedded systems. In that case, it is immensely advantageous to develop and run the same event-driven application code on both the host and the embedded targets.

So, the plan for today—just as in the previous lessons in this segment—is to begin by building and running the example applications on desktop Linux, and then on the embedded Linux target, which in this case is the wireless Raspberry Pi Zero. After that, we’ll examine how the Active Object framework integrates with POSIX.

As before, to keep the discussion concrete and anchored in real practice, I’ll be using the QP Active Object framework, one of the most mature and widely deployed frameworks of its kind. The QP frameworks have supported POSIX for more than two decades.

For this lesson, I’ll be working with the C++ edition because it tends to be more popular in POSIX-based applications. However, everything you see today applies equally to QP/C.

Since all code examples for today are already included in the QP/C++ framework, the code download for this lesson 57 will be the standard, open-source QP distribution, which you can obtain from the state-machine.com website as the QP-bundle for Linux.

Once you download the QP-bundle, it will likely end up in your home/Downloads directory.

From there, you can unzip it into a location of your choice. I unzip the qp-bundle one level up from Downloads, that is, into my home directory.

This creates the qp directory there with the qp-bundle content, which includes: the QP/C and QP/C++ frameworks, the qm modeling tool, and the qtools collection of host-based tools for software tracing and testing.

Now, to quickly explain the QP/C++ code organization for POSIX inside the qp/qpcpp directory, the POSIX port is located in the ports/posix directory. Actually, you have two different POSIX and POSIX-QV, and we'll take a closer look at both later.

Because first, I'd like to show you the examples for POSIX that are provided, as usual, in the examples directory.

Here, in the posix-win32 sub-directory, you can find several examples, the simplest being blinky, which consists of just one Active Object. A slightly more advanced example is DPP--the Dining Philosophers Problem--which demonstrates multiple communicating Active Objects. It consists of 5 active objects called Philo-zero through four, emulating the dining philosophers, and 1 active object called Table that manages the forks for hungry philosophers.

To build this example, change into the qp/qpcpp/examples/posix-win32/dpp directory and check that it contains the Makefile. So, you just type "make".

This should produce the executable `project` in the dbg directory (corresponding to the Debug build configuration). You can run it by typing dbg/project.

The DPP example prints the short intro and then the changing status of the dining philosophers. The application accepts input in the form of keystrokes. Specifically, you can press the p key to pause serving permission to eat to the Philosophers. This causes all of them to eventually become hungry.

You can resume serving the forks to philosophers by pressing the 'S' key.

You terminate the application by pressing the ESC key.

Regarding the structure of the DPP application, the main() function and the state machine code for the Active Objects are completely generic—they do not depend on the underlying operating system. In fact, the DPP code has been generated from the same QM model across all environments you’ve seen in earlier lessons, including Zephyr RTOS, the preemptive QK kernel, and the non-preemptive QV kernel.

All platform-specific differences are isolated in the Board Support Package, the bsp.cpp file. This is where you’ll find the error handler, the initialization code—including the startup of the Active Objects—and the display routines that produce the printouts you saw on the screen.

One aspect that you can also adjust in the BSP is the rate at which the QF::onClockTick() callback will be invoked. The recommended setting here is the "native" system clock tick interrupt rate of your Linux distribution, which is often 100 Hz, but you need to check on your system.

The QF::onClockTick() callback should always perform the framework's tick processing of the armed time events. But that callback can also invoke other services, such as console key stroke processing, as in this case, or software tracing processing.

Speaking of which, this BSP also demonstrates software tracing, which I covered in Lessons 45 and 46. Lesson 46 introduced the idea of minimally intrusive software tracing using binary protocols.

Normally, this software tracing is inactive. It becomes active only when you build the “spy” configuration.

The Makefile for the DPP example supports the spy build configuration via the parameter CONF=spy added to the make command.

However, this build will currently fail because the open-source QP-Bundle distribution no longer includes software-tracing support.

Unfortunately, widespread GPL non-compliance — accelerated in the AI era — has forced the removal of certain components, including software tracing, from the open-source distribution and therefore from any AI-accessible training sets.

The removed components are now provided separately as QP-Extras. If you're interested in this functionality, you can request an Evaluation license; such requests will be granted very liberally.

Once you submit your request, you’ll receive a download link to the “QP/C++ Extras” ZIP file, such as the one I already have in my Downloads directory 

You need to unzip it into the directory containing the matching QP/C++ version.

For example, I unzip my qpcpp-extras_8.1.5.zip into ../qp/qpcpp, which already contains QP/C++ 8.1.5.

The ZIP file is password-protected; the password is your license number, which you’ll receive with your evaluation.

Now, when you go back to the DPP example directory, you can successfully build the “spy” configuration.

Before running the spy build, though, you need to launch the host-component of the software tracing system to visualize the binary trace. This component is called qspy and is provided in the QTOOLS collection.

For easy access to this utility and for testing support, you might want to edit your .profile file and add the following two lines to it.

After this, in your second terminal, type: qspy.

If qspy doesn't launch, you might need to change its permissions to allow execution by typing: chmod +x qspy.

When qspy is up and running, in your first terminal you can launch spy/project build and view the produced software trace. Please note that it contains much more than just your own instrumentation because the framework itself is also instrumented.

With the qspy running, you can also build and run the trace-based tests, which were provided in the "QP Extras" and have been unzipped into the "tests" directory.

For example, let's go to the tests/TUN_QP_qep_hsm directory, which stands for Test-Unit for the qep_hsm module. Each such test is organized into a src directory for the code-under-test and a test directory for the test fixture and scripts. Here you can find subdirectories for testing on the host and various targets. Since we're on the Linux host, let's go to the host directory and type make because the Makefile is provided.

This builds and immediately runs the test, as is usual for Test-Driven Development (TDD).

At the end of the test, you also see the test coverage report, which shows 100% line coverage and 100% branch coverage (MC/DC).

The trace-based testing consists of sending various commands to the target and comparing the generated trace to the test expectations, also called the test oracle.

If you are interested in such a modern testing approach, the QuantumLeaps YouTube channel provides a Playlist "Tracing, Testing, and Debugging", and among others, my presentation "Testing of Event-Driven Embedded Software with Python" that I delivered at the Embedded Online Conference a few years back.


Ok, so this was a quick tour of running Active Objects on a Linux host computer. But you can also run the same example applications on an embedded Linux target, for example, a Raspberry Pi Zero Wireless.

The plan for that is to connect to the PI, send it the qpcpp code, and then build and run the examples there.

I can connect to my wireless PI through WiFi via ssh, like this:
ssh, user ql @ IP address of the PI. Of course, you need to adjust the username and IP address to your Linux target.

Because the PI is a smaller computer, let's not send the whole QP-bundle, but only the qpcpp directory. For that, I'll zip just that directory into the qpcpp_8.1.5.zip archive. Please note that it already contains the "QP Extras".

Now, I ssc this file to my wireless PI, with the same username and IP address as previously used to ssh. Again, you need to adjust these parameters to your specific linux target.

Now, on the PI I check that the file has been received, and I unzip it to the qp directory.

So, the directory qp has been created, and inside it is the qpcpp directory.

Now, I go into the DPP example inside: qpcpp/examples/posix-win32/dpp

Here, I first need to clean up the previous builds copied from the desktop, but otherwise I can just build the example by typing make.

Of course, my older PI zero first generation is a much slower computer than my host machine, so the build takes significantly longer.

But the dbg build directory is produced, and I can run the DPP example as before by typing dbg/project.

I can also run the example with superuser privileges, which allows the application to apply the SCHED_FIFO real-time scheduling policy for POSIX threads. I'll show you that option in the POSIX port later.

Speaking of the POSIX port, all demonstrations so far used the standard QP port to POSIX that runs every active object in its own, separate POSIX-thread. This is a good option if you want to take advantage of multicore, for example.

However, there is also another QP port called POSIX-QV, which runs all active objects in the main thread, just like the non-preemptive QV kernel. This is a good option if you want to economize the number of POSIX threads and you want to safely share resources among your active objects.

The DPP example and the provided Makefile support the POSIX-QV port, but you need to remember to clean up the previous build.

Now, you can build the POSIX-QV version by adding the QV=1 parameter to the make command.

The POSIX-QV version of DPP runs exactly the same as the multi-threaded version.

Finally, I can also build the spy configuration.

But to run it, I first need to ensure that QSPY host application is running, and then I need to provide the IP address of that QSPY instance to the executable.

Because remember, this is a remote Linux target, while QSPY runs on the host.

You can even run trace-based tests on such remote Linux targets, but this is a bit more involved. For details, please watch my dedicated video "QUTest Unit Testing for Embedded Systems: Running QUTest on Embedded Linux"

***
After you've seen the examples, let's talk about the QP port to POSIX.

This port is implemented in the QP framework’s Operating System Abstraction Layer (OSAL) -- which was introduced in Lesson 56.

The QP/C++ OSAL layer consists mainly of two files: qp_port.hpp and qf_port.cpp located in the directory qpcpp/ports/posix in this case.

The highlights of this port are as follows:
- All application code and QP framework run in the user space.
- Every active object runs in its own POSIX thread (p-thread), which executes the event loop.
- The event queue owned by each active object is the native QEQueue instance. The queue blocks on the POSIX condition variable when waiting for events.
- The critical section that QP uses to protect its own internal integrity is implemented as a POSIX mutex. The critical sections are not allowed to nest.
- The same critical section mutex is also used with the condition variables that throttle the event queues.  
- The system clock tick that handles the QP time events runs in the main thread.

Specifically, the call to QF::run() from main enters an endless loop, which calls the QF::onClockTick() callback.

If the tick period is configured, the main thread sleeps without drifting for a configured tick period and then calls QF::onClockTick().

Otherwise, the loop only calls QF::onClockTick() without sleeping. The responsibility for sleeping between the clock ticks is then deferred to the application, which can implement it some other way.

Either way, QF::onClockTick() callback must always call the QP time event processing. It can also call some other functions.

Regarding the internal POSIX threads of Active Objects created in the QActive::start() method, the port first attempts to apply the real-time SCHED_FIFO scheduling policy. However, this requires superuser privileges. If that attempt fails, the port defaults to the regular SCHED_OTHER policy.

I've demonstrated these aspects by launching the application example without and with superuser privileges.

Finally, regarding the p-thread priority assignment, which is relevant only for the SHED_FIFO policy, it is explained in the NOTEs at the bottom of the qf_port.cpp file.

***
In the last segment of this lesson, I'd like to briefly review the single-threaded POSIX-QV port, as promised. This port resides in the qpcpp/ports/posix-qv directory.

It also consists mostly of qp_port.hpp and qf_port.cpp files.

The highlights of the single-threaded POSIX port are as follows:
- The application code and QP framework run in the user space.
- All active objects run in the same main() thread, so the QF::run() function looks very much like the one in the non-preemptive QV kernel discussed in lesson 54.
- The event queue of each active object is the native QEQueue instance. The queue doesn't need to block in this port, so there is no need for per-active-object condition variables.
- The critical section that QP uses to protect its own internal integrity is implemented as a POSIX mutex. The critical sections are not allowed to nest.
- The system clock tick that handles the QP time events runs in its own p-thread, which is different than the main thread already used for executing active objects.

This "ticker thread" is created in QF::run() right before it enters the loop that cooperatively exectutes all active objects.

Now, regarding the noteworthy details:

When no events are available, the QF::run() function called from main in POSIX cannot perform busy-waiting and idling, as did the QV in the microcontroller version. Instead, the main thread must be efficiently blocked.

This is accomplished by waiting on a condition variable called QF::condVar_.

This condition variable is signaled by every event posted to any event queue.

The system clock tick processing in the POSIX-QV port also periodically calls the QF::onClockTick() callback as in the multi-threaded POSIX port, but the tick processing is no longer handled in the main thread but rather in the separate "ticker thread". 

Another important difference is that the "ticker thread" is created only when the tick period is configured. When it is set to zero, the ticker thread is not created at all, and QF::onClockTick() is never called. This means that the entire responsibility for periodic time event processing (and perhaps other periodic processing) is entirely deferred to the application.


This concludes this lesson about extending the modern event-driven paradigm to POSIX-compliant operating systems. 

Systems like embedded Linux make up a large share of today’s embedded applications, and being able to avoid the pitfalls of “naked,” unconstrained threads is a major advantage. The QP ports for POSIX and POSIX-QV give you the full power of modern event-driven Active Objects, hierarchical state machines, modeling, tracing, and testing across all these platforms.

If you enjoy this channel, please consider subscribing to support the ongoing production of new lessons. Thank you for watching!