QTools
8.0.0
Collection of Host-Based Tools
This milestone QTools release introduces several new features and updates to QSPY tracing host utility, QUTest testing harness, and QView monitoring system.
QSPY host utility
This QUTools release updates the QSPY host utility for the matching QP/C and QP/C++ version 8.0.0 as well as adds new features to the QUTest testing harness and the QView monitoring system.
QUTest testing harness
This release adds BDD (Behavioral Driven Development) support to QUTest.
See example BDD test script. Here is the generated QUTest output.
QUTest unit testing front-end 8.0.0 running on Python 3.10.1
Copyright (c) 2005-2024 Quantum Leaps, www.state-machine.com
Attaching to QSpy (localhost:7701)... OK
Run ID : 241011_103618
Target : remote
==================================[Group 1]==================================
test_sched.py
@uid{TST-QP-QK_02}
This test group verifies the preemption scenarios
in the QK preemptive kernel
[ 1]--------------------------------------------------------------------------
SCENARIO: ao->ao->ao (NO PTS)
GIVEN: priorities: (1,0), (2,0), (3,0)
WHEN: post TEST0_SIG to aoB[1]
THEN: expect the given event exchange
[ PASS ( 0.2s) ]
[ 2]--------------------------------------------------------------------------
SCENARIO: ao->ao->ao (PTS1)
GIVEN: priorities: (1,3), (2,3), (3,0)
WHEN: post TEST0_SIG to aoB[1]
THEN: expect the given event exchange
[ PASS ( 0.3s) ]
[ 3]--------------------------------------------------------------------------
SCENARIO: ao->ao->ao (PTS2)
GIVEN: priorities: (1,0), (2,3), (3,0)
WHEN: post TEST0_SIG to aoB[1]
THEN: expect the given event exchange
[ PASS ( 0.2s) ]
==================================[ SUMMARY ]=================================
Target ID : 241011_100041 (QP-Ver=800)
Log file :
Groups : 1
Tests : 3
Skipped : 0
Failed : 0
==============================[ OK ( 0.9s) ]===============================
QView monitoring
This release contains a re-designed QView, which now can be simply imported to the customization Python script, instead of the customization script being called from qview.py. This enables developers to import any other Python modules, which was not possible before.
QTools version info:
Bug fixes (in QUTest):
Feature (in QView):
QTools version info:
This release applies the pylint static analyzer to all Python utilities included in the QTools collection:
The biggest changes for compliance with pylint include migration from %-strings to f-strings (requires Python 3.6 or higher).
.cmake, .json, .pyi, .md5)QTools collection for Windows (qtools\bin directory) now includes:
executables to accompany the new cmake support added in QP/C 7.3.3.
QTools version info:
Added Python script qspy_kill.py to remotely terminate (kill) the QSPY host application.
new qspy_kill.py script
qtoos/qspy/ directory and can be used in the test automation to remotely terminate QSPY.QTools version info:
Updated QSPY Host Application for the latest QP/C/C++ 7.3.0.
QTools version info:
Introduced the QUTest interactive mode.
QTools version info:
QUTest testing framework:
This release represents a significant overhaul of the qutest.py test runner.
qutest.py script runner qutest.py command-line options have been expanded and changed. Unfortunately, it was not possible to preserve the backwards compatibility with the earlier versions. This means that various Makefiles for building and running tests need to be updated.qutest.py to better match the more verbose output generated by test scripts. (The minimal one-line-per test output could not be reconciled with various notes now generated by test scripts.)QView visualization and monitoring:
Documentation
QTools version info:
QTools version info:
Documentation
doxygen/ folder for generation of QM Manual in LaTeX (enables generation of PDF)QTools version info:
QSPY host application:
QS_SCHED_PREEMPT replaces deprecated QS_MUTEX_LOCKQS_SCHED_RESTORE replaces deprecated QS_MUTEX_UNLOCKQF_TIMEEVT_CTR_SIZE to 4 in QP/C and QP/C.Slightly restructured QSPY source code to better integrate directly with QP Applications. Specifically, moved QSPY_cleanup() to qspy.c, so it is available to the applications.
Changes in QSpy
Changes in QUTest
qutest.py has been updated to include new QS trace records and global filters for them. The following QS record groups have been added to the "QUTest DSL": Changes in QUView
qview.py has been updated to include new QS trace records and global filters for them. The following QS record groups have been added to QView: Changes in QSpy
-r<c0|c1|c2>.Changes in QUTest
Changes in QCalc
qcalc.py.Updated GNU-ARM Toolchain for Windows
Updated Python for Windows
Changes in QSpy
QSPY_printInfo() to qspy.c, to enable direct QSPY output to the console. Please see also discussion thread [QSPY to console in win32-qv and posix-qv ports] https://sourceforge.net/p/qpc/discussion/668726/thread/92e56e9480),Changes in QUTest
Changes in QView
Changes in QWin
Changes in QCalc
Updated Python for Windows
Changes in QUTest and QView
The UDP socket binding (for connection to QSPY host application) has been changed from "localhost" to "0.0.0.0". This is to improve access to QSPY running on remote hosts. This fixes the following bug:
Changes in QView
Changed the "Local Filter" dialog box to show "QS_id=..." instead of "AO-prio=..." for QS-IDs above 64.
Added QCalc
A new version of the popular QCalc Programmer's Calculator has been added again. This version is based on Python (whereas the previous was based on Tcl/Tk). The new version is console based and, among other enhancements, adds support for 64-bit range.
Updated GNU-ARM Toolchain for Windows
The GNU-ARM toolchain in QTools for Windows has been updated to the latest Version 10-2020-q4-major, released on December 11, 2020.
Updated Python for Windows
The Python interpreter in QTools for Windows has been updated to Python 3.9.
The main purpose of this release is to adjust QUTest™ Unit Testing Harness and QView™ Visualization & Monitoring to the new QS Local Filter design implemented in QP/C and QP/C++ 6.9.1. Specifically, the loc_filter() and ao_filter() functions in QUTest scripts and in QView scripts has been re-designed. Additionally, the QView Local Filter menu have been adjusted accordingly.
Changes in QSPY
The QSPY host application has been updated to handle the redesigned local-filters and the new predefined #QS_QF_NEW_ATTEMPT trace record. Also the default version compatibility with the QS target-resident component (the -v command-line option) has been increased from 6.2 to 6.6.
Additionally, the QS_U64() and QS_I64() data elements have been made available for all types of CPUs, whereas previously they were available only on 64-bit CPUs (see also feature#181).
Additionally, QSPY now can format application-specific data elements in hexadecimal. Here are a few examples of QS trace records in the Target and the generated QSPY output:
Implemented Feature Requests
This release brings the new QView™ Visualization & Monitoring component, which replaces QSpyView written originally in Tcl/Tk. The new QView is written in Python (3.3.+) and brings much commonality with the QUTest™ Unit Testing Harness, which has also been re-structured and improved.
Changes in Directory Structure
The new QView and the updated QUTest components are now located directly under the qtools folder and are no longer in the qtools\qspy folder. Here is the new qtools directory structure:
qutest.py script.Changes in QSPY:
Bug Fixes:
Bug Fixes:
Also, improved some comments in QUTest scripting documentation (qutest.py).
This QTools release changes the build process for the QSPY, QCLEAN, and QFSGEN utilities on Windows. Specifically, these programs are now built using Visual Studio 2019 instead of the MinGW compiler for Windows. Also, the source code of all Quantum Leaps utilities has been reviewed and all "unsafe" calls to standard C library have been replaced with their "safe" counterparts (e.g., strcpy()->strcpy_s(), strcat()->strcat_s(),fprintf()->fprintf_s(), etc.) All these changes were made to avoid the malware warnings that were issued by some anti-virus software against executables built with MinGW.
This release also replaces the MinGW toolchain (8.2.0) with the latest mingw32 (9.2.0) installed with MSYS2 (see also https://sourceforge.net/p/qpc/discussion/668726/thread/06b89ba2d5/ ).
Finally, this release updates the QWIN Prototyping Toolkit (qtools\qwin directory) to work with the latest Visual Studio 2019 and specifically with the Resource Editor now available in VS 2019. The qwin-demo project has been updated to build with Visual Studio 2019.
The main purpose of this release is the change in distribution of the QTools collection, which is now bundled together with the QP Frameworks (QP/C, QP/C++ and QP-nano) as well as QM into "QP-bundle". This "QP-bundle" provides a single, streamlined and simplified download and installation of all QP Frameworks and all the accompanying tools.
Also this release brings significant changes to QTools for Windows. Specifically, the locations of the MinGW compiler and Tcl/Tk have been changed to separate directories (as opposed to being co-located in the qtools/bin directory). This facilitates easier upgrades of these third-party tools, and indeed, they have been upgraded in this QTools/Windows release as follows:
qtools/bin directory and therefore the QTools installer for Windows adds to the PATH both the qtools/bin and qtools/MinGW/bin directories.Also, this release makes the following changes to the QSPY host application:
qspy is equivalent to the former qspy -u -t. In other words, QSPY by default opens the UDP port and opens the TCP/IP port for Target connection.-v 660 command-line option, but is in preparation for the future changes in QP/C and QP/C++ frameworks (the upcoming QP/C/C++ 6.6.0).This release adds debug mode to QUTest scripting (in Python). Specifically, you can now provide a special value DEBUG to the qutest.py Python script launcher, in which case qutest.py will start in debug mode, in which it will not launch the host executable and will not reset the Target. Instead, qutest.py will wait for the Target reset and other responses from the Target.
Additionally, this release improves the QTools installation on Windows. Specifically the Windows installer now installs Python and has been re-designed to automatically set up the QTOOLS environment variable and add QTOOLS%\bin;QTOOLS%\python to the PATH. The uninstaller reverses these changes.
This release speeds up the QSPY host application by tweaking the timing constants in the TCP and UDP communication interfaces. The resulting QSPY 6.4.0 runs QUTest tests significantly faster (at least twice as fast) as before.
The main purpose of this release is the update of the GNU-ARM toolchain (included in QTools for Windows) to the latest GCC 8. Specifically, this QTools release switches to the official GNU-ARM Embedded Toolchain for Arm Cortex-M and Cortex-R processors maintained by ARM Ltd..
qtools\gnu_arm-none-eabi and contains tools with the prefix arm-none-eabi-. This is different from the GNU-ARM toolchain used previously (which was located in qtools\gnu_arm-eabi and used the tools prefix arm-eabi-). This change requires adjusting existing Makefiles and other build commands for the GNU-ARM toolchain, which has been done in the matching QP/C/C++/nano release 6.3.8.Additionally, this release makes further improvements to the "qutest.py" Python interface. Specifically, the skip() command can now be used anywhere in the test scripts to skip the commands. (Previously, the skip() command could only legitimately be used just before the test() command).
This release brings a completely re-designed and simplified Python interface for the QUTest unit testing harness. Specifically, this release adds a new scripting engine called simply qutest.py, which replaces the previous "qspypy".
The new qutest.py Python interface is a simple single-threaded application that no longer requires PyTest or other such external Python packages, as did "qspypy" previously. The new Python interface is now also compatible with both Python 2 (2.7+) and Python 3 (3.4+), which means that virtually any contemporary Python version without extensions can be used for running QUTest Python scripts.
Additionally, this release fixes the problem in the QSPY host utility to correctly format signed integers (I8, I16, I32), which didn't work correctly on 64-bit Linux platforms. Also, the QSPY utility on Windows now uses the newer Windows socket library "ws2_32", which replaced the old "wsock32".
Finally, the QTools release for Windows updates the provided MinGW GNU-C/C++ compiler to version 6.3.0.
The main purpose of this release is to improve QUTest support. Specifically, this release adds a feature of querying and reporting the status of the current objects, such as the current State-Machine (SM), Active-Object (AO), current Memory-Pool (MP), current Event-Queue (EQ), and current Time-Event (TE). The feature impacts the following facilities:
query_curr()qutest.query_curr().Also, the following bugs related to Python scripting have been fixed in the qspypy 6.3.6 included in this release:
qutest.expect() implementation more closely matches the behavior of the expect() directive from Tcl scripting. The modified qutest.expect() compares correctly the timestamp fields (as opposed to ignoring them) and also handles special character sequences '?', '*', and [chars] (like the Tcl string match command).pip install %QTOOLS%\qspyFinally, this release of the QTools collection contains the Unity unit testing harness (framework).
This release adds official support for writing QUTest test scripts in Python. Also, this release fixes some minor typos in error messages generated by QSpy.
This minor release corrects the QSPY utility. Specifically, the application-specific output from the macros QS_FUN(), QS_OBJ() and QS_SIG() contained extra space (' ') before the function/object/signal name. This extra space has been now removed.
This release simplifies the usage of the QUTest (qutest.tcl) and QSpyView (qspyview.tcl) utilities by reducing the number of parameters required by these scripts. Specifically, the "local_port" parameter might now be omitted and still every instance of qutest.tcl / qspyview.tcl will be given a unique local UDP port. Additionally, the "port" parameter has been combined with the "host" parameter in the form host[:port].
Here is the new usage of the qutest.tcl script:
Here is the new usage of the qspyview.tcl script:
For example, to attach to QSPY running on the host 192.168.1.100 and port 7705, you now can launch these scripts as follows:
Modified files:
Also, this release adds the LMFlash utility to QTools for Windows for flash programming of TM4C MCUs.
This release updates the QSPY host utility as well as the qutest.tcl and qspyview.tcl scripts for the following new QS records introduced in QP/C/C++ 6.2.0:
The global filter settings in the qutest.tcl and qspyview.tcl scripts have been updated to the augmented and re-organized QS trace records.
Additionally, the new commands post() and publish() have been added to the QUTest "testing DSL" (qutest.tcl script). Examples of use of these new commands are provided in the QP/C/C++ directory <qpc|qpcpp>\examples\qutest\defer\.
This release fixes the following bug in QSPY:
Additionally, this release simplifies the format of the external dictionary files, so that they can potentially be generated from .map files or .elf files. The new format no longer requires storing the number of entries in the dictionary and the entries don't need to be sorted by the key-value. The new dictionary format also allows adding comments and empty lines between the dictionary blocks.
Additionally, the QSPY for POSIX (Linux, MacOS, etc.) corrects the problem with reading input from a file (-f[bin_file] command-line option).
Finally, this release increases the QCLEAN maximum size limit to 10MB per file.
This release improves the QSPY command-line option processing by allowing option parameters to be separated by spaces from the option letter (including the optional parameters). For example, the following command-line would not work in the previous version, but will work now:
qspy -t 6602
At the same time, the option parameters can follow immediately the option letter, so the following command line will work as well (backwards compatibility):
qspy -t6602
This release also improves the handling of external dictionary files in QSPY host application. Specifically, the -d command-line option has been extended to allow no parameter, in which case an external dictionary file is opened automatically as soon as QSPY receives the target reset or target information. When the -d <file> option is used, the provided dictionary file is used right away, but the dictionary information might get discarded when the target information does not match the configuration and time-stamp obtained from the dictionary file.
Fixed compilation and linkage on MacOS for qspy, qclean, qfsgen.
Fixed bug#180 qspyview.tcl for peek is not in sync with the qs_rx parsing of peek.
Used unsigned integer math in qcalc.tcl.
In QTools for Windows, updated the GNU-ARM toolchain to the latest available GCC 7.2.0 adapted from SysProgs Prebuilt GNU toolchain for arm-eabi:
http://gnutoolchains.com/arm-eabi/.
Fixed bug#175 "QS_QF_ACTIVE_GET & QS_QF_EQUEUE_GET Record Mislabeled in QSPY Output" (https://sourceforge.net/p/qpc/bugs/175/ ).
Added bin/qcalc.tcl to the GIT repository (so that it shows up on GitHub).
Added the GNU-ARM (EABI) toolset to the QTools Collection for Windows in the directory qtools/gnu_arm-eabi. The addition of the GNU-ARM toolset matches the changes made to Makefiles in QP/C/C++/nano 5.9.2.
To reduce the size of the QTools for Windows download, the self-extracting archive file qtools_win32_<ver>.exe has been prepared with the 7-Zip utility.
This release adds the QUTest (pronounced 'cutest') Unit Testing support to QP/Spy software tracing. Specifically, this release adds a new head-less (console-based) QSPY front-end, which runs unit tests.
This release also adapts the QSPY host utility to support QUTest unit testing. Several new commands have been added and the structure of the code has been expanded.
Also, the QSpyView Visualization extension has been moved to the tcl\ sub-directory (the Tcl script tcl\qspyview.tcl).
The other Quantum Leaps utilities, like QClean, QFSGen, and QCalc, have been updated and greatly improved.
Finally, all utilities in the QTools collection have been documented in the new QTools Manual, available online at https://www.state-machine.com/qtools.
Corrected the QSPY software tracing host application to properly display floating point numbers in user-defined trace records (QS_F32() and QS_F64() macros). The problem was incompatibility between Microsoft VC++ and GCC floating-point format specifications. In the GCC software build (which produces the QSPY executable in the qtools/bin directory), the MS-VC++ floating point format resulted in all zeros (e.g., 0.0000e+000).
Added QWIN GUI↑ to the collection (sub-directory qwin_gui).
Updated the QSPY software tracing host application for the QS trace record name changes introduced in QP 5.6.2.
Extended the QSPY software tracing host application for bi-directional communication with embedded targets (output and input into the target). Added a UDP socket to QSPY, as an extensibility mechanism for adding external unit testing, GUIs and other "front-ends" to control the embedded targets.
Provided new QSpyView Tcl/Tk extension of the QSPY host application for control testing, and visualization of the real-time tracing data from embedded targets at real-time. QSpyView enables developers to rapidly build both GUI-based and "headless" scripts for their specific applications (see https://www.state-machine.com/qpc/arm-cm_dpp_ek-tm4c123gxl.html).
Corrected the version representation from hex to decimal, to match the change in the QP Framework. The version representation mismatch caused problems in parsing newly modified trace records, when the qspy.c implementation was inserted directly into the projects.
Added new trace records to the QSPY host application: QS_QEP_TRAN_HIST, QS_QEP_TRAN_EP, and QS_QEP_TRAN_XP. Changed labels for standard records from Q_ENTRY, Q_EXIT, Q_INIT to ENTRY, EXIT, INIT.
Fixed the bug in the QSPY host application, which didn't handle correctly object/functions/signal names longer than 32 characters. The name limit has been raised to 64 characters and this version also correctly truncates the names at the limit without printing any garbage characters.
Modified QSPY utility to support changes in QP 5.1.x:
-improved handling of target resets by adding an empty QS record before the QS_QP_RESET record. The empty record provides the frame delimiter in case the last record in incomplete, so that the following QS_QP_RESET record can be recognized.
-improved handling of internal object/function/signal dictionaries so that symbolic information is displayed for all occurrences of an object/function/signal, for which a dictionary record was received.
Modified the QSPY utility to support changes in QP 5.0.x:
-modified the standard trace records QS_QF_TICK and QS_QF_TIMEEVT_* to contain the tick-rate number.
-added trace records QS_TEST_RUN and QS_TEST_FAIL for unit testing.
-added version compatibility level 5.0, whereas specifying version -v 4.5 runs qspy in compatibility mode with QP 4.5.x.
-added Find And Replace Text (FART) utility for Windows.
Re-designed the QSPY interface to support more flexible parsing of the trace records in desktop-based simulations (such as Windows or Qt). Users can provide a custom parsing callback function to QSPY_config(). Also added QS_RESET() macro to reset the internal dictionaries (and other cleanup in the future) when the target resets.
Added the QS_USR_DICTIONARY() entry for storing dictionaries of the user trace records. Replaced all remaining sprintf() calls with snprintf().
Re-designed the implementation of the QSPY host application, so that it can be convenienty included as part of the QP library. This allows direct QS tracing output to the screen for QP Applications running on the desktop. The QSPY application has been converted from C++ to plain C for easier integration with QP/C.
This QSPY version matches the changes to the critical section macros made in QP 4.3.00. The QS record names QS_QF_INT_LOCK and QS_QF_INT_UNLOCK have been replaced with QS_QF_CRIT_ENTRY and QS_QF_CRIT_EXIT, respectively.
This QSPY version fixes the bug of incorrect reporting function or object pointers for which the dictionary records are not provided and which are repeated in one format line (bug #3405904). For example, trace record AO.FIFO would report (incorrectly) as follows:
0014004078 AO.FIFO: Sndr=200009B4 Obj=200009B4 Evt(Sig=00000009,Obj=200009B4, Pool= 0, Ref= 0) Queue(nFree= 5, nMin= 5)
The Sndr= and Obj= are reported to be the same, but they were not.
This QSPY version adds generation of sequence diagrams as files to be processed by MscGen (www.mcternan.me.uk/mscgen/). This version adds the option -g<msc_file> to generate a .msc file.
Also, this version of QSPY for Windows allows COM ports larger than COM9.
This QSPY version matches the changes made to QS target code in QP/C/C++ 4.2.xx. These changes include sending the additional byte of sender priority in trace records:
Additional changes include sending the poolID and refCtr of events in two bytes instead of just one byte. The changes affect the following trace records:
Also, for compatibility with QP 4.2.xx, this version changes the defaults as follows:
signal size (-S) from 1 byte to 2 bytes, and
baud rate (-b) from 38400 to 115200.
This version adds the following trace record:
QS_QF_TIMEEVT_CTR.
The version also adds compatibility with 64-bit targets (such as 64-bit linux). This version can accept 8-byte pointers (both object pointers and function pointers) as well as 64-bit integers sent in user-defined trace records.
This version also adds the hex format for uint32_t integers sent in user-defined trace records.
Finally, this version adds a command-line option -v to specify the corresponding QP version running on the target. The default version is -v4.2, but specifying version -v4.1 or -v4.0 will switch QSPY into backwards-compatibility mode with the earlier versions of QP.
This is the initial standalone release of the QSPY host pplication. QSPY is still available in the QP/C and QP/C++ distributions, but other rapid prototyping platforms (such as mbed or Arduino) do not use the standard QP distributions and have no easy access to the QSPY tool. For these users, this pre-compiled standalone release is more convenient.