QTools
8.0.4
Collection of Host-Based Tools
A QUTest test script contains a group of related tests (a test group). The basic job of these tests is to send commands to the test fixture running in the Target and to compare the QSPY textual output produced by the Target with the test expectations.
The QUTest test scripts are executed in the host by the QUTest script runner that communicates with the QSPY host application via the UDP interface. The QUTest script runner is itself implemented in a Python module qutest.py.
The Python module qtools/qspy/py/qutest.py defines a small Domain Specific Language (DSL) for writing test scripts in Python. The structure of this DSL is very simple. Each test script consists of two sections:
The "preamble" section of a test script file specifies include scripts and defines callback functions common to all tests in this file. It can contain the following functions:
The on_...() callback functions can call any of the test commands.
The "tests" section of a test script file contains the actual tests. Each test starts with the test() command and can contain any number of the test commands. The test continues until another test() command, or the skip() command.
The "tests" section can also contain tests written in the more structured "BDD-style" (BDD stands for Behavior-Driven Development). In this approach, tests are called "scenarios" and consist of sections: "given", "when", and "then".
QUTest supports two alternative ways of specifying the BDD scenarios.
The alternative #1 is based on the following group of commands, which you can simply mix with other QUTest commands:
The alternative #2 is based on the Python decorators, where every section of a scenario becomes a separate Python function. QUTest script can apply the following Python decorators:
The commands that you can place in the tests (as well as inside the callback functions) are:
Commands that don't communicate with the Target:
Testing in QUTest is based on comparing the output sent to the QSpy host utility with the test expectations codified in the test script. Often, instead of writing the test expectations directly, it is much easier to copy the raw QSpy output and paste it into the test script (scrutinizing the output for correctness). However, the raw QSpy output still needs to be converted to the valid Python commands (e.g., expect("@timestamp ...")), which is tedious. Fortunately, the QUTest directory contains a Python script named qutestify.py, which automates the conversion process. The following subsections illustrate the steps of the process:
The following test script posts an event to a given Active Object. Please note that the post() command is not followed by any test expectations (qutest_dsl::expect() "expect()" commands).
The following console output from the trial run of this test, which fails because the test generates output that is not explicitly expected:
C:\qp-dev\safe-qpc\v-v\TUN_QP_qf_time\test>make TESTS=test_xyz.py python3 C:\qp\qtools/qutest/qutest.py -ebuild_host/qf_time.exe -q -l -o -- test_xyz.py QUTest unit testing front-end 8.0.4 running on Python 3.12.9 Copyright (c) 2005-2025 Quantum Leaps, www.state-machine.com Attaching to QSpy (localhost:7701)... OK Run ID : 250612_113944 Target : build_host/qf_time.exe,localhost:6601 ==================================[Group 1]================================== test_xyz.py File: qf_qtime.c [ 1]-------------------------------------------------------------------------- TUN_QP_qf_time_01 exp: end-of-test got: "0000000001 MP-Get Obj=EvtPool1,Free=9,Min=9" ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ![ FAIL ( 0.1s) ] ==================================[ SUMMARY ]================================= Target ID : 250612_113054 (QP-Ver=804) Log file : Groups : 1 Tests : 1 Skipped : 0 FAILED : 1 [ 1 ] ==============================[ FAIL ( 3.2s) ]=============================== make: *** [Makefile:241: run] Error 1
The following console output shows the QSpy run of this (failing) test run:
<F-END> Attached Chan=00000010
Run ID : 250612_113944
Target : build_host/qf_time.exe,localhost:6601
==================================[Group 1]==================================
File: qf_qtime.c
[ 1]--------------------------------------------------------------------------
TUN_QP_qf_time_01
<COMMS> TCP-IP Connected to Host=127.0.0.1,Port=59614
Trg-RST QP-Ver=804,Build=250612_113054
Obj-Dict 0x00411500->QS_RX
. . . .
Fun-Dict 0x00404A7B->QActive_postLIFO_
Fun-Dict 0x0040D8B9->QS_processTestEvts_
QF_RUN
Trg-Ack QS_RX_TEST_SETUP
Trg-Ack QS_RX_CURR_OBJ
Trg-Ack QS_RX_EVENT
0000000001 MP-Get Obj=EvtPool1,Free=9,Min=9
0000000002 QF-New Sig=TRIG1_SIG,Size=8
0000000003 AO-Post Sdr=QS_RX,Obj=the_TstAO,Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>,Que<Free=10,Min=10>
0000000004 AO-GetL Obj=the_TstAO,Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>
0000000005 Disp===> Obj=the_TstAO,Sig=TRIG1_SIG,State=TstAO_stateA
0000000006 TE0-Arm Obj=TstAO_inst.te0A,AO=the_TstAO,Tim=1,Int=0
0000000007 TE0-Arm Obj=TstAO_inst.te0B,AO=the_TstAO,Tim=2,Int=0
0000000008 TE0-Arm Obj=TstAO_inst.te0C,AO=the_TstAO,Tim=2,Int=2
===RTC===> St-Entry Obj=the_TstAO,State=TstAO_stateB
0000000009 ===>Tran Obj=the_TstAO,Sig=TRIG1_SIG,State=TstAO_stateA->TstAO_stateB
0000000010 QF-gc Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>
0000000011 MP-Put Obj=EvtPool1,Free=10
0000000012 Trg-Done QS_RX_EVENT
Trg-Ack QS_RX_TEST_TEARDOWN
[ 1]! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ![ FAIL ( 0.1s) ]
<COMMS> TCP-IP Disconn from Host=127.0.0.1,Port=59614
==================================[ SUMMARY ]=================================
Target ID : 250612_113054 (QP-Ver=804)
Log file :
Groups : 1
Tests : 1
Skipped : 0
FAILED : 1 [ 1 ]
==============================[ FAIL ( 3.2s) ]===============================
<F-END> Detached ######################################
Now, you select and copy to the Clipboard the raw QSpy output generated by the original post() command (lines from 0000000001... to 0000000012 Trg-Done QS_RX_EVENT).
The following listing shows the test script from step-0 after pasting the raw QSpy output:
Please note that the lines from 0000000001... to 0000000012 Trg-Done QS_RX_EVENT pasted from the raw QSpy output are not valid Python.
The following console output shows the run of the qutestify.py utility on the test_xyz.py test script (Please note the changed lines: 13 output, which informs you how many lines of the script have been changed.):
C:\qp-dev\safe-qpc\v-v\TUN_QP_qf_time\test>qutestify test_xyz.py QUTestestify utility 8.0.4 Copyright (c) 2005-2025 Quantum Leaps, www.state-machine.com qutestifying: test_xyz.py changed lines: 13.
The following listing shows the updated test script (after refreshing the editor):