QTools  8.0.4
Collection of Host-Based Tools
Loading...
Searching...
No Matches
QUTest Script Reference

QUTest Fixture ReferenceQView Visualization & Monitoring

A QUTest test script contains a group of related tests (a test group). The basic job of these tests is to send commands to the test fixture running in the Target and to compare the QSPY textual output produced by the Target with the test expectations.

The QUTest test scripts are executed in the host by the QUTest script runner that communicates with the QSPY host application via the UDP interface. The QUTest script runner is itself implemented in a Python module qutest.py.

QUTest Testing DSL

The Python module qtools/qspy/py/qutest.py defines a small Domain Specific Language (DSL) for writing test scripts in Python. The structure of this DSL is very simple. Each test script consists of two sections:

Preamble

The "preamble" section of a test script file specifies include scripts and defines callback functions common to all tests in this file. It can contain the following functions:

The on_...() callback functions can call any of the test commands.

Tests

The "tests" section of a test script file contains the actual tests. Each test starts with the test() command and can contain any number of the test commands. The test continues until another test() command, or the skip() command.

BDD-Style Tests

The "tests" section can also contain tests written in the more structured "BDD-style" (BDD stands for Behavior-Driven Development). In this approach, tests are called "scenarios" and consist of sections: "given", "when", and "then".

QUTest supports two alternative ways of specifying the BDD scenarios.

Note
BDD-Style scenrios can be mixed with regular tests in the same test script, however a given scenario must consistently use the chosen BDD alternative.

The alternative #1 is based on the following group of commands, which you can simply mix with other QUTest commands:

The alternative #2 is based on the Python decorators, where every section of a scenario becomes a separate Python function. QUTest script can apply the following Python decorators:

Commands

The commands that you can place in the tests (as well as inside the callback functions) are:

Simple Commands:

Complex Commands:

Commands that don't communicate with the Target:

Script Converter Utility (qutestify.py)

Testing in QUTest is based on comparing the output sent to the QSpy host utility with the test expectations codified in the test script. Often, instead of writing the test expectations directly, it is much easier to copy the raw QSpy output and paste it into the test script (scrutinizing the output for correctness). However, the raw QSpy output still needs to be converted to the valid Python commands (e.g., expect("@timestamp ...")), which is tedious. Fortunately, the QUTest directory contains a Python script named qutestify.py, which automates the conversion process. The following subsections illustrate the steps of the process:

Original Test Script

The following test script posts an event to a given Active Object. Please note that the post() command is not followed by any test expectations (qutest_dsl::expect() "expect()" commands).

# test-script for QUTest unit testing harness
# see https://www.state-machine.com/qtools/qutest.html
note("File: qf_qtime.c")
#=============================================================================
# preamble...
#=============================================================================
# tests...
#-----------------------------------------------------------------------------
test("TUN_QP_qf_time_01")
current_obj(OBJ_AO, "the_TstAO")
post("TRIG1_SIG")

Trial Test Run

The following console output from the trial run of this test, which fails because the test generates output that is not explicitly expected:

C:\qp-dev\safe-qpc\v-v\TUN_QP_qf_time\test>make TESTS=test_xyz.py
python3 C:\qp\qtools/qutest/qutest.py -ebuild_host/qf_time.exe -q -l -o -- test_xyz.py

QUTest unit testing front-end 8.0.4 running on Python 3.12.9
Copyright (c) 2005-2025 Quantum Leaps, www.state-machine.com
Attaching to QSpy (localhost:7701)... OK

Run ID    : 250612_113944
Target    : build_host/qf_time.exe,localhost:6601

==================================[Group  1]==================================
test_xyz.py
File: qf_qtime.c
[ 1]--------------------------------------------------------------------------
TUN_QP_qf_time_01
exp: end-of-test
got: "0000000001 MP-Get   Obj=EvtPool1,Free=9,Min=9"
! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ![ FAIL (  0.1s) ]

==================================[ SUMMARY ]=================================

Target ID : 250612_113054 (QP-Ver=804)
Log file  :
Groups    : 1
Tests     : 1
Skipped   : 0
FAILED    : 1 [ 1 ]

==============================[ FAIL (  3.2s) ]===============================
make: *** [Makefile:241: run] Error 1

Copying the Raw QSpy Output

The following console output shows the QSpy run of this (failing) test run:

   <F-END> Attached Chan=00000010
Run ID    : 250612_113944
Target    : build_host/qf_time.exe,localhost:6601

==================================[Group  1]==================================

File: qf_qtime.c
[ 1]--------------------------------------------------------------------------
TUN_QP_qf_time_01
   <COMMS> TCP-IP   Connected to Host=127.0.0.1,Port=59614
           Trg-RST  QP-Ver=804,Build=250612_113054
           Obj-Dict 0x00411500->QS_RX
           . . . .
           Fun-Dict 0x00404A7B->QActive_postLIFO_
           Fun-Dict 0x0040D8B9->QS_processTestEvts_
           QF_RUN
           Trg-Ack  QS_RX_TEST_SETUP
           Trg-Ack  QS_RX_CURR_OBJ
           Trg-Ack  QS_RX_EVENT
0000000001 MP-Get   Obj=EvtPool1,Free=9,Min=9
0000000002 QF-New   Sig=TRIG1_SIG,Size=8
0000000003 AO-Post  Sdr=QS_RX,Obj=the_TstAO,Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>,Que<Free=10,Min=10>
0000000004 AO-GetL  Obj=the_TstAO,Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>
0000000005 Disp===> Obj=the_TstAO,Sig=TRIG1_SIG,State=TstAO_stateA
0000000006 TE0-Arm  Obj=TstAO_inst.te0A,AO=the_TstAO,Tim=1,Int=0
0000000007 TE0-Arm  Obj=TstAO_inst.te0B,AO=the_TstAO,Tim=2,Int=0
0000000008 TE0-Arm  Obj=TstAO_inst.te0C,AO=the_TstAO,Tim=2,Int=2
===RTC===> St-Entry Obj=the_TstAO,State=TstAO_stateB
0000000009 ===>Tran Obj=the_TstAO,Sig=TRIG1_SIG,State=TstAO_stateA->TstAO_stateB
0000000010 QF-gc    Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>
0000000011 MP-Put   Obj=EvtPool1,Free=10
0000000012 Trg-Done QS_RX_EVENT
           Trg-Ack  QS_RX_TEST_TEARDOWN
[ 1]! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ! ![ FAIL (  0.1s) ]
   <COMMS> TCP-IP   Disconn from Host=127.0.0.1,Port=59614

==================================[ SUMMARY ]=================================

Target ID : 250612_113054 (QP-Ver=804)
Log file  :
Groups    : 1
Tests     : 1
Skipped   : 0
FAILED    : 1 [ 1 ]

==============================[ FAIL (  3.2s) ]===============================
   <F-END> Detached ######################################

Now, you select and copy to the Clipboard the raw QSpy output generated by the original post() command (lines from 0000000001... to 0000000012 Trg-Done QS_RX_EVENT).

Pasting to the Test Script

The following listing shows the test script from step-0 after pasting the raw QSpy output:

# test-script for QUTest unit testing harness
# see https://www.state-machine.com/qtools/qutest.html
note("File: qf_qtime.c")
#=============================================================================
# preamble...
#=============================================================================
# tests...
#-----------------------------------------------------------------------------
test("TUN_QP_qf_time_01")
current_obj(OBJ_AO, "the_TstAO")
post("TRIG1_SIG")
0000000001 MP-Get Obj=EvtPool1,Free=9,Min=9
0000000002 QF-New Sig=TRIG1_SIG,Size=8
0000000003 AO-Post Sdr=QS_RX,Obj=the_TstAO,Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>,Que<Free=10,Min=10>
0000000004 AO-GetL Obj=the_TstAO,Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>
0000000005 Disp===> Obj=the_TstAO,Sig=TRIG1_SIG,State=TstAO_stateA
0000000006 TE0-Arm Obj=TstAO_inst.te0A,AO=the_TstAO,Tim=1,Int=0
0000000007 TE0-Arm Obj=TstAO_inst.te0B,AO=the_TstAO,Tim=2,Int=0
0000000008 TE0-Arm Obj=TstAO_inst.te0C,AO=the_TstAO,Tim=2,Int=2
===RTC===> St-Entry Obj=the_TstAO,State=TstAO_stateB
0000000009 ===>Tran Obj=the_TstAO,Sig=TRIG1_SIG,State=TstAO_stateA->TstAO_stateB
0000000010 QF-gc Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>
0000000011 MP-Put Obj=EvtPool1,Free=10
0000000012 Trg-Done QS_RX_EVENT

Please note that the lines from 0000000001... to 0000000012 Trg-Done QS_RX_EVENT pasted from the raw QSpy output are not valid Python.

Running qutestif.py Test Converter

The following console output shows the run of the qutestify.py utility on the test_xyz.py test script (Please note the changed lines: 13 output, which informs you how many lines of the script have been changed.):

C:\qp-dev\safe-qpc\v-v\TUN_QP_qf_time\test>qutestify test_xyz.py

QUTestestify utility 8.0.4
Copyright (c) 2005-2025 Quantum Leaps, www.state-machine.com
qutestifying: test_xyz.py
changed lines: 13.

The following listing shows the updated test script (after refreshing the editor):

# test-script for QUTest unit testing harness
# see https://www.state-machine.com/qtools/qutest.html
note("File: qf_qtime.c")
#=============================================================================
# preamble...
#=============================================================================
# tests...
#-----------------------------------------------------------------------------
test("TUN_QP_qf_time_01")
current_obj(OBJ_AO, "the_TstAO")
post("TRIG1_SIG")
expect("@timestamp MP-Get Obj=EvtPool1,Free=9,Min=9")
expect("@timestamp QF-New Sig=TRIG1_SIG,Size=8")
expect("@timestamp AO-Post Sdr=QS_RX,Obj=the_TstAO,Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>,Que<Free=10,Min=10>")
expect("@timestamp AO-GetL Obj=the_TstAO,Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>")
expect("@timestamp Disp===> Obj=the_TstAO,Sig=TRIG1_SIG,State=TstAO_stateA")
expect("@timestamp TE0-Arm Obj=TstAO_inst.te0A,AO=the_TstAO,Tim=1,Int=0")
expect("@timestamp TE0-Arm Obj=TstAO_inst.te0B,AO=the_TstAO,Tim=2,Int=0")
expect("@timestamp TE0-Arm Obj=TstAO_inst.te0C,AO=the_TstAO,Tim=2,Int=2")
expect("===RTC===> St-Entry Obj=the_TstAO,State=TstAO_stateB")
expect("@timestamp ===>Tran Obj=the_TstAO,Sig=TRIG1_SIG,State=TstAO_stateA->TstAO_stateB")
expect("@timestamp QF-gc Evt<Sig=TRIG1_SIG,Pool=1,Ref=1>")
expect("@timestamp MP-Put Obj=EvtPool1,Free=10")
expect("@timestamp Trg-Done QS_RX_EVENT")

QUTest Fixture ReferenceQView Visualization & Monitoring