Repeatability of test-case SETTE tests
Context
When using the default build locations for SETTE tests (variable CUSTOM_DIR
undefined), the SETTE testing of the test cases in the main
branch is not repeatable without intervention, even if the "clean" option (-c
) has been selected.
Analysis
The default build path for test cases is set to the value of variable CONFIG_DIR0
, i.e. to the cfgs
subdirectory, which differs from the path used by makenemo
to create the actual test subdirectories when testing test cases. As a consequence, identically named test subdirectories are created inside both the cfgs
and tests
subdirectories, which causes makenemo
to exit prematurely with an error message in subsequent attempts to SETTE test the corresponding test case.
Fix
Unless a buld location is explicitly specified via variable CUSTOM_DIR
, it is recommended to suppress the -t
option in the makenemo
calls throughout all SETTE scripts.