Some of the unit tests defined with the new test infrastructure[1]
initialize the QApplication; for these tests, it is necessary to ensure
that the QApplication instance is destroyed before exit() begins,
otherwise we get a segfault when run_test_suite terminates (see [2]).
In order to prevent this segfault from happening, call
flightgear::shutdownQtApp() unconditionally from
fgtest::shutdownTestGlobals() as long as Qt support is compiled in
(flightgear::shutdownQtApp() is safe to call even if the QApplication
hasn't been initialized).
[1] Currently, the three PosInitTests::* tests.
[2] https://bugreports.qt.io/browse/QTBUG-48709
The exceptions that occur after this call can be handled by CppUnit, but the
exit() results in the test suite silently exiting without restoring stdout and
stderr.
This allows for finer control of the test suite execution. The default of ON
will cause the test-suite to be executed after compiling the test_suite target.
This includes the following help message detailing all of the new options:
"""
Usage: run_test_suite [options]
Options:
-h, --help show this help message and exit.
Test selection options:
-s, --system-tests execute the system/functional tests.
-u, --unit-tests execute the unit tests.
-g, --gui-tests execute the GUI tests.
-m, --simgear-tests execute the simgear tests.
The -s, -u, -g, and -m options accept an optional argument to perform a
subset of all tests. This argument should either be the name of a test
suite or the full name of an individual test.
Full test names consist of the test suite name, the separator '::' and then
the individual test name. The test names can revealed with the verbose
option.
Verbosity options:
-v, --verbose verbose output including names and timings for all
tests.
"""
The CppUnit source is built as a static library and linked to the run_test_suite
executable. The FlightGear CppUnit library is now build using CMake rather than
using the Autotools build system.
This required many time related include files, functions, and variables to be
tested for via CMake, and converted to macros via #cmakedefine, so that they
would be automatically set for the test-mktime.cxx file.
This old standalone test, from Curt back in 2001, was converted to the format
required for the CppUnit test suite. As most of simgear/math/sg_geodesy.hxx no
longer exists, the code was updated using SGGeod and SGGeoc. Hardcoded values
are used to permanently lock in the values currently calculated by simgear.
Each of the global logstream priorities are captured into its own
std::ostringstream stream, all held together in the test suite global _iostreams
class instance. This object can be obtained by calling getIOstreams(). The
streams are captured using the StreamLogCallback class, which is a simple
modification of the simgear FileLogCallback class, registered with the global
logstream's addCallback() function.
When tests fail, all of contents the different simgear logstreams are now
reported. The failure report consists of the following sections:
- Failure information.
- SG_BULK simgear logstream (all messages).
- SG_BULK only simgear logstream.
- SG_DEBUG only simgear logstream.
- SG_INFO only simgear logstream.
- SG_WARN only simgear logstream.
- SG_ALERT only simgear logstream.
- Combined STDOUT and STDERR streams.
Any empty sections, except for SG_BULK, will not be shown.
A large number of improvements have been made by subclassing the CppUnit classes
and overriding the base functions, including CompilerOutputter, TestListener,
and TextTestRunner. All IO has also been captured by the test runner.
The result is that during the test suite run, only the characters '.', 'F', and
'E' are output for a pass, failure and error state. At the end of each test
suite category, all failures and errors are reported in full detail, including
the different captured IO streams. A final synopsis is printed out, improving
the overview in the case of too many tests failing.
For the fgCompilerOutputter class, the printSuccess(), printFailureReport(),
printFailureDetail(), and printSuiteStats() functions have been replaced to
implement the above printout design. The class also stores the std::vector of
TestIOCapt structures for the final printouts.
The fgTestListener class handles the events from the running of the test suite.
The startTest() and endTest() functions are used for IO capture. The IO is
placed into a TestIOCapt data structure, with one std::string for holding the
combined STDOUT and STDERR IO, and another for the SG_LOG IO. If failures
occur, the TestIOCapt structure is appended to the fgCompilerOutputter vector.
The startTest() and endTest() functions are also used for starting and stopping
a timer to allow the full test suite to be timed. And the addFailure() function
simply registers test failures or errors.
The fgTestRunner class overrides the CppUnit::TextTestRunner::run() function,
simply to prevent the base method from spawning a second test listener, causing
the test output to be duplicated.
Some auxiliary formatting functions have been added to print out titles,
sections, and synopsis summary lines.
This includes the basic CMake infrastructure for building and executing the test
suite. Four test categories have been added - unit, system/functional, GUI, and
simgear unit tests. The test suite is run by typing 'make test_suite'.
All of the fgfs sources are included in the new run_test_suite executable,
excluding the bootstrap routine and its main() function. The test suite
currently consists of a single dummy unit test for the NasalSys subsystem, and a
single demonstration simgear/props unit test.
Since the new _storagePath data member internally contains the add-on
id, changing _id after _storagePath has been initialized would make both
data members inconsistent. As changing the add-on id is probably not a
very useful operation, the simplest way to prevent such an inconsistency
from happening is to make Addon's _id data member const (as is already
the case for _storagePath), and thus remove Addon::setId().
Consequently, remove the Addon default constructor too, since add-ons
with an empty id would be ill-formed and couldn't be changed (_id being
const now). This leaves us with one Addon constructor:
Addon(std::string id, AddonVersion version = AddonVersion(),
SGPath basePath = SGPath(), std::string minFGVersionRequired = "",
std::string maxFGVersionRequired = "",
SGPropertyNode* addonNode = nullptr);
New methods Addon::createStorageDir() and Addon::getStoragePath() with
corresponding Nasal bindings in the addons.Addon ghost:
createStorageDir() method (returns the dir, doesn't fail if it already
exists)
storagePath read-only attribute to get the dir
The directory reserved for each add-on is
$FG_HOME/Export/Addons/ADDON_ID, but please use the above methods (or
the corresponding C++ ones) to avoid hardcoding such paths in your code.
Also create directory $FG_HOME/Export/Addons in fgInitConfig() as a way
of reserving the namespace, in order to prevent future failures in case
someone would have the strange idea to create it as a file...
It is the call to exit that causes FG to lock up and become a zombie.
Alternative is to throw an exception, i.e.:
throw sg_error(std::string("YASim SOLUTION FAILURE:") + a->getFailureMsg(););
This adds the fix for non converging aircraft. Henning has performed a thorough analysis of all of FGAddon and other repositories and there are a few models that fail, however there have always been a few that fail - but with this patch this situation is improving.
