It is not needed anymore to hardcode the number of lines of
$FG_ROOT/Airports/apt.dat.gz. The new method, which relies on
SGPath::sizeInBytes() and sg_gzifstream::approxOffset(), works as well
for the other dat files.
Add a new NavCache rebuild phase, REBUILD_READING_APT_DAT_FILES, since
the process is now made of two parts.
Rename NavDataCachePrivate::getDatFilesPaths() to
NavDataCachePrivate::findDatFiles(), because it now returns a
DatFilesGroupInfo (new struct) instead of a PathList. For the same
reason, rename NavDataCachePrivate::aptDatPaths to
NavDataCachePrivate::aptDatFilesInfo. Adapt signatures, etc.
This requires up-to-date SimGear and FGData.
Add checks to avoid read-past-vector-bounds errors.
Make the current line number in each apt.dat file available to methods
such as parseRunwayLine810(), parseRunwayLine850(), etc. for useful
error messages.
In APTLoader::parseCommLine():
- use 'rowCode' instead of 'lineId' for consistency with the apt.dat
spec and the rest of the code;
- use 'unsigned int' for the type since row codes are always
non-negative;
- add a missing 'return' statement and improve an error message.
Small optimization in APTLoader::parseAirportLine().
This will be clearer since it is a PositionedID and we are also dealing
with airport identifiers in the sense of the apt.dat spec in the same
class.
Rename 'line_id' to 'rowCode' in src/Airports/apt_loader.cxx
"Row code" is the expression consistently used for this number in the
apt.dat format spec. Since we already deal with airport *identifiers*,
this should make this piece of code easier to read.
For the same reason, rename the 'num' parameter of
APTLoader::parsePavementNodeLine850() to 'rowCode'.
- APTLoader::parseViewpointLine(): new method.
- APTLoader::parseViewpointLine(): check vector size to prevent
past-bounds reading.
More accurate signature for fptypeFromRobinType()
Row codes in the apt.dat spec are always non-negative -> take an
unsigned int instead of just an int.
Use an initialization list for remaining members of APTLoader::APTLoader()
This is slightly more efficient in general. Swap the order of
declarations for the 'cache' and 'currentAirportID' members of
APTLoader to have the initialization list in the same order as the
member declarations (cf. g++'s -Wreorder option and
<http://stackoverflow.com/questions/1828037/whats-the-point-of-g-wreorder>).
Fix detection of blank lines by APTLoader::isBlankOrCommentLine()
In
<https://sourceforge.net/p/flightgear/flightgear/merge-requests/39/#cea6>,
it was decided to let the main apt.dat line reading loop give out
"lines" that may end with '\r', because most of the downstream code will
automatically get rid of this character thanks to its use of
simgear::strutils::split(). However, APTLoader::isBlankOrCommentLine()
didn't detect blank lines properly due to that trailing '\r', which
could cause bad behavior because the subsequent atoi() call could return
anything from a string containing only whitespace (the "anything" in
question being then interpreted as an apt.dat row code...).
Add method APTLoader::cleanLine()
This method returns a copy of the input line with trailing '\r' char(s)
removed.
APTLoader::loadAirports(): clean message when finding an unknown row code
The start of the log message could previously be overwritten by later
text because of the '\r' at the end of input lines (now obtained from
APTLoader::readAptDatFile()). Quite confusing! Use the new
APTLoader::cleanLine() to prevent this from happening.
It is now allowed to have the same airport appear in several apt.dat
files ($scenery_path/NavData/apt/*.dat[.gz] for each scenery path, plus
the default $FG_ROOT/Airports/apt.dat.gz, coming last). Airports found
in earlier files(*) take precedence over those found later, in case
several apt.dat files define the same airports. Airports that are
skipped due to this mechanism are logged with
SG_LOG(SG_GENERAL, SG_INFO, ...).
(*) using 1) FG_SCENERY order (followed by $FG_ROOT/Airports/apt.dat.gz)
and 2) lexicographic order inside each $scenery_path/NavData/apt
folder
With this commit, APTLoader::parseAPT() is replaced by two methods:
readAptDatFile() and loadAirports():
- APTLoader::readAptDatFile() reads airport definitions from an
apt.dat file into APTLoader's 'airportInfoMap' member variable,
discarding duplicate definitions due to overlapping apt.dat files
('airportInfoMap' is an std::unordered_map instance in C++11 and
later, an std::map otherwise);
- APTLoader::loadAirports() reads each airport definition from
'airportInfoMap' and loads it into the NavCache, the same way as
APTLoader::parseAPT() used to do.
The airportDBLoad() function is not useful anymore, and is thus removed
(in NavDataCache::doRebuild(), APTLoader::readAptDatFile() is now called
once per apt.dat file, but APTLoader::loadAirports() is only called
once at the end, after duplicate airports have been discarded; the class
interface is much better suited to this scheme, because it can cleanly
retain the state between these calls).
By the way, this commit fixes an old bug: APTLoader's member variable
'last_apt_id' was used in several places but never assigned to, except
in APTLoader::APTLoader() as the empty string.
Thanks to Alan Teeder for his feedback and testing.
Load every file matching the pattern NavData/apt/*.dat[.gz] inside each
scenery path. These files are loaded in the same order as the components
of globals->get_unmangled_fg_scenery() they reside in. Inside a given
component, the order is determined by pathSortPredicate() in
simgear/misc/sg_dir.cxx (lexicographic order at the time of this
writing). For compatibility with existing scenery,
$FG_ROOT/Airports/apt.dat.gz is also loaded last.
The idea is that such files will have the same precedence order as the
globals->get_unmangled_fg_scenery() scenery components they come from.
This commit doesn't handle this fully yet, though: it blindly loads all
these files. A future commit will ensure that no airport is loaded twice
due to overlapping apt.dat files. This commit however handles all the
logic of navdata cache rebuilding when the list, the order of apt.dat
files, or any of their timestamps changes.
Although only apt.dat files receive a new treatment in this commit, the
changes to NavDataCache.[ch]xx are already generic so that extension of
this method to fix.dat, nav.dat, etc. will require almost no change to
NavDataCache.[ch]xx (however, changes will probably be needed in the
various loaders: in fixlist.[ch]xx, navdb.[ch]xx, etc.).
src/Navaids/CacheSchema.h:
- increment the SCHEMA_VERSION by 1. This ensures among others that if
someone uses a FlightGear version posterior to this change with
new-style scenery (having NavData/apt/*.dat[.gz] files inside
scenery paths), then goes back to a FlightGear version anterior to
this change, his NavCache is rebuilt ignoring the in-scenery-paths
NavData/apt/*.dat[.gz] files, as expected with the old FlightGear
version.
src/Navaids/NavDataCache.cxx:
- NavDataCachePrivate: replace aptDatPath (SGPath) with aptDatPaths
(PathList).
- NavDataCachePrivate::getDatFilesPaths(): new method that returns the
list of $scenery_path/NavData/<type>/*.dat[.gz] files found inside
scenery paths (where <type> is one of 'apt', 'fix', etc.), plus the
historical file (e.g., $FG_ROOT/Airports/apt.dat.gz for the 'apt'
type).
- NavDataCachePrivate::areDatFilesModified(): new method that tells
whether any of these files (for a given type) has changed since the
last NavCache rebuild, or if their ordered list has changed.
- NavDataCachePrivate::isCachedFileModified(): minor changes.
- NavDataCache::updateListsOfDatFiles(): new method that updates the
lists of dat files used for NavCache freshness checking and
rebuilding, i.e. currently sets/updates d->aptDatPaths using the new
method d->getDatFilesPaths(), and d->metarDatPath, d->navDatPath,
d->fixDatPath, d->poiDatPath, etc. as usual. This method will be
useful for instance in the built-in launcher after updating scenery
paths and before calling NavDataCache::isRebuildRequired().
- NavDataCache::NavDataCache(): use
NavDataCache::updateListsOfDatFiles() to initialize d->aptDatPaths,
d->metarDatPath, d->navDatPath, d->fixDatPath, d->poiDatPath, etc.
- NavDataCache::isRebuildRequired(): use
NavDataCachePrivate::areDatFilesModified() instead of just checking
$FG_ROOT/Airports/apt.dat.gz.
- NavDataCache::doRebuild(): load all apt.dat files listed in
d->aptDatPaths, instead of only $FG_ROOT/Airports/apt.dat.gz. Write
their ordered list and timestamps in the NavCache.
src/Navaids/NavDataCache.hxx:
- declare the new method NavDataCache::updateListsOfDatFiles().
- NavDataCache::DatFileType: new enum with values DATFILETYPE_APT,
DATFILETYPE_METAR, DATFILETYPE_AWY, DATFILETYPE_NAV,
DATFILETYPE_FIX, DATFILETYPE_POI, DATFILETYPE_CARRIER and
DATFILETYPE_TACAN_FREQ. Maybe some of the corresponding files won't
have to be moved to scenery paths, but simply listing them in the
enum doesn't change how they are dealt with. Those for which
per-scenery-path locations doesn't make sense can just be removed
from the enum.
- NavDataCache::datTypeStr: new static string_list giving an
std::string such as 'apt' for each value of the
NavDataCache::DatFileType enum.
- NavDataCache::defaultDatFile: new static string_list giving a path
(relative to $FG_ROOT) to the historical/default file for each value
of the NavDataCache::DatFileType enum.
src/Airports/apt_loader.cxx and src/Airports/apt_loader.hxx:
- always include a path to the apt.dat file being processed in log
messages, since they can now apply to many files;
- be clearer about code 99: it should normally be at the end of
apt.dat files, but technically, it is not an EOF;
- use the expression "row code" consistently with the apt.dat format
spec (for now: only in places where there is another change to do).
src/GUI/QtLauncher.cxx and src/GUI/QtLauncher_private.hxx:
- turn QtLauncher::setSceneryPaths() into a static method and call it
in runLauncherDialog() before instantiating NavDataCache, so that
NavDataCache::updateListsOfDatFiles() (called from NavDataCache's
constructor) can see all configured scenery paths.
This will avoid problems in case stampCacheFile() or
isCachedFileModified() is passed a relative path, or a path containing
.. components, etc. Among others, it ensures the stat_cache table only
contains absolute paths.
NavDataCache: read and write methods preserving order for string list props
New methods: NavDataCache::readOrderedStringListProperty() and
NavDataCache::writeOrderedStringListProperty().
add missing includes; proper interface for the APTLoader class
remove unused members 'last_apt_info' and 'token'
The APTLoader methods defined in src/Airports/apt_loader.cxx are now at
top level instead of being inside a 'class' definition block, therefore
they don't have to be indented anymore.
APTLoader::throwExceptionIfStreamError(): better API and implementation
- loadFixes() becomes FixesLoader::loadFixes();
- the previous 'cache' variable (NavDataCache *) is now a private member
of FixesLoader, named '_cache'.
- Line numbering was incorrect, because "in >> lat >> lon >> ident;"
happily skipped over blank lines without increasing the line number.
- Safe stream input handling: first, std::getline() tries to read data,
then we check whether that was successful (via the input stream's
bool() method, implicitly called in the 'for' loop's exit check), and
only if this is the case, we process the data that was read. The main
problem with the previous code is that checking the stream's eofbit
can't possibly predict whether a _future_ read will be successful---it
may fail due to an I/O error, at least.
- Currently, the code uses atof() to parse the latitude and longitude
fields. This should be fast, though not good at detecting errors in
the input; however, this is not worse than the previous code which
didn't handle such cases at all.
- Correctly deal with input lines containing a number of fields
different from 3 (except for the header and the special '99' line):
log a warning, ignore the line and continue. This adresses the problem
described in
<https://sourceforge.net/p/flightgear/mailman/flightgear-devel/thread/87shydeen1.fsf%40frougon.crabdance.com/#msg35034274>.
In short, the previous code entered an endless loop trying to process
31.815914 -106.281897 EL PA
from recent earth_fix.dat
(obtained from <http://gateway.x-plane.com/navaids/LatestNavFix.zip>).
Expand existing —-config option to read either property-XML files (as
it previously did) but for non-XML files, to parse them as command
line argument files.
Bypass the options system to set location from the launcher; this
allows the same code to be used in-sim for repositioning, while
keeping compatibility with other repositions approaches.
With Simgear commit d7d59b08a2f1a77a4247ec1a89d6ff48ed73f5c7, this
allows terrasync to be initialised from files in the install data,
which avoids downloading them again.
