From 31ec7278725f7d489de87b3941dd9427e111d92b Mon Sep 17 00:00:00 2001 From: Julian Smith <jules@op59.net> Date: Thu, 24 Dec 2020 14:21:38 +0000 Subject: [PATCH] Added record/replay of extra properties, with specific support for window size/position and view settings. Recording of extra properties is only supported in Continuous recordings. Modified Continuous recording format to allow future forwards compatibility. See docs-mini/README-recordings.md for details. This breaks compatibility with previously-generated Continuous recordings, but this only affects next. To reduce overhead we record all extra property values in the first frame and then later frames contain only extra property changes. When replaying, if the user jumps backwards we replay all extra property changes since the start of the recording. Similarly if the user jumps forwards, we replay any intervening extra property changes. Recording extra properties: This is enabled by: /sim/replay/record-extra-properties The extra properties that are recorded are identified by the property paths in the values of /sim/replay/record-extra-properties-paths/path[] properties. We record the entire tree for each specified path. Recording of main window position size: We have specific support for record and replay of main window position/size. This is enabled by: /sim/replay/record-main-window Recording of main window view: We have specific support for recording the view type and direction/zoom settings. This is enabled by: /sim/replay/record-main-view We record the /sim/current-view/ property tree, excluding some subtrees that continuously vary but are not required for replaying of the view. When replaying, we allow separate control of what extra property changes are replayed, with: /sim/replay/replay-extra-property-changes /sim/replay/replay-extra-property-removal /sim/replay/replay-main-window-position /sim/replay/replay-main-window-size /sim/replay/replay-main-view We work around some problems caused by the use of tied properties when replaying changes to view-number. Window position issue: When replaying window position and size changes, things get a little tricky because osgViewer::GraphicsWindow::setWindowRectangle() takes a position for the entire window, but osgGA::GUIEventAdapter::RESIZE events contain the position of the interior of the window; for example the y values will differ by the height of the window's titlebar. This can cause windows to move progressively further down each time they are positioned or resized. There doesn't seem to be a way of finding the size of a window's furniture directly. So instead this commit adds a new method osgGA::GUIEventAdapter::setWindowRectangleInteriorWithCorrection() which wraps osgViewer::GraphicsWindow::setWindowRectangle(). We listen for the following osgGA::GUIEventAdapter::RESIZE event and calculate corrections for x and y position that are used for subsequent calls. docs-mini/README-recordings.md: Updated to document new Continuous format. scripts/python/recordreplay.py: New script to test various aspects of record/replay. Other: We now create convenience softlink to most recent continuous recording, using SGPath::makeLink(). Note that SGPath::makeLink() currently does nothing on Windows. Changed format of Continuous recordings to contain a single property tree in header. This is much simpler than having separate Config and Meta trees. --- docs-mini/README-recordings.md | 78 +++- scripts/python/recordreplay.py | 395 +++++++++++++++++ src/Aircraft/flightrecorder.cxx | 292 ++++++++++++- src/Aircraft/flightrecorder.hxx | 21 +- src/Aircraft/replay.cxx | 752 +++++++++++++++++++++++++------- src/Aircraft/replay.hxx | 60 ++- src/Viewer/FGEventHandler.cxx | 45 +- src/Viewer/FGEventHandler.hxx | 11 + src/Viewer/renderer.cxx | 20 +- src/Viewer/renderer.hxx | 4 +- 10 files changed, 1467 insertions(+), 211 deletions(-) create mode 100755 scripts/python/recordreplay.py diff --git a/docs-mini/README-recordings.md b/docs-mini/README-recordings.md index f3b9c4a9d..eacf1d75c 100644 --- a/docs-mini/README-recordings.md +++ b/docs-mini/README-recordings.md @@ -10,13 +10,29 @@ As of 2020-12-22, there are three kinds of recordings: Normal recordings are compressed and contain frames at varying intervals with more recent frames being closer together in time. They are generated from the in-memory recording that Flightgear always maintains. They may contain multiplayer information. -Continuous recordings are uncompressed and written directly to a file while Flightgear runs, giving high resolution with near unlimited recording time. They may contain multiplayer information, and (as of 2020-12-23) may contain information about arbitrary properties. +Continuous recordings are uncompressed and written directly to a file while Flightgear runs, giving high resolution with near unlimited recording time. They may contain multiplayer information. As of 2020-12-23 they may contain information about extra properties, allowing replay of views and main window position/size. Recovery recordings are essentially single-frame Continuous recordings. When enabled, Flightgear creates them periodically to allow recovery of a session if Flightgear crashes. +## Properties + +* `/sim/replay/tape-directory` - where to save recordings. +* `/sim/replay/record-multiplayer` - if true, we include multiplayer information in Normal and Continuous recordings. +* Normal recordings: + * `/sim/replay/buffer/high-res-time` - period for high resolution recording. + * `/sim/replay/buffer/medium-res-time` - period for medium resolution. + * `/sim/replay/buffer/low-res-time` - period for low resolution. + * `/sim/replay/buffer/medium-res-sample-dt` - sample period for medium resolution. + * `/sim/replay/buffer/low-res-sample-dt` - sample period for low resolution. +* Continuous recordings: + * `/sim/replay/record-continuous` - if true, do continuous record to file. + * `/sim/replay/record-extra-properties` - if true, we include selected properties in recordings. +* Recovery recordings: + * `/sim/replay/record-recovery-period` - if non-zero, we update recovery recording in specified interval. + ## Code -The code that creates recordings is a not particularly clean or easy to work with. +The code that creates recordings is not particularly clean or easy to work with. It consists mainly of two source files: @@ -29,43 +45,63 @@ Despite their names these files are both involved with record and replay. `src/A ## File format -Flightgear recording files have the following structure: +### Normal recordings * Header: - * A zero-terminated magic string: `FlightGear Flight Recorder Tape` (see the `FlightRecorderFileMagic`). - - * A Meta property tree containing a single `meta` node with various child nodes. + * A zero-terminated magic string: `FlightGear Flight Recorder Tape` (variable `FlightRecorderFileMagic`). + * A Meta property tree containing a `meta` node with various child nodes. + * A Config property tree containing information about what signals will be contained in each frame. Each signal is a property; signals are used for the main recorded information such as position and orientation of the user's aircraft, and positions of flight surfaces, gear up/down etc. Aircraft may define their own customised set of signals. - The Meta and Config property trees are each written as a binary length integer followed by a text representation of the properties. - - For Continuous recordings, the header is written by `FGReplay::continuousWriteHeader()`. + The Meta and Config property trees are each written as `<length:64><text>` where `<text>` is a text representation of the properties. `<text>` is terminated with a zero which is included in the `<length:64>` field. * A series of frames, each containg the data in a `FGReplayData` instance, looking like: * Frame time as a binary double. - * Signals information as described in the header's Config property tree, in a fixed-size binary format. + * Signals information as described in the header's `Config` property tree, represented as a 64-bit length followed by binary data. - * If the header's Meta property tree has a child node `multiplayer` set to true, we append recorded information about multiplayer aircraft. This will be a binary integer containing the number of multiplayer messages to follow then, for each message, the message size followed by the message contents. The message contents are exactly the multiplayer packet as received from the network. +All data after the header is in a single gzipped stream. - * **[Work in progress as of 2020-12-23]** (Continuous recordings only) If the header's Meta property tree has a child node `record-properties` set to true, we append a variable list of property changes: +### Continuous recordings - * Length of following property-change data. +* Header: - * `<length><path><length><string_value>` + * A zero-terminated magic string: `FlightGear Flight Recorder Tape` (variable `FlightRecorderFileMagic`). - * `...` + * A property tree represented as a `uint32` length followed by zero-terminated text. This contains: + + * A `meta` node with various child nodes. + + * `data[]` nodes describing the data items in each frame in the order in which they occur. Supported values are: + + * `signals` - core information about the user's aircraft. + * `multiplayer` - information about multiplayer aircraft. + * `extra-properties` - information about extra properties. + + * A `signals` node containing layout information for signals, in the same format as for Normal recordings. + + The header is written by `FGReplay::continuousWriteHeader()`. + +* A series of frames, each containg the data in a `FGReplayData` instance, looking like: + + * Frame time as a binary double. + + * A list of ordered `<length:32><data>` items as described by the `data[]` nodes in the header. This format allows Flightgear to skip data items that it doesn't understand if loading a recording that was created by a newer version. - Removal of a property is encoded as an item consisting of a zero integer followed by `<length><path>`. - - All numbers here are 16 bit binary. + * For `signals`, `<data>` is binary data for the core aircraft properties. + + * For `multiplayer`, `<data>` is a list of `<length:16><packet>` items where `<packet>` is a multiplayer packet as received from the network. -In Normal recordings, all data after the header is in a single gzipped stream. + * For `extra-properties`, `<data>` is a list of property changes, each one being: + + * `<length:16><path><length:16><value>` - property <path> has changed to `<value>`. + + Removal of a property is encoded as `<0:16><length:16><path>`. Continuous recordings do not use compression, in order to simplify random access when replaying. @@ -74,12 +110,14 @@ Continuous recordings do not use compression, in order to simplify random access When a Continuous recording is loaded, `FGReplay::loadTape()` first steps through the entire file, building up a mapping in memory from frame times to offsets within the file, so that we can support the user jumping forwards and backwards in the recording. +If the recording contains extra properties, we also build a cache of the locations of frames that have a non-empty extra-properties item, again to support jumping around in time. + ## Multiplayer ### Recording while replaying: -If the user replays part of their history while we are saving to a Continuous recording, and the Continuous recording includes multiplayer information, then we carry on receiving multiplayer information from the network and writing it to the Continuous recording. The user's aircraft is recorded as being stationary while the user was replaying. +If the user replays part of their history while we are saving to a Continuous recording, and the Continuous recording includes multiplayer information, then we carry on receiving multiplayer information from the network and writing it to the Continuous recording. The user's aircraft is recorded as being stationary for the period when the user was replaying. ### Replaying diff --git a/scripts/python/recordreplay.py b/scripts/python/recordreplay.py new file mode 100755 index 000000000..6ff6b5d42 --- /dev/null +++ b/scripts/python/recordreplay.py @@ -0,0 +1,395 @@ +#!/usr/bin/env python3 + +''' +Test script for record/replay. Only tested on Unix. + +E.g.: + + ./flightgear/scripts/python/recordreplay.py -f run_fgfs.sh + +Args: + --all + Run all tests (this is default). + --continuous BOOLS + --extra-properties BOOLS + --it-max <it> + Set min iteration (inclusive) to run; useful when fixing tests and + retrying. + --it-min <it> + Set max iteration (exclusive) to run; useful when fixing tests and + retrying. + --main-view BOOLS + --multiplayer + -f <fgfs> + The fgfs executable to use. Default assumes the Walk build system. + --f-old <fgfs-old> + A second fgfs executable. If specified we run all tests twice, first + using <fgfs-old> to create the recording and <fgfs> to replay it, + second the other way round. + + BOOLS is comma-sparated list of 0 or 1, with 1 activating the particular + feature. So for example '--continuous 0' tests normal recording/replay', + '--continuous 1' tests continuous recording/replay, and continuous 0,1' + tests both. + + We test all combinations of continuous, extra-properties, main-view and + multiplayer recordings. For each test we check that we can create a + recording, and replay it in a new fgfs instance. When replaying we check + a small number of basic things such as the recording length, and whether + extra-properties are replayed. +''' + +import os +import signal +import sys +import time + +import FlightGear + +def log(text): + print(text, file=sys.stderr) + sys.stderr.flush() + +g_cleanup = [] + +class Fg: + ''' + Runs flightgear. self.fg is a FlightGear.FlightGear instance, which uses + telnet to communicate with Flightgear. + ''' + def __init__(self, aircraft, args, env=None): + ''' + aircraft: + Specified as: --aircraft={aircraft}. This is separate from <args> + because we need to know the name of recording files. + args: + Miscellenous args either space-separated name=value strings or a + dict. + env: + Environment to set. If DISPLAY is not in <env> we add 'DISPLAY=:0'. + ''' + self.pid = None + self.aircraft = aircraft + args += f' --aircraft={aircraft}' + + port = 5500 + args += f' --telnet={port}' + args2 = args.split() + + if isinstance(env, str): + env2 = dict() + for nv in env.split(): + n, v = nv.split('=', 1) + env2[n] = v + env = env2 + environ = os.environ.copy() + if env: + environ.update(env) + if 'DISPLAY' not in environ: + environ['DISPLAY'] = ':0' + + # Run flightgear in new process, telling it to open telnet server. + self.pid = os.fork() + if self.pid==0: + log(f'calling os.exec(): {" ".join(args2)}') + os.execve(args2[0], args2, environ) + + # Connect to flightgear's telnet server. + timeout = 15 + t0 = time.time() + while 1: + time.sleep(1) + dt = time.time() - t0 + if dt > timeout: + text = f'Timeout trying to connect. timeout={timeout}' + log(text) + raise Exception(text) + try: + log('Connecting... ') + self.fg = FlightGear.FlightGear('localhost', port) + log(f'Connected. timeout={timeout} dt={dt:.1f}') + return + except Exception as e: + log(f'Failed to connect timeout={timeout} dt={dt:.1f}: {e}') + + def waitfor(self, name, value, timeout=30): + ''' + Waits for specified property to be <value>. Returns time taken. + ''' + t0 = time.time() + while 1: + time.sleep(1) + dt = time.time() - t0 + try: + v = self.fg[name] + log(f'Waiting for {name}={value} current value: {v}. timeout={timeout} dt={dt:.1f}') + except Exception as e: + log(f'Failed to get value of property {name}: {e}. timeout={timeout} dt={dt:.1f}') + v = None + if v == value: + return dt + if dt > timeout: + raise Exception(f'Timeout waiting for {name}={value}; current value: {v}. timeout={timeout}') + + def run_command(self, command): + self.fg.telnet._putcmd(command) + ret = self.fg.telnet._getresp() + log(f'command={command!r} ret: {ret}') + return ret + + def close(self): + assert self.pid + sig = signal.SIGTERM + log(f'Sending sig={sig} to pid={self.pid}') + os.kill(self.pid, sig) + log(f'Waiting for pid={self.pid}') + r = os.waitpid(self.pid, 0) + log(f'waitpid => {r}') + self.pid = None + + def __del__(self): + if self.pid: + log('*** Fg.__del__() calling self.close()') + self.close() + +def make_recording( + fg, + continuous=0, + extra_properties=0, + main_view=0, + length=5, + ): + ''' + Makes a recording, and returns its path. + + We check that the recording file is newly created. + ''' + t = time.time() + if continuous: + assert not fg.fg['/sim/replay/record-continuous'] + fg.fg['/sim/replay/record-continuous'] = 1 + t0 = time.time() + while 1: + if time.time() > t0 + length: + break + fg.run_command('run view-step step=1') + time.sleep(1) + fg.fg['/sim/replay/record-continuous'] = 0 + path = f'{fg.aircraft}-continuous.fgtape' + time.sleep(1) + else: + # Normal recording will have effectively already started, so we sleep + # for the remaining time. This is a little inaccurate though because it + # looks like normal recording starts a little after t=0, e.g. at t=0.5. + # + # Also, it looks like /sim/time/elapsed-sec doesn't quite track real + # time, so we sometimes need to sleep a little longer. + # + while 1: + t = fg.fg['/sim/time/elapsed-sec'] + log(f'/sim/time/elapsed-sec={t}') + if t > length: + break + time.sleep(length - t + 0.5) + log(f'/sim/time/elapsed-sec={t}') + fg.fg.telnet._putcmd('run save-tape tape-data/starttime= tape-data/stoptime=') + response = fg.fg.telnet._getresp() + log(f'response: {response!r}') + path = f'{fg.aircraft}.fgtape' + + # Check recording is new. + os.system(f'ls -lL {path}') + s = os.stat(path, follow_symlinks=True) + assert s.st_mtime > t + path2 = os.readlink(path) + log(f'path={path} path2={path2}') + return path + + +def test_record_replay( + fgfs_save, + fgfs_load, + multiplayer, + continuous, + extra_properties, + main_view, + length, + ): + if not fgfs_load: + fgfs_load = fgfs + log(f'=== save: {fgfs_save}') + log(f'=== load: {fgfs_load}') + log(f'=== --multiplayer {multiplayer} --continuous {continuous} --extra-properties {extra_properties} --main-view {main_view}') + log(f'===') + + aircraft = 'harrier-gr3' + args = f'--state=vto --airport=egtk' + args += f' --prop:bool:/sim/replay/record-extra-properties={extra_properties}' + args += f' --prop:bool:/sim/replay/record-main-view={main_view}' + args += f' --prop:bool:/sim/replay/record-main-window=0' + + # Start Flightgear. + fg = Fg(aircraft, f'{fgfs_save} {args}', + #env='SG_LOG_DELTAS=flightgear/src/Network/props.cxx=4', + ) + fg.waitfor('/sim/fdm-initialized', 1, timeout=45) + + assert fg.fg['sim/replay/record-extra-properties'] == extra_properties + assert fg.fg['sim/replay/record-main-view'] == main_view + log(f'sim/replay/record-extra-properties={fg.fg["sim/replay/record-extra-properties"]}') + + # Save recording: + path = make_recording(fg, + continuous=continuous, + extra_properties=extra_properties, + main_view=main_view, + length=length, + ) + + g_cleanup.append(lambda: os.remove(path)) + fg.close() + + # Load recording into new Flightgear. + path = f'{aircraft}-continuous.fgtape' if continuous else f'{aircraft}.fgtape' + fg = Fg(aircraft, f'{fgfs_load} {args} --load-tape={path}') + fg.waitfor('/sim/fdm-initialized', 1, timeout=45) + fg.waitfor('/sim/replay/replay-state', 1) + + # Check replay time is ok. + rtime_begin = fg.fg['/sim/replay/start-time'] + rtime_end = fg.fg['/sim/replay/end-time'] + rtime = rtime_end - rtime_begin + log(f'rtime={rtime_begin}..{rtime_end}, recording length: {rtime}, length={length}') + assert rtime > length-1 and rtime < length+2, \ + f'length={length} rtime_begin={rtime_begin} rtime_end={rtime_end} rtime={rtime}' + + num_frames_extra_properties = fg.fg['/sim/replay/continuous-stats-num-frames-extra-properties'] + log(f'num_frames_extra_properties={num_frames_extra_properties}') + if continuous: + if main_view: + assert num_frames_extra_properties > 1 + else: + assert num_frames_extra_properties == 0 + else: + assert num_frames_extra_properties in (0, None), \ + f'num_frames_extra_properties={num_frames_extra_properties}' + + time.sleep(length) + + fg.close() + + os.remove(path) + + log('Test passed') + + +if __name__ == '__main__': + + fgfs = f'./build-walk/fgfs.exe-run.sh' + fgfs_old = None + + continuous_s = [0, 1] + extra_properties_s = [0, 1] + main_view_s = [0, 1] + multiplayer_s = [0, 1] + fgfs_reverse_s = [0] + it_min = None + it_max = None + + if len(sys.argv) == 1: + do_all = True + + args = iter(sys.argv[1:]) + while 1: + try: + arg = next(args) + except StopIteration: + break + if arg == '--all': + do_all = True + elif arg == '--continuous': + continuous_s = map(int, next(args).split(',')) + elif arg == '--extra-properties': + extra_properties_s = map(int, next(args).split(',')) + elif arg == '--it-max': + it_max = int(next(args)) + elif arg == '--it-min': + it_min = int(next(args)) + elif arg == '--main-view': + main_view_s = map(int, next(args).split(',')) + elif arg == '--multiplayer': + multiplayer_s = map(int, next(args).split(',')) + elif arg == '-f': + fgfs = next(args) + elif arg == '--f-old': + fgfs_old = next(args) + fgfs_reverse = [0, 1] + else: + raise Exception(f'Unrecognised arg: {arg!r}') + + try: + if fgfs_old: + for fgfs1, fgfs2 in [(fgfs, fgfs_old), (fgfs_old, fgfs)]: + for multiplayer in 0, 1: + test_record_replay( + fgfs1, + fgfs2, + multiplayer, + continuous=0, + extra_properties=0, + main_view=0, + length=10, + ) + else: + its_max = len(multiplayer_s) * len(continuous_s) * len(extra_properties_s) * len(main_view_s) * len(fgfs_reverse_s) + it = 0 + for multiplayer in multiplayer_s: + for continuous in continuous_s: + for extra_properties in extra_properties_s: + for main_view in main_view_s: + for fgfs_reverse in fgfs_reverse_s: + if fgfs_reverse: + fgfs_save = fgfs_old + fgfs_load = fgfs + else: + fgfs_save = fgfs + fgfs_load = fgfs_old + + ok = True + if it_min is not None: + if it < it_min: + ok = False + if it_max is not None: + if it >= it_max: + ok = False + log('') + log(f'===') + log(f'=== {it}/{its_max}') + if ok: + test_record_replay( + fgfs_save, + fgfs_load, + multiplayer=multiplayer, + continuous=continuous, + extra_properties=extra_properties, + main_view=main_view, + length=10 + ) + it += 1 + finally: + pass + + # If everything passed, cleanup. Otherwise leave recordings in place, as + # they can be useful for debugging. + # + for f in g_cleanup: + try: + f() + except: + pass + + if 0: + # This path can be used to check we cleanup properly after an error. + fg = Fg('./build-walk/fgfs.exe-run.sh --aircraft=harrier-gr3 --airport=egtk') + time.sleep(5) + assert 0 diff --git a/src/Aircraft/flightrecorder.cxx b/src/Aircraft/flightrecorder.cxx index afcfcaecb..13df76db8 100644 --- a/src/Aircraft/flightrecorder.cxx +++ b/src/Aircraft/flightrecorder.cxx @@ -42,9 +42,190 @@ using namespace FlightRecorder; using std::string; + +// Appends raw data to <out>. +static void s_Append(std::vector<char>& out, const void* data, size_t data_len) +{ + size_t pos = out.size(); + out.resize(pos + data_len); + memcpy(&out[pos], data, data_len); +} + +// Appends int16 number to <out>. +static void s_AppendNumber(std::vector<char>& out, int16_t number) +{ + s_Append(out, &number, sizeof(number)); +} + +// Appends string to <out> as int16 length followed by chars. +static void s_AppendString(std::vector<char>& out, const std::string& s) +{ + s_AppendNumber(out, s.size()); + s_Append(out, s.c_str(), s.size()); +} + +// Updates property <b> so that it is identical to property <a> and writes +// information on all differences (creation, value changes and deletion) to +// <out>. +// +// We call ourselves recursively for child nodes. +// +// We are careful to preserve indices. +// +// We treat all property values as text. +// +static void s_RecordPropertyDiffs( + std::vector<char>& out, + SGPropertyNode* a, + SGPropertyNode* b, + const std::vector<const char*>* path_exclude_prefixes + ) +{ + assert(a); + assert(b); + assert(!strcmp(a->getName(), b->getName())); + assert(a->getPath() == b->getPath()); + assert(a->getIndex() == b->getIndex()); + + if (path_exclude_prefixes) { + for (const char* path_exclude_prefix: *path_exclude_prefixes) { + if (simgear::strutils::starts_with(a->getPath(true /*simplify*/), path_exclude_prefix)) { + SG_LOG(SG_SYSTEMS, SG_BULK, "Ignoring: " << a->getPath(true /*simplify*/)); + return; + } + } + } + // If values differ, write a's value to out and change b's value to a's value. + const char* a_value = a->getStringValue(); + const char* b_value = b->getStringValue(); + if (strcmp(a_value, b_value)) { + // Values are different so write out node <a> and update b. + SG_LOG(SG_SYSTEMS, SG_DEBUG, "recording property change:" + << a->getPath() + << ": " << b->getStringValue() + << " => " << a->getStringValue() + ); + s_AppendString(out, a->getPath(true /*simplify*/)); + s_AppendString(out, a->getStringValue()); + b->setStringValue(a_value); + } + + // Look at all child nodes of <b>, removing any that are not in <a>. + int bn = b->nChildren(); + for (int i=0; i<bn; ++i) { + SGPropertyNode* bc = b->getChild(i); + SGPropertyNode* ac = a->getChild(bc->getName(), bc->getIndex(), false /*create*/); + if (!ac) { + // Child node is in b but not in a; we write out special + // information about the deleted node and remove from b. + s_AppendString(out, ""); + s_AppendString(out, bc->getPath(true /*simplify*/)); + b->removeChild(bc); + } + } + + // Look at all child nodes of <a>, copying across to <b> as required. + // + int an = a->nChildren(); + for (int i=0; i<an; ++i) { + SGPropertyNode* ac = a->getChild(i); + SGPropertyNode* bc = b->getChild(ac->getName(), ac->getIndex(), true /*create*/); + // Recurse. + s_RecordPropertyDiffs(out, ac, bc, path_exclude_prefixes); + } +} + + +// Takes care of writing extra-properties to FGReplayData if we are doing a +// Continuous recording with extra-properties. +// +// We write different properties depending on properties such as +// /sim/replay/record-main-view. +// +struct RecordExtraProperties +{ + RecordExtraProperties() + : + m_record_extra_properties(fgGetNode("/sim/replay/record-extra-properties", true /*create*/)), + m_record_main_window(fgGetNode("/sim/replay/record-main-window", true /*create*/)), + m_record_main_view(fgGetNode("/sim/replay/record-main-view", true /*create*/)), + m_record_extra_properties_paths(fgGetNode("/sim/replay/record-extra-properties-paths", true /*create*/)) + { + } + + // Added info about property changes to *ReplayData if we are recording + // them. + // + void capture(SGPropertyNode_ptr RecordExtraPropertiesReference, FGReplayData* ReplayData) + { + if (m_record_extra_properties->getBoolValue()) { + // Record extra properties specified in + // /sim/replay/record-extra-properties-paths/path[]. + // + auto paths = m_record_extra_properties_paths->getChildren("path"); + for (SGPropertyNode* n: paths) { + // We don't try to handle indices or deletion for top-level + // property node. + // + std::string path = n->getStringValue(); + SGPropertyNode* a = globals->get_props()->getNode(path, false /*create*/); + if (!a) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "property does not exist: " << path); + continue; + } + SGPropertyNode* b = RecordExtraPropertiesReference->getNode(path, true /*create*/); + s_RecordPropertyDiffs(ReplayData->extra_properties, a, b, NULL /*path_exclude_prefixes*/); + } + } + + if (m_record_main_window->getBoolValue()) { + // Record size/position of main window. + // + static std::vector<const char*> s_paths = { + "sim/startup/xpos", + "sim/startup/ypos", + "sim/startup/xsize", + "sim/startup/ysize" + }; + for (const char* path: s_paths) { + SGPropertyNode* a = globals->get_props()->getNode(path, true /*create*/); + SGPropertyNode* b = RecordExtraPropertiesReference->getNode(path, true /*create*/); + s_RecordPropertyDiffs(ReplayData->extra_properties, a, b, NULL /*path_exclude_prefixes*/); + } + } + + if (m_record_main_view->getBoolValue()) { + // Record main window view. + // + static std::vector<const char*> s_excludes = { + "/sim/current-view/debug/", + "/sim/current-view/raw-orientation", + "/sim/current-view/viewer-" + }; + const char* path = "sim/current-view"; + SGPropertyNode* a = globals->get_props()->getNode(path, true /*create*/); + SGPropertyNode* b = RecordExtraPropertiesReference->getNode(path, true /*create*/); + s_RecordPropertyDiffs(ReplayData->extra_properties, a, b, &s_excludes); + } + } + + private: + + SGPropertyNode_ptr m_record_extra_properties; + SGPropertyNode_ptr m_record_main_window; + SGPropertyNode_ptr m_record_main_view; + SGPropertyNode_ptr m_record_extra_properties_paths; +}; + +static std::shared_ptr<RecordExtraProperties> s_record_extra_properties; + + + FGFlightRecorder::FGFlightRecorder(const char* pConfigName) : m_RecorderNode(fgGetNode("/sim/flight-recorder", true)), m_ReplayMultiplayer(fgGetNode("/sim/replay/multiplayer", true)), + m_RecordContinuous(fgGetNode("/sim/replay/record-continuous", true)), + m_RecordExtraProperties(fgGetNode("/sim/replay/record-extra-properties", true)), m_TotalRecordSize(0), m_ConfigName(pConfigName), m_usingDefaultConfig(false), @@ -54,6 +235,7 @@ FGFlightRecorder::FGFlightRecorder(const char* pConfigName) : FGFlightRecorder::~FGFlightRecorder() { + s_record_extra_properties.reset(); } void @@ -122,6 +304,8 @@ FGFlightRecorder::reinit(SGPropertyNode_ptr ConfigNode) // expose size of actual flight recorder record m_RecorderNode->setIntValue("record-size", m_TotalRecordSize); SG_LOG(SG_SYSTEMS, SG_INFO, "FlightRecorder: record size is " << m_TotalRecordSize << " bytes"); + + s_record_extra_properties.reset(new RecordExtraProperties); } /** Check if SignalList already contains the given property */ @@ -448,6 +632,17 @@ FGFlightRecorder::capture(double SimTime, FGReplayData* ReplayData) ReplayData->multiplayer_messages.push_back( MultiplayerMessage); } } + + // Add extra properties if we are doing continuous recording. + // + if (m_RecordContinuous->getBoolValue()) { + if (!m_RecordExtraPropertiesReference) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "m_RecordPropertiesReference is null"); + m_RecordExtraPropertiesReference = new SGPropertyNode; + } + s_record_extra_properties->capture(m_RecordExtraPropertiesReference, ReplayData); + } + ReplayData->UpdateStats(); // Note that if we are replaying, <ReplayData> will contain the last live @@ -500,10 +695,36 @@ weighting(TInterpolation interpolation, double ratio, double v1,double v2) } } +void +FGFlightRecorder::resetExtraProperties() +{ + SG_LOG(SG_SYSTEMS, SG_ALERT, "Clearing m_RecordExtraPropertiesReference"); + m_RecordExtraPropertiesReference = nullptr; +} + +// Converts string to int, ignoring errors and doing nothing if out-para <out> +// is null. +static void setInt(const std::string& value, int* out) +{ + if (!out) return; + try { + *out = std::stoi(value); + } + catch (std::exception& e) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "Ignoring failed conversion of '" << value << "' to int: " << e.what()); + } +} + /** Replay. * Restore all properties with data from given buffer. */ void -FGFlightRecorder::replay(double SimTime, const FGReplayData* _pNextBuffer, const FGReplayData* _pLastBuffer) +FGFlightRecorder::replay(double SimTime, const FGReplayData* _pNextBuffer, + const FGReplayData* _pLastBuffer, + int* main_window_xpos, + int* main_window_ypos, + int* main_window_xsize, + int* main_window_ysize + ) { const char* pLastBuffer = (_pLastBuffer) ? &_pLastBuffer->raw_data.front() : nullptr; const char* pBuffer = (_pNextBuffer) ? &_pNextBuffer->raw_data.front() : nullptr; @@ -611,7 +832,7 @@ FGFlightRecorder::replay(double SimTime, const FGReplayData* _pNextBuffer, const m_CaptureBool[i].Signal->setBoolValue(0 != (pFlags[i>>3] & (1 << (i&7)))); } } - + // Replay any multiplayer messages. But don't send the same multiplayer // messages repeatedly when we are called with a timestamp that ends up // picking the same _pNextBuffer as last time. @@ -623,6 +844,73 @@ FGFlightRecorder::replay(double SimTime, const FGReplayData* _pNextBuffer, const m_MultiplayMgr->pushMessageHistory(multiplayer_message); } } + + // Replay property changes. + // + + bool replay_extra_property_removal = globals->get_props()->getBoolValue("sim/replay/replay-extra-property-removal"); + bool replay_extra_property_changes = globals->get_props()->getBoolValue("sim/replay/replay-extra-property-changes"); + bool replay_main_view = globals->get_props()->getBoolValue("sim/replay/replay-main-view"); + bool replay_main_window_position = globals->get_props()->getBoolValue("sim/replay/replay-main-window-position"); + bool replay_main_window_size = globals->get_props()->getBoolValue("sim/replay/replay-main-window-size"); + + if (replay_extra_property_removal) { + for (auto extra_property_removed_path: _pNextBuffer->replay_extra_property_removals) { + SG_LOG(SG_SYSTEMS, SG_DEBUG, "replaying extra property removal: " << extra_property_removed_path); + globals->get_props()->removeChild(extra_property_removed_path); + } + } + + // Apply any changes to /sim/current-view/view-number* first. This is a + // hack to avoid problems where setting view-number appears to make the + // view code change other things such as pitch-offset-deg internally. So if + // the recorded property changes list view-number after pitch-offset-deg, + // we will end up overwriting the recorded change to pitch-offset-deg. + // + // The ultimate cause of this problem is that pitch-offset-deg is a tied + // property so when recording we don't always pick up all changes. + // + if (replay_main_view) { + for (auto prop_change: _pNextBuffer->replay_extra_property_changes) { + const std::string& path = prop_change.first; + const std::string& value = prop_change.second; + if (simgear::strutils::starts_with(path, "/sim/current-view/view-number")) { + SG_LOG(SG_SYSTEMS, SG_DEBUG, "*** SimTime=" << SimTime << " replaying view " << path << "=" << value); + globals->get_props()->setStringValue(path, value); + } + } + } + + for (auto prop_change: _pNextBuffer->replay_extra_property_changes) { + const std::string& path = prop_change.first; + const std::string& value = prop_change.second; + + if (0) {} + else if (path == "/sim/startup/xpos") { + if (replay_main_window_position) setInt(value, main_window_xpos); + } + else if (path == "/sim/startup/ypos") { + if (replay_main_window_position) setInt(value, main_window_ypos); + } + else if (path == "/sim/startup/xsize") { + if (replay_main_window_size) setInt(value, main_window_xsize); + } + else if (path == "/sim/startup/ysize") { + if (replay_main_window_size) setInt(value, main_window_ysize); + } + else if (simgear::strutils::starts_with(path, "/sim/current-view/")) { + if (replay_main_view) { + SG_LOG(SG_SYSTEMS, SG_DEBUG, "SimTime=" << SimTime + << " replaying view change: " << path << "=" << value); + globals->get_props()->setStringValue(path, value); + } + } + else if (replay_extra_property_changes) { + SG_LOG(SG_SYSTEMS, SG_DEBUG, "SimTime=" << SimTime + << " replaying extra_property change: " << path << "=" << value); + globals->get_props()->setStringValue(path, value); + } + } } int diff --git a/src/Aircraft/flightrecorder.hxx b/src/Aircraft/flightrecorder.hxx index 8314912e9..4573f5314 100644 --- a/src/Aircraft/flightrecorder.hxx +++ b/src/Aircraft/flightrecorder.hxx @@ -57,11 +57,19 @@ public: void reinit (void); void reinit (SGPropertyNode_ptr ConfigNode); FGReplayData* capture (double SimTime, FGReplayData* pRecycledBuffer); + + // Updates main_window_* out-params if we find window move/resize events + // and replay of such events is enabled. void replay (double SimTime, const FGReplayData* pNextBuffer, - const FGReplayData* pLastBuffer = NULL); - + const FGReplayData* pLastBuffer, + int* main_window_xpos, + int* main_window_ypos, + int* main_window_xsize, + int* main_window_ysize + ); int getRecordSize (void) { return m_TotalRecordSize;} void getConfig (SGPropertyNode* root); + void resetExtraProperties(); private: SGPropertyNode_ptr getDefault(void); @@ -78,7 +86,14 @@ private: SGPropertyNode_ptr m_RecorderNode; SGPropertyNode_ptr m_ConfigNode; SGPropertyNode_ptr m_ReplayMultiplayer; - + SGPropertyNode_ptr m_RecordContinuous; + SGPropertyNode_ptr m_RecordExtraProperties; + + // This contains copy of all properties that we are recording, so that we + // can send only differences. + // + SGPropertyNode_ptr m_RecordExtraPropertiesReference; + FlightRecorder::TSignalList m_CaptureDouble; FlightRecorder::TSignalList m_CaptureFloat; FlightRecorder::TSignalList m_CaptureInteger; diff --git a/src/Aircraft/replay.cxx b/src/Aircraft/replay.cxx index 0c93a9f22..95c29551a 100644 --- a/src/Aircraft/replay.cxx +++ b/src/Aircraft/replay.cxx @@ -29,6 +29,8 @@ #include <float.h> #include <string.h> +#include <osgViewer/ViewerBase> + #include <simgear/constants.h> #include <simgear/structure/exception.hxx> #include <simgear/props/props_io.hxx> @@ -40,6 +42,8 @@ #include <Main/fg_props.hxx> #include <MultiPlayer/mpmessages.hxx> +#include <Viewer/renderer.hxx> +#include <Viewer/FGEventHandler.hxx> #include "replay.hxx" #include "flightrecorder.hxx" @@ -146,6 +150,10 @@ FGReplay::FGReplay() : last_lt_time(0.0), last_msg_time(0), last_replay_state(0), + m_sim_startup_xpos(fgGetNode("sim/startup/xpos", true)), + m_sim_startup_ypos(fgGetNode("sim/startup/ypos", true)), + m_sim_startup_xsize(fgGetNode("sim/startup/xsize", true)), + m_sim_startup_ysize(fgGetNode("sim/startup/ysize", true)), replay_time_prev(-1.0), m_high_res_time(60.0), m_medium_res_time(600.0), @@ -166,7 +174,7 @@ static int PropertiesWrite(SGPropertyNode* root, std::ostream& out) { stringstream buffer; writeProperties(buffer, root, true /*write_all*/); - size_t buffer_len = buffer.str().size() + 1; + uint32_t buffer_len = buffer.str().size() + 1; out.write(reinterpret_cast<char*>(&buffer_len), sizeof(buffer_len)); out.write(buffer.str().c_str(), buffer_len); return 0; @@ -174,7 +182,7 @@ static int PropertiesWrite(SGPropertyNode* root, std::ostream& out) static int PropertiesRead(std::istream& in, SGPropertyNode* node) { - size_t buffer_len; + uint32_t buffer_len; in.read(reinterpret_cast<char*>(&buffer_len), sizeof(buffer_len)); std::vector<char> buffer( buffer_len); in.read(&buffer.front(), buffer.size()); @@ -182,13 +190,24 @@ static int PropertiesRead(std::istream& in, SGPropertyNode* node) return 0; } -/* Reads uncompressed vector<char> from file. */ -static void VectorRead(std::istream& in, std::vector<char>& out) +/* Reads uncompressed vector<char> from file. Throws if length field is +longer than <max_length>. */ +template<typename SizeType> +static SizeType VectorRead(std::istream& in, std::vector<char>& out, uint32_t max_length=(1<<31)) { - size_t length; + SizeType length; in.read(reinterpret_cast<char*>(&length), sizeof(length)); + if (sizeof(length) + length > max_length) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "recording data vector too long." + << " max_length=" << max_length + << " sizeof(length)=" << sizeof(length) + << " length=" << length + ); + throw std::runtime_error("Failed to read vector in recording"); + } out.resize(length); in.read(&out.front(), length); + return sizeof(length) + length; } static void popupTip(const char* message, int delay) @@ -199,12 +218,6 @@ static void popupTip(const char* message, int delay) globals->get_commands()->execute("show-message", args); } -enum FGTapeType -{ - FGTapeType_NORMAL, - FGTapeType_CONTINUOUS, - FGTapeType_RECOVERY, -}; // Returns a path using different formats depending on <type>: // @@ -212,29 +225,31 @@ enum FGTapeType // FGTapeType_CONTINUOUS: <tape-directory>/<aircraft-type>-<date>-<time>-continuous.fgtape // FGTapeType_RECOVERY: <tape-directory>/<aircraft-type>-recovery.fgtape // -static SGPath makeSavePath(FGTapeType type) +static SGPath makeSavePath(FGTapeType type, SGPath* path_timeless=nullptr) { const char* tapeDirectory = fgGetString("/sim/replay/tape-directory", ""); const char* aircraftType = fgGetString("/sim/aircraft", "unknown"); - + if (path_timeless) *path_timeless = ""; SGPath path = SGPath(tapeDirectory); path.append(aircraftType); - path.concat("-"); if (type == FGTapeType_RECOVERY) { - path.concat("recovery"); + path.concat("-recovery"); } else { + if (path_timeless) *path_timeless = path; time_t calendar_time = time(NULL); struct tm *local_tm; local_tm = localtime( &calendar_time ); char time_str[256]; - strftime( time_str, 256, "%Y%m%d-%H%M%S", local_tm); + strftime( time_str, 256, "-%Y%m%d-%H%M%S", local_tm); path.concat(time_str); } if (type == FGTapeType_CONTINUOUS) { path.concat("-continuous"); + if (path_timeless) path_timeless->concat("-continuous"); } path.concat(".fgtape"); + if (path_timeless) path_timeless->concat(".fgtape"); return path; } @@ -249,23 +264,38 @@ void FGReplay::valueChanged(SGPropertyNode * node) return; } - if (m_continuous_out) { + if (m_continuous_out.is_open()) { // Stop existing continuous recording. + SG_LOG(SG_SYSTEMS, SG_ALERT, "Stopping continuous recording"); m_continuous_out.close(); popupTip("Continuous record to file stopped", 5 /*delay*/); } if (continuous) { // Start continuous recording. - SGPropertyNode_ptr MetaData; - SGPropertyNode_ptr Config; - SGPath path = makeSavePath(FGTapeType_CONTINUOUS); - bool ok = continuousWriteHeader(m_continuous_out, MetaData, Config, path); - if (!ok) { + SGPath path_timeless; + SGPath path = makeSavePath(FGTapeType_CONTINUOUS, &path_timeless); + m_continuous_out_config = continuousWriteHeader(m_continuous_out, path, FGTapeType_CONTINUOUS); + if (!m_continuous_out_config) { SG_LOG(SG_SYSTEMS, SG_ALERT, "Failed to start continuous recording"); popupTip("Continuous record to file failed to start", 5 /*delay*/); return; } + + SG_LOG(SG_SYSTEMS, SG_ALERT, "Starting continuous recording"); + + // Make a convenience link to the recording. E.g. + // harrier-gr3-continuous.fgtape -> + // harrier-gr3-20201224-005034-continuous.fgtape. + // + // Link destination is in same directory as link so we use leafname + // path.file(). + // + path_timeless.remove(); + bool ok = path_timeless.makeLink(path.file()); + if (!ok) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "Failed to create link " << path_timeless.c_str() << " => " << path.file()); + } SG_LOG(SG_SYSTEMS, SG_DEBUG, "Starting continuous recording to " << path); popupTip("Continuous record to file started", 5 /*delay*/); } @@ -331,7 +361,7 @@ FGReplay::init() replay_looped = fgGetNode("/sim/replay/looped", true); replay_duration_act = fgGetNode("/sim/replay/duration-act", true); speed_up = fgGetNode("/sim/speed-up", true); - replay_multiplayer = fgGetNode("/sim/replay/multiplayer", true); + replay_multiplayer = fgGetNode("/sim/replay/record-multiplayer", true); recovery_period = fgGetNode("/sim/replay/recovery-period", true); // alias to keep backward compatibility @@ -590,7 +620,7 @@ static void MoveFrontMultiplayerPackets(replay_list_type& list) /** Save raw replay data in a separate container */ static bool -saveRawReplayData(gzContainerWriter& output, const replay_list_type& ReplayData, size_t RecordSize, bool multiplayer) +saveRawReplayData(gzContainerWriter& output, const replay_list_type& ReplayData, size_t RecordSize, SGPropertyNode* meta) { // get number of records in this stream size_t Count = ReplayData.size(); @@ -602,7 +632,7 @@ saveRawReplayData(gzContainerWriter& output, const replay_list_type& ReplayData, return false; } - // write the raw data (all records in the given list) + // read the raw data (all records in the given list) replay_list_type::const_iterator it = ReplayData.begin(); size_t CheckCount = 0; while ((it != ReplayData.end())&& @@ -613,13 +643,20 @@ saveRawReplayData(gzContainerWriter& output, const replay_list_type& ReplayData, output.write(reinterpret_cast<const char*>(&pRecord->sim_time), sizeof(pRecord->sim_time)); output.write(&pRecord->raw_data.front(), pRecord->raw_data.size()); - if (multiplayer) { - size_t num_messages = pRecord->multiplayer_messages.size(); - output.write(reinterpret_cast<const char*>(&num_messages), sizeof(num_messages)); - for ( auto message: pRecord->multiplayer_messages) { - size_t message_size = message->size(); - output.write(reinterpret_cast<const char*>(&message_size), sizeof(message_size)); - output.write(&message->front(), message_size); + for (auto data: meta->getNode("meta")->getChildren("data")) { + SG_LOG(SG_SYSTEMS, SG_DEBUG, "data->getStringValue()=" << data->getStringValue()); + if (!strcmp(data->getStringValue(), "multiplayer")) { + uint32_t length = 0; + for (auto message: pRecord->multiplayer_messages){ + length += sizeof(uint16_t) + message->size(); + } + output.write(reinterpret_cast<const char*>(&length), sizeof(length)); + for (auto message: pRecord->multiplayer_messages) { + uint16_t message_size = message->size(); + output.write(reinterpret_cast<const char*>(&message_size), sizeof(message_size)); + output.write(&message->front(), message_size); + } + break; } } CheckCount++; @@ -639,34 +676,36 @@ saveRawReplayData(gzContainerWriter& output, const replay_list_type& ReplayData, // Sets things up for writing to a normal or continuous fgtape file. // -// Extra: +// extra: // NULL or extra information when we are called from fgdata gui, e.g. with // the flight description entered by the user in the save dialogue. // path: // Path of fgtape file. We return nullptr if this file already exists. -// Duration: +// duration: // Duration of recording. Zero if we are starting a continuous recording. -// +// tape_type: +// // Returns: -// A new SGPropertyNode containing meta child with information about the -// aircraft etc plus Extra's user-data if specified. +// A new SGPropertyNode suitable as prefix of recording. If +// extra:user-data exists, it will appear as meta/user-data. // static SGPropertyNode_ptr saveSetup( - const SGPropertyNode* Extra, + const SGPropertyNode* extra, const SGPath& path, - double Duration + double duration, + FGTapeType tape_type ) { - SGPropertyNode_ptr MetaData; + SGPropertyNode_ptr config; if (path.exists()) { // same timestamp!? SG_LOG(SG_SYSTEMS, SG_ALERT, "Error, flight recorder tape file with same name already exists: " << path); - return MetaData; + return config; } - MetaData = new SGPropertyNode(); - SGPropertyNode* meta = MetaData->getNode("meta", 0, true); + config = new SGPropertyNode; + SGPropertyNode* meta = config->getNode("meta", 0 /*index*/, true /*create*/); // add some data to the file - so we know for which aircraft/version it was recorded meta->setStringValue("aircraft-type", fgGetString("/sim/aircraft", "unknown")); @@ -676,25 +715,43 @@ static SGPropertyNode_ptr saveSetup( meta->setStringValue("aircraft-version", fgGetString("/sim/aircraft-version", "(undefined)")); // add information on the tape's recording duration - meta->setDoubleValue("tape-duration", Duration); + meta->setDoubleValue("tape-duration", duration); char StrBuffer[30]; - printTimeStr(StrBuffer, Duration, false); + printTimeStr(StrBuffer, duration, false); meta->setStringValue("tape-duration-str", StrBuffer); // add simulator version copyProperties(fgGetNode("/sim/version", 0, true), meta->getNode("version", 0, true)); - if (Extra && Extra->getNode("user-data")) + if (extra && extra->getNode("user-data")) { - copyProperties(Extra->getNode("user-data"), meta->getNode("user-data", 0, true)); + copyProperties(extra->getNode("user-data"), meta->getNode("user-data", 0, true)); } - bool multiplayer = fgGetBool("/sim/replay/multiplayer", false); - meta->setBoolValue("multiplayer", multiplayer); + // We always record signals. + config->addChild("data")->setStringValue("signals"); + + if (tape_type == FGTapeType_CONTINUOUS) { + if (fgGetBool("/sim/replay/record-multiplayer", false)) { + config->addChild("data")->setStringValue("multiplayer"); + } + if (0 + || fgGetBool("/sim/replay/record-extra-properties", false) + || fgGetBool("/sim/replay/record-main-window", false) + || fgGetBool("/sim/replay/record-main-view", false) + ) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "Adding data[]=extra-properties." + << " record-extra-properties=" << fgGetBool("/sim/replay/record-extra-properties", false) + << " record-main-window=" << fgGetBool("/sim/replay/record-main-window", false) + << " record-main-view=" << fgGetBool("/sim/replay/record-main-view", false) + ); + config->addChild("data")->setStringValue("extra-properties"); + } + } // store replay messages - copyProperties(fgGetNode("/sim/replay/messages", 0, true), MetaData->getNode("messages", 0, true)); + copyProperties(fgGetNode("/sim/replay/messages", 0, true), meta->getNode("messages", 0, true)); - return MetaData; + return config; } // Opens continuous recording file and writes header. @@ -709,54 +766,77 @@ static SGPropertyNode_ptr saveSetup( // If path_override is not "", we use it as the path (instead of the path // determined by saveSetup(). // -bool -FGReplay::continuousWriteHeader( +SGPropertyNode_ptr FGReplay::continuousWriteHeader( std::ofstream& out, - SGPropertyNode_ptr& MetaData, - SGPropertyNode_ptr& Config, - const SGPath& path + const SGPath& path, + FGTapeType tape_type ) { - if (!MetaData) { - MetaData = saveSetup(NULL /*Extra*/, path, 0 /*Duration*/); - if (!MetaData) { - return false; - } - } - if (!Config) { - Config = new SGPropertyNode; - m_pRecorder->getConfig(Config.get()); - } + SGPropertyNode_ptr config = saveSetup(NULL /*Extra*/, path, 0 /*Duration*/, tape_type); + SGPropertyNode* signals = config->getNode("signals", true /*create*/); + m_pRecorder->getConfig(signals); out.open(path.c_str(), std::ofstream::binary | std::ofstream::trunc); out.write(FlightRecorderFileMagic, strlen(FlightRecorderFileMagic)+1); - PropertiesWrite(MetaData, out); - PropertiesWrite(Config, out); + PropertiesWrite(config, out); + + if (tape_type == FGTapeType_CONTINUOUS) { + // Ensure that all recorded properties are written in first frame. + // + m_pRecorder->resetExtraProperties(); + } + if (!out) { out.close(); - return false; + config = nullptr; } - return true; + return config; } // Writes one frame of continuous record information. // bool -FGReplay::continuousWriteFrame(FGReplayData* r, std::ostream& out) +FGReplay::continuousWriteFrame(FGReplayData* r, std::ostream& out, SGPropertyNode_ptr config) { + SG_LOG(SG_SYSTEMS, SG_BULK, "writing frame." + << " out.tellp()=" << out.tellp() + << " r->sim_time=" << r->sim_time + ); out.write(reinterpret_cast<char*>(&r->sim_time), sizeof(r->sim_time)); - size_t aircraft_data_size = r->raw_data.size(); - out.write(reinterpret_cast<char*>(&aircraft_data_size), sizeof(aircraft_data_size)); - out.write(&r->raw_data.front(), r->raw_data.size()); - - size_t multiplayer_num = r->multiplayer_messages.size(); - out.write(reinterpret_cast<char*>(&multiplayer_num), sizeof(multiplayer_num)); - for ( size_t i=0; i<multiplayer_num; ++i) { - size_t length = r->multiplayer_messages[i]->size(); - out.write(reinterpret_cast<char*>(&length), sizeof(length)); - out.write(&r->multiplayer_messages[i]->front(), length); + for (auto data: config->getChildren("data")) { + const char* data_type = data->getStringValue(); + if (!strcmp(data_type, "signals")) { + uint32_t signals_size = r->raw_data.size(); + out.write(reinterpret_cast<char*>(&signals_size), sizeof(signals_size)); + out.write(&r->raw_data.front(), r->raw_data.size()); + } + else if (!strcmp(data_type, "multiplayer")) { + uint32_t length = 0; + for (auto message: r->multiplayer_messages) { + length += sizeof(uint16_t) + message->size(); + } + SG_LOG(SG_SYSTEMS, SG_DEBUG, "data_type=" << data_type << " out.tellp()=" << out.tellp() + << " length=" << length); + out.write(reinterpret_cast<const char*>(&length), sizeof(length)); + for (auto message: r->multiplayer_messages) { + uint16_t message_size = message->size(); + out.write(reinterpret_cast<const char*>(&message_size), sizeof(message_size)); + out.write(&message->front(), message_size); + } + } + else if (!strcmp(data_type, "extra-properties")) { + uint32_t length = r->extra_properties.size(); + SG_LOG(SG_SYSTEMS, SG_DEBUG, "data_type=" << data_type << " out.tellp()=" << out.tellp() + << " length=" << length); + out.write(reinterpret_cast<char*>(&length), sizeof(length)); + out.write(&r->extra_properties[0], length); + } + else { + SG_LOG(SG_SYSTEMS, SG_ALERT, "unrecognised data_type=" << data_type); + assert(0); + } } bool ok = true; @@ -764,6 +844,13 @@ FGReplay::continuousWriteFrame(FGReplayData* r, std::ostream& out) return ok; } +struct FGFrameInfo +{ + size_t offset; + bool has_signals = false; + bool has_multiplayer = false; + bool has_extra_properties = false; +}; void FGReplay::update( double dt ) @@ -809,9 +896,9 @@ FGReplay::update( double dt ) if (m_continuous_in.is_open()) { SG_LOG(SG_SYSTEMS, SG_DEBUG, "Unloading continuous recording"); m_continuous_in.close(); - m_continuous_time_to_offset.clear(); + m_continuous_in_time_to_frameinfo.clear(); } - assert(m_continuous_time_to_offset.empty()); + assert(m_continuous_in_time_to_frameinfo.empty()); guiMessage("Replay stopped. Your controls!"); } @@ -959,7 +1046,7 @@ FGReplay::update( double dt ) } if (m_continuous_out.is_open()) { - continuousWriteFrame(r, m_continuous_out); + continuousWriteFrame(r, m_continuous_out, m_continuous_out_config); } if (replay_state == 0) @@ -976,6 +1063,8 @@ FGReplay::update( double dt ) s_last_recovery = t; } + // Write recovery recording periodically. + // if (t - s_last_recovery >= recovery_period_s) { s_last_recovery = t; @@ -984,7 +1073,7 @@ FGReplay::update( double dt ) // static SGPath path = makeSavePath(FGTapeType_RECOVERY); static SGPath path_temp = SGPath( path.str() + "-"); - static SGPropertyNode_ptr MetaData; + //static SGPropertyNode_ptr MetaData; static SGPropertyNode_ptr Config; SG_LOG(SG_SYSTEMS, SG_BULK, "Creating recovery file: " << path); @@ -996,8 +1085,9 @@ FGReplay::update( double dt ) (void) remove(path_temp.c_str()); std::ofstream out; bool ok = true; - if (ok) ok = continuousWriteHeader(out, MetaData, Config, path_temp); - if (ok) ok = continuousWriteFrame(r, out); + SGPropertyNode_ptr config = continuousWriteHeader(out, path_temp, FGTapeType_RECOVERY); + if (!config) ok = false; + if (ok) ok = continuousWriteFrame(r, out, config); out.close(); if (ok) { rename(path_temp.c_str(), path.c_str()); @@ -1140,6 +1230,7 @@ FGReplay::interpolate( double time, const replay_list_type &list) * Returns true when replay sequence has finished, false otherwise. */ + bool FGReplay::replay( double time ) { // cout << "replay: " << time << " "; @@ -1148,34 +1239,159 @@ FGReplay::replay( double time ) { replayMessage(time); - if (!m_continuous_time_to_offset.empty()) { - // Replay from uncompressed recording file. + if (!m_continuous_in_time_to_frameinfo.empty()) { + // We are replaying a continuous recording. // - auto p = m_continuous_time_to_offset.lower_bound(time); - if (p == m_continuous_time_to_offset.end()) { - // end. - --p; - replay( time, p->second); - return true; + // We need to detect whether replay() updates the values for the main + // window's position and size. + int xpos0 = m_sim_startup_xpos->getIntValue(); + int ypos0 = m_sim_startup_xpos->getIntValue(); + int xsize0 = m_sim_startup_xpos->getIntValue(); + int ysize0 = m_sim_startup_xpos->getIntValue(); + + int xpos = xpos0; + int ypos = ypos0; + int xsize = xsize0; + int ysize = ysize0; + + double t_begin = m_continuous_in_time_last; + if (m_continuous_in_extra_properties) { + // Continuous recording has property changes. + // + if (time < m_continuous_in_time_last) { + // We have gone backwards; need to replay all property changes + // from t=0. + t_begin = -1; + } } + double multiplayer_recent = 3; + if (m_continuous_in_multiplayer) { + double t = std::max(0.0, time - multiplayer_recent); + t_begin = std::min(t_begin, t); + } + + // Replay property changes for all t in t_prop_begin < t < time, and multiplayer + // changes for most recent multiplayer_recent seconds. + // + for (auto p = m_continuous_in_time_to_frameinfo.upper_bound(t_begin); + p != m_continuous_in_time_to_frameinfo.end(); + ++p) + { + if (p->first >= time) break; + SG_LOG(SG_SYSTEMS, SG_DEBUG, "Replaying extra property changes." + << " m_continuous_in_time_last=" << m_continuous_in_time_last + << " time=" << time + << " t_begin=" << t_begin + << " p->first=" << p->first + << " p->second.offset=" << p->second.offset + ); + + // Replaying a frame is expensive because we read frame data + // from disc each time. So we only replay this frame if it has + // extra_properties, or if it has multiplayer packets and we are + // within <multiplayer_recent> seconds of current time. + // + bool replay_this_frame = p->second.has_extra_properties; + if (!replay_this_frame) { + if (p->first > time - multiplayer_recent) { + if (p->second.has_multiplayer) { + replay_this_frame = true; + } + } + } + if (replay_this_frame) { + replay( + p->first, + p->second.offset, + 0 /*offset_old*/, + false /*replay_signals*/, + p->first > time - multiplayer_recent /*replay_multiplayer*/, + true /*replay_extra_properties*/, + &xpos, + &ypos, + &xsize, + &ysize + ); + } + } + + // Replay from uncompressed recording file. + // + auto p = m_continuous_in_time_to_frameinfo.lower_bound(time); + bool ret = false; + + size_t offset; + size_t offset_prev = 0; + + if (p == m_continuous_in_time_to_frameinfo.end()) { + // We are at end of recording; replay last frame. + --p; + offset = p->second.offset; + ret = true; + } else if (p->first > time) { // Look for preceding item. - if (p == m_continuous_time_to_offset.begin()) { - replay(time, p->second); - return false; + if (p == m_continuous_in_time_to_frameinfo.begin()) { + // <time> is before beginning of recording. + offset = p->second.offset; + } + else { + // Interpolate between items that straddle <time>. + auto prev = p; + --prev; + offset_prev = prev->second.offset; + offset = p->second.offset; } - auto prev = p; - --prev; - replay( time, p->second, prev->second); - return false; } else { // Exact match. - replay(time, p->second); - return false; + offset = p->second.offset; } + replay( + time, + offset, + offset_prev /*offset_old*/, + true /*replay_signals*/, + true /*replay_multiplayer*/, + true /*replay_extra_properties*/, + &xpos, + &ypos, + &xsize, + &ysize + ); + + if (0 + || xpos != xpos0 + || ypos != ypos0 + || xsize != xsize0 + || ysize != ysize0 + ) { + // Move/resize the main window to reflect the updated values. + globals->get_props()->setIntValue("/sim/startup/xpos", xpos); + globals->get_props()->setIntValue("/sim/startup/ypos", ypos); + globals->get_props()->setIntValue("/sim/startup/xsize", xsize); + globals->get_props()->setIntValue("/sim/startup/ysize", ysize); + + osgViewer::ViewerBase* viewer_base = globals->get_renderer()->getViewerBase(); + if (viewer_base) { + std::vector<osgViewer::GraphicsWindow*> windows; + viewer_base->getWindows(windows); + osgViewer::GraphicsWindow* window = windows[0]; + + // We use FGEventHandler::setWindowRectangle() to move the + // window, because it knows how to convert from window work-area + // coordinates to window-including-furniture coordinates. + // + flightgear::FGEventHandler* event_handler = globals->get_renderer()->getEventHandler(); + event_handler->setWindowRectangleInteriorWithCorrection(window, xpos, ypos, xsize, ysize); + } + } + + m_continuous_in_time_last = time; + + return ret; } if ( ! short_term.empty() ) { @@ -1235,40 +1451,138 @@ FGReplay::replay( double time ) { * given two FGReplayData elements and a time, interpolate between them */ void -FGReplay::replay(double time, FGReplayData* pCurrentFrame, FGReplayData* pOldFrame) +FGReplay::replay(double time, FGReplayData* pCurrentFrame, FGReplayData* pOldFrame, + int* xpos, int* ypos, int* xsize, int* ysize) { - m_pRecorder->replay(time,pCurrentFrame,pOldFrame); + m_pRecorder->replay(time, pCurrentFrame, pOldFrame, xpos, ypos, xsize, ysize); } +static int16_t read_int16(std::ifstream& in, size_t& pos) +{ + int16_t a; + in.read(reinterpret_cast<char*>(&a), sizeof(a)); + pos += sizeof(a); + return a; +} +static std::string read_string(std::ifstream& in, size_t& pos) +{ + int16_t length = read_int16(in, pos); + std::vector<char> path(length); + in.read(&path[0], length); + pos += length; + std::string ret(&path[0], length); + return ret; +} -/* Reads a FGReplayData from uncompressed file. */ -static std::unique_ptr<FGReplayData> ReadFGReplayData(std::ifstream& in, size_t pos) +/* Reads extra-property change items in next <length> bytes. Throws if we don't +exactly read <length> bytes. */ +static void ReadFGReplayDataExtraProperties(std::ifstream& in, FGReplayData* replay_data, uint32_t length) +{ + SG_LOG(SG_SYSTEMS, SG_BULK, "reading extra-properties. length=" << length); + size_t pos=0; + for(;;) { + if (pos == length) { + break; + } + if (pos > length) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "Overrun while reading extra-properties:" + " length=" << length << ": pos=" << pos); + assert(0); + } + SG_LOG(SG_SYSTEMS, SG_BULK, "length=" << length<< " pos=" << pos); + std::string path = read_string(in, pos); + if (path == "") { + path = read_string(in, pos); + SG_LOG(SG_SYSTEMS, SG_DEBUG, "property deleted: " << path); + replay_data->replay_extra_property_removals.push_back(path); + } + else { + std::string value = read_string(in, pos); + SG_LOG(SG_SYSTEMS, SG_DEBUG, "property changed: " << path << "=" << value); + replay_data->replay_extra_property_changes[path] = value; + } + } +} + +/* Reads all or part of a FGReplayData from uncompressed file. */ +static std::unique_ptr<FGReplayData> ReadFGReplayData( + std::ifstream& in, + size_t pos, + SGPropertyNode* config, + bool load_signals, + bool load_multiplayer, + bool load_extra_properties + ) { /* Need to clear any eof bit, otherwise seekg() will not work (which is pretty unhelpful). E.g. see: https://stackoverflow.com/questions/16364301/whats-wrong-with-the-ifstream-seekg */ + SG_LOG(SG_SYSTEMS, SG_BULK, "reading frame. pos=" << pos); in.clear(); in.seekg(pos); std::unique_ptr<FGReplayData> ret(new FGReplayData); in.read(reinterpret_cast<char*>(&ret->sim_time), sizeof(ret->sim_time)); - VectorRead(in, ret->raw_data); - /* Multiplayer information is a vector of vectors. */ - size_t n; - in.read(reinterpret_cast<char*>(&n), sizeof(n)); - ret->multiplayer_messages.resize(n); - for (size_t i=0; i<n; ++i) { - ret->multiplayer_messages[i].reset(new std::vector<char>); - VectorRead(in, *ret->multiplayer_messages[i]); + for (auto data: config->getChildren("data")) { + const char* data_type = data->getStringValue(); + SG_LOG(SG_SYSTEMS, SG_DEBUG, "in.tellg()=" << in.tellg() << " data_type=" << data_type); + uint32_t length; + in.read(reinterpret_cast<char*>(&length), sizeof(length)); + SG_LOG(SG_SYSTEMS, SG_DEBUG, "length=" << length); + if (load_signals && !strcmp(data_type, "signals")) { + ret->raw_data.resize(length); + in.read(&ret->raw_data.front(), ret->raw_data.size()); + } + else if (load_multiplayer && !strcmp(data_type, "multiplayer")) { + /* Multiplayer information is a vector of vectors. */ + ret->multiplayer_messages.clear(); + uint32_t pos = 0; + for(;;) { + SG_LOG(SG_SYSTEMS, SG_DEBUG, "length=" << length << " pos=" << pos); + assert(pos <= length); + if (pos == length) { + break; + } + std::shared_ptr<std::vector<char>> v(new std::vector<char>); + ret->multiplayer_messages.push_back(v); + pos += VectorRead<uint16_t>(in, *ret->multiplayer_messages.back(), length - pos); + } + } + else if (load_extra_properties && !strcmp(data_type, "extra-properties")) { + ReadFGReplayDataExtraProperties(in, ret.get(), length); + } + else { + SG_LOG(SG_GENERAL, SG_BULK, "Skipping unrecognised data: " << data_type); + in.seekg(length, std::ios_base::cur); + } } + return ret; } -/* Replays one iteration from uncompressed file. */ -void FGReplay::replay(double time, size_t offset, size_t offset_old) +// Replays one frame from uncompressed file. <offset> and <offset_old> +// are offsets in file of frames that are >= and < <time> respectively. +// <offset_old> may be 0, in which case it is ignored. +// +// We load the frame(s) from disc, omitting some data depending on +// replay_signals, replay_multiplayer and replay_extra_properties. Then call +// m_pRecorder->replay(), which updates the global state. +// +void FGReplay::replay( + double time, + size_t offset, + size_t offset_old, + bool replay_signals, + bool replay_multiplayer, + bool replay_extra_properties, + int* xpos, + int* ypos, + int* xsize, + int* ysize + ) { SG_LOG(SG_SYSTEMS, SG_BULK, "FGReplay::replay():" @@ -1276,19 +1590,40 @@ void FGReplay::replay(double time, size_t offset, size_t offset_old) << " offset=" << offset << " offset_old=" << offset_old ); - std::unique_ptr<FGReplayData> replay_data = ReadFGReplayData(m_continuous_in, offset); + std::unique_ptr<FGReplayData> replay_data = ReadFGReplayData( + m_continuous_in, + offset, + m_continuous_in_config, + replay_signals, + replay_multiplayer, + replay_extra_properties + ); std::unique_ptr<FGReplayData> replay_data_old; if (offset_old) { - replay_data_old = ReadFGReplayData(m_continuous_in, offset_old); + replay_data_old = ReadFGReplayData( + m_continuous_in, + offset_old, + m_continuous_in_config, + replay_signals, + replay_multiplayer, + replay_extra_properties + ); } - m_pRecorder->replay(time, replay_data.get(), replay_data_old.get()); + m_pRecorder->replay(time, replay_data.get(), replay_data_old.get(), xpos, ypos, xsize, ysize); } double FGReplay::get_start_time() { - if (!m_continuous_time_to_offset.empty()) { - double ret = m_continuous_time_to_offset.begin()->first; + if (!m_continuous_in_time_to_frameinfo.empty()) { + double ret = m_continuous_in_time_to_frameinfo.begin()->first; + SG_LOG(SG_SYSTEMS, SG_DEBUG, + "ret=" << ret + << " m_continuous_in_time_to_frameinfo is " + << m_continuous_in_time_to_frameinfo.begin()->first + << ".." + << m_continuous_in_time_to_frameinfo.rbegin()->first + ); return ret; } @@ -1310,8 +1645,15 @@ FGReplay::get_start_time() double FGReplay::get_end_time() { - if (!m_continuous_time_to_offset.empty()) { - double ret = m_continuous_time_to_offset.rbegin()->first; + if (!m_continuous_in_time_to_frameinfo.empty()) { + double ret = m_continuous_in_time_to_frameinfo.rbegin()->first; + SG_LOG(SG_SYSTEMS, SG_DEBUG, + "ret=" << ret + << " m_continuous_in_time_to_frameinfo is " + << m_continuous_in_time_to_frameinfo.begin()->first + << ".." + << m_continuous_in_time_to_frameinfo.rbegin()->first + ); return ret; } @@ -1361,12 +1703,21 @@ loadRawReplayData(gzContainerReader& input, FGFlightRecorder* pRecorder, replay_ ReplayData.push_back(pBuffer); if (multiplayer) { - size_t num_messages = 0; - input.read(reinterpret_cast<char*>(&num_messages), sizeof(num_messages)); - for (size_t i=0; i<num_messages; ++i) { - size_t message_size; + uint32_t length; + input.read(reinterpret_cast<char*>(&length), sizeof(length)); + uint32_t pos = 0; + for(;;) { + if (pos == length) break; + assert(pos < length); + uint16_t message_size; input.read(reinterpret_cast<char*>(&message_size), sizeof(message_size)); - std::shared_ptr<std::vector<char>> message(new std::vector<char>(message_size)); + pos += sizeof(message_size) + message_size; + if (pos > length) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "Tape appears to have corrupted multiplayer data"); + return false; + } + auto message = std::make_shared<std::vector<char>>(message_size); + //std::shared_ptr<std::vector<char>> message(new std::vector<char>(message_size)); input.read(&message->front(), message_size); pBuffer->multiplayer_messages.push_back( message); } @@ -1422,14 +1773,14 @@ FGReplay::saveTape(const SGPath& Filename, SGPropertyNode_ptr MetaData) size_t RecordSize = Config->getIntValue("recorder/record-size", 0); SG_LOG(SG_SYSTEMS, MY_SG_DEBUG, "Config:recorder/signal-count=" << Config->getIntValue("recorder/signal-count", 0) << " RecordSize: " << RecordSize); - bool multiplayer = MetaData->getBoolValue("meta/multiplayer", 0); + //bool multiplayer = MetaData->getBoolValue("meta/multiplayer", 0); if (ok) - ok &= saveRawReplayData(output, short_term, RecordSize, multiplayer); + ok &= saveRawReplayData(output, short_term, RecordSize, MetaData); if (ok) - ok &= saveRawReplayData(output, medium_term, RecordSize, multiplayer); + ok &= saveRawReplayData(output, medium_term, RecordSize, MetaData); if (ok) - ok &= saveRawReplayData(output, long_term, RecordSize, multiplayer); + ok &= saveRawReplayData(output, long_term, RecordSize, MetaData); Config = 0; } @@ -1443,11 +1794,26 @@ FGReplay::saveTape(const SGPath& Filename, SGPropertyNode_ptr MetaData) bool FGReplay::saveTape(const SGPropertyNode* Extra) { - SGPath path = makeSavePath(FGTapeType_NORMAL); - SGPropertyNode_ptr MetaData = saveSetup(Extra, path, get_end_time()-get_start_time()); + SGPath path_timeless; + SGPath path = makeSavePath(FGTapeType_NORMAL, &path_timeless); + SGPropertyNode_ptr MetaData = saveSetup(Extra, path, get_end_time()-get_start_time(), FGTapeType_NORMAL); bool ok = false; if (MetaData) { ok = saveTape(path, MetaData); + if (ok) { + // Make a convenience link to the recording. E.g. + // harrier-gr3.fgtape -> + // harrier-gr3-20201224-005034.fgtape. + // + // Link destination is in same directory as link so we use leafname + // path.file(). + // + path_timeless.remove(); + ok = path_timeless.makeLink(path.file()); + if (!ok) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "Failed to create link " << path_timeless.c_str() << " => " << path.file()); + } + } } if (ok) guiMessage("Flight recorder tape saved successfully!"); @@ -1465,34 +1831,60 @@ FGReplay::saveTape(const SGPropertyNode* Extra) bool FGReplay::loadTape(const SGPath& Filename, bool Preview, SGPropertyNode& MetaMeta) { + SG_LOG(SG_SYSTEMS, SG_DEBUG, "loading Preview=" << Preview << " Filename=" << Filename); { - /* Try to load as uncompressed first. */ - m_continuous_in.open( Filename.str()); + /* Try to load as uncompressed Continuous recording first. */ + std::ifstream in_preview; + std::ifstream& in(Preview ? in_preview : m_continuous_in); + in.open( Filename.str()); + if (!in) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "Failed to open Filename=" << Filename); + return false; + } std::vector<char> buffer(strlen( FlightRecorderFileMagic) + 1); - m_continuous_in.read(&buffer.front(), buffer.size()); + in.read(&buffer.front(), buffer.size()); if (strcmp(&buffer.front(), FlightRecorderFileMagic)) { - SG_LOG(SG_SYSTEMS, SG_DEBUG, "fgtape prefix doesn't match FlightRecorderFileMagic: " << Filename); + SG_LOG(SG_SYSTEMS, SG_ALERT, "fgtape prefix doesn't match FlightRecorderFileMagic: '" << &buffer.front() << "'"); + in.close(); } else { SG_LOG(SG_SYSTEMS, SG_DEBUG, "fgtape is uncompressed: " << Filename); - SGPropertyNode_ptr MetaDataProps = new SGPropertyNode(); - PropertiesRead(m_continuous_in, MetaDataProps.get()); - copyProperties(MetaDataProps->getNode("meta", 0, true), &MetaMeta); - SGPropertyNode_ptr Config = new SGPropertyNode(); - PropertiesRead(m_continuous_in, Config.get()); - + m_continuous_in_config = new SGPropertyNode; + try { + PropertiesRead(in, m_continuous_in_config.get()); + } + catch (std::exception& e) { + SG_LOG(SG_SYSTEMS, SG_ALERT, "Failed to read Config properties in: " << Filename); + in.close(); + return false; + } + SG_LOG(SG_SYSTEMS, SG_DEBUG, "m_continuous_in_config is:\n" + << writePropertiesInline(m_continuous_in_config, true /*write_all*/) << "\n"); + copyProperties(m_continuous_in_config->getNode("meta", 0, true), &MetaMeta); if (Preview) { - m_continuous_in.close(); + in.close(); return true; } - - m_pRecorder->reinit(Config); + m_pRecorder->reinit(m_continuous_in_config); clear(); fillRecycler(); time_t t = time(NULL); size_t pos = 0; + m_continuous_in_time_last = -1; + m_continuous_in_time_to_frameinfo.clear(); + int num_frames_extra_properties = 0; + int num_frames_multiplayer = 0; + // Read entire recording and build up in-memory cache of simulator + // time to file offset, so we can handle random access. + // + // We also cache any frames that modify extra-properties. + // + SG_LOG(SG_SYSTEMS, SG_DEBUG, "Indexing Continuous recording " << Filename); for(;;) { + SG_LOG(SG_SYSTEMS, SG_BULK, "reading frame." + << " m_continuous_in.tellg()=" << m_continuous_in.tellg() + ); pos = m_continuous_in.tellg(); m_continuous_in.seekg(pos); double sim_time; @@ -1503,38 +1895,62 @@ FGReplay::loadTape(const SGPath& Filename, bool Preview, SGPropertyNode& MetaMet << " m_continuous_in.tellg()=" << m_continuous_in.tellg() << " sim_time=" << sim_time ); + FGFrameInfo frameinfo; + frameinfo.offset = pos; - size_t length_aircraft; - m_continuous_in.read(reinterpret_cast<char*>(&length_aircraft), sizeof(length_aircraft)); - m_continuous_in.seekg(length_aircraft, std::ios_base::cur); - - size_t num_multiplayer; - m_continuous_in.read(reinterpret_cast<char*>(&num_multiplayer), sizeof(num_multiplayer)); - for ( size_t i=0; i<num_multiplayer; ++i) { - size_t length; + //bool frame_has_property_changes = false; + auto datas = m_continuous_in_config->getChildren("data"); + SG_LOG(SG_SYSTEMS, SG_DEBUG, "datas.size()=" << datas.size()); + for (auto data: datas) { + uint32_t length; m_continuous_in.read(reinterpret_cast<char*>(&length), sizeof(length)); + SG_LOG(SG_SYSTEMS, SG_BULK, + "m_continuous_in.tellg()=" << m_continuous_in.tellg() + << " Skipping data_type=" << data->getStringValue() + << " length=" << length + ); m_continuous_in.seekg(length, std::ios_base::cur); + if (length) { + if (!strcmp(data->getStringValue(), "signals")) { + frameinfo.has_signals = true; + } + else if (!strcmp(data->getStringValue(), "multiplayer")) { + frameinfo.has_multiplayer = true; + ++num_frames_multiplayer; + } + else if (!strcmp(data->getStringValue(), "extra-properties")) { + frameinfo.has_extra_properties = true; + ++num_frames_extra_properties; + } + } } - SG_LOG(SG_SYSTEMS, SG_BULK, "" << " pos=" << pos << " sim_time=" << sim_time - << " length_aircraft=" << length_aircraft - << " num_multiplayer=" << num_multiplayer + << " num_frames_multiplayer=" << num_frames_multiplayer + << " num_frames_extra_properties=" << num_frames_extra_properties ); if (!m_continuous_in) { + // EOF; we need to cope if last frame is incomplete, as + // this can easily happen if Flightgear was killed while + // recording. break; } - m_continuous_time_to_offset[sim_time] = pos; + m_continuous_in_time_to_frameinfo[sim_time] = frameinfo; } t = time(NULL) - t; - SG_LOG(SG_SYSTEMS, SG_DEBUG, "Indexed uncompressed recording" + SG_LOG(SG_SYSTEMS, SG_ALERT, "Indexed uncompressed recording" << ". time taken: " << t << "s" << ". recording size: " << pos - << ". numrecording items: " << m_continuous_time_to_offset.size() + << ". Number of frames: " << m_continuous_in_time_to_frameinfo.size() + << ". num_frames_multiplayer: " << num_frames_multiplayer + << ". num_frames_extra_properties: " << num_frames_extra_properties ); + fgSetInt("/sim/replay/continuous-stats-num-frames", m_continuous_in_time_to_frameinfo.size()); + fgSetInt("/sim/replay/continuous-stats-num-frames-extra-properties", num_frames_extra_properties); + fgSetInt("/sim/replay/continuous-stats-num-frames-multiplayer", num_frames_multiplayer); start(true /*NewTape*/); return true; } @@ -1659,9 +2075,13 @@ FGReplay::loadTape(const SGPath& Filename, bool Preview, SGPropertyNode& MetaMet << ", expected size was " << OriginalSize << "."); } - bool multiplayer = MetaMeta.getBoolValue("multiplayer", 0); - SG_LOG(SG_SYSTEMS, SG_DEBUG, "multiplayer=" << multiplayer); - + bool multiplayer = false; + for (auto data: MetaMeta.getChildren("data")) { + if (!strcmp(data->getStringValue(), "multiplayer")) { + multiplayer = true; + } + } + SG_LOG(SG_SYSTEMS, SG_ALERT, "multiplayer=" << multiplayer); if (ok) ok &= loadRawReplayData(input, m_pRecorder, short_term, RecordSize, multiplayer); if (ok) diff --git a/src/Aircraft/replay.hxx b/src/Aircraft/replay.hxx index 5830d71f0..9ce21d632 100644 --- a/src/Aircraft/replay.hxx +++ b/src/Aircraft/replay.hxx @@ -50,6 +50,9 @@ struct FGReplayData { // Incoming multiplayer messages. std::vector<std::shared_ptr<std::vector<char>>> multiplayer_messages; + std::vector<char> extra_properties; + std::map<std::string, std::string> replay_extra_property_changes; + std::vector<std::string> replay_extra_property_removals; // Updates static statistics defined below. void UpdateStats(); @@ -81,6 +84,15 @@ typedef struct { std::string speaker; } FGReplayMessages; +enum FGTapeType +{ + FGTapeType_NORMAL, + FGTapeType_CONTINUOUS, + FGTapeType_RECOVERY, +}; + +struct FGFrameInfo; + typedef std::deque < FGReplayData *> replay_list_type; typedef std::vector < FGReplayMessages > replay_messages_type; @@ -114,8 +126,27 @@ private: void clear(); FGReplayData* record(double time); void interpolate(double time, const replay_list_type &list); - void replay(double time, size_t offset, size_t offset_old=0); - void replay(double time, FGReplayData* pCurrentFrame, FGReplayData* pOldFrame=NULL); + void replay( + double time, + size_t offset, + size_t offset_old, + bool replay_signals, + bool replay_multiplayer, + bool replay_extra_properties, + int* xpos=nullptr, + int* ypos=nullptr, + int* xsize=nullptr, + int* ysize=nullptr + ); + void replay( + double time, + FGReplayData* pCurrentFrame, + FGReplayData* pOldFrame=nullptr, + int* xpos=nullptr, + int* ypos=nullptr, + int* xsize=nullptr, + int* ysize=nullptr + ); void guiMessage(const char* message); void loadMessages(); void fillRecycler(); @@ -129,13 +160,12 @@ private: bool listTapes(bool SameAircraftFilter, const SGPath& tapeDirectory); bool saveTape(const SGPath& Filename, SGPropertyNode_ptr MetaData); bool loadTape(const SGPath& Filename, bool Preview, SGPropertyNode& MetaMeta); - bool continuousWriteHeader( + SGPropertyNode_ptr continuousWriteHeader( std::ofstream& out, - SGPropertyNode_ptr& myMetaData, - SGPropertyNode_ptr& Config, - const SGPath& path + const SGPath& path, + FGTapeType tape_type ); - bool continuousWriteFrame(FGReplayData* r, std::ostream& out); + bool continuousWriteFrame(FGReplayData* r, std::ostream& out, SGPropertyNode_ptr meta); double sim_time; double last_mt_time; @@ -161,6 +191,11 @@ private: SGPropertyNode_ptr replay_multiplayer; SGPropertyNode_ptr recovery_period; + SGPropertyNode_ptr m_sim_startup_xpos; + SGPropertyNode_ptr m_sim_startup_ypos; + SGPropertyNode_ptr m_sim_startup_xsize; + SGPropertyNode_ptr m_sim_startup_ysize; + double replay_time_prev; // Used to detect jumps while replaying. double m_high_res_time; // default: 60 secs of high res data @@ -177,11 +212,16 @@ private: void valueChanged(SGPropertyNode * node); // Things for replaying from uncompressed fgtape file. - std::ifstream m_continuous_in; - std::map<double, size_t> m_continuous_time_to_offset; + std::ifstream m_continuous_in; + bool m_continuous_in_multiplayer; + bool m_continuous_in_extra_properties; + std::map<double, FGFrameInfo> m_continuous_in_time_to_frameinfo; + SGPropertyNode_ptr m_continuous_in_config; + double m_continuous_in_time_last; // For writing uncompressed fgtape file. - std::ofstream m_continuous_out; + SGPropertyNode_ptr m_continuous_out_config; + std::ofstream m_continuous_out; }; #endif // _FG_REPLAY_HXX diff --git a/src/Viewer/FGEventHandler.cxx b/src/Viewer/FGEventHandler.cxx index d45db6025..bf6d0550c 100644 --- a/src/Viewer/FGEventHandler.cxx +++ b/src/Viewer/FGEventHandler.cxx @@ -161,6 +161,26 @@ bool isMainWindow(const osgGA::GUIEventAdapter& ea, osgGA::GUIActionAdapter& us) } +void FGEventHandler::setWindowRectangleInteriorWithCorrection(osgViewer::GraphicsWindow* window, int x, int y, int width, int height) +{ + // Store (x y) in our state so that our handle() event handler can + // compare the requested position with the actual position and update + // (m_setWindowRectangle_delta_x m_setWindowRectangle_delta_y) accordingly. + // + SG_LOG(SG_VIEW, SG_DEBUG, "FGEventHandler::setWindowRectangle(): pos=(" << x << " " << y << ")" + << " delta=(" << m_setWindowRectangle_delta_x << " " << m_setWindowRectangle_delta_y << ")" + ); + m_setWindowRectangle_called = true; + m_setWindowRectangle_called_x = x; + m_setWindowRectangle_called_y = y; + window->setWindowRectangle( + x - m_setWindowRectangle_delta_x, + y - m_setWindowRectangle_delta_y, + width, + height + ); +} + bool FGEventHandler::handle(const osgGA::GUIEventAdapter& ea, osgGA::GUIActionAdapter& us) { @@ -274,8 +294,29 @@ bool FGEventHandler::handle(const osgGA::GUIEventAdapter& ea, return true; } CameraGroup::getDefault()->resized(); - if (resizable) - globals->get_renderer()->resize(ea.getWindowWidth(), ea.getWindowHeight()); + if (resizable) { + if (m_setWindowRectangle_called) { + // Update m_setWindowRectangle_delta_x and + // m_setWindowRectangle_delta_y so that our + // setWindowRectangle() compensates for the window furniture + // differences in future calls. + // + m_setWindowRectangle_called = false; + int error_x = ea.getWindowX() - m_setWindowRectangle_called_x; + int error_y = ea.getWindowY() - m_setWindowRectangle_called_y; + SG_LOG(SG_VIEW, SG_BULK, "m_setWindowRectangle_called is set:" + << " m_setWindowRectangle_delta=(" + << m_setWindowRectangle_delta_x << " " << m_setWindowRectangle_delta_y << ")" + << " m_setWindowRectangle_called_=(" + << m_setWindowRectangle_called_x << " " << m_setWindowRectangle_called_y << ")" + << " ea.getWindow=(" << ea.getWindowX() << " " << ea.getWindowY() << ")" + << " error=(" << error_x << " " << error_y << ")" + ); + m_setWindowRectangle_delta_x += error_x; + m_setWindowRectangle_delta_y += error_y; + } + globals->get_renderer()->resize(ea.getWindowWidth(), ea.getWindowHeight(), ea.getWindowX(), ea.getWindowY()); + } statsHandler->handle(ea, us); #ifdef SG_MAC // work around OSG Cocoa-Viewer issue with resize event handling, diff --git a/src/Viewer/FGEventHandler.hxx b/src/Viewer/FGEventHandler.hxx index 9e1bbbb24..b593ee6e1 100644 --- a/src/Viewer/FGEventHandler.hxx +++ b/src/Viewer/FGEventHandler.hxx @@ -99,6 +99,11 @@ public: void reset(); void clear(); + + // Wrapper for osgViewer::GraphicsWindow::setWindowRectangle() that takes + // coordinates excluding window furniture. + // + void setWindowRectangleInteriorWithCorrection(osgViewer::GraphicsWindow* window, int x, int y, int width, int height); static int translateKey(const osgGA::GUIEventAdapter& ea); static int translateModifiers(const osgGA::GUIEventAdapter& ea); @@ -122,6 +127,12 @@ protected: void handleStats(osgGA::GUIActionAdapter& us); bool changeStatsCameraRenderOrder; SGPropertyNode_ptr _display, _print; +private: + bool m_setWindowRectangle_called = false; + int m_setWindowRectangle_called_x = 0; + int m_setWindowRectangle_called_y = 0; + int m_setWindowRectangle_delta_x = 0; + int m_setWindowRectangle_delta_y = 0; }; bool eventToWindowCoords(const osgGA::GUIEventAdapter* ea, double& x, double& y); diff --git a/src/Viewer/renderer.cxx b/src/Viewer/renderer.cxx index 2720f1a51..569345a51 100644 --- a/src/Viewer/renderer.cxx +++ b/src/Viewer/renderer.cxx @@ -498,6 +498,8 @@ FGRenderer::init( void ) _xsize = fgGetNode("/sim/startup/xsize", true); _ysize = fgGetNode("/sim/startup/ysize", true); + _xpos = fgGetNode("/sim/startup/xpos", true); + _ypos = fgGetNode("/sim/startup/ypos", true); _splash_alpha = fgGetNode("/sim/startup/splash-alpha", true); _horizon_effect = fgGetNode("/sim/rendering/horizon-effect", true); @@ -905,16 +907,14 @@ FGRenderer::updateSky() } void -FGRenderer::resize( int width, int height ) +FGRenderer::resize( int width, int height, int x, int y ) { - int curWidth = _xsize->getIntValue(), - curHeight = _ysize->getIntValue(); SG_LOG(SG_VIEW, SG_DEBUG, "FGRenderer::resize: new size " << width << " x " << height); - if ((curHeight != height) || (curWidth != width)) { // must guard setting these, or PLIB-PUI fails with too many live interfaces - _xsize->setIntValue(width); - _ysize->setIntValue(height); - } + if (width != _xsize->getIntValue()) _xsize->setIntValue(width); + if (height != _ysize->getIntValue()) _ysize->setIntValue(height); + if (x != _xpos->getIntValue()) _xpos->setIntValue(x); + if (y != _ypos->getIntValue()) _ypos->setIntValue(y); // update splash node if present _splash->resize(width, height); @@ -925,6 +925,12 @@ FGRenderer::resize( int width, int height ) #endif } +void +FGRenderer::resize( int width, int height ) +{ + resize(width, height, _xpos->getIntValue(), _ypos->getIntValue()); +} + typedef osgUtil::LineSegmentIntersector::Intersection Intersection; SGVec2d uvFromIntersection(const Intersection& hit) { diff --git a/src/Viewer/renderer.hxx b/src/Viewer/renderer.hxx index 50054bf9c..e87afedae 100644 --- a/src/Viewer/renderer.hxx +++ b/src/Viewer/renderer.hxx @@ -58,7 +58,8 @@ public: void setupView(); - void resize(int width, int height ); + void resize(int width, int height); + void resize(int width, int height, int x, int y); void update(); @@ -116,6 +117,7 @@ protected: SGPropertyNode_ptr _textures; SGPropertyNode_ptr _cloud_status, _visibility_m; SGPropertyNode_ptr _xsize, _ysize; + SGPropertyNode_ptr _xpos, _ypos; SGPropertyNode_ptr _panel_hotspots, _sim_delta_sec, _horizon_effect, _altitude_ft; SGPropertyNode_ptr _virtual_cockpit; SGTimeStamp _splash_time;