Melchior FRANZ:
Here's a Perl implementation of a METAR proxy server. Tested on Linux only, but should work on all Unices, and possibly on Windows, too. Its purpose is to: - provide METAR data for machines without internet connection - centralize METAR fetching: one machine in a network runs the proxy, all other connect to the proxy - deliver defined and reproducible weather for educational purposes - save weather situations for later use in fgfs Quick instructions to download the world weather for the last 3 hours and run proxy and fgfs with it (~ 2MB download; for less bandwidth consumption see the --record mode): $ metarproxy --download 3h $ metarproxy -v -c & $ fgfs --proxy=localhost:5509 --time-offset=-2 --enable-real-weather-fetch
This commit is contained in:
parent
ded8b8f34e
commit
4ffb6c0fe9
2 changed files with 802 additions and 0 deletions
utils/metarproxy
280
utils/metarproxy/README
Normal file
280
utils/metarproxy/README
Normal file
|
@ -0,0 +1,280 @@
|
||||||
|
FlightGear METAR proxy server
|
||||||
|
=============================
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
metarproxy is a caching proxy server for METAR data strings written in
|
||||||
|
Perl. It can be used from the FlightGear flight simulator to:
|
||||||
|
|
||||||
|
- provide METAR data for machines without internet connection
|
||||||
|
- centralize METAR fetching: one machine in a network runs the proxy, all
|
||||||
|
other connect to the proxy
|
||||||
|
- deliver defined and reproducible weather for educational purposes
|
||||||
|
- save weather situations for later use
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Quick instructions to try out:
|
||||||
|
|
||||||
|
$ metarproxy --download 3h
|
||||||
|
$ metarproxy --color &
|
||||||
|
$ fgfs --proxy=localhost:5509 --time-offset=-2 --enable-real-weather-fetch
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
To make use of the proxy, you have to:
|
||||||
|
|
||||||
|
1. check if you want to use the default cache directory
|
||||||
|
and other default settings, or change them accordingly
|
||||||
|
2. make sure the cache is filled with METAR strings
|
||||||
|
3. start the proxy server
|
||||||
|
4. run fgfs with appropriate time and proxy settings
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
1. Basic setup and preparing the cache
|
||||||
|
======================================
|
||||||
|
If you are happy with the defaults, you can well skip to the
|
||||||
|
next section.
|
||||||
|
|
||||||
|
|
||||||
|
1a. The cache directory
|
||||||
|
-----------------------
|
||||||
|
All metarproxy operation modes need access to a cache, either for
|
||||||
|
storing or retrieving METAR strings. By default, the cache directory
|
||||||
|
is $FG_HOME/metar, whereby $FG_HOME is either to be set as environment
|
||||||
|
variable, or defaults to $HOME/.fgfs. $HOME, in turn, defaults to "."
|
||||||
|
(the current working directory). In other words: if no provisions are
|
||||||
|
made, you end up with /home/$USER/.fgfs/metar as your cache directory
|
||||||
|
on Linux-like operating systems, and ./.fgfs/metar elsewhere.
|
||||||
|
|
||||||
|
There are several ways to change the cache path:
|
||||||
|
|
||||||
|
- change one of the environment variables, ideally $FG_HOME. This can
|
||||||
|
be done in the system configuration in MS Windows, and in ~/.bashrc
|
||||||
|
or ~/.profile etc. on Linux-like systems
|
||||||
|
|
||||||
|
export FG_HOME=/var/tmp/metar
|
||||||
|
|
||||||
|
- or on the command line when running metarproxy:
|
||||||
|
|
||||||
|
$ FG_HOME=/var/tmp/metar metarproxy
|
||||||
|
|
||||||
|
- you can also set the cache directory directly as a command line option
|
||||||
|
--base or -b:
|
||||||
|
|
||||||
|
$ metarproxy --base=/var/tmp/metar
|
||||||
|
|
||||||
|
- this command line option can, together with any of the other metarproxy
|
||||||
|
options, be stored again in an environment variable METARPROXY
|
||||||
|
|
||||||
|
export METARPROXY="-c -vv -b/var/tmp/metar"
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
1b. set metarproxy's proxy server
|
||||||
|
---------------------------------
|
||||||
|
metarproxy isn't only a proxy server itself, it can also use one to
|
||||||
|
download METAR strings. By default it uses the one defined in the
|
||||||
|
environment variable http_proxy (which is commonly used on Linux-like
|
||||||
|
systems, and is, for instance, used by the lynx browser), or none if
|
||||||
|
unset. To set a particular proxy server for HTTP download, use one of
|
||||||
|
these methods:
|
||||||
|
|
||||||
|
- set http_proxy globally: EXPORT http_proxy=http://localhost:3128/
|
||||||
|
- or on the command line: $ http_proxy=http://localhost:3128/ metarproxy
|
||||||
|
- unset http_proxy: $ http_proxy= metarproxy
|
||||||
|
- use the command line option: $ metarproxy --proxy=http://localhost:3128/
|
||||||
|
- set the option globally: EXPORT METARPROXY="-yhttp://localhost:3128"
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
2. Fill the cache with METAR data
|
||||||
|
=================================
|
||||||
|
|
||||||
|
There are three operation modes to do that:
|
||||||
|
|
||||||
|
2a. --download mode to download worldwide data sets
|
||||||
|
2b. --install mode to install files from your system
|
||||||
|
2c. --record mode to record a selection of stations over some period
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
2a. --download mode
|
||||||
|
-------------------
|
||||||
|
You can download worldwide sets of METAR strings, each in a file of about
|
||||||
|
1MB size from weather.noaa.gov[1]. This can be done with a separate ftp
|
||||||
|
client or web browser, but it can also be done by metarproxy:
|
||||||
|
|
||||||
|
$ metarproxy --download 3h ... download last three hours (~ 3MB)
|
||||||
|
|
||||||
|
Note that the file for the *current* hour is only partly filled! You can
|
||||||
|
use from 1h up to 24h. Alternatively, you can request particular hours:
|
||||||
|
|
||||||
|
$ metarproxy --download 0 ... download first hour after midnight GMT
|
||||||
|
|
||||||
|
Ranges are allowed, too:
|
||||||
|
|
||||||
|
$ metarproxy --download 0-2 ... download first three hours after
|
||||||
|
midnight GMT
|
||||||
|
|
||||||
|
These three methods can be use in combination:
|
||||||
|
|
||||||
|
$ metarproxy --download 6h 0-2 4
|
||||||
|
|
||||||
|
Files downloaded this way aren't stored on your systems in the same form
|
||||||
|
as they are offered under [1], but are already stored in the cache in a
|
||||||
|
different way (see section 5). Redundant strings are not stored, so it's safe
|
||||||
|
to --download the same hours more than once. This won't create duplicates.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
2b. --install mode
|
||||||
|
------------------
|
||||||
|
The --download mode needs a sufficiently cheap and fast internet
|
||||||
|
connection. Sometimes it may be desirable to download the files directly
|
||||||
|
from the links (see [1]) on one computer, to burn them on a CD and then
|
||||||
|
to install them on the laptop. The downloaded files have names like
|
||||||
|
00Z.TXT to 23Z.TXT, whereby the number stands for the hour when they
|
||||||
|
were started. Only the last 24 hours are available for download.
|
||||||
|
If GMT is 1800, then 18Z.TXT will be the currently written and most
|
||||||
|
recent file. 19Z.TXT is already 23 hours old and will be overwritten
|
||||||
|
in one hour. To install such files in the cache, do this:
|
||||||
|
|
||||||
|
$ metarproxy --install 00Z.TXT 01Z.TXT
|
||||||
|
|
||||||
|
or
|
||||||
|
|
||||||
|
$ metarproxy --install ??Z.TXT
|
||||||
|
|
||||||
|
etc.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
2c. --record mode
|
||||||
|
-----------------
|
||||||
|
To record a set of stations over a period, without the need to download
|
||||||
|
several megabytes of data, you can use the record mode:
|
||||||
|
|
||||||
|
$ metarproxy --record KSFO KOAK KNUQ KSJC KCCR
|
||||||
|
|
||||||
|
The stations are then checked every 15 minutes and the METAR data
|
||||||
|
stored in the cache. Additionally, you can specify one or more files
|
||||||
|
with station IDs:
|
||||||
|
|
||||||
|
$ metarproxy --record --file=$FG_HOME/station-list
|
||||||
|
$ metarproxy --record EDDM --file=tmp/Austria --file=/tmp/Hungary
|
||||||
|
|
||||||
|
These files simply contain station IDs separated by spaces in one
|
||||||
|
or more lines:
|
||||||
|
|
||||||
|
$ cat /tmp/Austria
|
||||||
|
LOWL LOWI LOWS LOWW LOWK LOWG
|
||||||
|
LOXL LOXA LOXT
|
||||||
|
|
||||||
|
Some of the IDs are logically assigned, so that you can create a list
|
||||||
|
of, lets say, all Austrian METAR stations from FlightGear's METAR list:
|
||||||
|
|
||||||
|
$ zgrep "^LO" $FG_ROOT/Airports/metar.dat.gz > /tmp/Austria
|
||||||
|
$ zgrep "^ED" $FG_ROOT/Airports/metar.dat.gz > /tmp/Germany
|
||||||
|
$ zgrep "^EG" $FG_ROOT/Airports/metar.dat.gz > /tmp/UK
|
||||||
|
$ zgrep "^K" $FG_ROOT/Airports/metar.dat.gz > /tmp/USA
|
||||||
|
|
||||||
|
Quit the --record mode by Ctrl-C or killing the program.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
3. run the metarproxy server
|
||||||
|
============================
|
||||||
|
|
||||||
|
assuming that the cache directory is already set, you just need to
|
||||||
|
run the proxy:
|
||||||
|
|
||||||
|
$ metarproxy&
|
||||||
|
|
||||||
|
or with colored output and more log messages:
|
||||||
|
|
||||||
|
$ metarproxy -c -vv
|
||||||
|
|
||||||
|
The proxy listens to port 5509 by default, but you can easily let
|
||||||
|
it use another port. As you can see, the proxy is quite liberal
|
||||||
|
with respect to option syntax:
|
||||||
|
|
||||||
|
$ metarproxy --port 1234
|
||||||
|
$ metarproxy --port=1234
|
||||||
|
$ metarproxy -p 1234
|
||||||
|
$ metarproxy -p1234
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
4. let fgfs use the metar proxy
|
||||||
|
===============================
|
||||||
|
|
||||||
|
All you need to do is point FlightGear to the metar proxy and let
|
||||||
|
it run at a simulated time for which you actually have cached METAR
|
||||||
|
data:
|
||||||
|
|
||||||
|
$ fgfs --proxy=localhost:5509 --start-date-lat=2005:01:12:12:00:00
|
||||||
|
|
||||||
|
FlightGear will then fetch the metar data from the proxy as if it
|
||||||
|
were weather.noaa.gov. If no appropriate data set is found at all,
|
||||||
|
the proxy sends a default string. If data are found but older than
|
||||||
|
250 minutes, then the last successful data are sent again.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
5. the cache organization
|
||||||
|
=========================
|
||||||
|
|
||||||
|
metarproxy puts all data for KSFO on 2005/1/19 into a directory
|
||||||
|
2005-01-19/K/KS/KSFO. The date directory name is used to find all
|
||||||
|
data for this day, but metarproxy will also look at the date in
|
||||||
|
particular METAR strings. So, renaming the directory to 2005/1/20
|
||||||
|
won't make the cached data available for the next day! You need
|
||||||
|
to set fgfs' GMT date to 2005/1/19. Also, if the simulated GMT
|
||||||
|
is midnight, then you will get midnight weather. You can't
|
||||||
|
enjoy midnight weather at daylight. The cache always delivers
|
||||||
|
the (past) real weather at simulated GMT.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
6. download addresses
|
||||||
|
=====================
|
||||||
|
Download addresses for the last 24 hours:
|
||||||
|
|
||||||
|
http://weather.noaa.gov/pub/data/observations/metar/cycles/
|
||||||
|
ftp://weather.noaa.gov/data/observations/metar/cycles/
|
||||||
|
|
||||||
|
Addresses for the most recent METAR data strings of particular
|
||||||
|
stations:
|
||||||
|
|
||||||
|
http://weather.noaa.gov/pub/data/observations/metar/stations/
|
||||||
|
ftp://weather.noaa.gov/data/observations/metar/stations/
|
||||||
|
|
||||||
|
|
||||||
|
$Id$
|
||||||
|
Melchior FRANZ <mfranz@aon.at>, 2005/1/24
|
||||||
|
|
522
utils/metarproxy/metarproxy
Executable file
522
utils/metarproxy/metarproxy
Executable file
|
@ -0,0 +1,522 @@
|
||||||
|
#!/usr/bin/perl -w
|
||||||
|
# FlightGear METAR proxy server
|
||||||
|
# Melchior FRANZ (c) 2005, <mfranz@aon.at>, GPL V2
|
||||||
|
# $Id$
|
||||||
|
#
|
||||||
|
# typical use
|
||||||
|
# 1) fill cache, for example with:
|
||||||
|
# $ metarproxy --download 3h
|
||||||
|
#
|
||||||
|
# 2) run proxy with FlightGear:
|
||||||
|
# $ metarproxy -c -v &
|
||||||
|
# $ fgfs --enable-real-weather-fetch --proxy=localhost:5509 --start-date-lat=2005:01:11:12:00:00
|
||||||
|
|
||||||
|
use strict;
|
||||||
|
use IO::Socket;
|
||||||
|
use Net::hostent;
|
||||||
|
use Time::Local;
|
||||||
|
|
||||||
|
my $HOME = $ENV{'HOME'} || ".";
|
||||||
|
my $FG_HOME = $ENV{'FG_HOME'} || $HOME . "/.fgfs";
|
||||||
|
my $BASE = $FG_HOME . "/metar";
|
||||||
|
my $SERVER = "weather.noaa.gov";
|
||||||
|
my $PORT = 5509;
|
||||||
|
my $PROXY = $ENV{'http_proxy'};
|
||||||
|
my $METAR_MAX_AGE = 250 * 60;
|
||||||
|
my $METAR_DEFAULT = "00000KT 15KM CLR 15/00 A3000";
|
||||||
|
my @COLOR = ("31;1", "31", "32", "", "36;1");
|
||||||
|
my $USECOLOR = 0;
|
||||||
|
|
||||||
|
|
||||||
|
my $help = <<EOF;
|
||||||
|
Usage:
|
||||||
|
metarproxy [-v] [-b <path>] [-p <port>] [--serve]
|
||||||
|
metarproxy [-v] [-b <path>] [-y <proxy>] --download <list of: all|7|0-10|6h>
|
||||||
|
metarproxy [-v] [-b <path>] [-y <proxy>] --record [<list of station IDs>] [-f <path>]
|
||||||
|
metarproxy [-v] [-b <path>] --install <list of metar files>
|
||||||
|
metarproxy [-V]
|
||||||
|
metarproxy [-h]
|
||||||
|
|
||||||
|
server mode:
|
||||||
|
-s|--serve start proxy server (default)
|
||||||
|
-p|--port set port (default: $PORT)
|
||||||
|
|
||||||
|
download mode:
|
||||||
|
-d|--download <list of hours>
|
||||||
|
"all" ... whole day (24 files)
|
||||||
|
<number> ... this hour (example: 6)
|
||||||
|
<range> ... these hours (example: 2-5)
|
||||||
|
<period> ... last n hours (example: 3h)
|
||||||
|
-y|--proxy use proxy (default: off)
|
||||||
|
|
||||||
|
install mode:
|
||||||
|
-i|--install <list of files to install>
|
||||||
|
|
||||||
|
record mode:
|
||||||
|
-r|--record <list of METAR station IDs (ICAO)>
|
||||||
|
-f|--file <file containing list of station IDs>
|
||||||
|
-y|--proxy use proxy (default: off)
|
||||||
|
|
||||||
|
all modes:
|
||||||
|
-b|--base set base directory (default: \$FG_HOME/metar)
|
||||||
|
-c|--color toggle color mode (default: off)
|
||||||
|
-v|--verbose increase verbosity level (default: off; maximum: -vvvv)
|
||||||
|
|
||||||
|
-q|--quiet only show error messages
|
||||||
|
-h|--help this help
|
||||||
|
-V|--version return version number
|
||||||
|
|
||||||
|
Environment:
|
||||||
|
FG_HOME ... FlightGear home directory (default: \$HOME/.fgfs)
|
||||||
|
METARPROXY ... default options (e.g. export METARPROXY='-vv --color')
|
||||||
|
http_proxy ... system wide proxy setting (currently: '$PROXY')
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
\$ metarproxy -b\$HOME/.fgfs/metar --download 3h
|
||||||
|
\$ metarproxy --proxy=http://localhost:3128 --download all
|
||||||
|
\$ metarproxy --download 3h 7 21-23
|
||||||
|
\$ metarproxy --record -f/tmp/list LOWW LOWL
|
||||||
|
\$ metarproxy -b/var/tmp/metar --install /tmp/*Z.TXT
|
||||||
|
\$ metarproxy -p5600 & fgfs --proxy=localhost:5600 --enable-real-weather-fetch
|
||||||
|
\$ http_proxy= metarproxy --record LOXL
|
||||||
|
|
||||||
|
Sources:
|
||||||
|
http://weather.noaa.gov/pub/data/observations/metar/{stations,cycles}/
|
||||||
|
ftp://weather.noaa.gov/data/observations/metar/{stations,cycles}/
|
||||||
|
EOF
|
||||||
|
|
||||||
|
|
||||||
|
my $ERR = 0;
|
||||||
|
my $WARN = 1;
|
||||||
|
my $INFO = 2;
|
||||||
|
my $BULK = 3;
|
||||||
|
my $DEBUG = 4;
|
||||||
|
my $VERBOSITY = $INFO;
|
||||||
|
|
||||||
|
my @ITEMS;
|
||||||
|
my $PROXYHOST;
|
||||||
|
my $PROXYPORT;
|
||||||
|
|
||||||
|
|
||||||
|
# main =======================================================================
|
||||||
|
|
||||||
|
|
||||||
|
sub parse_options()
|
||||||
|
{
|
||||||
|
sub argument {
|
||||||
|
map { return $_ if defined $_ and $_ ne "" } @_;
|
||||||
|
shift @ARGV;
|
||||||
|
return $ARGV[0];
|
||||||
|
}
|
||||||
|
my $mode = 4;
|
||||||
|
unshift @ARGV, split / /, $ENV{'METARPROXY'} if defined $ENV{'METARPROXY'};
|
||||||
|
while (1) {
|
||||||
|
$_ = $ARGV[0];
|
||||||
|
defined $_ or last;
|
||||||
|
# dissolve glued together short options (e.g. -cvv)
|
||||||
|
if (/^-([^-]{2,})$/) {
|
||||||
|
shift @ARGV;
|
||||||
|
map { unshift @ARGV, "-$_" } split //, $1;
|
||||||
|
next;
|
||||||
|
}
|
||||||
|
if (!/^-/) {
|
||||||
|
push @ITEMS, $_;
|
||||||
|
} elsif (/^(-d|--download)$/) {
|
||||||
|
$mode = 1;
|
||||||
|
} elsif (/^(-i|--install)$/) {
|
||||||
|
$mode = 2;
|
||||||
|
} elsif (/^(-r|--record)$/) {
|
||||||
|
$mode = 3;
|
||||||
|
} elsif (/^(-s|--server?)$/) {
|
||||||
|
$mode = 4;
|
||||||
|
} elsif (/^(-b(.*)|--base(=(.*))?)/) {
|
||||||
|
my $path = &argument($2, $4);
|
||||||
|
defined $path or &fatal("-b|--base option lacks <path> argument");
|
||||||
|
$path =~ s/^~/$HOME/;
|
||||||
|
$BASE = $path;
|
||||||
|
&log($BULK, "set option --base: '$BASE'");
|
||||||
|
} elsif (/^(-f(.*)|--file(=(.*))?)$/) {
|
||||||
|
my $file = &argument($2, $4);
|
||||||
|
defined $file or &fatal("-f|--file option lacks <path> argument");
|
||||||
|
&log($BULK, "set option --file: '$file'");
|
||||||
|
&read_icao_file($file);
|
||||||
|
} elsif (/^(-p(.*)|--port(=(.*))?)$/) {
|
||||||
|
$PORT = &argument($2, $4);
|
||||||
|
defined $PORT or &fatal("--port option lacks <port number> argument");
|
||||||
|
&log($BULK, "set option --port: '$PORT'");
|
||||||
|
} elsif (/^(-y(.*)|--proxy(=(.*))?)$/) {
|
||||||
|
$PROXY = &argument($2, $4);
|
||||||
|
defined $PROXY or &fatal("--proxy option lacks <host> definition");
|
||||||
|
&log($BULK, "set option --proxy: '$PROXY'");
|
||||||
|
} elsif (/^(-v|--verbose)$/) {
|
||||||
|
$VERBOSITY++;
|
||||||
|
} elsif (/^(-q|--quiet)$/) {
|
||||||
|
$VERBOSITY = 0;
|
||||||
|
} elsif (/^(-h|--help)$/) {
|
||||||
|
print $help;
|
||||||
|
return 0;
|
||||||
|
} elsif (/^(-V|--version)$/) {
|
||||||
|
($_ = '$Revision$') =~ s/.*(\d+\.\d+).*/print "$1\n"/e;
|
||||||
|
return 0;
|
||||||
|
} elsif (/^(-c|--color)$/) {
|
||||||
|
$USECOLOR = !$USECOLOR;
|
||||||
|
} else {
|
||||||
|
&fatal("unknown option $_");
|
||||||
|
}
|
||||||
|
shift @ARGV;
|
||||||
|
}
|
||||||
|
return $mode;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
sub main()
|
||||||
|
{
|
||||||
|
undef $PROXY if $PROXY eq "";
|
||||||
|
my $mode = &parse_options();
|
||||||
|
exit if $mode == 0;
|
||||||
|
|
||||||
|
-d $FG_HOME or mkdir $FG_HOME or &fatal("cannot create directory $FG_HOME ($!)");
|
||||||
|
-d $BASE or mkdir $BASE or &fatal("cannot create directory $BASE ($!)");
|
||||||
|
|
||||||
|
if (defined $PROXY) {
|
||||||
|
$PROXY =~ m|^(http://)?([a-zA-Z][a-zA-Z0-9-.]*):(\d+)/?| or &fatal("invalid proxy address: '$PROXY'");
|
||||||
|
($PROXYHOST, $PROXYPORT) = ($2, $3);
|
||||||
|
}
|
||||||
|
|
||||||
|
my $ret = 0;
|
||||||
|
if ($mode == 1) {
|
||||||
|
$ret = &download;
|
||||||
|
} elsif ($mode == 2) {
|
||||||
|
$ret = &install();
|
||||||
|
} elsif ($mode == 3) {
|
||||||
|
$ret = &record();
|
||||||
|
} elsif ($mode == 4) {
|
||||||
|
&log($ERR, "ignoring command line args: " . (join ", ", @ITEMS)) if @ITEMS;
|
||||||
|
$ret = &serve();
|
||||||
|
}
|
||||||
|
exit $ret;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
sub read_icao_file($)
|
||||||
|
{
|
||||||
|
my $path = shift;
|
||||||
|
$path =~ s/^\~/$HOME/;
|
||||||
|
|
||||||
|
if (!open(F, "<$path")) {
|
||||||
|
&log($ERR, "cannot open station list $path ($!)");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
while (<F>) {
|
||||||
|
s/\s+$//;
|
||||||
|
foreach (split) {
|
||||||
|
if (/^[A-Z][A-Z0-9]{3}$/) {
|
||||||
|
push @ITEMS, $_;
|
||||||
|
} else {
|
||||||
|
&log($ERR, "discarding suspicious station from $path: $_");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
close F or &log($ERR, "cannot close station list $path ($!)");
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# download ===================================================================
|
||||||
|
|
||||||
|
|
||||||
|
sub download()
|
||||||
|
{
|
||||||
|
my %h;
|
||||||
|
sub norm {
|
||||||
|
my $i = shift;
|
||||||
|
$i = 0 if $i < 0;
|
||||||
|
$i = 23 if $i > 23;
|
||||||
|
return $i;
|
||||||
|
}
|
||||||
|
foreach (@ITEMS) {
|
||||||
|
if (/^all$/) {
|
||||||
|
map { $h{$_} = 1 } (0 .. 23);
|
||||||
|
} elsif (/^(\d+)-(\d+)$/) {
|
||||||
|
map { $h{$_} = 1 } (&norm($1) .. &norm($2));
|
||||||
|
} elsif (/^(\d+)h$/) {
|
||||||
|
my $to = (gmtime(time))[2];
|
||||||
|
my $from = $to - &norm($1) + 1;
|
||||||
|
if ($from < 0) {
|
||||||
|
map { $h{$_} = 1 } ((24 + $from) .. 23);
|
||||||
|
$from = 0;
|
||||||
|
}
|
||||||
|
map { $h{$_} = 1 } ($from .. $to);
|
||||||
|
} elsif (/^(\d+)$/) {
|
||||||
|
$h{&norm($1)} = 1;
|
||||||
|
} else {
|
||||||
|
&log($ERR, "illegal download argument '$_' ignored");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
@ITEMS = sort { $a <=> $b } keys %h;
|
||||||
|
@ITEMS or &fatal("nothing to download");
|
||||||
|
&log($INFO, "downloading: " . (join ", ", @ITEMS));
|
||||||
|
foreach (@ITEMS) {
|
||||||
|
my $file = sprintf "/pub/data/observations/metar/cycles/%02dZ.TXT", $_;
|
||||||
|
&install_metar_http($SERVER, "80", $file);
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# install ====================================================================
|
||||||
|
|
||||||
|
|
||||||
|
sub install()
|
||||||
|
{
|
||||||
|
foreach my $file (@ITEMS) {
|
||||||
|
&log($INFO, "installing $file");
|
||||||
|
if (! -f $file) {
|
||||||
|
&log($ERR, "file $file doesn't exist");
|
||||||
|
next;
|
||||||
|
}
|
||||||
|
if (!open (IN, "<$file")) {
|
||||||
|
&log($ERR, "cannot open $file ($!)");
|
||||||
|
next;
|
||||||
|
}
|
||||||
|
local $/ = "";
|
||||||
|
&install_metar($_) foreach <IN>;
|
||||||
|
close IN or &log($ERR, "cannot close $file ($!)");
|
||||||
|
}
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# install a METAR string KSFO under $FG_HOME/metar/2005-01-12/K/KS/KSFO
|
||||||
|
sub install_metar($)
|
||||||
|
{
|
||||||
|
my $metar = shift;
|
||||||
|
return unless $metar =~ /^(\d{4})\/(\d+)\/(\d+)\s(\d+):(\d+).*\015?\012([A-Z])([A-Z0-9])([A-Z0-9]{2})\s/s;
|
||||||
|
|
||||||
|
my $name = sprintf "$BASE/%04d-%02d-%02d", $1, $2, $3;
|
||||||
|
-d $name or mkdir $name or &fatal("cannot create directory $name ($!)");
|
||||||
|
$name .= "/$6";
|
||||||
|
-d $name or mkdir $name or &fatal("cannot create directory $name ($!)");
|
||||||
|
$name .= "/$6$7";
|
||||||
|
-d $name or mkdir $name or &fatal("cannot create directory $name ($!)");
|
||||||
|
$name .= "/$6$7$8";
|
||||||
|
|
||||||
|
my $found;
|
||||||
|
if (open(F, "<$name")) {
|
||||||
|
local $/ = "";
|
||||||
|
while (<F>) {
|
||||||
|
if (m|^$1/$2/$3 $4:$5\s|s) {
|
||||||
|
$found = 1;
|
||||||
|
last;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
close F or &log($ERR, "cannot close file $name ($!)");
|
||||||
|
return if defined $found;
|
||||||
|
}
|
||||||
|
&log($INFO, "writing to $name");
|
||||||
|
|
||||||
|
open(F, ">>$name") or &fatal("cannot append to file $name ($!)");
|
||||||
|
print F $metar;
|
||||||
|
close F or &log($ERR, "cannot close file $name ($!)");
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
sub install_metar_http($$$)
|
||||||
|
{
|
||||||
|
my ($server, $port, $addr) = @_;
|
||||||
|
&log($INFO, "installing data from http://$server:$port$addr");
|
||||||
|
if (defined $PROXYHOST) {
|
||||||
|
&log($INFO, "via proxy http://$PROXYHOST:$PROXYPORT");
|
||||||
|
$addr = "http://$server" . $addr;
|
||||||
|
($server, $port) = ($PROXYHOST, $PROXYPORT);
|
||||||
|
}
|
||||||
|
|
||||||
|
my $socket = IO::Socket::INET->new(Proto => "tcp", PeerAddr => $server, PeerPort => $port);
|
||||||
|
$socket or &fatal("cannot connect to http://$server:$port$addr/ ($!)");
|
||||||
|
$socket->autoflush(1);
|
||||||
|
my $get = "GET $addr HTTP/1.0";
|
||||||
|
print $socket "$get\015\012\015\012";
|
||||||
|
&log($DEBUG, ":$get:");
|
||||||
|
|
||||||
|
# skip header
|
||||||
|
while (<$socket>) {
|
||||||
|
s/\s*$//;
|
||||||
|
last if /^$/;
|
||||||
|
&log($DEBUG, "[$_]");
|
||||||
|
}
|
||||||
|
local $/ = "";
|
||||||
|
foreach (<$socket>) {
|
||||||
|
&install_metar("$_\n");
|
||||||
|
}
|
||||||
|
close($socket) or &log($ERR, "cannot close INET socket ($!)");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# record =====================================================================
|
||||||
|
|
||||||
|
|
||||||
|
sub record()
|
||||||
|
{
|
||||||
|
@ITEMS or &fatal("no stations given");
|
||||||
|
|
||||||
|
my %h;
|
||||||
|
# check for validity and remove duplicates
|
||||||
|
foreach (@ITEMS) {
|
||||||
|
if (/^[A-Z][A-Z0-9]{3}$/) {
|
||||||
|
$h{$_} = 1;
|
||||||
|
} else {
|
||||||
|
&log($ERR, "discarding invalid station '$_'");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
@ITEMS = sort keys %h;
|
||||||
|
|
||||||
|
&log($INFO, "recording stations @ITEMS");
|
||||||
|
while (1) {
|
||||||
|
foreach (@ITEMS) {
|
||||||
|
&install_metar_http($SERVER, "80", "/pub/data/observations/metar/stations/$_.TXT");
|
||||||
|
}
|
||||||
|
&log($INFO, "sleeping ...");
|
||||||
|
sleep 15 * 60
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# serve ======================================================================
|
||||||
|
|
||||||
|
|
||||||
|
sub serve()
|
||||||
|
{
|
||||||
|
my $server = IO::Socket::INET->new(Proto => 'tcp', LocalPort => $PORT, Listen => SOMAXCONN, Reuse => 1);
|
||||||
|
$server or &fatal("cannot setup server ($!)");
|
||||||
|
&log($BULK, "server $0 accepting clients on port $PORT");
|
||||||
|
|
||||||
|
while (my $client = $server->accept()) {
|
||||||
|
$client->autoflush(1);
|
||||||
|
my $hostinfo = gethostbyaddr($client->peeraddr);
|
||||||
|
my $clientname = $hostinfo->name || $client->peerhost;
|
||||||
|
my $addr = inet_ntoa(inet_aton($clientname));
|
||||||
|
|
||||||
|
my ($icao, $epoch);
|
||||||
|
while (<$client>) {
|
||||||
|
s/\s+$//;
|
||||||
|
&log($DEBUG, $_);
|
||||||
|
|
||||||
|
if (m|^GET\s+http://weather.noaa.gov/.*/([A-Z][A-Z0-9]{3}).TXT\s+HTTP/|) {
|
||||||
|
$icao = $1;
|
||||||
|
} elsif (/X-Time: (\d+)/) {
|
||||||
|
$epoch = $1;
|
||||||
|
} elsif (/^$/) {
|
||||||
|
last;
|
||||||
|
} else {
|
||||||
|
&log($INFO, "$_") if $VERBOSITY < $DEBUG;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (defined $icao and defined $epoch) {
|
||||||
|
my ($min, $hour, $day, $mon, $year) = (gmtime($epoch))[1 .. 5];
|
||||||
|
$year += 1900;
|
||||||
|
$mon++;
|
||||||
|
&log($BULK, sprintf "client '$clientname' [$addr] requests data for station $icao "
|
||||||
|
. "at %04d/%02d/%02d %02d:%02d", $year, $mon, $day, $hour, $min);
|
||||||
|
|
||||||
|
my ($metar, $age) = &get_metar($icao, $epoch);
|
||||||
|
if (defined $metar) {
|
||||||
|
if ($age <= $METAR_MAX_AGE) {
|
||||||
|
&log($BULK, "found (" . int($age / 60) . " min old)");
|
||||||
|
$metar =~ s/\s*$//s;
|
||||||
|
$METAR_DEFAULT = $metar;
|
||||||
|
$METAR_DEFAULT =~ s/.*\015?\012[A-Z0-9]{4}\s+[0-9]{6}Z\s+//s;
|
||||||
|
&log($DEBUG, "setting default to '$METAR_DEFAULT'");
|
||||||
|
$metar =~ s/\015?\012/\015\012/g;
|
||||||
|
} else {
|
||||||
|
&log($INFO, "found, but too old (" . int($age / 60) . " min)");
|
||||||
|
undef $metar;
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
&log($WARN, "not found!");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!defined $metar) {
|
||||||
|
&log($INFO, "sending last successful data again");
|
||||||
|
$metar = sprintf "%04d/%02d/%02d %02d:%02d\015\012",
|
||||||
|
$year, $mon, $day, $hour, $min;
|
||||||
|
$metar .= sprintf "$icao %02d%02d%02dZ $METAR_DEFAULT",
|
||||||
|
$day, $hour, $min;
|
||||||
|
}
|
||||||
|
print $client "Content-Type: text/plain\015\012"
|
||||||
|
. "X-MetarProxy: nasse Maus\015\012"
|
||||||
|
. "\015\012"
|
||||||
|
. "$metar\015\012";
|
||||||
|
&log($INFO, $metar);
|
||||||
|
} else {
|
||||||
|
&log($WARN, "incomplete request");
|
||||||
|
}
|
||||||
|
&log($BULK, "closing connection");
|
||||||
|
close $client;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
sub get_metar($$)
|
||||||
|
{
|
||||||
|
my $icao = shift;
|
||||||
|
my $rq_epoch = shift;
|
||||||
|
$icao =~ /^([A-Z])([A-Z0-9])([A-Z0-9]{2})$/;
|
||||||
|
|
||||||
|
sub scan_file($$) {
|
||||||
|
my $time = shift;
|
||||||
|
my $list = shift;
|
||||||
|
my ($hour, $day, $mon, $year) = (gmtime($time))[2 .. 5];
|
||||||
|
my $name = sprintf "$BASE/%04d-%02d-%02d/$1/$1$2/$1$2$3", $year + 1900, $mon + 1, $day;
|
||||||
|
if (open (F, "<$name")) {
|
||||||
|
&log($BULK, "reading $name");
|
||||||
|
local $/ = "";
|
||||||
|
push @$list, <F>;
|
||||||
|
close F or &log($ERR, "cannot close file $name ($!)");
|
||||||
|
} else {
|
||||||
|
&log($BULK, "no file $name to read ($!)");
|
||||||
|
}
|
||||||
|
return $hour < 2;
|
||||||
|
}
|
||||||
|
my @list; # "today" (and maybe "yesterday")
|
||||||
|
&scan_file($rq_epoch, \@list) and &scan_file($rq_epoch - 24 * 60 * 60, \@list);
|
||||||
|
|
||||||
|
my $age = 99999999;
|
||||||
|
my ($epoch, $metar);
|
||||||
|
foreach (@list) {
|
||||||
|
/^(\d{4})\/(\d+)\/(\d+)\s(\d+):(\d+).*\015?\012$icao\s/s or next;
|
||||||
|
$epoch = timegm(0, $5, $4, $3, $2 - 1, $1 - 1900);
|
||||||
|
next if $epoch > $rq_epoch; # lies in the future
|
||||||
|
next if $rq_epoch - $epoch > $age; # older than previous entry
|
||||||
|
$metar = $_;
|
||||||
|
$age = $rq_epoch - $epoch;
|
||||||
|
}
|
||||||
|
return ($metar, $age);
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
# ==================================================================
|
||||||
|
|
||||||
|
|
||||||
|
sub fatal()
|
||||||
|
{
|
||||||
|
&log($ERR, "$0: @_");
|
||||||
|
exit -1;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
sub log()
|
||||||
|
{
|
||||||
|
my $v = shift;
|
||||||
|
return if $v > $VERBOSITY;
|
||||||
|
$v = 4 if $v > 4;
|
||||||
|
print "\033[$COLOR[$v]m" if $USECOLOR;
|
||||||
|
print "@_";
|
||||||
|
print "\033[m" if $USECOLOR;
|
||||||
|
print "\n";
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
main
|
||||||
|
|
Loading…
Add table
Reference in a new issue