fly/fgmeta
1
0
Fork 0
Code Activity

FGmeta/Nasa2FGearthview

Browse source 
-----------------------
* merge contribution of eatdirt <chris@ringeval.com>
  add download, conversion and generation of normalmaps
  Big thanks Chris!
* update README and some minor updates to the script
This commit is contained in:
chris_blues 2019-03-25 18:20:43 +01:00
parent 1cf284bd6f
commit 7b51c99439
2 changed files with 734 additions and 241 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

223
Nasa2FGearthview/README.md
View file

@ -1,96 +1,123 @@
# Nasa2FGearthview # Nasa2FGearthview
A bash-script to convert NASA satellite images to ready-to-use A bash-script to convert NASA satellite images to ready-to-use
textures for FG's EarthView using ImageMagick. textures for FG's EarthView using ImageMagick and normalmap
For info about FGearthview, see the forum thread: You can get "normalmap" there:
https://github.com/planrich/normalmap
For info about FGearthview, see the forum thread:\
https://forum.flightgear.org/viewtopic.php?f=6&t=15754 https://forum.flightgear.org/viewtopic.php?f=6&t=15754
or this FG-wiki-page:
or this FG-wiki-page:\
http://wiki.flightgear.org/Earthview http://wiki.flightgear.org/Earthview
### Caution!
Don't use this script on a server! It will most likely cause
Denial-of-service (DoS). When working on these huge images, the
harddisk throughput will cease occasionally and CPU / RAM usage will
spike tremendously! So, only use this script on your home desktop
computer, or if you don't mind several long server-outages...
------------------------------------ ------------------------------------
About: ## About:
This script runs on Linux (maybe Mac also?) in a Bash This script runs on Linux (maybe Mac also?) in a Bash
(Bourne Again Shell) - Windows is not supported (by the nature of the (Bourne Again Shell) - Windows is not supported (by the nature of the
script). Maybe it works on windows as well, I don't know, feel free script). Maybe it works on windows as well, I don't know, feel free
to try, and please let me know! :) to try, and please let me know! :)
This will download the raw images from http://visibleearth.nasa.gov -
their server is not very fast, so I provide an alternative download
location: https://musicchris.de/download/FG/EarthView/raw-data-NASA.7z
This one is much quicker! If you really want the images directly from
NASA, then provide "nasa" to the script (see below)
In the end you will have 8 world-textures in .png and .dds format. In the end you will have 8 world-textures in .png and .dds format.
Generally .dds is better in performance, but it won't work on some Generally .dds is better in performance, but it won't work on some
graphics cards. If this is the case for you, then try the .png files. graphics cards. If this is the case for you, then try the .png files.
For further information see: For further information see:\
http://wiki.flightgear.org/index.php?title=DDS_Textures_in_FlightGear&redirect=no http://wiki.flightgear.org/index.php?title=DDS_Textures_in_FlightGear&redirect=no
If you also converted the clouds, then you'll also find 8 cloud- If you also converted the clouds and the height maps, then you'll also
textures in the format .png. Because the .dds-format has trouble with find 8 cloud- and 8 height textures (as well as their conversion to
rendering heavy alpha images, which is because of it's compression normal maps) in the format .png. Because the .dds-format has trouble
algorythm [1], I think it's useless to also build faulty files. with rendering heavy alpha images, which is because of it's
However, this is not entirely true! It is possible to switch off the compression algorythm [1], I think it's useless to also build faulty
.dds/DXT compression. But this results in huge files and is rather files. However, this is not entirely true! It is possible to switch
heavy on the GPU's RAM. off the .dds/DXT compression. But this results in huge files and is
rather heavy on the GPU's RAM.
Buckaroo has created a nice overview on dds-compression: Buckaroo has created a nice overview on dds-compression:
[1] http://www.buckarooshangar.com/flightgear/tut_dds.html [1] http://www.buckarooshangar.com/flightgear/tut_dds.html
------------------------------------ ------------------------------------
Installation and usage: ## Installation and usage:
Simply copy "convert.sh" into a folder of your liking and run it: Simply copy "convert.sh" into a folder of your liking and run it:
$ ./convert.sh ```shell
./convert.sh
```
This will show a help text, since you didn't specify any target(s). This will show a help text, since you didn't specify any target(s).
Possible targets are: Possible targets are:
* world * world
* clouds * clouds
* heights
* all * all
Additionally, there are some options you could specify (further Additionally, there are some options you could specify (further
explained below): explained below):
* 1k | 2k | 4k | 8k | 16k * 1k | 2k | 4k | 8k | 16k
* nasa * download | no-download
* no-download * world
* clouds
* heights
* cleanup * cleanup
* rebuild * rebuild
* check * check
So your call could look sth like this: So your call could look sth like this:
$ ./convert.sh world no-download cleanup 8k ```shell
./convert.sh world download alt cleanup 8k
```
------------------------------------ ------------------------------------
Requirements: ## Requirements:
WARNING! WARNING!
This script uses a *lot* disk space! Make sure you have at least 90GB This script uses a *lot* disk space! In my last test run, which
available! generated all maps in all resolutions, the disk usage was about 330GB!
Beware!\
Also, I wouldn't recommend doing this on a SSD! While SSDs are
generally faster, they also get more wear-and-tear when write such
huge files. So this script might cause your SSD to die earlier as it
should. Generally speaking, this won't kill your SSD, but it might
cause it to die earlier. HDDs are much more robust in that respect.
Also, this script will run for a *very long* time! It might be best to Also, this script will run for a *very long* time! It might be best to
let it run over night - your computer might become unresponsive from let it run over night - your computer might become unresponsive from
time to time, due to the heavy CPU and memory load, which tends to time to time, due to the heavy CPU and memory load, which tends to
occur, when converting 54000x27000 images. ;-) occur, when converting 54000x27000 images. ;-)
I also recommend to deactivate swapping!
$ sudo swapoff -a
To reactivate swapping do:
$ sudo swapon -a
This script relies on wget and imagemagick. Both are easily installed I also recommend to deactivate swapping!
by your systems package-management-system. ```shell
(On Debian/Ubuntu this is "apt-get") sudo swapoff -a
```
To reactivate swapping do:
```shell
$ sudo swapon -a
```
This script relies on wget, ImageMagick and, for converting the height
maps to normal maps, on "normalmap". Some of these programs are easily
installed by your systems package-management-system.\ (On
Debian/Ubuntu this is "apt-get").
So, on Debian for instance, you only need to put the following into So, on Debian for instance, you only need to put the following into
the console: the console:
$ sudo apt-get install wget imagemagick ```shell
sudo apt-get install wget imagemagick
```
Depending on your distro, the package names might differ slightly! Use Depending on your distro, the package names might differ slightly! Use
a search engine of your choice to find out, how the packages are named a search engine of your choice to find out, how the packages are named
@ -98,70 +125,92 @@ in your distro!
You may want to check: You may want to check:
$ apt search imagemagick ```shell
apt search imagemagick
```
### IMPORTANT!
Check out your ```/etc/ImageMagick-6/policy.xml```
On some distros, there are limits set, which will cause IM to abort
the conversion of images larger than
[rediculously small images](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=860763).
Edit and set to our needs:
* width: at least 55000
* height: at least 55000
* area: less than your free RAM
### Normalmap
For normalmap, you can download and compile it from
https://github.com/planrich/normalmap
You can install the binary into your system, or just copy it next to
convert.sh - both should work.
------------------------------------ ------------------------------------
Targets: ## Targets:
world **world**\
Generates the world tiles, needed to run FG with EarthView. Generates the world tiles, needed to run FG with EarthView.
You will find the results in output/[$resolution]/*. Copy You will find the results in output/[$resolution]/\*. Copy
these into $FGDATA/Models/Astro/*. More about the installation these into $FGDATA/Models/Astro/\*. More about the installation
of these textures can be found here: of these textures can be found here:
http://wiki.flightgear.org/Earthview#Customization http://wiki.flightgear.org/Earthview#Customization
clouds **clouds**\
Generates the cloud tiles, needed to run FG with EarthView. Generates the cloud tiles, needed to run FG with EarthView.
The locations are the same as the other textures mentioned The locations are the same as the other textures mentioned
above. Note that clouds are only available with up to 8k above. Note that clouds are only available with up to 8k
resolution, due to the available data at NASA. resolution, due to the available data at NASA.
all **heights**\
Converts everything needed for a full-blown earthview texture Generates the height tiles, which are then converted to the
set. Does the same as: normal maps needed to run FG with EarthView. The locations are
$ ./convert.sh world clouds the same as the other textures mentioned above. Note that
heights are only available with up to 8k resolution, due to the
available data at NASA.
**all**\
Converts everything needed for a full-blown earthview texture
set. Does the same as:
```./convert.sh world clouds heights```
Options: ## Options:
1k | 2k | 4k | 8k | 16k **1k | 2k | 4k | 8k | 16k**\
Lets you specify a desired resolution of the textures. Lets you specify a desired resolution of the textures.
Possible values are 1k, 2k, 4k, 8k and 16k. If nothing is Possible values are 1k, 2k, 4k, 8k and 16k. If nothing is
specified, the script will generate all of the resolutions. specified, the script will generate all of the resolutions.
16k is only available for earth textures. 16k is recommended only for earth textures, it will induce
oversampling from clouds and height maps.
nasa **download**\
Causes the script to download directly from Causes the script to download the needed data, this is the
http://visibleearth.nasa.gov . If omitted the script will default behavior (and can be omitted).
download from
https://musicchris.de/download/FG/EarthView/raw-data-NASA.7z
which is much faster!
Uses wget either way.
no-download **no-download**\
Causes the script to skip the download function. If you Causes the script to skip the download function. If you
already have the source images, then you don't need to already have the source images, then you don't need to
re-download them. (About 2.4GB!) re-download them. (About 2.4GB!)
If omitted, the script will download the source images from If omitted, the script will download the source images from
https://musicchris.de/download/FG/EarthView/raw-data-NASA.7z the default location.
cleanup **cleanup**\
Deletes the temporary files created during texture generation. Deletes the temporary files created during texture generation.
These can be found in tmp/ These can be found in tmp/
Note: if for some reason you later want some other resolution, Note: if for some reason you later want some other resolution,
then it's good to have the data there. So only do this, when then it's good to have the data there. So only do this, when
you're quite sure that you're done. you're quite sure that you're done.
Frees up a lot of disk-space! Which would have to be Frees up a lot of disk-space! Which would have to be
regenerated if needed again. regenerated if needed again.
rebuild **rebuild**\
Deletes only the temporary files of the given target. So if Deletes only the temporary files of the given target. So if
you call './convert.sh rebuild world' the script will delete you call ```./convert.sh rebuild world``` the script will delete
all corresponding temp-files of the target world, which will all corresponding temp-files of the target world, which will
trigger a complete regeneration of the relevant (instead of trigger a complete regeneration of the relevant (instead of
skipping existing files) skipping existing files)
check **check**\
Creates mosaics of the tiles, so you can look at them and see Creates mosaics of the tiles, so you can look at them and see
if all went well. if all went well.

752
Nasa2FGearthview/convert.sh
View file

File diff suppressed because it is too large Load diff