                            CubeMap v1.16
                             Quick Guide

(C) 2011-2020, Vladimir Romanyuk, admin@spaceengine.org

================================================================================
About
================================================================================

CubeMap is a software package to convert the texture of a cylindrical
projection to quadrilateralized spherical cube projection (for short cubemap
projection). It is support input textures of any resolution, any number of
channels per pixel, 8-bit or 16-bits signed or unsigned integers. The format of
the input texture is uncompressed raw data. The format of the output texture is
raw and / or array of tiles of different levels of detail in formats raw, tga,
dds, jpg, tif or png.

File list:

CubeMap.exe         - the tool to convert the texture in a cylindrical
                      projection into a cubemap projection.
Glue.exe            - the tool for bonding tiles to a single raw file.
RawConw.exe         - the tool for preprocessing of raw textures.
default_cub.ccf     - example configuration file for CubeMap.exe.
default_glue.gcf    - example configuration file for Glue.exe.
readme_eng_1.16.txt - this file.

================================================================================
Configuration file and command line
================================================================================

All program parameters are specified in the file default_cub.ccf in the program
folder. It is a simple text file containing a set of strings of the form

var value # comment

'var' is a parameter name, 'value' is a parameter's value, # char separates
comments. Comments may be either at the beginning of the line, is this case the
whole line is comment, either at end of the line after the 'value'. Parameter
value is available in four forms:

string - character string without quotation marks and spaces;
int - integer;
float - a real number;
bool - the binary value: true or 1 for true, false or 0 for false.

All utilities can work with any other configuration file, if you pass
it as the first command line parameter:

CubeMap   myconfig.ccf
Glue      myconfig.gcf

In this case, the file default_cub.ccf (or default_glue.gcf for Glue) is
ignored. It is recommended to associate system files with the extension *.ccf
with the program CubeMap.exe, then click on *.ccf file will automatically start
CubeMap with parameters from this file. Files *.ccf can be considered as scripts
for texture processing with the CubeMap. Extension .ccf can be replaced by any
other, so you can associate each tool with its own file type.

At the command prompt, you can specify the path to the input file and output
folder, then appropriate options from the config file are ignored:

CubeMap   [myconfig.ccf] [-i InputFile] [-o OutFolder] [-h]
Glue      [myconfig.gcf] [-i InputFolder] [-o OutFile] [-h]
RawConw   [options] [-i InputFile] [-o OutFile] [-a AlphaFile]

Option -h displays help on the command line usage.


================================================================================
Files organization
================================================================================

Path to the initial cylindrical texture is specified by InputFile, and not
contain spaces. Path to the output folder for the cubemap texture is given by
OutFolder, and also does not contain spaces. Folder must exist. There are
folders named neg_x, neg_y, neg_z, pos_x, pos_y, and pos_z will automatically
created inside it.

The original cylindrical texture should be in uncompressed raw format. Raw
format is simple two-dimensional array of pixels, whose format is described by
the following config options:

InputWidth     - width of input image
InputHeight    - height of the input image
InputChannels  - number of channels (Grayscale - 1, RGB - 3, RGBA - 4, etc.)
Input16bit     - capacity: 16 or 8 bits per channel
InputByteSwap  - for 16-bit: little endian (MAC format)
InputUnsigned  - for 16-bit: unsigned or signed values
InputLatOffset - the global shift in longitude in degrees

The first two parameters are integers, the rest are boolean.

Cubic texture is a set of files and folders organized
as follows:

cubemap ----------- folder that points in the parameter OutFolder
        neg_x ----- folder containing the tiles for neg_x face of cube
              0_0_0.jpg ----- 1 tile of level 0
              1_0_0.jpg - \
              1_0_1.jpg --+-- 4 tile of Level 1
              1_1_0.jpg - |
              1_1_1.jpg - /
              2_0_0.jpg - \
              2_0_1.jpg - |
              2_0_2.jpg - |
              2_0_3.jpg --+-- 16 tiles of level 2
              2_1_0.jpg - |
              2_1_1.jpg - |
              2_1_2.jpg - |
              2_1_3.jpg - |
              .........
              level0.raw - \
              level1.raw - + - temporary files
              level2.raw - |
              .........
        neg_y ----- folder containing the tiles for neg_x face of cube
        neg_z ----- folder containing the tiles for neg_z face of cube
        pos_x ----- folder containing the tiles for pos_x face of cube
        pos_y ----- folder containing the tiles for pos_y face of cube
        pos_z ----- folder containing the tiles for pos_z face of cube

Cubic texture consists of 6 faces, names given to them in accordance with axes
of coordinate system, which they cross. Let the origin is in the center of the
planet. Then the Y axis passes through the poles of the planet, is direction
from the bottom to the up, i.e. the North Pole is at pos_y face, South Pole is
at neg_y face. The X axis passes through the left to the right, i.e. the "left"
side of globe is at neg_x face, "right" side is at pos_x face. The Z axis
passes through the observer to forward, i.e. farthest side of globe is at neg_z
face, nearest side is at pos_z face. At the initial cylindrical texture, this
corresponds to: the upper boundary is the North Pole, the lower boundary is the
South Pole, the left and right boundaries are runs through the neg_x face
vertically, the central meridian is runs through the pos_x face vertically.
The scan of the cubemap faces is standard:

         +-------+
         |       |
         | pos_y |
         |       |
         +-------+
+-------++-------++-------++-------+
|       ||       ||       ||       |
| neg_x || pos_z || pos_x || neg_z |
|       ||       ||       ||       |
+-------++-------++-------++-------+
         +-------+
         |       |
         | neg_y |
         |       |
         +-------+

Resolution of the face is calculated as a quarter of the width of the original
texture, rounded to the nearest larger power of 2. I.e., for example, if the
resolution of the original texture is 86400 x 43200, the face resolution will
be 32768 x 32768 (32768 = 131072 / 4).

Each face is organized as a set of tiles. Resolution of the tile is determined
by tile TileWidth parameter and must be no larger than the resolution of the
face. For example, let the TileWidth is 256, and the resolution of the face is
32768. Then there will be 8 levels of tiles: Log2(32768 / 256) + 1 = 8. Levels
are numbered from 0 to 7, the number of level is the first number in the tile
file name tile (0_0_0.jpg, ..., 7_127_127.jpg). The resolution of each next
level has consistently increased by the factor of 2, while the number of tiles
by the factor of 4. Thus, we have one tile of 0 level (0_0_0.jpg) with
resolution of 256 x 256, containing the image of the entire face; 4 tiles of
level 1 (1_0_0.jpg ... 1_1_1.jpg), each containing a quarter of the image of
the entire face, with total resolution of 512 x 512, thus the resolution of
each tile is again 256 x 256; 16 tiles of level 2, etc. For the most detailed
level 7 it will be 128*128 = 16384 tiles, with total resolution of 32768x32768.
The second and third numbers in the tile's filename are vertical index (v) and
horizontal index (u) respectively. The indexes are increasing from left to
right and top to down. There is an example of the first three levels
organization:

level0
+-------+
|       |
| 0_0_0 |
|       |
+-------+

level1
+-------++-------+
|       ||       |
| 1_0_0 || 1_0_1 |
|       ||       |
+-------++-------+
+-------++-------+
|       ||       |
| 1_1_0 || 1_1_1 |
|       ||       |
+-------++-------+

level2
+-------++-------++-------++-------+
|       ||       ||       ||       |
| 2_0_0 || 2_0_1 || 2_0_2 || 2_0_3 |
|       ||       ||       ||       |
+-------++-------++-------++-------+
+-------++-------++-------++-------+
|       ||       ||       ||       |
| 2_1_0 || 2_1_1 || 2_1_2 || 2_1_3 |
|       ||       ||       ||       |
+-------++-------++-------++-------+
+-------++-------++-------++-------+
|       ||       ||       ||       |
| 2_2_0 || 2_2_1 || 2_2_2 || 2_2_3 |
|       ||       ||       ||       |
+-------++-------++-------++-------+
+-------++-------++-------++-------+
|       ||       ||       ||       |
| 2_3_0 || 2_3_1 || 2_3_2 || 2_3_3 |
|       ||       ||       ||       |
+-------++-------++-------++-------+


================================================================================
The CubeMap tool
================================================================================

Converting a cylindrical texture to a cubemap is performed in several stages.
There is possibility to transform only certain faces. This controlled by the
following parameters in the config file (setting the parameter to 0 or false
disables all convertion phases for the corresponding face):

Create_NEG_X 1
Create_POS_X 1
Create_NEG_Y 0 # this face will be skipped
Create_POS_Y 0 # this face will be skipped
Create_NEG_Z 1
Create_POS_Z 1

Execution of each stage can be disabled by setting the corresponding parameter
in the config to 0 or false (see below).

CreateBaseTex 1  # create a basic texture
ComputeFaces  1  # convert the selected faces
DownsizeFaces 1  # get the LOD-s selected faces
TileFaces     1  # get the tiles from the time raw-files
OptimizeTiles 0  # remove flat/uniform tiles from the chain

Consider all the steps in order.

1) Create a basic cylindrical texture.

Set the parameter CreateBaseTex to 1 to perform this operation.

SpaceEngine program uses the small cylindrical textures called base textures
for planets that are far from the camera. Creating a base texture implemented
by simple reduction of the original cylindrical texture by averaging pixels in
square blocks. Resolution of the base texture is the same as the resolution of
tiles (TileWidth). The BaseTexFormat parameter specifies the format of the base
texture.

2) Creating of a cubemap texture faces from the cylindrical texture.

Set the parameter ComputeFaces to 1 to perform this operation.

The transformation is performed by bilinear filtering: to get the color the
current pixel on the face of the cube, there are calculated real coordinates of
the corresponding point on the cylindrical texture, and color is obtained by
linear interpolation of the four pixels, closest to this point. This method
gives very good result, artifacts may be noticeable only very close to the
poles.

The program uses a cache memory for storing parts of the original cylindrical
texture. It is therefore possible to perform complete transformation of
original texture without loading it into memory. This is great for large
textures, such as NASA Earth texture "Blue Marble Next Generation", witch has
a size of 86400 x 43200 x 3 = 10.4 GiB. If system has enough free RAM at the
moment of the Cubemap start to load the source texture, it will do that.
Otherwise the cache will be used. Cubemap will allocate as much RAM for the
cache as possible, or amount, specified by MaxMem parameter in the config, if
free memory is too small. In case of using cache, processing speed will be
slower, because Cubemap will load chunks of the source file multiple times.

A temporary subfolder called raw will be created in the temporary folder
specified by the TempFolder parameter. Inside it, subfolders neg_x, neg_y,
neg_z, pos_x, pos_y, pos_z will be created, where raw-files of the
corresponding cubemap faces will be saved. In our example, each face raw-file
will have a name of level6.raw (because we will have 7 levels, starting from 0)
with resolution of 32768 o 32768.

If the TileBorder parameter is set to 2, a border 2 pixel-wide is added to the
perimeter of each tile. This border contains pixels from adjacent tiles. The
resolution of tiles remains the same, i.e. border pixels are added from inside,
with corresponding scaling of the central part of the tile texture. The
temporary file level*.raw will have the same resolution as well, in our
example, it will be 32768 x 32768.

SpaceEngine requires a border width of 2 pixels to eliminate gaps in the
terrain geometry and textures. It is important to save elevation map tiles to
the lossless format, the best one is 16-bit png.

Normalization for elevation maps (InputChannels 1) can be used to better fit
into the full dynamic range of the texture format. For example, for a 16-bit
height map gebco_bathy.21601x10801.bin, the minimum and maximum pixel values
are -10577 and 8430, that does not completely fits the dynamic range of 16-bit
integer (-32768 ... 32767). Setting Normalize parameter to true forces
re-calculation of heights to fits this dynamic range. Moreover, if the
parameter EnterNormData is set to false, the minimum and maximum values are
computed by reading all the pixels in the input texture. It can take a while
for large textures, so it is optimal to perform computing of the min/max values
once, and then set EnterNormData to true and enter the data from previous pass:

Normalize       true   # normalize the output image (height map)
EnterNormData   true   # to enter data manually for the normalization
NormMinValue    -10577 # the minimum value
NormMaxValue    8430   # the maximum value
NormIgnoreValue -32768 # skip this value while reading normalization data

Some maps uses special value to mark "no data" pixels. This may break out
reading of normalization data. For example, if your map is 16-bit signed
integer, you may find what normalization data reading stage reports
NormMinValue of -32768, while you know what real heights on this map starts
from something like -10000. In this case, set the parameter NormIgnoreValue
to -32768. This will tell the program to ignore this value during normalization
data reading stage.

3) Creating LODs down to zero.

Set the parameter DownsizeFaces to 1 to perform this operation.

The temporary file resolution reduces twice while tile resolution reached. In
our example, there are new temporary file appears in the corresponding face
folder: level6.raw, level5.raw, level4.raw, level3.raw, level2.raw, level1.raw
and level0.raw, with resolution from 16384 x 16384 to 256 x 256 respectively.

4) Creation of the tiles from the temporary raw files.

Set the parameter TileFaces to 1 to perform this operation.

Temporary files are cut to the appropriate number of tiles with resolution of
TileWidth x TileWidth and are saved to the folder named after face, in one of
the following formats::

raw - 8 or 16 bits, any number of channels, uncompressed
tga - 8 bits, 1, 3, 4-channel, uncompressed
dds - 8 bits, 1, 3, 4-channel, uncompressed
jpg - 8 bits, 1, 3, channel, lossy compression
tif - 8 or 16 bit, 1, 3, 4 channels, lossy compression
png - 8 or 16 bit, 1, 3, 4 channels, lossless compression

Tiles format is controlled by the following parameters:

OutFormat       - the tile format (raw, tga, dds, jpg, tif, png)
OutFormat2      - the tile format (raw, tga, dds, jpg, tif, png) for the
                  low-byte, or for the alpha-channel (see SeparateHiLo and
                  SeparateRGBA)
TileWidth       - the tile resolution, default for SE: 256
TileBorder      - the width of a tile border, default for SE: 2
Out16bit        - capacity: 16 or 8 bits per channel (only for raw and png)
OutByteSwap     - to 16-bit: little endian (MAC format)
OutUnsigned     - to 16-bit: unsigned or signed values
OutInvertAlpha  - invert the alpha channel pf RGBA-image
OutJPEGquality  - the quality of JPEG compression (0 ... 100)
MaxOversampling - don't save tiles of the last level it its oversampling rate
                  is greater than this value

The MaxOversampling parameter is used to avoid generating of tiles with
excessive resolution. This saves conversion time and disk space. For example, if
the original texture had a resolution of 36000x18000, it will be effectively
upscaled to the nearest power of two upwards, i.e. 65536х32768. This will make
the last level look very blurry; its oversampling rate will be 65536/36000=1,82.
If it is higher than the MaxOversampling value, raw files and tiles of this
level will not be created (although internally CubeMap will work in the
resolution of 65536).

Another possible scenario for using this option is converting a texture with
exact power-of-two dimensions (for example 32768x16384) and using a non-zero
TileBorder. In this case, CubeMap will slightly stretch the original texture to
a resolution of 32768 + 4 * (32768/256) = 33280 to take into account the slight
compression of the tile area visible in SpaceEngine (256 - 2 * 2 = 252 pixels).
Therefore, the working resolution will be 65536 (the nearest greater power-of-
two), and the oversampling value will be 65536/33280 = 1,969. Without the use of
MaxOversampling, these would create the last level of tiles that would look very
blurry.

To disable this limiting and save tiles of all levels, simply specify
MaxOversampling 2.

It is necessary to dwell on the parameters OutFormat2, SeparateRGBA and
SeparateHiLo. SpaceEngine supports a special way to store RGBA texture of the
planetary surface in the form of two separate texture files - a color files for
the RGB components and a black-and-white files for the alpha channel. This can
be used to save disk space. For example, the Earth surface requires RGBA
textures with color specified in the RGB channels, and a water mask specified in
the alpha channel. The compressed texture formats which support 4 channels
(RGBA) are only png and tif. But files of these formats are quite large, while
the color information in could be compressed in jpg with a sufficiently good
quality, and the water mask could be compressed in the lossless png very well.
Such a combined set of two textures jpg + png takes much less space than a
single 4-channel png. To use this method, set the SeparateRGBA true, and specify
the format for the RGB texture by the OutFormat, and for the alpha texture by
the OutFormat2:

OutFormat    jpg
OutFormat2   png
SeparateRGBA true

With this configuration, the CubeMap will save two files for each tile: the RGB
file with the suffix _c and the alpha file with the suffix _a, for example:

3_2_16_c.jpg
3_2_16_a.png

For the 16-bit elevation maps, it is also can be used the separation of the
16-bit tile image into two files: a low-byte and a high-byte. Sometimes saving
them separately into a 8-bit files gives an advantage in size compared to saving
in a single 16-bit image (png or tif). To do this, set the SeparateHiLo true,
and specify the format for the high-byte texture by the OutFormat, and for the
low-byte texture by the OutFormat2:

OutFormat    png
OutFormat2   jpg
SeparateHiLo true

With this configuration, the CubeMap will save two files for each tile: the
high-bytes file with the suffix _h and the low-bytes file with the suffix _l,
for example:

3_2_16_h.png
3_2_16_l.jpg

Note: the high-bytes must be saved in a lossless format, such as png. Otherwise,
the terrain will have large artifacts! The lower-bytes may be saved in a format
with lossy compression, such as jpg, but it also leads to artifacts. Only the
combination of png + png does not gives the artifacts, and sometimes it makes an
advantage in a file size; in this case when this method is recommended to use.

SpaceEngine automatically detects and loads the dual-file textures, no
additional tweaking of the planet script is required.

WARNING: if tile files are already exist in the folder, they WILL BE
overwritten.

5) Optimize tiles

Set the parameter OptimizeTiles to 1 to perform this operation.

The SpaceEngine program supports the tiled cubemap texture of any level of
detail in any place of planet. This means that you can have for example the
Earth texture of resolution 6 x 32768 x 32768 (obtained from the NASA "Blue
Marble Next Generation" texture), but in some places, like in the cities, the
level of detail can be increased by adding tiles of 7, 8 etc. levels, or vice
versa, in the vast homogeneous areas with no details - the oceans - you can
remove the tiles up to 2-3 level, thus saving disk space and increase loading
speed. The optimizer tries to identify these homogeneous regions and remove
them.

Determination of homogeneity is performed by calculating the variation of color
of the pixels of the tile - calculating a weighted sum of absolute difference
of current pixel color and average color of the tile. If this variation is less
than the value of MinVar parameter from the config, the tile is considered to
be homogeneous and moved to a temporary folder.

The temporary folders are created in a folder OutFolder and have names of
del_neg_x, del_neg_y, del_neg_z, del_pos_x, del_pos_y, and del_pos_z. There are
homogeneous tiles moved from the folders of neg_x, neg_y, neg_z, pos_x, pos_y
and pos_z respectively. Temporary folder is a candidate for deletion.

A general guidelines for working with the utility are the next. You start with
some MinVar value and run optimizer for one face, selecting by the parameters
Create_NEG_X ... Create_POS_Z. After that, open the temporary folders and look
on the tiles were in, then correct MinVar, move the tiles back (manually) and
repeat this process several tiles until a satisfactory result. Then run
optimizer for other faces with optimal MinVar value you have found.

The MinOptLevel specifies from which level the optimization process should
start. SpaceEngine requires at least 2-3 levels of textures even in flat areas
like oceans to maintain sphere tessellation at a good level to prevent visual
artifacts.

The current version (1.16) does not use a very good algorithm, so there may be
mistakes - such as tile with a small island and the ocean around it uniformly
may reckoned entirely homogeneous, while the tile with a slight color gradient
may reckoned inhomogeneous, although there is should be the vise versa.
Therefore, I suggest you view all the tiles in the temporary folder and move
back those who thought not fit to remove. Also, for large texture is
recommended to save temporary folders somewhere, so you can quickly restore
incorrectly deleted tiles without starting CubeMap tool again.

Optimization is not supported for the dual-file tiles (SeparateRGBA or
SeparateHiLo).


================================================================================
The Glue tool
================================================================================

The Glue tool is designed for merging tiles back to a single raw file. It is
good idea to use a separate config file (*-glue.gcf) with the following
parameters:

InputFolder - the folder in which the tiles are searched
OutFile     - output image file
TilePrefix  - string, the prefix before the name of tile (level number)
TileExt     - string, file name extension
StartU      - int, the starting U (horizontal) tile index
StartV      - int, the starting V (vertical)   tile index
EndU        - int, the ending   U (horizontal) tile index
EndV        - int, the ending   V (vertical)   tile index
SwapUV      - boolean, swap U and V indices
TileWidth   - width  of the tile in pixels
TileHeight  - height of the tile in pixels (Glue can process non-square tiles)
TileBorderWidth  - width  of border in pixels, which Glue will skip
TileBorderHeight - height of border in pixels, which Glue will skip
ChannelsPerPixel - number of channels of the tiles (Grayscale - 1, RGB - 3 etc)
BytesPerChannel  - number of bytes per channel

The tile's file name is constructed as follows: TilePrefix_v_u.TileExt, for
example, if TilePrefix is 3_, StartU is 2, StartV is 1, and TileExt is jpg,
then the tiles will have names 3_1_2.jpg, 3_1_3.jpg, etc. Extension .*
indicates that the program will try to load any of the supported formats, in
the following priority order:
.dds, .jpg, .jpeg, .tif, .tiff, .png, .tga, .raw
I.e., for example, if a folder has two files 3_1_2.jpg and 3_1_2.tga, it will
load the first one.

Tiles in RAW format can have any number of channels (ChannelsPerPixel) and any
number of bytes per channel (BytesPerChannel). For example, 16-bit RGBA image
should have ChannelsPerPixel 4 and BytesPerChannel 2.

Glue can skip borders on sides of the tiles (useful for merging back tile of
some maps, processed by Cubemap before). The parameter TileBorderWidth defines
width of the border, which will be skipped on both left and right sides of the
tile; TileBorderHeight do the same for top and bottom sides.


================================================================================
The RawConv tool
================================================================================

The RawConv tool is a simple utility useful for preprocessing of the raw files.
Usage:

RawConv [options] -i input_file -o out_file [-a alpha_file]

Options determines what operation are performed over the input_file:

-t: insert 8-bit grayscale image alpha_file into 8-bit RGB image input_file as
    an alpha channel.
-s: convert signed 16-bit image to unsigned 16-bit image or back.
-b: swap byte order of 16-bit image (convert between Mac and Windows format).

The result is saved to out_file. All input and output files must be in raw
format. Options -s and -b can be used together; their order defines order of
operations performed on every 16-bit word of the input file.


================================================================================
General Recommendations
================================================================================

1) Placing the input and output textures on different physical disks can
significantly increase performance, especially is using SSD drives. Performance
also increases if the input files is de-fragmented.

2) Write information about the format of the input raw-textures in its
filename, for example:
Earth-surface-32k-RGBA.raw - RGBA texture with 32768 x 16384 resolution;
Mars-bump-16k-16bit.raw - 16-bit grayscale texture with 16384 x 8192 resolution.
This will help to avoid confusion.

3) Convert the Texture of any format to raw can be done in Photoshop, IrfanView
and other programs. There also you can view the temporary raw-files.

4) Save the config files for each of the transformed texture, they might be
useful in the future. Associate files with the extension .ccf with program
CubeMap.exe, then double click on the .ccf file will automatically start the
CubeMap tool. You can also use drag and drop the .ccf file on the shortcut of
the CubeMap or Glue executable to run it.
an always manually remove them.

5) It is recommended to view the result of conversion of the polar faces pos_y
and neg_y, and, if there are noticeable artifacts present, edit the raw-files
of the polar faces in the graphics editor. To do this, run the ComputeFaces,
edit the raw-files, disable ComputeFaces and perform the rest operations.

6) In order to avoid the loss of precision it is not recommended to normalize
the height map, the dynamic range of which is "almost implemented", for example
(1 ... 252) for the 8-bit and (-32000 ... 32000) for the 16-bit.

7) If you want to save the tiles into a format not supported by the Cubemap, it
is necessary to perform the conversion into a lossless format such as tga, and
then perform a batch conversion of all tiles into the desired format using
third-party software such as IrfanView or ImageMagic.
