Batched texture conversion to ktx ?

Improving the artwork in Warzone2100 - not for mod discussions
Post Reply
Vincent
Trained
Trained
Posts: 103
Joined: 06 Aug 2016, 17:24

Batched texture conversion to ktx ?

Post by Vincent »

Hi,

in order to support mipmapping in Vulkan and use some advanced textures compression scheme (BC6/7) or layout (texture array or cube texture) Warzone 2100 textures will need to be converted to a more 3d graphic friendly format before being loaded by the game.
Currently all textures are stored in png format which is one of the best format for edition purpose : while compressed it's loseless and can hold an alpha channel. However this isn't a great format when the texture are actually used by wz2100 :
first the png format is not directly useable for rendering purpose which means wz2100 has to uncompress the file before actually using the content. This isn't that expensive but can takes several milisecond per file which add up when loading a level.
Second the texture is recompressed by the driver to a gpu friendly format after being loaded. Unfortunatly this compression is lossy ; several compression format are provided with different compromise in minds for different usage, see here : http://www.reedbeta.com/blog/understand ... n-formats/ (especially the comparison table) Currently there is little control over the compression scheme to use, as far as I know all textures use the S3TC scheme. And while uncompressing a texture is fast and always yield the same result, compression is slow and depends of the underlying algorithm even if the format is fixed. Drivers may choose different speed over quality ratio and thus textures may vary between drivers. And despites likely favoring compression speed the more advanced BC6 and BC7 format requires several minute to compress a single texture which means this format can't be used at all despite being the most Advanced one.
Third png only store a single texture. Mipmapping need to be regenerated on the fly by driver. This is usually fine since mipmap levels are expected to follow a standard filtering reduction (block filtering) so there isn't variance between drivers. Unfortunatly Vulkan dropped the functionnality which means that either a shader or some c++ code need to compute the various mipmap level.

The solution here is to use a format that stores texture in raw format that can be loaded directly into gpu memory. I know 2 formats for this, one is DDS and the other one that was suggested by Per (viewtopic.php?f=32&t=12594#p135098) is KTX (https://github.com/KhronosGroup/KTX) which is likely easier to use in a cross Platform way.
Both formats consist of a small header that describes texture dimension (width, height, depth/array layers), mipmap count and format, and the raw texture content.

I think it would be helpful to have a script that will execute on all .png files in data/texpage a command that generates mipmaps and saves the format into a compressed format that is widely supported by hardware (like S3TC). This script would be part of the compile process.
Ideally there would be some way (separate xml file ? name suffix convention ?) to tell that some texture is a normal map one or a diffuse one so that the script would pick an appropriated compressed format, and that script could be parametrized with some target flag so that for instance it could use the ASTC format when build for mobile.

The libktx provides a command line utility that does the mipmaping and compression that could be used.
Per
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 3780
Joined: 03 Aug 2006, 19:39

Re: Batched texture conversion to ktx ?

Post by Per »

That sounds like a good plan to me.
Jorzi
Regular
Regular
Posts: 2063
Joined: 11 Apr 2010, 00:14

Re: Batched texture conversion to ktx ?

Post by Jorzi »

How would this affect the file size of my mod? (it's already over 40MB) Isn't the most time consuming part to actually read the files from disk? Also, at what stage will this be done, since I like to be able to have a rapid feedback loop between warzone and my image editor.
However, different compression types for different maps is always good.
ImageImage
-insert deep philosophical statement here-
Post Reply