Vulkan dependencies

[Vincent]

Hi,
I mostly finished rewriting the vulkan code on top of the wrap_gl branch (it works on Intel but not on amd/nvidia due to rgb format support missing on the latters) ; it makes vulkan implementation less invasive and doesn't touch rendering algorithm.
There is however an issue : mipmaps need to be generated outside of Vulkan. It's possible to write a loop averaging texels for power of 2 textures but it's tedious and non power of 2 textures may require some hack. The "correct" solution is to provide textures as dds or ktx file which are composed of a binary header and raw Texel data. This means generating such texture with a third party tool like nvidia-texture-Tools as part of build process and then loading them with a lib (like gli which is a header only lib).

I'm planning to integrate Vulkan into warzone 2100 in 2 steps : the first one is bringing a working Vulkan backend which lacks texture mipmap levels and hence only support bilinear and not trilinear filter. This would ensure that it's properly working for various hardware. This part is the one which is almost finished. It requires 3 new dependencies : Vulkan obviously, Vulkan hpp (a C++ wrapper that reduces vulkan verbosity) and glslc. Vulkan is already provided in Ubuntu as far as I can tell, but I don't think MXE has it. I don't know if Vulkan hpp is but it's only an header so it may be possible to add it in 3rdparty/ directory. Glslc (or shaderc) is a glsl to spir-v compiler. It's a wrapper on top of glslang the official compiler from Khronos and adds some useful features like preprocessor directive (read: #include). As far as I know it's not in Ubuntu at the moment.

[pastdue]

Hi Vincent,

Before you continue with all of this work, I wanted to raise the subject of alternatives.

LibANGLE, a BSD-licensed library which is used by a number of games as well as in Firefox and Chrome (and even Qt), provides an OpenGL ES 2.0 API layer with OpenGL, Vulkan, and Direct3D backends.

Utilizing LibANGLE would:

• Require minimal modification to the existing code. (Essentially all of the OpenGL APIs that are currently used in Warzone 2100 are supported in OpenGL ES 2.0, and do the same things. Just a handful of lines of code would require changing or using slightly renamed functions or extensions.)
• Work today with SDL, which has support for LibANGLE (with a small amount of additional code required for Mac support).
• Provide native Direct3D support on Windows. (Resolving a large percentage of outstanding issues and crashes running Warzone on Windows caused by buggy OpenGL drivers and graphics hardware.)
• Provide the continued benefit of all of the workarounds and fixes that the LibANGLE project includes for graphics driver bugs, hardware bugs, operating system bugs, and more.

LibANGLE would enable a single, unified graphics codebase in Warzone, that is compilable on every platform, uses the preferred graphics API for that platform (Windows, Mac, Linux, etc), and supports all graphics hardware. And, thus, only *one* backend in Warzone that needs to be maintained. (With the benefit of constant improvements provided by the LibANGLE project.)

[Per]

MXE isn't a huge problem - we could just choose to not build the Vulkan backend for Windows initially. (Or we could circumvent it by runtime linking, but this is somewhat icky.)

I assume we make the SPIRV compiler a build dependency, not a runtime dependency? That is, we generate the SPIRV during the build process, and then store the binaries. We can git submodule the compiler into the thirdparty directory.

Textures - yes, the correct way is to generate mipmaps beforehand - both for GL and Vulkan, actually. Either permanently or during the build step, not sure which is better. There are some nice libs we could use - again, not sure which is better.

ANGLE - I'm not sure that it is a viable alternative, mostly because GLES 2 is a rather old and anemic API.

[pastdue]

ANGLE - I'm not sure that it is a viable alternative, mostly because GLES 2 is a rather old and anemic API.

Having been playing around with a branch already that uses it, the subset of the OpenGL API that Warzone currently uses is 99% supported by OpenGL ES 2.0. In addition, ANGLE supports OpenGL ES 3.0/3.1, which is comparable to OpenGL 4.3+, so this opens up even more modern OpenGL API usage in the future (on all platforms, without having to worry about the platform / drivers explicitly supporting OpenGL 4.3+). But as far as things are now, using OpenGL ES 3.0 APIs isn't even necessary to get things working with ANGLE.

As a general note, Vulkan is unavailable on Mac, and is unlikely to be supported by Apple.

[Per]

I'd be happy with both ANGLE and Vulkan backends. ANGLE still lists GLES3 support as "in progress" and the list of extensions looks short - but less short than last time I looked, and possibly has what we need now (haven't had time to look very closely).

[moltengear]

Hi all. I think it would be better if we improve the work with OpenGL.
OpenGL supports the DDS format.

GL_COMPRESSED_RGBA_S3TC_DXT1_EXT
GL_COMPRESSED_RGBA_S3TC_DXT3_EXT
GL_COMPRESSED_RGBA_S3TC_DXT5_EXT
FPS +20%

It is also possible to use shaders instead of glm.
Glm is not needed. This is the load on the CPU.

FPS +

OpenGL ES!
Netbooks don't pull this game. Android devices also can't.

[Per]

Hi all. I think it would be better if we improve the work with OpenGL.
OpenGL supports the DDS format.
...
It is also possible to use shaders instead of glm.
Glm is not needed. This is the load on the CPU.

DDS is a file format, has nothing really to do with OpenGL. DDS supports multiple texture formats, as does OpenGL. Running every calculation on the GPU would be insane - so, yes, something like GLM is needed.

[Vincent]

GLM is a lib for matrix computation. It doesn't have anything to do with OpenGL, it can be used with direct3d.

Others projects had lot of trouble getting ANGLE working, see godot for instance : https://github.com/godotengine/godot/issues/3634. It's also slow since it has to translate GL calls to D3D ones ; and ANGLE on top of Vulkan removes the purpose of Vulkan since stripped driver logic will go into an all purporse wrapper Library instead of the engine logic. Plus Angle would forbid any multithreaded rendering attempt.

[pastdue]

Others projects had lot of trouble getting ANGLE working, see godot for instance : https://github.com/godotengine/godot/issues/3634.

A lot of projects / games / game engines that weren't using it, now are - including Godot.

It's also slow since it has to translate GL calls to D3D ones ; and ANGLE on top of Vulkan removes the purpose of Vulkan since stripped driver logic will go into an all purporse wrapper Library instead of the engine logic

Not sure where you are getting the "slow". ANGLE is high-performance and is used in game engines, games, and widely used as part of Google Chrome (which uses ANGLE for all graphics rendering on Windows, including the accelerated Canvas2D implementation and WebGL).

There are a number of steps that a solution like ANGLE can take to convert from a certain API interface (that just happens to be OpenGL ES) to a Direct3D backend with high-performance. ANGLE uses them.

Plus Angle would forbid any multithreaded rendering attempt.

This has zero to do with ANGLE, and everything to do with Warzone and how graphics APIs work in general. As I mentioned in the other thread, there are already blockers preventing this from safely being implemented arbitrarily in the Warzone codebase, and the workarounds to those problems don't care whether the underlying graphics backend is being provided by OpenGL, ANGLE, etc.

More importantly, as I mentioned in the other thread, there is very little benefit to multithreading most of the Warzone codebase (in fact, you may make things slower). I'm working on a patch that permits rendering with display scaling on high-DPI displays, and running at 250% on a 2x high-DPI display at a gigantic window resolution, which is pushing dramatically more pixels and data and rendering, the game is barely using 30% of one CPU core at 60fps (with vsync enabled). This is on a 2012 laptop CPU / GPU.

Now, said patch *does* dramatically improve the text rendering subsystem. But it does so because I profiled Warzone, and discovered that was a key bottleneck slowing everything else down. The fix wasn't multi-threading.

[vexed]

Never used ANGLE personally, so, no idea what is needed for the code translation or what kind of a performance hit there would be.

My only concern would be maintenance of said changes, and how exactly can you add new features, if ANGLE doesn't support it yet?

I know Apple don't support Vulkan, since they want their only proprietary API (Metal), so, we would be using multiple rendering paths anyway, or use the MoltenVK wrapper?

Looking at https://github.com/google/angle it seems Vulkan is only "in progress" for Windows, nothing else, but planned for linux, but not Macs.
For Mac support, I only see "Desktop GL" as 'in progress', nothing else is planned, so, we would still be using multiple rendering paths even if we did switch to ANGLE.
So, the only thing ANGLE will help in, is for those crappy intel systems with broken openGL drivers?

If that is the case, then, I guess the best course of action would be to use API agnostic function calls, and within that, based on what is being compiled call the correct API routine?

Something like this crappy pseudo code

Code: Select all

WZ_DrawCube(...)
{
#if HAS_VULKAN
    #if not_a_mac
     (call Vulkan routines)
     #else
     (call MoltenVK routines [no idea how this wrapper works...])
     #endif
#else
  (call openGL rotuines)
#endif 
}

That reminds me, SDL was just updated to support Vulkan as well, (not sure what it does on the Mac side though).

Sorry for the rambling, and since I am traveling, don't expect a timely reply.

[Vincent]

Unfortunatly MoltenVK is non free as far as I know. The best solution is to implement a Metal rendering path imo since GL (and hence Angle) on OSX forbid compute shader usage.

[moltengear]

libsquish
https://github.com/Cavewhere/squish

Don't need to do a change of texture files.
The program automatically converts files at startup.
If there is support from the video cards.

Support.jpg (54.6 KiB) Viewed 3304 times

[Vincent]

nvidia texture tools is using libsquish for BC1 up to BC3. Basically I'd like to provide offline compressed textures ; this way more format can be supported (bc 6 and bc7 which can take several min per texture to compress) and ASTC format for soc can be added with a few line change in the CMakeLists.txt.
Textures will always be avaible on the repo as png file, they are compressed during the compilation stage.

[Berg]

What ever you do don’t go crushing the textures they have already been messed with over the years and they are not good now.

[moltengear]

I was able to load dxt textures! Can't render in dxt texture. It is necessary to divide the load of different types of textures.
The texture of the sky has alpha transparency. Therefore, it is impossible to apply dxt1 compression.
I use dxt5 compression for everything.

1) It is also necessary that the width and height of the textures be a multiple of four.

How DXT works.gif (41.05 KiB) Viewed 3076 times

2) square textures are needed for mobile processors.