Vulkan dependencies

For code related discussions and questions

Vulkan dependencies

Postby Vincent » 05 Sep 2017, 22:23

Hi,
I mostly finished rewriting the vulkan code on top of the wrap_gl branch (it works on Intel but not on amd/nvidia due to rgb format support missing on the latters) ; it makes vulkan implementation less invasive and doesn't touch rendering algorithm.
There is however an issue : mipmaps need to be generated outside of Vulkan. It's possible to write a loop averaging texels for power of 2 textures but it's tedious and non power of 2 textures may require some hack. The "correct" solution is to provide textures as dds or ktx file which are composed of a binary header and raw Texel data. This means generating such texture with a third party tool like nvidia-texture-Tools as part of build process and then loading them with a lib (like gli which is a header only lib).

I'm planning to integrate Vulkan into warzone 2100 in 2 steps : the first one is bringing a working Vulkan backend which lacks texture mipmap levels and hence only support bilinear and not trilinear filter. This would ensure that it's properly working for various hardware. This part is the one which is almost finished. It requires 3 new dependencies : Vulkan obviously, Vulkan hpp (a C++ wrapper that reduces vulkan verbosity) and glslc. Vulkan is already provided in Ubuntu as far as I can tell, but I don't think MXE has it. I don't know if Vulkan hpp is but it's only an header so it may be possible to add it in 3rdparty/ directory. Glslc (or shaderc) is a glsl to spir-v compiler. It's a wrapper on top of glslang the official compiler from Khronos and adds some useful features like preprocessor directive (read: #include). As far as I know it's not in Ubuntu at the moment.
Vincent
Trained
Trained
 
Posts: 75
Joined: 06 Aug 2016, 17:24

Re: Vulkan dependencies

Postby pastdue » 06 Sep 2017, 00:01

Hi Vincent,

Before you continue with all of this work, I wanted to raise the subject of alternatives.

LibANGLE, a BSD-licensed library which is used by a number of games as well as in Firefox and Chrome (and even Qt), provides an OpenGL ES 2.0 API layer with OpenGL, Vulkan, and Direct3D backends.

Utilizing LibANGLE would:
  • Require minimal modification to the existing code. (Essentially all of the OpenGL APIs that are currently used in Warzone 2100 are supported in OpenGL ES 2.0, and do the same things. Just a handful of lines of code would require changing or using slightly renamed functions or extensions.)
  • Work today with SDL, which has support for LibANGLE (with a small amount of additional code required for Mac support).
  • Provide native Direct3D support on Windows. (Resolving a large percentage of outstanding issues and crashes running Warzone on Windows caused by buggy OpenGL drivers and graphics hardware.)
  • Provide the continued benefit of all of the workarounds and fixes that the LibANGLE project includes for graphics driver bugs, hardware bugs, operating system bugs, and more.

LibANGLE would enable a single, unified graphics codebase in Warzone, that is compilable on every platform, uses the preferred graphics API for that platform (Windows, Mac, Linux, etc), and supports all graphics hardware. And, thus, only *one* backend in Warzone that needs to be maintained. (With the benefit of constant improvements provided by the LibANGLE project.)
pastdue
Rookie
Rookie
 
Posts: 16
Joined: 13 Aug 2017, 17:44

Re: Vulkan dependencies

Postby Per » 06 Sep 2017, 08:21

MXE isn't a huge problem - we could just choose to not build the Vulkan backend for Windows initially. (Or we could circumvent it by runtime linking, but this is somewhat icky.)

I assume we make the SPIRV compiler a build dependency, not a runtime dependency? That is, we generate the SPIRV during the build process, and then store the binaries. We can git submodule the compiler into the thirdparty directory.

Textures - yes, the correct way is to generate mipmaps beforehand - both for GL and Vulkan, actually. Either permanently or during the build step, not sure which is better. There are some nice libs we could use - again, not sure which is better.

ANGLE - I'm not sure that it is a viable alternative, mostly because GLES 2 is a rather old and anemic API.
Per
Warzone 2100 Team Member
Warzone 2100 Team Member
 
Posts: 3604
Joined: 03 Aug 2006, 19:39

Re: Vulkan dependencies

Postby pastdue » 06 Sep 2017, 17:57

Per wrote:ANGLE - I'm not sure that it is a viable alternative, mostly because GLES 2 is a rather old and anemic API.


Having been playing around with a branch already that uses it, the subset of the OpenGL API that Warzone currently uses is 99% supported by OpenGL ES 2.0. In addition, ANGLE supports OpenGL ES 3.0/3.1, which is comparable to OpenGL 4.3+, so this opens up even more modern OpenGL API usage in the future (on all platforms, without having to worry about the platform / drivers explicitly supporting OpenGL 4.3+). But as far as things are now, using OpenGL ES 3.0 APIs isn't even necessary to get things working with ANGLE.

As a general note, Vulkan is unavailable on Mac, and is unlikely to be supported by Apple.
pastdue
Rookie
Rookie
 
Posts: 16
Joined: 13 Aug 2017, 17:44

Re: Vulkan dependencies

Postby Per » 06 Sep 2017, 19:50

I'd be happy with both ANGLE and Vulkan backends. ANGLE still lists GLES3 support as "in progress" and the list of extensions looks short - but less short than last time I looked, and possibly has what we need now (haven't had time to look very closely).
Per
Warzone 2100 Team Member
Warzone 2100 Team Member
 
Posts: 3604
Joined: 03 Aug 2006, 19:39

Re: Vulkan dependencies

Postby moltengear » 07 Sep 2017, 15:50

Hi all. I think it would be better if we improve the work with OpenGL.
OpenGL supports the DDS format.

GL_COMPRESSED_RGBA_S3TC_DXT1_EXT
GL_COMPRESSED_RGBA_S3TC_DXT3_EXT
GL_COMPRESSED_RGBA_S3TC_DXT5_EXT
FPS +20%

It is also possible to use shaders instead of glm.
Glm is not needed. This is the load on the CPU.

FPS +

OpenGL ES!
Netbooks don't pull this game. Android devices also can't.
moltengear
Greenhorn
 
Posts: 8
Joined: 22 Jul 2017, 15:05

Re: Vulkan dependencies

Postby Per » 08 Sep 2017, 16:33

moltengear wrote:Hi all. I think it would be better if we improve the work with OpenGL.
OpenGL supports the DDS format.
...
It is also possible to use shaders instead of glm.
Glm is not needed. This is the load on the CPU.

DDS is a file format, has nothing really to do with OpenGL. DDS supports multiple texture formats, as does OpenGL. Running every calculation on the GPU would be insane - so, yes, something like GLM is needed.
Per
Warzone 2100 Team Member
Warzone 2100 Team Member
 
Posts: 3604
Joined: 03 Aug 2006, 19:39

Re: Vulkan dependencies

Postby Vincent » 09 Sep 2017, 17:40

GLM is a lib for matrix computation. It doesn't have anything to do with OpenGL, it can be used with direct3d.

Others projects had lot of trouble getting ANGLE working, see godot for instance : https://github.com/godotengine/godot/issues/3634. It's also slow since it has to translate GL calls to D3D ones ; and ANGLE on top of Vulkan removes the purpose of Vulkan since stripped driver logic will go into an all purporse wrapper Library instead of the engine logic. Plus Angle would forbid any multithreaded rendering attempt.
Vincent
Trained
Trained
 
Posts: 75
Joined: 06 Aug 2016, 17:24

Re: Vulkan dependencies

Postby pastdue » 11 Sep 2017, 02:54

Vincent wrote:Others projects had lot of trouble getting ANGLE working, see godot for instance : https://github.com/godotengine/godot/issues/3634.


A lot of projects / games / game engines that weren't using it, now are - including Godot.

Vincent wrote:It's also slow since it has to translate GL calls to D3D ones ; and ANGLE on top of Vulkan removes the purpose of Vulkan since stripped driver logic will go into an all purporse wrapper Library instead of the engine logic


Not sure where you are getting the "slow". ANGLE is high-performance and is used in game engines, games, and widely used as part of Google Chrome (which uses ANGLE for all graphics rendering on Windows, including the accelerated Canvas2D implementation and WebGL).

There are a number of steps that a solution like ANGLE can take to convert from a certain API interface (that just happens to be OpenGL ES) to a Direct3D backend with high-performance. ANGLE uses them.

Vincent wrote:Plus Angle would forbid any multithreaded rendering attempt.


This has zero to do with ANGLE, and everything to do with Warzone and how graphics APIs work in general. As I mentioned in the other thread, there are already blockers preventing this from safely being implemented arbitrarily in the Warzone codebase, and the workarounds to those problems don't care whether the underlying graphics backend is being provided by OpenGL, ANGLE, etc.

More importantly, as I mentioned in the other thread, there is very little benefit to multithreading most of the Warzone codebase (in fact, you may make things slower). I'm working on a patch that permits rendering with display scaling on high-DPI displays, and running at 250% on a 2x high-DPI display at a gigantic window resolution, which is pushing dramatically more pixels and data and rendering, the game is barely using 30% of one CPU core at 60fps (with vsync enabled). This is on a 2012 laptop CPU / GPU.

Now, said patch *does* dramatically improve the text rendering subsystem. But it does so because I profiled Warzone, and discovered that was a key bottleneck slowing everything else down. The fix wasn't multi-threading.
pastdue
Rookie
Rookie
 
Posts: 16
Joined: 13 Aug 2017, 17:44


Return to Coding