Reintegrating DirectX - Critiques, Issues, and ??

[Lord Apocalypse]

While hijacking a thread (sorry about that) Emdek and I started going off on a journey away from that of wrecks and into the dos and don'ts, nuts and bolts, etc etc of the current engine. Possible limitations were brought up as was the problems with lack of good OpenGL support from low end graphics drivers such as those from Intel.

So, the idea of putting directx back into warzone was brought up. While probably not an easy task it would solve some issues such as those pesky white textures that show up due to (mostly) intel's lack of proper opengl support. For now as an unsupported option you can try using TitaniumGL. It sounds like it will fix some issues, but has not been tested (as I have a Geforce 260). The other developers working on Vega Strike have decided to have people try and use it so I figured might as well say soemthing here. I hope it will be a temporary fix for both projects until intel can fix its opengl support and/or I and maybe Emdek can get DX back into warzone (provided the other WZ devs go support the idea).

But alas, for now this will just become another code thread for people to post thoughts, ideas, but not general dislike for microsoft. In the process of adding DX back in there will be a lot of issues as well the possible benifits of being able to optimize the current rendering engine.

First problem: winmain - depending on how the current code works this may not be a big issue or it could bog down into a function/variable hunt to find everything. I don't know.
Second problem: shaders - Since DX and OpenGL use different shader languages new shaders will have to be created. One possible solution is to use the Cg shading language.
Third problem: Gui - This probably isn't as big of an issue as I'm worried it will be, but best to bring it up anyway.

Now that some of this is taken care of we can move on to final thoughts. Hmm.. I really don't have any. I don't know what kind of hardware many of you have (for windows users anyway) but I hope that no one is using anything that isn't direct x 9 compatible.

So everyone share your thoughts, code prototype, ideas, and things you would like to see happen while this modestly grueling refactoring is thought out, played out, debated, flamed, and hijacked from time to time.

[Lord Apocalypse]

One quick note. Cg will compile out both HLSL (DirextX) and GLSL (OpenGl) shaders. http://developer.nvidia.com/cg-faq

*EDIT* This all started from this point in the wrecks thread viewtopic.php?f=30&t=9435#p103542

[Emdek]

That TitaniumGL is bit controversial, apparently it is adware under Linux.

Back to topic, the biggest issue for me (which would be nice to have resolved anyway) is that AFAIK some of game logic is mixed with rendering path (I would like to hear comments from developers about that, to make it clear) which would be main initial obstacle in abstracting rendering engine.
As I've stated before ideally it should be possible to switch rendering engine without need to recompile / have separate binaries, like make renderer a plugin that is loaded by checking which one is most suitable or forced through command line one. As I would like it not only limited to DX9 and OpenGL 2.0, if it would make more sense to have separate paths (without ifdefing etc. but using some common library if needed) for OpenGL 3.x , 4.x or at least for OpenGL ES.
We would need API interface which each backend would need to implement, that would include API for creating GUI (drawing and receiving events only, logic should be common, if possible). When we would have such API then those who would work on additional backends wouldn't (or at least would much less, there might show up need to modify that interface from time to time) interrupt others.
Yes, that task will require lots of work, but for me is something that is crucial for long term vitality of WZ engine.
And the only alternatives would be switching to another rendering engine or investing in current one sacrificing possible optimizations available in newer versions of specs and / or support for older hardware.

About DirectX9 support, AFAIR 9c was newest one which I was able to install for my i915 Intel card (released circa 2004) so in theory it should have probably at least broader support than OpenGL 2.x. I'm not familiar with DX API at all so I don't know if they deal somehow with hardware that doesn't support some of features (like using CPU for missing features), but I guess that they overcome that somehow.

[Per]

Adding an additional rendering path is an insane idea. However, what we should do is to port Warzone to OpenGL ES 2. If we do that, then we can add support for Google's ANGLE project (http://code.google.com/p/angleproject/), which emulates OpenGL ES 2 under DirectX. People with only OpenGL 1.x support on Linux would be screwed, though. But with the rise of $25 computers with OpenGL ES 2 support that could (presumably) run Warzone if we ported it to OpenGL ES 2, catering to the tiny minority with arcane hardware becomes more and more insane.

[Emdek]

Per, that would be good too (for me that is no issue, since my main computer is no longer crippled by i915, not sure if Radeon x200 like card would support that though... but probably yes, since it has full OpenGL 2.1 support with elements of 3.x), although separating rendering path would make some things cleaner and moving to OpenGL ES would require many work anyway and give an opportunity to clean up some things which would let others (if someone would like to) materialize that insane idea.
And advent of many cheap but quite powerful computers is not enough, since it won't magically replace hundreds of thousands integrated Intel (which had biggest GPU share in 2006, and all of them weren't OpenGL 2.0 capable, even worse, usually up to 1.3 / 1.4) only computers alone (there are also ancient cards, but those won't be supported anyway, since there are no functional drivers for recent OSes). Though since projects like Qt5 chosen to require at least OpenGL (ES) 2.0 (though it could be implemented in software, at least using some Mesa drivers that utilize CPU and some independent developers started to work on DirectX backend recently, which is also based on ANGLE) so such switch should be only a matter of time. But with abstraction we could at least let some "insane" people to try to support older configurations.

[cybersphinx]

until intel can fix its opengl support

The problem is only partially Intel's drivers, but also people who don't have current drivers installed.

and/or I and maybe Emdek can get DX back into warzone (provided the other WZ devs go support the idea).

I don't support it.

The part about cleaning up the code so that rendering and engine are properly separated is good. That would also make other things easier, like a dedicated server. But then instead of adding another renderer that adds code, needs maintenance and divides the available manpower, using an external engine sounds like a much better idea to me. That way we have less code and get all the features (GUI) of and the manpower of those contributing to the engine.

Or, since we don't have anyone actively working on it at the moment iirc, just enhance the OpenGL renderer to make that run on low-end systems, be more efficient etc. Putting that work into a Direct3D renderer instead might make Warzone work better on some Windows systems, but wouldn't fit our goal of platform independence (especially if at some point D3D adds something and nobody fixes the OGL part (like what happened to the software renderer after OGL was added) it becomes effectively Windows-only).

[Emdek]

cybersphinx, I'm also supporting idea of considering use of external engine, since every possible path other than "maintain current code only" does require lots of work and initial steps for each one are very similar.
But having custom backends doesn't even need to mean that they would need to even live in upstream repository and be distributed with official installers. Those could live as kind of addons, maintained and distributed by third parties, in most extreme case at least.

[Lord Apocalypse]

That TitaniumGL is bit controversial, apparently it is adware under Linux.

I'm only suggesting this for windows. There are a lot of OpenGL games that have issues on windows (even properly programmed) such as the white page/texture.

As I've stated before ideally it should be possible to switch rendering engine without need to recompile / have separate binaries, like make renderer a plugin that is loaded by checking which one is most suitable or forced through command line one.

So pretty much separate the renderer completely as a dynamic library. By this train of thought are you talking about have one lib for opengl one for dx etc? If so its not that great an idea. Now if your talking about changing from one rendering engine to another (say this WZ engine to ogre or irrlicht thats much harder to do than would be warranted.

As I would like it not only limited to DX9 and OpenGL 2.0...

This would be easier to do. We might be able to have to do something like DX11/GL 4 with DX9/GL2 fallback.

if it would make more sense to have separate paths (without ifdefing etc. but using some common library if needed) for OpenGL 3.x , 4.x or at least for OpenGL ES.

Haven't looked into GL ES so don't know anything about it yet.. will look it up. Our best option is to make the render use a GL/DX wrapper. That way we don't have to worry about one tech or the other getting left by the wayside. And yes 9.0c was the last in the DX9 release line.

@Per Uh where can I get me a $25 computer?? The idea isn't to create two separate rendering path but a way to have 1 call for DX and GL. I think I should go over the Ogre (doesn't ogre also run both??)and irrlect source and see how they handle calling dx and gl from the same renderer. Would give a good idea of how they handle it and maintain both. As for ANGLE... how far a long is it and whats up with all the WebGL talk.

@cybersphinx Even with current drivers installed a lot of these problems still exsist. I know there isn't a lot of people working on the code here, but to say that DX ruins the platform independence of WZ is hogwash. It adds complexity yes, but it can also make things easier for supporting windows users (lame I know). Provided we can get a functioning wrapper for DX/GL I don't see why we can't have both. WZ would only have ONE engine that would push calls into the DX/GL warapper. If a new graphics feature gets added, just make sure it can run on both (which it should since OpenGL is now caught up to DirectX).

As for the software renderer.. even in 2004, if you couldn't run in hardware it was time to upgrade that fossil. My thoughts on the graphics end.. If it doesn't support OpenGL 2.1 or DirectX 9.0c its time for an upgrade. After 3.1 WZ should not support anything older than GL 2.1 How many people still run hardware that supports only 1.4 anyway? How many open source projects have ogl support for ancient hardware??

...especially if at some point D3D adds something and nobody fixes the OGL part (like what happened to the software renderer after OGL was added) it becomes effectively Windows-only

Ok, who controls what goes into WZ currently? How many people have access to push anything into the git repo? This is a non issue for a team/project of this size. It would be easy to deny any patch that doesn't support both. Even if another 10 or more people started providing code patches it wouldn't be hard to accept/reject anything that isn't cross-platform.

On the mention of Mesa drivers... I will tell you this based on work from over at Vega Strike. Mesa drivers SUCK. We have nothing but problems with them and the main graphics developer klauss works in linux.In fact Vega Strike was coded for linux first and support for windows second. Probably why windows support under 64bit windows is crappy.

Anyway, ideas are always good even when some seem to be dead set against a given idea. So, keep the thoughts and ideas rolling. Maybe by the end of this we will have a good idea of what should be supported or added to the engine for 3.2 or perhaps earlier. I love WZ and don't want to see this game stuck in a rut.

[Emdek]

OGRE also looks interesting:

The class library abstracts the details of using the underlying system libraries like Direct3D and OpenGL and provides an interface based on world objects and other high level classes.

The worst thing is to decide which way to go, after that it will be easier, when the goal will be clear.
But in this case it looks like we are going to go OpenGL 2.0 ES for now, for upstream at least.


About how many of computers do not support OpenGL 2.0, I would says that maybe even up to 10%, maybe more, maybe less, but that is pretty significant number still, and worst cases are laptops and PC without PC Express x16 slots (quite recently I was using Dell workstation with something even older than i915 and without such slot).

[vexed]

From the start of the source release, it was pretty much a bunch of ideas, and nobody to do the ideas, or a few that wanted to do them, but it ended badly for whatever reason.
It took a few dedicated devs to re-energize Warzone, and make it cross-platform, and 64bit compatible, and now, deterministic.
Yeah, there were/are huge bumps in the road, but this is where we stand now.

[Going the custom DX dll route]
Making the code handle either DX or GL isn't that hard at all, it just takes someone to actually do the work.
Once that work is completed, what happens if there are DX bugs, and the linux devs are the only ones left ? How will that end up working, in short, it won't, and it would get dropped, pretty much the same way the old software renderer died.

[Going the 3rd party engine route]
It is really nice to have someone else maintain something that can handle rendering (and a GUI would be a huge +), be it Irrlicht, or one of the others, can kill a few birds with one stone. The only problem here is, WZ's data would need a major overhaul, and lots of time to convert it over, plus the source changes.
When we were looking around at 3rd party engines, some really nice ones didn't support the mac, so that limited it somewhat.
The other issue is, that they must support a cross-compiler.

Cg is crap, and produces horrible code for anything but nvidia hardware, yeah, big shock.
Quite frankly, I am not thrilled about GL 2.0 ES either. It is too neutered for my liking.


Having said all that, it still all boils down to, who is actually going to help, and I don't mean "help", I mean actually get their hands dirty, trying to improve Warzone ?
There have been many threads in the past, and even talk on IRC, but all it ends up being is just that, talk, and that don't really help Warzone improve at all...

[Lord Apocalypse]

Well, as I've stated in another thread I plan on working on a fork that updates to DX9 first then starts adding in other goodies. If I can get that done I see no problem in trying to get DX back into the current code. To me I don't care either way. Stay OpenGL or go DX/GL I just don't want to see the graphics side of things stay stagnant while the data and logic are the only things really being updated.

The whole point of this was adding DX and its gone slightly askew when others started bringing up plugging in ogre or irrlicht(however its spelled). While either would be nice it would require far more work than just the data. All the models and maps would need updating as I know Ogre uses its own graphics format.

For now, let us just focus on just adding directx back into the mix, and how we might go about doing that with the least amount of work and possible complecations.

[vexed]

A 3.1 fork that adds DX 9 ?
That would be ... interesting.
For the most part, it shouldn't be that difficult, you might have to swap x,y,z around for some things.
Would you be going the dll route, or the if (DX) ... route ?

[Lord Apocalypse]

Yeah I think one goes front to back the other goes back to front... I can't remember the name or specifics.. I just know its different. Best source on how to properly add a DX/GL wrapper though would be the 2 engines listed as they support both.

My plan though is to keep openAL. I see no reason to drop it over directsound. So it really only replaces SDL and OpenGL stuff. Keyboard and mouse support should be the easiest to swap over. Not sure if the graphics or gui will end up being harder.

[Rman Virgil]

.

Old man river here again with that bloody good Yoggi Berra non sequitur to kick it off:

"It's like deja-vu all over again."

Let's just say upfront that I donot buy into the argument that forks should be discouraged because they dillute potential manpower. I KNOW they are healthy and generative. Besides, the fuel for this type work is inherent passion and that translates simply as following your heart's desire, your vision, and trying to subvert that with intellectually based reasons is pretty much a fools errand more often than not (speaking within the framework of being a law abiding citizen, that is).

Moving on.....

vexed:

.... Having said all that, it still all boils down to, who is actually going to help, and I don't mean "help", I mean actually get their hands dirty, trying to improve Warzone ? There have been many threads in the past, and even talk on IRC, but all it ends up being is just that, talk, and that don't really help Warzone improve at all ..

That is exactly the bottom line. In all approaches, for all visions that would be made real, here as in RL, let's be clear.

=======>

Just a quick side-bar as they say in the legal profession... A wee bit WZ Source Schism History.

=======>

What happened in 2004-05 is clearly documented in the rts.net forum archives which are still extant.

Since I was there, and in the middle of it, I can provide a quick summary of what actually happened for any who may value historical accuracy over fanciful suppositions or selective exposure mythos.

The original WZ community had more than ideas about how to proceed with the source it was intrumental in liberating over the course of 6 years. There was a clearly defined and practical strategy and enough people willing and able to execute it. There was just one hitch. It had not taken into account any of what turned out to be a vast and aggressive interest outside of the WIN platform (dubbed the Slashdot.org effect) AND outside the original WIN community that had stuck together through thick and thin for all those non-negligible years with zero gaurantees that their efforts to liberate the source (and keep the game alive and still evolving, sans source, in the interim) would ever bear fruit. Talk about faith-based dogged dedication, passion, and getting your hands dirty...

The majority of the original WIN community had close to nil interest in pursueing a cross-platform dev strategy. A couple saw the wisdom of changing course and lent their support to the new cross-platform approach while the majority of the old crew withdrew with a mixture of dissappointment and anger determined to not be an active part of WZ anymore. Very sad but unavoidable collateral damage, including friendships.

That said, cross-platform was the right course '05 and I still believe remains so now. Regression is not an option but forking is certainly a viable option.

And yes, it does come down to dev manpower, getting your hands dirty so to speak, but that dev manpower is affected by other critical values and skill sets that have zero to do with coding. I would think that obvious by now.

I've already touched upon some of those values and skill sets in another thread earlier today in a different but related context so I will leave it there.
.

[cybersphinx]

Let's just say upfront that I donot buy into the argument that forks should be discouraged because they dillute potential manpower.

My point was that I don't see us interested in an additional D3D renderer, and if the goal is integration, from my point of view the effort would be better spent on other things. Of course I can't keep people from doing what they want to do, and in the end, it's the actual implementation that has to be judged (and not only by me).