Page 1 of 1

Warzone 2100 [part 1 of ?]

PostPosted: 25 Dec 2016, 05:01
by vexed
This is more or less a peek inside the dev cycle and how things got done, and how things are currently getting done now.
I am unsure at this time how many of these I would do, but, I feel I need to clear up some things that people didn't know/understand before.
No, I don't expect comments on these posts, that is why it is locked. :)

Anyway... here is part 1.

Yes, this game was an actual retail game.
It was made by Pumpkin studios, and published by Eidos.

Unfortunately for Pumpkin Studio, they went out of business. :(
However, the good news (for us at least) was, they (& Eidos) released the source code into the wild!

I will skip the first few incarnations of what became of those versions, since, I wasn't around then.
So, I'll pick this story up from 2.3 onward.

Warzone was hosted on Berlios, then on SourceForge using svn repositories.
SVN was a version control system that made things much easier to keep track of changes and what not, however, after a few releases from the 2.x series, trying to do massive changes with SVN proved to be VERY difficult at best, and often failed when we were attempting to do merges.
We tried to work with SourceForge, but, our project was just too big to be handled via SVN.
We tried Hg, bazzar, and git.
At that time, for widows, Hg was a much more solid product than git or bazzar.
However, the decision was made to stick with git, so, ever since that, that is what we have been using.
We also tried some different git hosts, until we finally settled on Github.

We attempted to do a 3.0 release, which was basically 2.3 with some upgrades do various things, and that ended up being very buggy, so, we never actually made a 3.0 release.

Our basic layout was, we had a "master" branch (aka trunk on SVN), and a 3.1 branch where we would do releases from.
It then became evident that this wasn't optimal, so we then with a 'bugfixes' branch.
The logic went somewhat like this.
We would use 3.1 for releases, but, try to keep it bug fixes only.
Master was used as the testing grounds so to speak.
Bugfixes was just that, it was meant that once we would fix a bug, we would commit to that branch, and then we would have bots commit the same bug fix into both 3.1 & master.
It seemed logical at the time, and it did work as planned, up until it didn't.

The issue then became how do we do new releases now, since 3.1 and master were diverging rapidly.
A ton of work was being done on master, yet, minimal testing was being done, and trying to get ANYONE to test master was very, VERY difficult. Yeah, there were a few people that did actually run master, and report issues to us, and for those people we deeply appreciated their efforts! :)

So, here we are with a more or less "stable" 3.1, and a vastly untested master wanting to be unleashed as 3.2.
Some thought it was better to keep on 3.1, and backport stuff (basically means to apply some of the newer patches from master into 3.1) from master, yet, others wanted to just release master as 3.2.0.

Yes, it was obvious that quite a few things was broken/buggy, and some downright broken altogether, like the Campaign.
In the end more people wanted to release master than those than wanted the alternative.

We are really a small group of people, with perhaps 1 or 2, sometimes 3 main contributors all working on Warzone 2100 on their free time (No, nobody is getting paid at all! The donations FULLY go to keeping the server alive, which runs forums, lobby server, addons, betaguide, building Warzone 2100 builds, and all that good stuff), and now and then we get some contributors.

Anyway, our old model of maintaining 3 branches (master [for new stuff], bugfixes [for bug fixes that belong in both master & 3.1], and 3.1 [stable]) went out the window, and it was thought it would make much more sense to keep doing releases from master.

It was argued that having featured branches as an expansion of what we have would help, and these feature branches would have specific changes made, and once fully tested, it would be merged back into the main master branch, and the feature branch would be removed. This has not happened yet.
Instead, everything is getting piled into master (for better or worse).

This is where we sit today.

Re: Warzone 2100 [part 1 of ?]

PostPosted: 07 Jan 2017, 02:36
by vexed
The engine.

When Warzone 2100 was out in retail, one of the biggest complaints was that you couldn't make enough units.
So, pretty much the first thing that was done when the source was released was to increase the unit cap.

Rejoice right?
Well, not so fast.
Turns out that there was a pretty good reason why Pumpkin had made the unit cap what it was.
It all boils down to, you can only do so much before the game gets slower and slower.
The more units the game has to take care of, the slower the game gets.

No problem you say, with today's machines, we can have more units!
Well, sorta. Yeah, a beefy CPU can process more things, but, not everyone has a beefy CPU.
If you start doing MP games, one person has a really fast CPU, and the other don't, turned out that the slower CPU would pretty much always lose.

Speaking of MP games, *ALL* versions of the game before 3.x has a slight issue.
That was, the game was NEVER in sync.
That means, people could see different things at different times depending on what they did.
That was the reason for all those tank explosions on one screen, when nothing was around them.
In fact, there WAS something around them, and they were being attacked, it is just that one player's screen never showed it, but, on the other player's screen, it did show it.
Fun times! ;)

{continued later...}

Re: Warzone 2100 [part 1 of ?]

PostPosted: 10 Jan 2017, 05:59
by vexed
Memory Pools!

Sounds fun, doesn't it?
No? Well, then, this isn't really for you, move along...

Pumpkin Studios followed the standard industry practice and had one big memory pool that they used for pretty much everything.

Using a memory pool for games makes things slightly more difficult, in that, the program now must keep track of allocations, instead of the OS, but, that minor penalty is well worth it for a variety of reasons.

If memory pools are good, then why was it removed, and what problems did that cause?
Well, glad you asked!

I have no idea what the motivation was for removing the memory pool, I wasn't around then.
I do know that the consequences of removing the memory pool caused a ton of problems though, and still causing issues to this day. Using C++ containers does help, but, there really is no fix for code that is, shall we say sloppy.

You have to understand that Warzone's code base is huge. It is pretty complex, hard to follow, little 'documentation' except what you read via comments in the source code (or downright wrong 'documentation'/comments) and gives new meaning to the word spaghetti code.
Basically, it is pretty difficult to know the exact consequences of what happens when you tinker with one part of the code, to other parts of the code.
This boils down to using pointers for everything. Pointers inside pointers inside pointers inside pointers...and so on.
Pointers are a must though, there is no denying that.

So, what went wrong?
Again, Warzone is really complex, and the use of multi-level pointers is rather common among all games. Changing over from a memory pool model to standard allocations showed just how much of the code was hiding issues/bugs.

With memory pools, all Pumpkin ever had to do was to zero out the pool, and BAM, everything was reset back to defaults.
Without memory pools, what was once a simple one call to reset everything turned into a hunt to find all the locations a certain pointer has been stored in, since using pointers that are freed is very, very bad, and usually causes crashes.

Needless to say, nobody wanted more crashes.
A large portion of (our limited) time was spent to find these bugs, and try to fix them. While a ton of them have indeed been fixed, or, rather bypassed (using asserts, more pointer checking and so on) it still isn't as good as the original release was, even to this day in terms of clearing out all the game data.

As I mentioned above, using C++ containers does indeed help in lots of areas, and this is rather new compared to the 100% C codebase that Warzone was originally written in.
The vast majority of the code is still written in C though, with only some of the newer/updated parts written in C++.

{more to come...when I get more time}

Re: Warzone 2100 [part 1 of ?]

PostPosted: 08 Feb 2017, 05:53
by vexed
Sync, or what made Warzone 2100 MP sane.
(the abbreviated version)

In all previous version of Warzone 2100 before 3.1.x, all MP games were out of sync.
That basically means, that many times, the game would behave differently, depending on who was running it, how fast their machine was, and their ping times.
You had tanks that exploded for no reason, and magically got back to life, or tanks that once existed vanished, or tanks that bounce back to their location. It was a mess.

Cyp was the main dev that brought sync to Warzone 2100, followed by some contributions from other devs.
What this did, for the first time in Warzone's history, all games could be the same across all platforms, and all machines.
That is where the 2+2=5 icon you may see from time to time signifies, that the game became out of sync if you saw that.
Trying to make sure that all clients sync up was (and still is) pretty difficult.
It isn't perfect yet, but, it is very close.

{more to come...when I get more time}