WDG filename hashing algorithm

[kage]

i'm putting serious consideration into an automatic wdg -> wz converter that "just works" or provides warnings for unconvertable changes -- was going to look for the hashing algo that strata, and i think perhaps kevin (and possibly others) reverse engineered. however, the rts.net forums are down, and i don't even remember how to get to the p-2 forum backups, if we ever had them.

that said, does anyone have any particulars of that hashing algorithm (or if need be, just the reverse algorithm that involves brute forcing)? without it, all there is to use are the hashmap that comes with wdg explorer, which is no good if someone used their own filenames.

[DevUrandom]

DGO?

[Buginator]

i'm putting serious consideration into an automatic wdg -> wz converter that "just works" or provides warnings for unconvertable changes -- was going to look for the hashing algo that strata, and i think perhaps kevin (and possibly others) reverse engineered. however, the rts.net forums are down, and i don't even remember how to get to the p-2 forum backups, if we ever had them.

that said, does anyone have any particulars of that hashing algorithm (or if need be, just the reverse algorithm that involves brute forcing)? without it, all there is to use are the hashmap that comes with wdg explorer, which is no good if someone used their own filenames.

I actually looked into this a bit, and here is what I found.

I know the hash function(s) is(are) not injective.  (They have a case sensitive, and a case insensitive version.)
I also know that all that you see in the warzone.wz & mp.wz files were converted from the wdgs.  So this utility is already made.  One version is on the mailing list, but if I recall correctly, that wasn't the final version of it.  However, that is the same one that Rodzilla used with some kind of script files to generate the infamous map pack*.  It would seem it works for the most part.
We don't got source, or access to the author who made it though.

From what I can tell, the utility uses all the names that warzone ever used (at the time), and it looks like it just does some sort of  lookup function to generate the names that are known in the data file, and generate the files in question.
Also note that there are also embedded files in the .wdgs that tell the game what to do. That would be the *.wrf files.
I think there are also some other things I found out about them, but I don't recall what they were at this time.
Perhaps I can dig up my notes, just got to find them...I bet they are somewhere close to the notes I did for the warzone wardrive cam or whatever it is called, along with lots of other stuff.  I just don't got the time to finish those things.

The other issue with making such a utility is the blasted licensing issues.  I know this keeps coming back to haunt us again & again, but there is no real way around this, *IF* you want to stay legit.  I guess there could be a web site that can host the converted versions, but that is up to the web master.


*seems that all those converted maps are basically in the state of flux, since it was (is) impossible** to get the OK from the original authors.  That is the reason why it is not included AFAIK.

**seems we have at least one author available, and I am going to make a post about that in a bit.

[Rman Virgil]

DGO?

* Direct Games Online.

- RV

[kage]

licensing issues? my intent for this utility is to allow command line conversion of a working 1.10 wdg into a working wzr wz file, with some versioning support, with warning/error output as needed (in other words, a whole lot more than just translating hashes back into filenames). the primary target audience for this would be modders who want to create wzr compatible versions of their old mods, as this community has *very few* compatible mods that i'm aware of. I also plan to allow a --add-gpl switch to inject a gpl license into the created wz (though only people who want to gpl their mods should use this), and if practical, some sort of mod signing support.

i do not intend for this to be used to pull and repackage proprietary content for rehosting -- the only people who should be using this are the owners of the content, or someone with explicit permission from the owners to do so.

EDIT:

hash creation algorithm as discovered by stratadrake via (presumably) disassembling MakeWDG.exe:

I did it! I cracked the algorithm! Does this make me a hacker? ^_^

Buried deep within the confines of MakeWDG, invisible to the naked eye, are 19 little instructions that look like:

:00402690 mov edx, dword ptr [esp+04]
:00402694 xor eax, eax
:00402696 mov cl, byte ptr [edx]
:00402698 test cl, cl
:0040269A je 004026C3
:0040269C shl eax, 04
:0040269F and ecx, 000000DF
:004026A5 add eax, ecx
:004026A7 mov ecx, eax
:004026A9 and ecx, F0000000
:004026AF je 004026BB
:004026B1 and eax, 0FFFFFFF
:004026B6 shr ecx, 18
:004026B9 xor eax, ecx
:004026BB mov cl, byte ptr [edx+01]
:004026BE inc edx
:004026BF test cl, cl
:004026C1 jne 0040269C
:004026C3 ret

How it works is:

0 - Calling function pushes a pointer to a null-terminated string onto the stack, then calls this function.
1 - Function verifies that the string pointer is not zero-length.
2 - A loop is set up as follows:
2a -- Read one byte from the string
2b -- Convert to lowercase, subtract 0x20 (or in short, "and cl, 0xDF")
2c -- Shift the running total (register eax) one hex-digit left, then add the byte (register ecx) to it
2d -- Separate the first (highest-order) hex digit from the rest and XOR it into the 2nd-to-last hex digit. To improve performance, this step is skipped if the running total has a highest-order hex digit of zero.
2e - Grab the next byte from the input string.
3 - Conclude the loop when the terminating null byte is read and the hash code is returned in eax.

the whole thread can be found from http://www.directgames.net/pumpkin2/for ... topic=1225

my thanks to cowboy for hosting the forum backup.

EDIT 2: and many thanks to RV for direction to the P-2 backups (thought i made a separate post for that, but oh well. apologies).

[DevUrandom]

Are you maybe searching for this?
https://trac.mortis.eu/warzone/browser/ ... dg.c?rev=1
(Credits for finding it go to EvilGuru.)

[EvilGuru]

The hashing algorithm 'HashStringIgnoreCase' can be found here:
https://trac.mortis.eu/warzone/browser/ ... ev=1#L1389

No dodgy reverse engineering or leaked code required.

Regards, Freddie.

[cybersphinx]

I also know that all that you see in the warzone.wz & mp.wz files were converted from the wdgs.  So this utility is already made.

Nope. The data we use was already in separate files with useful names in the source archive.

[Buginator]

Nope. The data we use was already in separate files with useful names in the source archive.

I don't believe that is correct, at least not all the data.  I know it was mentioned in one of the logs or e-mails  or maybe it was a forum post I have read in the past.  I think it was the patches or mp stuff or something like that.  Not 100% sure at this time.

But the main point I was making was they did use a utility to covert some .wdg files.

[Chojun]

Someone had to have converted wdgs to .wz because the data files in the original source release (1.10) corresponded with v 1.0.

For Warzone 2100 1.11 I had to manually go through and extract & convert all the files in the wdgs.  I had to get counseling.

[cybersphinx]

Someone had to have converted wdgs to .wz because the data files in the original source release (1.10) corresponded with v 1.0.

For Warzone 2100 1.11 I had to manually go through and extract & convert all the files in the wdgs.  I had to get counseling.

I think there was a so-called Dev-CD with the patches as separate files as well, and I seem to remember someone using those to update our data to 1.10. And someone auto-converted some maps into .wz format, but for those there are not that many necessary file names, so auto-guessing should have worked.