Stats data format I wish to have.

[Emdek]

We just need proper tool support, and we have none for master, and the current tools still work for 3.1

I can create one when it will be finally decided which format will be used...

[Reg312]

We just need proper tool support, and we have none for master, and the current tools still work for 3.1

my tool works for master version, and if i want i can simply extend it to any stat file in ini-format
i was forced to write small ini-parser

if noone will do nothing for 1-2 months i think i'll get time to write small ini/csv editor for stats with QT (cross-platform)
i just want to write something with QT

[Emdek]

if noone will do nothing for 1-2 months i think i'll get time to write small ini/csv editor for stats with QT (cross-platform)
i just want to write something with QT

You can help with mine (no repository yet, but general architecture was already designed).
Ii would be good someone who knows how old one, CSV based, looks like in practice, like dependencies between columns etc.

BTW, QT means QuickTime, you should use lowercase "t" always.

[Reg312]

@Emdek i like help people
but may be i'm not good programmer

architecture of my program (not ideal, but finally it works):
1) read stats structures from 2 csv-files
* if structure was changed i need only change these files and restart program
2) use loading function to parse wz-file to given structure
* unzip / or read list of files in directory and subfolders
* search for files (use short filename as key field)
* load files which was found using parse-function (separate for ini and for csv)
3) generate edit form run-time (when user select what table he want to see)

data stored in-memory using built-it data class with simple functions for serching,editing etc

in my tool data structures stored in 2 files, you can get it and read in your program
1) filelist.csv
https://github.com/crabster/Warzone-Sta ... lelist.csv
columns: short filename (as key), filename related to mp folder, bool field: is csv-file contains header, name of file with names, key columns index, miximum lines, description
some of these columns not fully used (messed up a bit in code)
2) stats_metadata.csv
https://github.com/crabster/Warzone-Sta ... tadata.csv
columns: short filename, column name, data type (int, bool or empty which means string), [internal: special use: default or expression or foreign key], [internal: defaule value, expression,], Caption, Description, Interface: group name, Interface group position, Interface order position

for foreign keys in stats_metadata.csv i have formatted string with properties like this:
link_table, link_file=researchfunctions.txt; lfile=functions.txt; rkey=Code; link_rkey=ResearchCode; link_lkey=FunctionCode; lkey=Name; linkcolumns=FunctionName|Name
list from file,lfile=body.txt;lKeyColumn=Code;lNameColumn=WZStats_Name,Body
link_table - when 1 line contains one or many foreign keys (table to table to table)
list from file - 1 line contains 1 foreign key (table to table)

also i have some problems with my architecture - files with item names are processed separately

[Duha]

i was forced to write small ini-parser

What wrong with existing parsers?

also i have some problems with my architecture - files with item names are processed separately

May be this trouble in game architecture. It will be better to have names inside stats files like done in body.ini


@all:
IMHO all data should be in one format:

One of ini disadvantages is many-to many relations.
I don`t see good way to show techology prerequirements in ini.
tech3 tech1
tech3 tech2

[aubergine]

Code: Select all

<tech id="someId">
  <name>canonical name</name>
  <enables type="tech">someTechId</enables>
  <enables type="structure">someStructureId</enables>
  ...etc...
</tech>

[Reg312]

What wrong with existing parsers?

if you know good ini parser for c# let me know, may be i'm a bit lazy to search

May be this trouble in game architecture. It will be better to have names inside stats files like done in body.ini

we still have body names in names.txt

[Duha]

Code: Select all

<tech id="someId">
  <name>canonical name</name>
  <enables type="tech">someTechId</enables>
  <enables type="structure">someStructureId</enables>
  ...etc...
</tech>

I was talking about ini.

What wrong with existing parsers?

if you know good ini parser for c# let me know, may be i'm a bit lazy to search

You lazy to search but not lazy to write and test some code? I am usualy test avalible libs and only after that invents own bicycle

[iap]

Code: Select all

<tech id="someId">
  <name>canonical name</name>
  <enables type="tech">someTechId</enables>
  <enables type="structure">someStructureId</enables>
  ...etc...
</tech>

Exactly!
Hierarchy and attributes give so many advantages it's shame to miss. You can't find it in ini

[Emdek]

iap, you can, especially since there is no real standard for them you can create a way to have multilevel hierarchy there.
Like:

Code: Select all

[someId]
name=canonical name
enables[tech][]=someTechId
enables[structure][]=someStructureId

AFAIR (I've needed to create custom parser anyway) in case of PHP it will be parsed correctly into multilevel arrays hierarchy without issues (as long you won't hit stupid behavior on special keys like no etc. - this is one of reasons why I've needed custom parser).

[iap]

Oh... well, then if it's more comfortable then ok.

[Emdek]

iap, that is relative, and with proper editor is meaningless (but INI and JSON are probably easiest to parse without adding external dependencies).

[Duha]

iap, you can, especially since there is no real standard for them you can create a way to have multilevel hierarchy there.
Like:

Code: Select all

[someId]
name=canonical name
enables[tech][]=someTechId
enables[structure][]=someStructureId

Looks not good. Not self explaining for new users. Has no documentation about syntax. Needs custom parser.

AFAIR (I've needed to create custom parser anyway) in case of PHP it will be parsed correctly into multilevel arrays hierarchy without issues (as long you won't hit stupid behavior on special keys like no etc. - this is one of reasons why I've needed custom parser).

There is no need to invent new bycicle when you can just deform existed.

XML: validation. You can write less code to validate outer data. Esay to make little changes (no need to compile)

Ohter formats has no big advatages via csv.

But changing format will need new parser

Using own format is not good:
You need to write and test code of parser and make some documentation. (using xml parser will need less code).
It will take more time to other members to get in this parser. (almost all programmers can parse xml correctly, and if they will have question they found answear fast)

If no xml then it will be better to update csv parser. (remove unused and add names)

iap, that is relative, and with proper editor is meaningless (but INI and JSON are probably easiest to parse without adding external dependencies).

in PHP or C++?

[Emdek]

Looks not good. Not self explaining for new users. Has no documentation about syntax. Needs custom parse

I've said above, that is relative, for many users it will be easier to understand as it doesn't look "scary" - XML documents are very verbose and if use extremely short / weird tags and don't have indentation then can look like binary data (AFAIR that was one of points against Microsoft Office Open XML). Existing entries are best example explaining how to write new ones and there can be always some README file or comments in header, why not. And custom parsers not always are bad, especially if they span on less than 100 lines of code, but you are sure that it will work as it should.

AFAIR (I've needed to create custom parser anyway) in case of PHP it will be parsed correctly into multilevel arrays hierarchy without issues (as long you won't hit stupid behavior on special keys like no etc. - this is one of reasons why I've needed custom parser).

There is no need to invent new bycicle when you can just deform existed.

In this case you can't unless you want to create custom C based PHP extension and you will be only able to use that on dedicated server as probably nobody will trust loading possibly unsafe binary files on their hosting.

iap, that is relative, and with proper editor is meaningless (but INI and JSON are probably easiest to parse without adding external dependencies).

in PHP or C++?

In whatever, but C++ makes most sense (Python also could be good choice).

INI is easiest format to edit in simplest editor, for above example you only need to have keys that use only characters from range a-zA-Z0-9_\ (well, in fact everything except =, [, ], and new line - to make it more readable) and for value you only need to escape new line characters (for example \n) and escape character (\) to avoid situations when there was already \n in raw value. The main and probably only advantage over JSON is that it forces placing one value per line which makes it more readable out of the box and doesn't require closing groups (which is easy to forget when editing by hand using things like Windows Notepad, at least that from XP) as values are terminated simply by end of line.
Later we can place JSON (with proper indentation and description how to escape characters that can break it without escaping).
Then CSV, which is pretty plain but requires lots of attention for long rows of values.
XML is another story and it's main disadvantage is that it requires additional dependencies. Custom parser could be easy to do though (but reinventing validation schemes doesn't make sense at all).

All depends on if we want to make it as easy and as fast as possible to modify using simplest possible text editor (Notepad++, Kate and others have built in tools that could assist in work with some of mentioned formats) or encourage to use specialized tool.

[iap]

I use text editor to write XML all the time. But this argument is redundant because the format already been decided. And if it will cost you less to make ini parser then use some premade XML then by all means, I'm in no condition to tell you otherwise.
Just think that for the long run it will be very good for the project.