Datafile-Format discussion

Post by **DevUrandom** » 18 Dec 2006, 16:46

The Lua tables would be human readable and with mod_lua you can even create a website directly out of it.

Next week (or the one after) I'll have a look at this stuff and see if it is really that bad as you think.

ratarf · Post by **ratarf** » 18 Dec 2006, 20:01

Yeah, we should examine all different approaches before starting anything. It's better to spend more time on preparation than starting all over again afterwards.

Post by **Watermelon** » 24 Dec 2006, 20:18

Just for your info,I am working on a very simple custom plain text reader function to make the txt files easier to read/edit.

example plain text data:
[ad f ad sf] nameA = 1; nameB = 2; someStr= "321";

will be read as:
object1:
name = adfadsf
object1 properties:
nameA = 1
nameB = 2

object2:
name = BA
object2 properties:
someStr = 321(string)

syntaxs of this custom plain text:
[ object name starter/new object starter/previous object end
] object name end
= left 'operand' property name,right 'operand' property value
; end value
" string
all space ' ' will be skipped.

Now I am trying to find a way to convert from old 'comma delimited sscanf' txt format to the new txt format,the sscanf thing with all data type mixed up togther is really complicated to convert it seems to me...

Post by **Kamaze** » 24 Dec 2006, 20:43

Watermelon wrote: ...

A normal *.ini file!?

Post by **Watermelon** » 24 Dec 2006, 21:23

Kamaze wrote: A normal *.ini file!?

no it's just some simple string read functions I started from scratch,I looked at libini and it seems to be a bit complex and it uses '\n' as 'somevar = somevalue' end mark,which would probably create a havoc if we use it to replace wz txt comma's.(200+ lines with 30 values each line delimited by comma becomes (200 * 30) lines txt file with ini...)

Post by **Watermelon** » 25 Dec 2006, 17:32

Finished the old format to new format converter for weapons.txt.

I am wondering if I should incorporate it into wz stats load functions or use it as an external tool to convert from/convert to old format that wz uses.

Here is the weapons.txt(SP and MP) in new format:

kage · Post by **kage** » 26 Dec 2006, 13:17

Watermelon wrote: I am wondering if I should incorporate it into wz stats load functions or use it as an external tool to convert from/convert to old format that wz uses.

to answer a question with another question:

are we designing warzone to support multiple versions of the engine? -- as in, a frontend loader that allows the player to join an mp game of any warzone version downloaded, and have the frontend automatically switch between versions.

in other words, if we do intend to allow the above functionality, then we would want to implement a standardized and extendable api for interfacing data loading modules to the engine, so that all data-loading modules may be forwards compatible: in this case, we don't permanently convert any data, but just convert it in memory at runtime.

if we don't care to support a version-agnostic front-end, then it's probably a whole lot better to use an external tool with an internal raw metalanguage that is used to translate between any two supported data-file versions.

in either case, it'd probably be a good thing if every single data file henceforth started listing a minimum required engine version so that parsing will be easy -- everything before now was pretty much the same and unchanged, while everything after now can list that version to determine if a mod can run on the available engine. either way, some of the benefits of xml and similarly-featured formats, such as being supposedly backwards compatible (ignore any options you see but don't know about) are lost, since a mod would automatically become unbalanced if those ignored features were taken into account during the design of the mod, which is another good reason, even in xml, to force a minimum required engine release.

Post by **Watermelon** » 26 Dec 2006, 21:02

kage wrote: to answer a question with another question:

are we designing warzone to support multiple versions of the engine? -- as in, a frontend loader that allows the player to join an mp game of any warzone version downloaded, and have the frontend automatically switch between versions.

in other words, if we do intend to allow the above functionality, then we would want to implement a standardized and extendable api for interfacing data loading modules to the engine, so that all data-loading modules may be forwards compatible: in this case, we don't permanently convert any data, but just convert it in memory at runtime.

if we don't care to support a version-agnostic front-end, then it's probably a whole lot better to use an external tool with an internal raw metalanguage that is used to translate between any two supported data-file versions.

in either case, it'd probably be a good thing if every single data file henceforth started listing a minimum required engine version so that parsing will be easy -- everything before now was pretty much the same and unchanged, while everything after now can list that version to determine if a mod can run on the available engine. either way, some of the benefits of xml and similarly-featured formats, such as being supposedly backwards compatible (ignore any options you see but don't know about) are lost, since a mod would automatically become unbalanced if those ignored features were taken into account during the design of the mod, which is another good reason, even in xml, to force a minimum required engine release.

I think the sscanf is one of the most efficient ways to 'parse' data in memory(maybe only a little slower than 'preformatted' or so-called serialization'ed binary data?),so it's better to keep the current format for now for the sake of optimal loading speed,and use some tools to convert the data to human-readable to make mods,then convert the data back to 'wz txt format'(fields delimited by comma's and each line represents an 'object') after the modifications are done.

Post by **DevUrandom** » 26 Dec 2006, 21:20

The minimum required engine is a nice idea. I'll try to implement that in my efforts on the modinfo files.

JorgeAldo · Post by **JorgeAldo** » 23 Jan 2007, 22:48

What about this http://ndevilla.free.fr/iniparser/ ?

Its MIT License. An INI File parser.

Should ease the maintenance cause java, c++ and delphi have ini file parsers included in their object classes...

Post by **DevUrandom** » 23 Jan 2007, 23:26

Well, I don't think that we need external code to parse an ini file...

JorgeAldo · Post by **JorgeAldo** » 24 Jan 2007, 00:23

reinventing the wheel for simple things sometimes get us stuck without any gains...

Warzone 2100 Forums