Converting from INI to JSON

[Per]

I'm seriously considering changing all the .ini format stuff to JSON format instead.

There are two reasons for this. The first is that we are currently using QSettings with a Qt4 virtual filesystem, which hacks any QVariant into a .ini content. This works quite well, even if it is somewhat hacky. It converts even rather complicated javascript data into somewhat readable content in savegames. However, in Qt5 the Qt people removed the virtual filesystem, and that means we cannot use QSettings anymore. That leaves us with two alternatives - write our own ini file parser that does the same thing, or change the file format. Since we depend on serializing javascript to file for savegames, the only reasonable alternative is JSON. Qt5 has native JSON support (and, importantly, it can load JSON from a memory buffer).

The second reason is that once we start looking at more complicated data structures, .ini just falls flat. And it is nowhere as readable as JSON for serialized javascript content. We are running into issues when integrating .ini content with javascript already, where it would be really nice to have a more expressive data format (see discussion with crabster in another thread). I suspect .ini is a dying format, with less and less supporting tools and libraries, while JSON has tons of support in editors, tools and libraries.

I've already talked about this with the people I believe would be the ones would be the most affected at the moment, and I think we are in agreement that this is something that should be explored further.

The ini diff feature will not translate to the JSON world. However, if we just use dictionaries for all higher level stuff, then allowing a small JSON file to replace portions of a parent JSON file will be quite simple by just changing named properties directly.

Until we port over to Qt5, we can use QJson. However, I think we can port to Qt5 before 3.2 release.

My biggest worry is the lack of support in the Flame map editor, as nobody is maintaining that code anymore. (Anyone know any Visual Basic developers who want to help us?)

I've committed some tools to convert most of our existing INI files to JSON already, in master tools/conversion/ini2json/

I'll attach some preliminary JSON converted files to this post so that people can see what it might look like.

Feedback wanted.

[crab_]

We also use .ini files in maps. Do you plan to turn map files to json-format?

Can you upload all mp-stats files in JSON-format? I can test json-stats in my WZ Guide. It can show errors or inconsistence.

I think someone here can fix Flame and make it use json.
Do we have github repository for Flame?


[update]
I see you removed IDs of stuff.
I can expect IDs can be used in Flame and some other stuff. So i suggest to keep IDs (at least as field of object). I use IDs as filenames of icon-files in my Guide.

hmm... you removed IDs but did not changed references
"Heavy Cannon": {
"requiredResearch": [
"R-Wpn-Cannon-Damage05",
"R-Struc-Factory-Module",
"R-Vehicle-Metals03",
"R-Wpn-Cannon2Mk1"
],

"Heavy Cannon" has required research "R-Struc-Factory-Module"
but there is no more research with ID "R-Struc-Factory-Module". We do not have research with key "R-Struc-Factory-Module".
Same problem with other fields. IDs are still used in all properties of research objects.
Removing IDs is not easy task.
I'm afraid some IDs can have use in Warzone source. Need to check it.

[Terminator]

JASON will be cool for future Master server protocol.

[Per]

Pushed fix for IDs still being used as references. Also added 'id' field with old ID name for every item for backwards compatibility, as requested.

Attached zip of multiplayer stats as JSON files.

I am considering changing all the icon IDs into filename references as well, since it makes no sense to keep them as indirect references now that we have done away with storing icons as texture atlases on the filesystem. Currently the mapping from the 'icon ID' to icon is hard-coded in the source code.

[crab_]

I have 1 question.

In 3.1 we have structure functions.
What happened with structure functiuons? For example, i want to change power icome of oil derrick.
In 3.1 Oil Derrick has "Resource" function attached.


Oh 1 more question, Can you please upload other files (like TerrainTable, features etc.)
In my Wz Guide i have following list of stats:

Weapons;
Bodies;
Propulsion;
Structures;
PropulsionModifiers;
StructureModifiers;
PropulsionType;
TerrainTable;
Researches;
Repair;
Construction;
ECM;
Sensor;
Features;
Templates;


[update]

Also added 'id' field with old ID name for every item for backwards compatibility, as requested.

I do not see old ID.
I opened research.json and do not see field like "oldID". Same with weapons.json.
May be i'm wrong? I did not searched deep..

Another issue - properties are messed up.
Better have sorted properties.
In ini we had placed periodicalDamage stuff in a row.
But now we have following:

"muzzleGfx": "FXCan20m.PIE",
"rotate": 180,
"periodicalDamageRadius": 48,
"weaponEffect": "FLAMER",
"periodicalDamage": 28,

I suggest make properties sorted by meaning of properties or alphabetically
"periodicalDamage": 28,
"periodicalDamageRadius": 48,
"periodicalDamageTime": 50,

[Iluvalar]

The ressource function from power derrick was meant to mesure the amount of oil it contain before it get depleted. The idea was thrown away.

There is no way to change the power from the derricks. (it would be great tough). You can still change the power output of the power generator and modules. not sure if it help ?

[Per]

The old structure functions have been changed into fields - powerPoints, productionPoints, rearmPoints, repairPoints and researchPoints. This was done in the conversion to ini.

The old ID was in the field named 'id', but it was misplaced. Fixed. I've also added alphabetical sorting by the keys now.

I have not converted all the .ini files yet. The bodypropulsionimd.ini I plan to merge into body.json. The propulsionsounds.ini and propulsiontype.ini I plan to merge into one. I suppose we should also have a closer look at the others as well.

New zip attached.

[cowboy]

Working on a more modern version of FlaME ...

https://github.com/bchavez/SharpFlame

Source coming soon.

[Rman Virgil]

Working on a more modern version of FlaME ...

https://github.com/bchavez/SharpFlame

Source coming soon.

Fantastic ! And whata New Years suprise, cowboy.

Cheers, R.J.
.

[Per]

Working on a more modern version of FlaME ...

Oh, that would be really good news, indeed.

[cowboy]

Hi there Rman!

Great to see you. Yes, happy new years!

Finished porting FlaME from VB to C#. Now I can get to refactoring and modernizing the code.

viewtopic.php?f=6&t=11536

-cowboy

[crab_]

Hello.
I have tried use new json-files in my WZ Guide
I found one issue.

research.json
-sometimes requredResearch is array.
-and sometimes requredResearch is string.

"AA Target Prediction Computer": {
"requiredResearch": [
"AA Target Acquisition Shells",
"Dedicated Synaptic Link Data Analysis Mk3"
],

"AA Chainfeed Loader Mk2": {
"requiredResearch": "AA Chainfeed Loader",

I think it is wrong.
For example. in templates.json we have
"weapons": [
"Super Pulse Laser Cyborg"
]
- array with one item. Field "weapons" is array always.

[update]
Found the same issue with field "resultComponents".

Per, can you just add brackets to these fields? I think each field should have one type.
"resultComponents": ["Avenger SAM"],

[Per]

I thought I was clever when I made it (turning all multi-item entries into lists, leaving all others as non-list), since it looked much more neat that way. However, I realize now that this is just causing grief for whoever wants to use it. So I've changed it to always be lists for list type values.

[crab_]

Thank you.

I almost finished work on turning my Guide to use JSON-files. (I think it will work, but i did not tried last version of json-files).
Can we discuss plans on other files (TerrainTable, Features, WeaponModifiers, PropulsionType)?
I'm afraid you want to change structure of these files.

[update]
Found we have weapons and research with same names.
So now its is impossible to determite "Hurricane AA Turret" is weapon or "Hurricane AA Turret" is research.
It is possible via old ID
nevermind.. it is not issue

[update2]
research,json - missing result structures
for example
"Scourge Missile Tower": {
"iconID": "IMAGE_RES_DEFENCE",
"id": "R-Defense-GuardTower-ATMiss",
"msgName": "RES_EMP_ATMTOW",
"redStructures": [
"Lancer Tower"
],
"requiredResearch": [
"Scourge Missile"
],
"researchPoints": 7200,
"researchPower": 225,
"statID": "Scourge Missile Tower"
},

[Per]

Missing resultStructures added. I will have to look at the other files later, hopefully soon.