Converting from INI to JSON

Discuss the future of Warzone 2100 with us.
Per
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 3774
Joined: 03 Aug 2006, 19:39

Converting from INI to JSON

Post by Per » 31 Dec 2013, 17:08

I'm seriously considering changing all the .ini format stuff to JSON format instead.

There are two reasons for this. The first is that we are currently using QSettings with a Qt4 virtual filesystem, which hacks any QVariant into a .ini content. This works quite well, even if it is somewhat hacky. It converts even rather complicated javascript data into somewhat readable content in savegames. However, in Qt5 the Qt people removed the virtual filesystem, and that means we cannot use QSettings anymore. That leaves us with two alternatives - write our own ini file parser that does the same thing, or change the file format. Since we depend on serializing javascript to file for savegames, the only reasonable alternative is JSON. Qt5 has native JSON support (and, importantly, it can load JSON from a memory buffer).

The second reason is that once we start looking at more complicated data structures, .ini just falls flat. And it is nowhere as readable as JSON for serialized javascript content. We are running into issues when integrating .ini content with javascript already, where it would be really nice to have a more expressive data format (see discussion with crabster in another thread). I suspect .ini is a dying format, with less and less supporting tools and libraries, while JSON has tons of support in editors, tools and libraries.

I've already talked about this with the people I believe would be the ones would be the most affected at the moment, and I think we are in agreement that this is something that should be explored further.

The ini diff feature will not translate to the JSON world. However, if we just use dictionaries for all higher level stuff, then allowing a small JSON file to replace portions of a parent JSON file will be quite simple by just changing named properties directly.

Until we port over to Qt5, we can use QJson. However, I think we can port to Qt5 before 3.2 release.

My biggest worry is the lack of support in the Flame map editor, as nobody is maintaining that code anymore. (Anyone know any Visual Basic developers who want to help us?)

I've committed some tools to convert most of our existing INI files to JSON already, in master tools/conversion/ini2json/

I'll attach some preliminary JSON converted files to this post so that people can see what it might look like.

Feedback wanted.
Attachments
templates.json
(46.4 KiB) Downloaded 334 times
research.json
(171.52 KiB) Downloaded 298 times
construction.json
(557 Bytes) Downloaded 270 times

crab_
Trained
Trained
Posts: 349
Joined: 29 Jul 2013, 18:09

Re: Converting from INI to JSON

Post by crab_ » 01 Jan 2014, 01:29

We also use .ini files in maps. Do you plan to turn map files to json-format?

Can you upload all mp-stats files in JSON-format? I can test json-stats in my WZ Guide. It can show errors or inconsistence.

I think someone here can fix Flame and make it use json.
Do we have github repository for Flame?


[update]
I see you removed IDs of stuff.
I can expect IDs can be used in Flame and some other stuff. So i suggest to keep IDs (at least as field of object). I use IDs as filenames of icon-files in my Guide.

hmm... you removed IDs but did not changed references
"Heavy Cannon": {
"requiredResearch": [
"R-Wpn-Cannon-Damage05",
"R-Struc-Factory-Module",
"R-Vehicle-Metals03",
"R-Wpn-Cannon2Mk1"
],


"Heavy Cannon" has required research "R-Struc-Factory-Module"
but there is no more research with ID "R-Struc-Factory-Module". We do not have research with key "R-Struc-Factory-Module".
Same problem with other fields. IDs are still used in all properties of research objects.
Removing IDs is not easy task.
I'm afraid some IDs can have use in Warzone source. Need to check it.
Warzone2100 Guide - http://betaguide.wz2100.net/

User avatar
Terminator
Regular
Regular
Posts: 1066
Joined: 05 Aug 2006, 13:46
Location: Ukraine
Contact:

Re: Converting from INI to JSON

Post by Terminator » 01 Jan 2014, 02:03

JASON will be cool for future Master server protocol.
Death is the only way out... sh*t Happens !

Russian-speaking Social network Group http://vk.com/warzone2100

Per
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 3774
Joined: 03 Aug 2006, 19:39

Re: Converting from INI to JSON

Post by Per » 01 Jan 2014, 17:28

Pushed fix for IDs still being used as references. Also added 'id' field with old ID name for every item for backwards compatibility, as requested.

Attached zip of multiplayer stats as JSON files.

I am considering changing all the icon IDs into filename references as well, since it makes no sense to keep them as indirect references now that we have done away with storing icons as texture atlases on the filesystem. Currently the mapping from the 'icon ID' to icon is hard-coded in the source code.
Attachments
mp-jsons.zip
(35.08 KiB) Downloaded 288 times

crab_
Trained
Trained
Posts: 349
Joined: 29 Jul 2013, 18:09

Re: Converting from INI to JSON

Post by crab_ » 01 Jan 2014, 17:36

I have 1 question.

In 3.1 we have structure functions.
What happened with structure functiuons? For example, i want to change power icome of oil derrick.
In 3.1 Oil Derrick has "Resource" function attached.


Oh 1 more question, Can you please upload other files (like TerrainTable, features etc.)
In my Wz Guide i have following list of stats:

Weapons;
Bodies;
Propulsion;
Structures;
PropulsionModifiers;
StructureModifiers;
PropulsionType;
TerrainTable;
Researches;
Repair;
Construction;
ECM;
Sensor;
Features;
Templates;


[update]
per wrote:Also added 'id' field with old ID name for every item for backwards compatibility, as requested.
I do not see old ID.
I opened research.json and do not see field like "oldID". Same with weapons.json.
May be i'm wrong? I did not searched deep..

Another issue - properties are messed up.
Better have sorted properties.
In ini we had placed periodicalDamage stuff in a row.
But now we have following:

"muzzleGfx": "FXCan20m.PIE",
"rotate": 180,
"periodicalDamageRadius": 48,
"weaponEffect": "FLAMER",
"periodicalDamage": 28,


I suggest make properties sorted by meaning of properties or alphabetically
"periodicalDamage": 28,
"periodicalDamageRadius": 48,
"periodicalDamageTime": 50,
Warzone2100 Guide - http://betaguide.wz2100.net/

User avatar
Iluvalar
Regular
Regular
Posts: 1798
Joined: 02 Oct 2010, 18:44

Re: Converting from INI to JSON

Post by Iluvalar » 01 Jan 2014, 18:46

The ressource function from power derrick was meant to mesure the amount of oil it contain before it get depleted. The idea was thrown away.

There is no way to change the power from the derricks. (it would be great tough). You can still change the power output of the power generator and modules. not sure if it help ?
Heretic 2.3 improver and proud of it.

Per
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 3774
Joined: 03 Aug 2006, 19:39

Re: Converting from INI to JSON

Post by Per » 01 Jan 2014, 21:30

The old structure functions have been changed into fields - powerPoints, productionPoints, rearmPoints, repairPoints and researchPoints. This was done in the conversion to ini.

The old ID was in the field named 'id', but it was misplaced. Fixed. I've also added alphabetical sorting by the keys now.

I have not converted all the .ini files yet. The bodypropulsionimd.ini I plan to merge into body.json. The propulsionsounds.ini and propulsiontype.ini I plan to merge into one. I suppose we should also have a closer look at the others as well.

New zip attached.
Attachments
mp-jsons.zip
(37.93 KiB) Downloaded 238 times

cowboy
Rookie
Rookie
Posts: 29
Joined: 14 Jun 2010, 01:42
Contact:

Re: Converting from INI to JSON

Post by cowboy » 03 Jan 2014, 23:31

Working on a more modern version of FlaME ...

https://github.com/bchavez/SharpFlame

Source coming soon.

User avatar
Rman Virgil
Professional
Professional
Posts: 3812
Joined: 25 Sep 2006, 01:06
Location: USA

Re: Converting from INI to JSON

Post by Rman Virgil » 04 Jan 2014, 00:22

cowboy wrote:Working on a more modern version of FlaME ...

https://github.com/bchavez/SharpFlame

Source coming soon.
Fantastic ! 8) And whata New Years suprise, cowboy. :3

Cheers, R.J. XD
.
Last edited by Rman Virgil on 04 Jan 2014, 13:22, edited 1 time in total.

Per
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 3774
Joined: 03 Aug 2006, 19:39

Re: Converting from INI to JSON

Post by Per » 04 Jan 2014, 10:31

cowboy wrote:Working on a more modern version of FlaME ...
Oh, that would be really good news, indeed.

cowboy
Rookie
Rookie
Posts: 29
Joined: 14 Jun 2010, 01:42
Contact:

Re: Converting from INI to JSON

Post by cowboy » 05 Jan 2014, 03:58

Hi there Rman!

Great to see you. Yes, happy new years!

Finished porting FlaME from VB to C#. Now I can get to refactoring and modernizing the code.

viewtopic.php?f=6&t=11536

-cowboy

crab_
Trained
Trained
Posts: 349
Joined: 29 Jul 2013, 18:09

Re: Converting from INI to JSON

Post by crab_ » 08 Jan 2014, 15:37

Hello.
I have tried use new json-files in my WZ Guide
I found one issue.

research.json
-sometimes requredResearch is array.
-and sometimes requredResearch is string.

"AA Target Prediction Computer": {
"requiredResearch": [
"AA Target Acquisition Shells",
"Dedicated Synaptic Link Data Analysis Mk3"
],

"AA Chainfeed Loader Mk2": {
"requiredResearch": "AA Chainfeed Loader",


I think it is wrong.
For example. in templates.json we have
"weapons": [
"Super Pulse Laser Cyborg"
]

- array with one item. Field "weapons" is array always.

[update]
Found the same issue with field "resultComponents".

Per, can you just add brackets to these fields? I think each field should have one type.
"resultComponents": ["Avenger SAM"],
Warzone2100 Guide - http://betaguide.wz2100.net/

Per
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 3774
Joined: 03 Aug 2006, 19:39

Re: Converting from INI to JSON

Post by Per » 09 Jan 2014, 00:43

I thought I was clever when I made it (turning all multi-item entries into lists, leaving all others as non-list), since it looked much more neat that way. However, I realize now that this is just causing grief for whoever wants to use it. So I've changed it to always be lists for list type values.
Attachments
mp-json.zip
(37.92 KiB) Downloaded 232 times

crab_
Trained
Trained
Posts: 349
Joined: 29 Jul 2013, 18:09

Re: Converting from INI to JSON

Post by crab_ » 09 Jan 2014, 00:52

Thank you.

I almost finished work on turning my Guide to use JSON-files. (I think it will work, but i did not tried last version of json-files).
Can we discuss plans on other files (TerrainTable, Features, WeaponModifiers, PropulsionType)?
I'm afraid you want to change structure of these files.

[update]
Found we have weapons and research with same names.
So now its is impossible to determite "Hurricane AA Turret" is weapon or "Hurricane AA Turret" is research.
It is possible via old ID :)
nevermind.. it is not issue

[update2]
research,json - missing result structures
for example
"Scourge Missile Tower": {
"iconID": "IMAGE_RES_DEFENCE",
"id": "R-Defense-GuardTower-ATMiss",
"msgName": "RES_EMP_ATMTOW",
"redStructures": [
"Lancer Tower"
],
"requiredResearch": [
"Scourge Missile"
],
"researchPoints": 7200,
"researchPower": 225,
"statID": "Scourge Missile Tower"
},
Warzone2100 Guide - http://betaguide.wz2100.net/

Per
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 3774
Joined: 03 Aug 2006, 19:39

Re: Converting from INI to JSON

Post by Per » 09 Jan 2014, 01:22

Missing resultStructures added. I will have to look at the other files later, hopefully soon.
Attachments
mp-json.zip
(38.34 KiB) Downloaded 241 times

Post Reply