Need some data mining?

For code related discussions and questions
User avatar
Duha
Trained
Trained
Posts: 287
Joined: 25 Mar 2012, 20:05
Location: SPb, Russia

Need some data mining?

Post by Duha »

Hello, I am make tool for editing stats. (viewtopic.php?f=5&t=9260)
I have easy API to data mining. And visualization tools.

Issues found:
ZNULLWEAPON has WEAPORN_CLASS: MISC (both base and mp)

This is obsolete variant. Should be HEAT (As I understand from code)
Spoiler:
http://addons.wz2100.net/ developer
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Need some data mining?

Post by aubergine »

Would you be able to create a spreadsheet that contains all the research items, and their effects, etc. from the data you have?

Getting a "single source of truth" has been a major blocker for AI developers - specifically, at the moment we need to hard-code things like research paths. I can't, for example, say "Make the armour better on tanks" without hard-coding a research item.

I started a manual process to try and achieve a single source of truth but it was too laborious and very unreliable:

https://warzone.atlassian.net/wiki/disp ... preadsheet

You'll note from that spreadsheet that I was also trying to categorise research items, for example PATH_AAGUN is associated with anything that will improve gun-based anti-aircraft weapons, which in turn would allow my AI script to quickly identify all research in that line.

I started writing some notes on the sort of info I'd need:

https://warzone.atlassian.net/wiki/disp ... et+columns

But again, it was all taking too long and there were far too many unknowns. One of the other big things missing was a column that defines prerequisite research for each item - it seems the online research guide is out of date (maybe wz 2.10?) which further hindered progress That thwarted my attempts to create a tech tree in XML format:

https://warzone.atlassian.net/wiki/disp ... +Tech+Tree (the .xlogic files are XML representations of the tech tree).

If you could help generate any of those things it would be a massive leap forward for AI developers because we could use that info to create JSON objects which in turn would allow AIs to make much more dynamic algorithms for doing research based on observed gamestate (rather than always requiring the developer to try and define too much what should happen).
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
Duha
Trained
Trained
Posts: 287
Joined: 25 Mar 2012, 20:05
Location: SPb, Russia

Re: Need some data mining?

Post by Duha »

aubergine wrote:....
If you could help generate any of those things it would be a massive leap forward for AI developers because we could use that info to create JSON objects which in turn would allow AIs to make much more dynamic algorithms for doing research based on observed gamestate (rather than always requiring the developer to try and define too much what should happen).
Lets do it.
First step is to make data loader.

Can you write short description for all files in research folder?
prresearch.txt
researchfunctions.txt
resultcomponent.txt
redcomponents.txt
researchstruct.txt
resultstructure.txt
redstructure.txt
research.txt
http://addons.wz2100.net/ developer
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Need some data mining?

Post by aubergine »

Is this what you need?

http://developer.wz2100.net/wiki/TxtEditing

Other than that I don't really know what half the fields mean, which was one of the main blockers to me getting anything done.
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Need some data mining?

Post by aubergine »

Also, this topic might be better moved to the coding forum?
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
Reg312
Regular
Regular
Posts: 681
Joined: 25 Mar 2011, 18:36

Re: Need some data mining?

Post by Reg312 »

aubergine wrote: Other than that I don't really know what half the fields mean, which was one of the main blockers to me getting anything done.
you can see in WZ source all you need
hm, i think i can make more detailed description of stat fields, but they all can be changed in any moment (for example body.txt was moved to body.ini and unused/old fields ware removed)
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Need some data mining?

Post by aubergine »

I imagine that there won't be too many changes from one version to the next - it's fairly rare that these files are changed I would assume?
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
Duha
Trained
Trained
Posts: 287
Joined: 25 Mar 2012, 20:05
Location: SPb, Russia

Re: Need some data mining?

Post by Duha »

Reg312 wrote:
aubergine wrote: Other than that I don't really know what half the fields mean, which was one of the main blockers to me getting anything done.
you can see in WZ source all you need
hm, i think i can make more detailed description of stat fields, but they all can be changed in any moment (for example body.txt was moved to body.ini and unused/old fields ware removed)
Code does not get some answears:

I can understand what field in this fiels for. But I dont understand meaning of this files.
resultcomponent.txt
redcomponents.txt
researchstruct.txt
redstructure.txt
http://addons.wz2100.net/ developer
User avatar
Staff
Global Moderator
Global Moderator
Posts: 387
Joined: 27 May 2011, 04:08

Re: Need some data mining?

Post by Staff »

aubergine wrote:Also, this topic might be better moved to the coding forum?
Yes, it should. Moved.
Sorry, this account has been deactivated.
Please post issues in Website feedback instead, only Moderators can read things there.
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Need some data mining?

Post by aubergine »

Not sure if this is any use, but found a link in my bookmarks with loads of info on various setting files: viewtopic.php?f=5&t=9118&p=98362#p98356

EDIT: That post explains what many of the files are (the ones you were asking about above).
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
Duha
Trained
Trained
Posts: 287
Joined: 25 Mar 2012, 20:05
Location: SPb, Russia

Re: Need some data mining?

Post by Duha »

aubergine wrote:Not sure if this is any use, but found a link in my bookmarks with loads of info on various setting files: viewtopic.php?f=5&t=9118&p=98362#p98356
Thanks.
http://addons.wz2100.net/ developer
User avatar
Duha
Trained
Trained
Posts: 287
Joined: 25 Mar 2012, 20:05
Location: SPb, Russia

Re: Need some data mining?

Post by Duha »

1. Looks like that data in research.txt includes data result_component and result_structure.

Research ID,unused,Subgroup icon,Major tech,Main icon,IMD,IMD2,MsgName,StructName,CompName,CompType,ResearchPoints,keyTopic,unused,unused,unused,unused,unused,unused,unused

StructName is duplicated in resultstructure
CompName,CompType is duplicated in resultcomponent

Are this field should be unused too?

2.
And can you explain how game find out if first string is headers or real data? base/cam1 and mp/multiplayer differs.
http://addons.wz2100.net/ developer
User avatar
Iluvalar
Regular
Regular
Posts: 1828
Joined: 02 Oct 2010, 18:44

Re: Need some data mining?

Post by Iluvalar »

StructName,CompName,CompType are only a graphical informations about how to depict the research.
Heretic 2.3 improver and proud of it.
User avatar
Duha
Trained
Trained
Posts: 287
Joined: 25 Mar 2012, 20:05
Location: SPb, Russia

Re: Need some data mining?

Post by Duha »

Iluvalar wrote:StructName,CompName,CompType are only a graphical informations about how to depict the research.
Thanks.

There I can find this images in game? It is inside PIE or somethere else?
http://addons.wz2100.net/ developer
User avatar
Iluvalar
Regular
Regular
Posts: 1828
Joined: 02 Oct 2010, 18:44

Re: Need some data mining?

Post by Iluvalar »

Yeah if, there is a structure name, it will depict that, with his structure pie, his base pie and any weapon or sensor already on it.
If it's a component, it will do the same. mount + nuzzle + base. for weapons.
That's why it's useful, because different component can have more than one pie simultaneous :).
Heretic 2.3 improver and proud of it.
Post Reply