Exposing the research tree to JS API

[aubergine]

Summary of this post: JS API needs full detail on what a tech does (and to what it does it to), and what the research paths are

There are already a few JS API features that provide access to certain basic information about technologies in the research tree, however none of them allow an AI to gain enough information about a given technology or research path to make algorithmic decisions about what to research next... the human developer always has to pre-define the research path to some extent.

Until recently this has proved sufficient due to the relative stability of the WZ tech tree, however we now see a number of mods that drastically change the tech tree both in structure and in content:

* Contingency -- lots of new techs and very different paths
* Step research -- same techs, but very different paths
* CRS -- notion of a "meta tree" which creates "research eras"
* NRS -- new techs and different paths, as well as new concepts such as "loans" & "interest"

AI developers can simply choose to stick to the vanilla WZ tech tree and ignore the rest, however their AI will fail to deliver any meaningful results to increasing numbers of players trying out research mods.

Thus, we are already seeing the first signs of a trend to create "personalities" of AIs that are turned to specific research trees, albeit that the user must install and then choose the relevant AI for the research mod they are using to enable them -- AIs currently have no way of knowing what research tree is in effect.

As time goes on, it's going to be increasingly difficult for AI developers to keep up not only with changes in WZ itself and the ever evolving needs of their core AI code, but also the research mods that are both increasing in number and updates.

A research mod, unless it involves changing WZ source code, is by definition a number of txt/ini/whatever files that define a data structure that WZ can process. As such, that data must give indication as to what any particular tech will achieve (although I imagine there will be a few hard-coded "special cases" in there somewhere).

Thus, if more of the data about a given tech, and also the ability to inspect the research paths, is made available to the JS API, it would stand to reason that AI developers could create a more algorithmic approach to research.

I have no idea what sort of data there is for a given tech, or how that data defines what the tech will affect, or how research paths are defined. All those things are on my to-do list as I familiarise myself with the stats folder and various other WZ data files, and I'll document everything I learn along the way on the wiki.

However, there are obviously a growing number of people who understand the tech tree, and other stats, in great detail. Maybe there could be a gathering of minds to find some eloquent approach to exposing more comprehensive data to the JS API, which in turn would enable AI developers to make more adaptive scripts?

[Per]

I'd be happy to expose more data to javascript, but I really don't think we should be creating simulationist AIs that try to understand changes to the rules. I've been down that road before with another game AI, and that road is dark and full of horrors.

[aubergine]

It would make working with the existing tree easier as well IMHO. There are certain things where currently AI devs are forced to create big lists of techs to research. Having the extra research data would enable a lot of that stuff to be done more algorithmically, and if code is written in a transparent style it will be clear to see what the AI developer's intent was should others wish to branch or extend their work at a later date.

[Shadow Wolf TJC]

I've recently been experimenting with a modular tier-limiting script that's designed to limit which tiers of technology are available during matches:

Code: Select all

var tierLimit = 3;
include ("tierlimiter.js");
...
...
...
if (tierLimit >= 3) //Should we enable T3 technologies?
{
	completeResearch("R-T2Limiter",playnum);
}
if (tierLimit >= 2) //Should we enable T2 technologies?
{
	completeResearch("R-T1Limiter",playnum);
}

"R-T1Limiter" and "R-T2Limiter" are both custom research objects that do nothing other than to serve as a prerequisites for all T2 and T3 technology respectively. They cannot be researched, though as prerequisites to all T2 or T3 technology, they determine whether or not technologies from certain tiers can be researched.

tierlimiter.js is a custom JavaScript file that "needs" to be included in mods for the script to work. By default, it does nothing. However, other mods can overwrite this file with their own tierlimiter.js file. For example:

Code: Select all

if (tierLimit > 1)
{
	tierLimit = 1;
}

This would restrict the tech tree to just T1 technology.

[aubergine]

Yup, that sounds like what Illuvar's "research eras" in CRS mod is doing as well.

The pursueResearch() (and also findResearch() in 3.2) are useful for dealing with research paths. Both functions need to be given a target to reach though, and it's for this reason that having much more information about technologies and what they effect would be so beneficial for AI scripts.

From the perspective of an AI script, I want to be able to find out things like:

* What armour upgrades are available for a given propulsion category (eg. what will increase armour on a tank)
* What upgrades are available for a given weapon class, and what effect do they have

So, I'm looking for a way to determine what target to aim for. Once I know the target, pursueResearch() and findResearch() will help me deal with the research path to get there.

Unfortunately I've not yet done enough digging to know what all the possible effects of a technology are, or how to determine what will be effected. Otherwise I'd provide a much more concrete example of what I'm looking to achieve.

@Shadow: Do you have such a list, or know where I can get one? Specifically:

* What defines the scope of a technology (what things the tech applies to)?
* What are the possible effects of a technology within that scope?

[Shadow Wolf TJC]

Sorry. I don't know which list you're talking about, though if it's a research list other than the tech tree that we have in the guide section, then the answer would be no, sadly.

As for what technologies are defined as T1, T2, or T3, it could vary from person to person. Personally, I'd consider Dedicated Synaptic Research Data Analysis, and any other technologies that required it, as T2, and Neural Synapse Research Brain, and any other technologies that required it, as T3, but that's just me.

[aubergine]

In terms of the list, I don't know if any list exists, however I do know that in the various stats files there must be some way to define a) what a tech applies to and b) what the effect of the tech is to the things it applies to. It's that stuff I'd want exposing to the JS API.

In terms of tech level, I think having technologies that enable a tech level (like you and Iluvalar are working on, I'll refer to this as "tech era") would be a huge leap forward for numerous reasons...

Knowing what tech levels are available

If the tech era techs use a consistent ID both in standard WZ and any mods, eg. R-TechEra-T0, R-TechEra-T1, etc., then it's going to be clear in any research tree a) what tech levels are available and b) what their effect is.

Adapting UI to available tech levels

Knowing what tech levels are available could allow more dynamic UI for choosing tech level - eg. if there's no T3 in current tree, don't show T3 link. Also, the data associated with the tech era data has a name so the UI could show friendly captions like "T0 (Scavenger Era), T1 (Project Era)" etc.

Setting tech era for all players

Currently tech levels are applied via a mix of rules.js (which sets T0 to some extent) and monstrous cam2tech.vlo and cam3tech.vlo. Once we have tech eras, we can move all tech-level setting code in to rules.js and it will be trivial to do, for example:

Code: Select all

// helper function to complete research of a specific tech for all players (except scavs)
// used by applyTechLevel()
function giveTechToAllPlayers(tech) {
  var player = maxPlayers;
  while (-1<--player) completeResearch(tech,player);
}

// apply tech level to all players (except scavs)
function applyTechLevelToAllPlayers(customEra) {
  // assume a new techLevel global defines desired tech level for the game
  var techLevelDesired = customEra || techLevel; // what is desired tech level
  var techLevelData; // research object associated with that tech level

  do {

    // get the data for the tech era research object
    techLevelData = getResearch("R-TechEra-T" + TechLevelDesired);

    // I'm going pretend getResearch() returns null when research not found (IRL it throws an error - yuk!)
    // ...anyway, if tech not found, try a lower tech level
    if (!techLevelData) techLevelDesired -= 1;

  } while (techLevelDesired);

  // we now found closest match to desired tech level, and got data about it
  // so, let's set the tech era!

  // get all pre-requisite research...
  // note: findResearch() includes target tech in the array
  var requiredResearch = findResearch("R-TechEra-T"+techLevelDesired);

  // give the list of techs to all players
  requiredResearch.forEach(giveTechToAllPlayers);
}

// run:
applyTechLevelToAllPlayers();

// or at any time (eg. based on some game event), update tech era:
applyTechLevelToAllPlayers(4); // set T4 era

Determine current tech era for a player

Assuming that T0 always exists, and is pre-req for all higher tech levels scripts can easily find out what tech era a player is at:

Code: Select all

function getPlayerTechEra(player) {
  var era = 0; // start at T0
  var data;
  do {
    // note: getResearch() doesn't yet have a "player" param so that would need adding to JS API
    // ...and again I'm pretending getResearch() returns null if tech not found ;)
    data = getResearch("R-TechEra-T" + era, player);
    era += 1;
  } while (data && data.done);
  return --era;
}

Tech tree visualisations

Clearly defined tech eras could make visualisation of tech trees much better.

In particular, it could be mandated that T0 era is the sole root of every tech tree - a clear starting point for all code that processes the tree.

Deprecating old tech

It would be a lot cleaner to disable old techs when a new era is reached.

The list goes on and on, the concept is full of win IMHO

[karamel]

Just a few words, as a very quick fix isn't it possible to make something like a #define in mods for AI to see which tech tree is used?

This still requires modders to adapt AI for multiple tech trees but I far as I know modifying the tech tree affects the whole playing style and thus I don't feel it shocking that AIs are not adapted for this or this mod.

This could be a wonderful world if AI modders and tech/units modders worked hand in hand (AI modders makes the strategy decisions and tech/units modders adapt the AI to give units and researches pathes to do so) but it will work only with a limited amound of mods.


Anyway setting once for all which techs are T0, T1, T2 or T3 in a mod can be helpfull since these eras are a core feature (there is an unused column in research.txt that says Level-one, Level-two… wasn't that for it?).

[aubergine]

In my experiments with task driven AIs (none released yet as they don't do anything end-users would consider meaningful) I'm approaching a point where an AI should be able to adapt to whatever tech tree it is faced with.

So long as there are a few "known" facts, like techs that lead to a new tech level/era, and the ability to get detailed data about any tech (what does it do, and what gets affected by what it does) then the AI script will be more than capable of making good decisions. The initial coding effort is quite time consuming and intricate, however once it's done it can be wrapped in a js include file and easily used by other AI or scripts.

[NoQ]

This all won't make much sense without exposing all stats to JS API.

[Per]

This all won't make much sense without exposing all stats to JS API.

If we stored the stats as JSON, then exposing it all to the JS API would be trivial...

[aubergine]

I assume the stats are pulled from the txt/ini files in to some sort of C++ object - could that object be exposed to the JS env?

[Per]

I assume the stats are pulled from the txt/ini files in to some sort of C++ object - could that object be exposed to the JS env?

Yes... but it is a lot of boring work... and would quickly get out of sync if something changed in the C++ object and people forget to update the JS bridge...