Script interface as dll/so interface

[Omnivore]

Hello, not sure whether this should go in coding, scripting, suggestion, or here so moderators please relocate as necessary.

I've been looking at the trunk migration from wzscript to javascript and, while I'm glad to see the move, I'd like to suggest separating out the interface between the core and the scripts into a dll/so interface. This would allow for a wzscript dll/so, a javascript dll/so, a <insert script language name here> dll/so, as well as allow scripts to include external libs for ai purposes like a fuzzy logic interference engine, a portal to a common community wide genetic algorithm history, a neural net library, etc.

I'd think it would also help in debugging and documentation, being a single point of C language code providing the linkage for all scripts regardless of particular language.

I'm willing to pitch in on this, though I'd need quite a bit of warzone specific assistance as I'm new to the codebase.

[Per]

This would seriously complicate the scripting interface, and we already have way too much work ahead of us on that front with too few developers. I'm sorry, but I do not think that is a good idea.

Perhaps if you described what you want to do, we could discuss another way of achieving it?

[cybersphinx]

Also, this seems like a good way to break scripts when a needed library is not installed (and maybe not even available for a certain platform, because nobody compiled it).

[Omnivore]

Thank you gentlemen for your quick and courteous response.

I was looking into the possibility of doing a "Hammer's Slammers" mod which would require the ability to extend the core and also looking to get back into a bit of AI programming possibly using some external libs I wrote back in the early 90's. Of current scripting languages, I'm most comfortable overall with Python for this type of application though I, of course, understand it isn't a universal choice.

After further review of the source code of trunk and Illuvar's 2.3.8 based fork, I believe I'll celebrate my 50th birthday by toying around with a new project derived from Illuvar's fork once I add in sufficient unit tests. I'm medically retired but have to scratch that programming itch, 30+ years of practice won't let me stop entirely, lol.

I hope you all won't mind if I pose a question here from time to time.

Thanks again.

[vexed]

Thank you gentlemen for your quick and courteous response.

I was looking into the possibility of doing a "Hammer's Slammers" mod which would require the ability to extend the core and also looking to get back into a bit of AI programming possibly using some external libs I wrote back in the early 90's. Of current scripting languages, I'm most comfortable overall with Python for this type of application though I, of course, understand it isn't a universal choice.

After further review of the source code of trunk and Illuvar's 2.3.8 based fork, I believe I'll celebrate my 50th birthday by toying around with a new project derived from Illuvar's fork once I add in sufficient unit tests. I'm medically retired but have to scratch that programming itch, 30+ years of practice won't let me stop entirely, lol.

I hope you all won't mind if I pose a question here from time to time.

Thanks again.

Greetings.

I am curious as why you would wish to work with any of the 2.3 codebases, since, as was mentioned we are in the process of improving the scripting system.
The old scripting system is very unforgiving, in both the script side, and the backend.

In response to your original question, having a shared library approach does have benefits, but, you also then run into version hell, not to mention all the other things needed to make sure you are using the correct shared library while loading a save game. There are lots of stuff that warzone does as direct memory dumps, and it isn't pretty.
It still would be a very interesting (although difficult) project though.

A side note, trunk was svn, and if you are still using that, then that is the old codebase. We have moved to git, where the old trunk is now called master.

[Omnivore]

I cloned both master (sorry for calling it trunk) and Illuvar's 2.3 fork from git, sorry for any confusion. As to why 2.3 over master, for what I have in mind with the Hammer's Slammers mod, I'll need to make changes pretty deep into the core and I have a small bit of experience with SDL. Not that I'm by any stretch a graphics programmer, haven't done anything serious since the late 90's, just I'd be more comfortable hacking SDL than OpenGL based code if <shudder> the need arises.

Versioning problems with DLLs has reached urban myth proportions, yes they can easily come to exist if you ignore the possibilities, but it is possible to avoid them. First off, the DLL/SO interface specification would be under control of master/core code. The standard script interface would use, presumably, the javascript engine under development. That engine would be shipped with the core, more or less part and parcel of master though sitting just outside the fence.

Using convention over configuration where possible, let the script's extension specify the engine: .js would use the standard javascript engine by default. Part of the interface specification should include a discovery method whereby the script engine DLL/SO being loaded can specify its standard extension(s). A mod could override this convention by specifying a particular engine to be used with its scripts, but the mod is then responsible for supplying the engine dll and any required external libs. The same goes for extension libraries used by a mod. In other words, the responsibility for version control is in the hands of the modder not the core.

Avoiding versioning problems with the core script interface is rather straightforward, always add, never remove. Any additions must behave reasonably in the presence of older script versions (iow defaults). Save games could, and should, include script engine and mod specification including version. If an attempt is made to load a save game for which the required script engine/mod is unavailable, the user should be notified.

The gain from all of this is not only greatly enhanced flexibility for mods, but it opens up the possibility for much higher reliability on all sides given that standard unit tests (and necessary mocks) can be implemented on both sides of the interface. Now granted, you'd not expect the average modder to utilize unit tests but the advanced modder could, and indeed should. As for the core, unit tests and regression tests need to become mandatory for all but UI/graphics output else the bug pit grows ever deeper and less resolvable as time progresses.

Even aside from the script dll interface idea, more unit and regression testing would be a really great idea. Yeah I'm a 'test first' fanatic (TDD/BDD) but I used to hate the idea of writing tests always leaving them until last and half-baked. Having worked on one maintenance nightmare after another though, I finally saw the light lol. Personally I find I'm far more productive in the long run by describing the desired behavior and writing the tests first before writing 'the real code'.

Oops, sorry 'bout the rant!

[Per]

2.3 and master contain pretty much identical amounts of OpenGL code. So that should not be an argument. We do not use SDL for any actual drawing.

If you want to use the javascript interface, you need to work on master, since this is not present in 2.3, and will be very hard to backport.