Unit testing

For AI and campaign script related discussions and questions
Per
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 3780
Joined: 03 Aug 2006, 19:39

Re: Unit testing

Post by Per »

aubergine wrote:Really need a dump() function in JS API to make it easier to export test results!
Ok. Let's talk about that :)

Where should dump() dump its data, under what filename?

I'm somewhat paranoid about giving the scripting environment writing permissions to the file system. So I'd prefer that the c++ code is in command of filenames and filepaths.
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Unit testing

Post by aubergine »

Yes, I agree.

I think the file should always be created in the logs folder (the one that's neighbours with the mods folder and config file).

I'd like control over the filename, but not the extension. We don't want people creating .exe or .ini files on windows etc.

IMHO the file extension should be .dump or .txt.
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
dak180
Trained
Trained
Posts: 288
Joined: 01 Nov 2009, 23:58
Location: Keeper of the Mac Builds

Re: Unit testing

Post by dak180 »

Per wrote:Where should dump() dump its data, under what filename?

I'm somewhat paranoid about giving the scripting environment writing permissions to the file system. So I'd prefer that the c++ code is in command of filenames and filepaths.
aubergine wrote:Yes, I agree.

I think the file should always be created in the logs folder (the one that's neighbours with the mods folder and config file).

I'd like control over the filename, but not the extension. We don't want people creating .exe or .ini files on windows etc.

IMHO the file extension should be .dump or .txt.
.log seems like a better choice than .dump.

As for the name something based on the local date/time + the name of the calling script seems like it would do the job.
User:dak180
Keeper of the Mac Builds
Per
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 3780
Joined: 03 Aug 2006, 19:39

Re: Unit testing

Post by Per »

Implemented in 500c153a3f9344ece46e61939b30d35c26ea9d7e.

Log is cleared when script is started. Note that if you have multiple scripts with the same name, they log to the same file. Not sure if that is a good idea or not.
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Unit testing

Post by aubergine »

Woohoo! Will that make it in to 3.1 branch?

Test API v1.1 will be out later tonight, with massively improve module() functionality -- startup and teardown functions, module-scoped settings, etc. Docs for new module stuff here:

* Test.module(): https://warzone.atlassian.net/wiki/page ... d=16515312
* Module lifecycle: https://warzone.atlassian.net/wiki/disp ... cle+Object

The REQUIRE( ) signal has been upgraded to integrate with modules:

* REQUIRE( ): https://warzone.atlassian.net/wiki/page ... d=16515131

Also, there is much cleaning up of variables/properties available to unit test, startup and teardown functions -- still need to update docs for that.

I did some extensive code refactoring, by hand, so you can imagine how many mistakes I made. After dealing with a few compile-time errors, the API started running and it's unit tests started throwing up all sorts of failed assertions allowing me to rapidly track down my remaining mistakes. In short, the unit testing not only makes it easier to find where things are broken, it boosts confidence to make radical alterations to code because you know the unit tests (if they provide near-100% coverage) will find broken stuff quickly.
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
Per
Warzone 2100 Team Member
Warzone 2100 Team Member
Posts: 3780
Joined: 03 Aug 2006, 19:39

Re: Unit testing

Post by Per »

aubergine wrote:Woohoo! Will that make it in to 3.1 branch?
No.

I noticed that it is currently quite a resource hog. Apparently the physicsfs library does an implicit fsync() call on every log() now, which is ... not good. However, this made me notice a horrid bug in semperfi that probably would have gone unnoticed for a bit longer otherwise ;-)
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Unit testing

Post by aubergine »

Would it be possible to add it anyway? It's not something that's going to be used during normal gameplay, it's more to help with development if AI scripts, etc. Being able to dump stuff to a text file for later review will save lots of time, particularly as there is current no easy way to communicate with an AI in 3.1 branch (no chat features) so having a way to generate a blob of text and dump it to a file would help a great deal.
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Unit testing

Post by aubergine »

Here's v1.1...
TestDemo_v1.1.zip
Test API demo v1.1
(15.5 KiB) Downloaded 259 times
I've set the timer to start dumping results to console at 11 seconds instead of 15. As usual, change to player 1 to see their console. Unit tests updated to test the new module() and REQUIRE stuff.
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Unit testing

Post by aubergine »

Having multiple scripts output to same file is going to be problematic IMHO. Eg. If I'm in a game against 4 of my AIs, and I want to run tests or diagnostics, having all 4 of them dump to the same file is going to make things very confusing.

Could the player ID be included in filename? ... "logs/" + scriptName + "." + playerID + ".log"

Example: logs/EggPlant.3.log

Also, if the extension is going to be ".log" then it would probably be worth renaming dump() to log()
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Unit testing

Post by aubergine »

I've set up a git repo for my APIs: https://github.com/aubergine10/EggPlant

Still learning how to use git so would appreciate any feedback on what's uploaded so far.

Not quite sure how to get the local git folder moved in to my mods/autoload/EggPlant folder etc.
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
NoQ
Special
Special
Posts: 6226
Joined: 24 Dec 2009, 11:35
Location: /var/zone

Re: Unit testing

Post by NoQ »

Not quite sure how to get the local git folder moved in to my mods/autoload/EggPlant folder etc.
Add a zip command (that produces a playable mod) to your warzone shortcut, whatever it is called in your mac thing? (:
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Unit testing

Post by aubergine »

I'll have a go at making a shell script - apparently if you change their extension to .command they can be run from the GUI on a mac. Ideally I need to get updated to the latest version of OS X so I can use sexy Github GUI app.

Anyway, Test API v1.2 is now out, adding:

* Moddable output methods
* Test events

The config for the output methods, and default onFinish() event handler, can be found in the imaginatively named Config folder.

I've not yet worked out how to create a STABLE branch in git, but as soon as I do (lots of RTFM for me!) I'll get the Test API moved in to that branch.

My focus is now turning to getting test scripts written for the Util API so I can get that released. It has a dependency checker and lots of other useful stuff that all my other APIs rely on.
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
Duha
Trained
Trained
Posts: 287
Joined: 25 Mar 2012, 20:05
Location: SPb, Russia

Re: Unit testing

Post by Duha »

aubergine wrote:I've set up a git repo for my APIs: https://github.com/aubergine10/EggPlant

Still learning how to use git so would appreciate any feedback on what's uploaded so far.

Not quite sure how to get the local git folder moved in to my mods/autoload/EggPlant folder etc.
I dont like this test style. It is not simple. Manual counting tests is not pretty. IMHO test should be easy to write.

There is basic course about git Try git on codeacademy and github.

You can use simlink to put you code inside proper place. If git in git just .gitignore.
http://addons.wz2100.net/ developer
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Unit testing

Post by aubergine »

So what style of testing should I be using? Can't just come out saying "its not pretty" and then not mention the better alternative?

EDIT: If you're thinking of something along the lines of cucumber, I considered that but rejected it as this project has no corporate clients. As for BDD style testing, I'm already considering and mod to the Test API that will add BDD-style chaining, etc. If you had something else in mind, please enlighten me...
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
User avatar
aubergine
Professional
Professional
Posts: 3459
Joined: 10 Oct 2010, 00:58
Contact:

Re: Unit testing

Post by aubergine »

Did some digging in to symlinks and they are exactly what I was looking for - thanks for the tip Duha!!

Note to any other mac users trying this -- you need to use symlinks, not aliases.
"Dedicated to discovering Warzone artefacts, and sharing them freely for the benefit of the community."
-- https://warzone.atlassian.net/wiki/display/GO
Post Reply