Unit testing

[aubergine]

Just sneaked out v1.3:

* added: simularTo( ) and notSimilarTo( ) assertions
* fixed: bug in Test.UNIT_TIMEOUT value

Right, back to Util API...

[Duha]

So what style of testing should I be using? Can't just come out saying "its not pretty" and then not mention the better alternative?

It just prototypy of sintax.

Code: Select all

In wz code

Test.run (test1)

test1 is nameof test function
function test1()
  function setup
  function tearDown
  function testThat 
  function dosomthing

For each function that starts with test you call setup function before and teardown.
Test function can contain many assertation and other things. If assert fails it just raise exception.

It is style of junit of java and unittest in python.
Each test class or test method can be launched separetly.

[aubergine]

Ok, I can probably implement something like that - do you have any links to examples of that style of test script?

Also, what do you think of BDD style test scripts, something like this: https://github.com/joshuaclayton/specit ... E.markdown

[Duha]

Ok, I can probably implement something like that - do you have any links to examples of that style of test script?

Also, what do you think of BDD style test scripts, something like this: https://github.com/joshuaclayton/specit ... E.markdown

Looks like you link is example.

In my python projects use unittest. I like it for easy syntax.
http://docs.python.org/2/library/unittest.html

[aubergine]

Just spotted huge blunder in assertions - the actual/expected params were wrong way round in all of them.

I've updated documentation but not yet uploaded v1.4 of the API - will upload new versions to git tomorrow.

Apologies for any pain caused.

[aubergine]

Test API v1.4 uploaded to git: https://github.com/aubergine10/EggPlant

Note the order of expected/actual params on all assertions has been changed to actual, expected (was expected, actual) -- if you've made any test scripts using this API, please update them. Sorry for the pain, but it's to keep the assertions in same format as QUnit where possible.

With the similarTo() and notSimilarTo() assertions, I've added some constants: Test.FOUND and Test.NOT_FOUND as detailed in the docs:

* https://warzone.atlassian.net/wiki/page ... d=16777272
* https://warzone.atlassian.net/wiki/page ... d=16777274

With regard to using the constants on the notSimilarTo() assertion, I'm keen to get feedback as to whether they "sound right" when used in that context?

For example, these tests will pass because actual value is not similar to expected value:

Code: Select all

notSimilarTo( {a:1, c:3}, {a:5}, "actual.a != 5" );
notSimilarTo( {a:1, c:3}, {b:Test.FOUND}, "actual.b != found" );
notSimilarTo( {a:5, b:1, c:3, d:2}, {d:Test.NOT_FOUND}, "actual.d != not found" );

I wasn't sure whether to invert the meaning of the constants when used in notSimilarTo() -- any comments?

I've left similarTo() and notSimilarTo() marked as "beta" in the docs for now, they are both tested and work, but I want to see if anyone has any better ideas about the constants before treating them as final.

[aubergine]

@Duha - could you take a look at this and let me know what you think? https://warzone.atlassian.net/wiki/disp ... +direction

[Per]

I have been thinking about making a unit testing map with its own javascript that would run various calls automatically, and then ask the user to do various things to trigger things that cannot easily be tested automatically. This map could be added to the distribution and be run as part of the commit changes to scripting / official release procedure.

If anyone want to have a go at this, it would be nice. I can add a hackAssert(condition, message...) function for it.

[aubergine]

Could that not be done with a challenge map and do the unit testing in JS?

What would the hackAssert() function do?

EDIT: BTW, I've been pondering something similar, perhaps even using audio prompts and console messages to guide the user through the testing process. I was also thinking similar technique could be used to create a series of tutorial maps...

[Duha]

@Duha - could you take a look at this and let me know what you think? https://warzone.atlassian.net/wiki/disp ... +direction

I was on vacations. mobile device is not good to inspect code.

I write some code as example of that I like.

Big peace of code just insert it to js test. (it can be run in any js enviroment just change "console" to other output.) (dont try to read it, coffescript interpretator writes cod not for reading )

Spoiler:

Code: Select all

(function() {
  var Assertation, Runner, SimpleTest, TestCase, assertationKeyword, res, test,
    __hasProp = {}.hasOwnProperty,
    __extends = function(child, parent) { for (var key in parent) { if (__hasProp.call(parent, key)) child[key] = parent[key]; } function ctor() { this.constructor = child; } ctor.prototype = parent.prototype; child.prototype = new ctor(); child.__super__ = parent.prototype; return child; };

  assertationKeyword = 'AssertationError:';

  Runner = (function() {

    function Runner(tests) {
      this.tests = tests;
      this.result = [];
    }

    Runner.prototype.run = function() {
      var test, _i, _len, _ref, _results;
      _ref = this.tests;
      _results = [];
      for (_i = 0, _len = _ref.length; _i < _len; _i++) {
        test = _ref[_i];
        _results.push(this.result.push(test.run()));
      }
      return _results;
    };

    Runner.prototype.getResultText = function() {
      var res, testResult, text, _i, _j, _len, _len1, _ref;
      testResult = [];
      _ref = this.result;
      for (_i = 0, _len = _ref.length; _i < _len; _i++) {
        res = _ref[_i];
        for (_j = 0, _len1 = res.length; _j < _len1; _j++) {
          text = res[_j];
          testResult.push("" + text["class"] + "." + text.test + " " + text.result + " " + text.error);
        }
      }
      return testResult.join('\n');
    };

    Runner.prototype.toString = function() {
      return 'Testrunner';
    };

    return Runner;

  })();

  Assertation = (function() {

    function Assertation() {}

    Assertation.prototype.assert = function(val) {
      if (!val) {
        throw "" + assertationKeyword + " assertation of " + val + " fail";
      }
    };

    return Assertation;

  })();

  TestCase = (function(_super) {

    __extends(TestCase, _super);

    function TestCase(_name) {
      var key, value;
      this._name = _name != null ? _name : 'unnamed test';
      this._tests = [];
      for (key in this) {
        value = this[key];
        if (key.slice(0, 4) === 'test') {
          this._tests.push(key);
        }
      }
      this.result = [];
    }

    TestCase.prototype.preTest = function() {};

    TestCase.prototype.postTest = function() {};

    TestCase.prototype.run = function() {
      var key, _i, _len, _ref;
      _ref = this._tests;
      for (_i = 0, _len = _ref.length; _i < _len; _i++) {
        key = _ref[_i];
        this.preTest();
        this.result.push(this.doTest(key));
        this.postTest();
      }
      return this.result;
    };

    TestCase.prototype.doTest = function(key) {
      var msg;
      try {
        this[key]();
      } catch (error) {
        msg = error.toString();
        if (msg.slice(0, (assertationKeyword.length - 1) + 1 || 9e9) === assertationKeyword) {
          return {
            'result': 'fail',
            'class': this._name,
            'test': key,
            'error': msg.slice(assertationKeyword.length)
          };
        } else {
          return {
            'result': 'error',
            'class': this._name,
            'test': key,
            'error': msg
          };
        }
      }
      return {
        'result': 'pass',
        'class': this._name,
        'test': key,
        'error': ''
      };
    };

    return TestCase;

  })(Assertation);

  SimpleTest = (function(_super) {

    __extends(SimpleTest, _super);

    function SimpleTest() {
      return SimpleTest.__super__.constructor.apply(this, arguments);
    }

    SimpleTest.prototype.testFoo = function() {
      return this.assert(1);
    };

    SimpleTest.prototype.testNegative = function() {
      return this.assert(0);
    };

    return SimpleTest;

  })(TestCase);

  test = new SimpleTest('zzzzz');

  res = new Runner([test]);

  res.run();

  console(res.getResultText());

}).call(this);

Sample of the test

Spoiler:

Code: Select all

// test class can be writted in more pretty way
SimpleTest = (function(_super) {

    __extends(SimpleTest, _super);

    function SimpleTest() {
      return SimpleTest.__super__.constructor.apply(this, arguments);
    }

   // test method. It can be more pretty too, return is not needed. It exception catched  test failed or errored, else passed
   SimpleTest.prototype.testFoo = function() {
    // you can put any code and any number of assertation to here.  
    return this.assert(1);
    };

    SimpleTest.prototype.testNegative = function() {
      return this.assert(0);
    };

    return SimpleTest;

  })(TestCase);

  // this part should be integrated. For example search all instances of TestCase in code and use them.  
  test = new SimpleTest('zzzzz');
  res = new Runner([test]);
  
  // running tests
  res.run();
  // printing test output
  console(res.getResultText());

coffescript

Spoiler:

Code: Select all

assertationKeyword = 'AssertationError:'

class Runner
  constructor: (@tests) ->
    @result = []

  run: () ->
    for test in @tests
      @result.push test.run()

  getResultText: () ->
    testResult = []
    for res in @result
      for text in res
        testResult.push("#{text.class}.#{text.test} #{text.result} #{text.error}")
    testResult.join('\n')

  toString: ()->
    'Testrunner'

class Assertation
  assert: (val) ->
    if !val
      throw  "#{assertationKeyword} assertation of #{val} fail"


class TestCase extends Assertation
  constructor: (@_name='unnamed test')->
    @_tests = []
    for key, value of @
      if key[..3] == 'test'
        @_tests.push(key)
    @result  = []



  preTest: ->
  postTest: ->

  run: () ->
    for key in @_tests
      @preTest()
      @result.push @doTest(key)
      @postTest()
    return @result

  doTest: (key) ->
    try
      @[key]()
    catch error
      msg = error.toString()
      if msg[..assertationKeyword.length-1] == assertationKeyword
        return {'result': 'fail', 'class': @_name, 'test': key, 'error': msg[assertationKeyword.length..] }
      else
        return {'result': 'error', 'class': @_name, 'test': key, 'error': msg }
    return {'result': 'pass', 'class': @_name, 'test': key, 'error': '' }





class SimpleTest extends TestCase
  testFoo: () ->
    @assert 1
  testNegative: () ->
    @assert 0



test = new SimpleTest('zzzzz')
res =  new Runner([test])
res.run()
console res.getResultText()

[aubergine]

And what would happen if you had lots of test scripts and wanted to split them out in to separate .js files? Variables like __extends, etc., would need to be put on to the global scope, unless you propose that all tests are put in to one massive .js file?

Anyway, I can see the type of syntax & extensibility you are looking for and will add it to my list of future updates to my Test API. My current focus is just on getting more of my APIs released with testing under the current Test API framework.

[Duha]

And what would happen if you had lots of test scripts and wanted to split them out in to separate .js files? Variables like __extends, etc., would need to be put on to the global scope, unless you propose that all tests are put in to one massive .js file?

You just need to import all test classes to you scope.

I this is manual way, but it is should not be used.

Code: Select all

test1 = new RobotTest('robot');
test2 = new TruckTest('truck');
res = new Runner([test1, test2]);

Normal way should be lookes something like that:

Code: Select all

wz.exe -test // all test
wz.exe -test RobotTest // all test in class
wz.exe -test RobotTest.testOne //current test

and get output to console. In that case autotest will be real autotests. And can be integrated to build system.

And manual running from code somthing like that:

Code: Select all

test.run()
test.run('RobotTest')
test.run('RobotTest.testOne')