Tag Archives: unit testing

Making TDD even more efficient

Kent Beck is still kind of working on unit testing tools — a couple of simple but great ideas that are baked into his re-launched JUnitMax:

  1. When running a test suite after a failure, give the most useful feedback first — by running the failures first before the rest of the suite
  2. Otherwise, run the fastest tests first. This gives you 99% of the feedback as fast as possible
  3. Alas, only currently available for Eclipse.

How to create an NUnit test project that is also a self-running console app .exe

  1. Create your NUnit Test project as a Windows Console Application, not as a Class Library.
  2. Then make your main Program look like this:
    [TestFixture]
    public class Program
    {
        static void Main(string[] args)
        {
            NUnit.ConsoleRunner.Runner.Main(
                new[]{Assembly.GetExecutingAssembly().Location }
                );
        }
    
        [TestCase("Aa - Bb")]
        public void WhenValidatingForename_should_accept_valid_characters(string validInput)
        {
            var result= new ClassUnderTest().Validate(validInput);
            Assert.IsTrue(result);
        }
    
        [TestCase("X<")]
        public void WhenValidatingForename_should_reject_invalid_characters(string invalidInput)
        {
            var result= new ClassUnderTest().Validate(invalidInput);
            Assert.IsFalse(result);
        }
    }
  3. Then, add a reference not only to nunit.framework.dll but also to nunit-console-runner.dll

You now have a self-running executable that runs all your tests, but still behaves in the usual way in a build script, or when running tests inside Visual Studio with TestRunner or Resharper or similar.
NB You may need to check if your build scripts are hard-coded to expect a Test project to be a .dll.

Software Quality — the Whole not the Parts

Ben Higginbottom makes a significant point right at the top of this question about the Knight Capital fiasco:

Was the Knight Capital fiasco related to Release Management? On August 1, 2012, Knight Capital Group had a very bad day, losing $440 million in forty-five minutes. More than two weeks later, there has been no official detailed explanation of …

Ben Higginbottom: Nightmare scenarios like this are not the result in the failing in any one discrete components that can be ‘fixed’ simply and sweetly by improving the process, but by small failures in multiple domains (I’m kind of reminded of a fatal accident investigation). So coding error, gap in the unit testing, weak end to end testing and poor post release testing coupled with a lack of operational checks when live. I can understand the desire for a ‘silver bullet’ fix, but in any complex system I’ve never known them to exist.

Sometimes, it’s the whole, not the parts, that needs fixing.

A javascript unit testing framework in 50 lines – plus some assertions

Surprised by a TDD session at XP-Man in which we rolled our own unit test framework from scratch – using the growing framework itself to test itself as it grew – I added some basic assertions to my 50 line javascript test runner.

I suggest that assertions should be thought of as separate to a test runner. Assertions are at the heart of – and should grow into – your domain specific testing language. So you should think of assretions as owned by your application, not the testing framework. As your application grows, so domain specific assertions should grow. A testing framework should probably provide minimal assertions for the language it runs on: the ‘primitive’ true/false/equal/notEqual assertions which will be used to define more domain specific assertions; and doing the legwork of tedious but common cases, such as testing equality by value of complex objects and collections.

In the case of javascript, the below might do. The tricky thing for assertions is to come up with something that is fluent and not verbose. C# extension methods are perfect for this. In javascript one could add methods to Object.prototype, but that doesn’t have the same scoping restriction as C# so feels too much like trampling all over everyone else’s garden: it’s poor namespacing and a failure of good manners.

/* Assertions for a test framework. http://www.cafe-encounter.net/p756/ */

function Assert(actual, assertionGender){
    this.actual=actual;
    this.notted=!assertionGender;
    this.Expected=  assertionGender ? "Expected" : "Did not expect";
}

function assert(actual){return new Assert(actual, true);}
function assertNot(actual){return new Assert(actual, false);}

Assert.prototype.IsTrue = function(msg){
    if( this.actual == this.notted ){ throw msg||this.Expected + " true.";}
};

Assert.prototype.Equals = function(expected,msg){
    if( (this.actual==expected) == this.notted ){
        throw msg|| this.Expected + " " + this.actual + " to equal " + expected ;
    }
};

Assert.prototype.EqualsByPropertyValue = function(expected,msg){
    var actualValue =  JSON.stringify( this.actual );
    var expectedValue = JSON.stringify( expected );
    if( (actualValue == expectedValue) == this.notted ){
        throw msg|| this.Expected + " " + actualValue + " to equal " + expectedValue + " by value";
    }
};

$(function(){
    new Specification("SpecificationsForPrimitiveAssertionPasses",{
        isTrueShouldBeTrueForTrue : function(){
            assert(true).IsTrue();
        },
        equalsShouldPassForEqualArguments : function(){
            assert( 1+1 ).Equals( 2 );
        },
        equalsByPropertyValueShouldPassForValueEqualArguments : function(){
            assert( { a: "a", b : function(){ return "b";}} )
                .EqualsByPropertyValue( { a: "a", b : function(){ return "b";}} );
        },
        assertNotShouldReverseTheOutcomeOfAnAssertion : function(){
            assertNot(false).IsTrue();
            assertNot(1+1).Equals(3);
            assertNot({ a: "a"}).EqualsByPropertyValue( { a: "b"});
        }
    }).runTests();

    new Specification("SpecificationsForPrimitiveAssertion_Failures",{
        isTrueShouldNotBeTrueForFalse : function(){
            assert(false).IsTrue();
        },
        equalsShouldFailForNonEqualArguments : function(){
            assert(1).Equals(2);
        },
        equalsByPropertyValueShouldFailForDifferingPropertyValues : function(){
            assert( { a: "a" } ).EqualsByPropertyValue( { a: "A" } );
        },
        equalsByPropertyValueShouldFailForDifferingProperties : function(){
            assert( { a: "a" } ).EqualsByPropertyValue( { aa: "a" } );
        },
        assertNotShouldReverseTheOutcomeOfIsTrue : function(){
            assertNot(true).IsTrue();
            assertNot({ a: "a"}).EqualsByPropertyValue( { a: "a"});
        },
        assertNotShouldReverseTheOutcomeOfEquals : function(){
            assertNot(1+1).Equals(2);
        },
        assertNotShouldReverseTheOutcomeOfEqualsByValue : function(){
            assertNot({ a: "a"}).EqualsByPropertyValue( { a: "a"});
        },
        assertTrueFailureWithCustomerMessageShouldOutputCustomMessage : function(){
            assert(false).IsTrue("This assertion failure should output this custom message");
        },
        assertEqualsFailureWithCustomerMessageShouldOutputCustomMessage : function(){
            assert(1).Equals(2, "This assertion failure should output this custom message");
        },
        assertEqualsByValueFailureWithCustomerMessageShouldOutputCustomMessage : function(){
            assert({a:"a"}).EqualsByPropertyValue( 1, "This assertion failure should output this custom message");
        }

    }).runTests();
});

A javascript unit testing framework in 50 lines

Last week at xp-man we did some TDD practise – by writing a tdd framework from scratch. The surprise was, how easy it was. So here, to my surprise, a very-bare-bones javascript unit test framework in 50 lines of code including the passing unit tests for the framework itself.

/*  www.cafe-encounter.net/p734/ */

function Specification( name, tests, outputContainer ){
    this.name= name;
    this.tests= tests;
    this.outputContainer= outputContainer || $('#testResults');
}

Specification.prototype = {
    name : 'UnnamedSpecification',
    tests : {},
    outputContainer : null,

    addMyselfToUI : function(){
        this.outputContainer.append( "<li><h3>" + this.name +"</h3><ul id='" + this.name + "'></ul></li>");
    },

    testDomId:function (test) {
        return this.name + "_" + test;
    },

    addTestToUI : function addTestToUI( test ){
        $('#' + this.name).append( "<li id='" + this.testDomId(test) + "'>" + test + "<input type='checkbox'/></li>");
    },

    markTestPassed : function( test ){
        $("#" + this.testDomId(test) + " input").prop("checked",true);
    },

    markTestFailed : function( test, ex ){
        $("#" + this.testDomId(test) + " input").prop("checked",false);
        $("#" + this.testDomId(test) + " input").after("<ul>Output:<li>" + ex.toString() + "</li></ul>");
    },

    runTest : function (test) {
        this.addTestToUI(test);
        try{
            this.tests[test]();
            this.markTestPassed(test)
        } catch(ex)  {
            this.markTestFailed(test, ex);
        }
    },

    runTests : function(){
        this.addMyselfToUI();

        for (var test in this.tests) {
            if (this.tests.hasOwnProperty(test)) this.runTest( test );
        }
    }
};

$(function(){
     new Specification("TestFrameworkSpecifications", {

        aTestShouldAddItselfInTheTestResults:function() {},

        aPassingTestShouldTickTheCheckbox : function() {},

        aFailingTestShouldUntickTheCheckbox:function () {
            throw "This thrown exception should appear in the output, indicating a failure";
        }
    }).runTests();
});

And the html doc to load and run it:

<html>
<head>
    <title>Minimal Javascript Test Framework</title>
    <script type="text/javascript" src="js/jquery-1.8.2.js"></script>
    <script type="text/javascript" src="js/wsl-testframework.js"></script>
</head>
<body>
<h1>Tests</h1>
<ul id="testResults">
</ul>
</body>
</html>

As written, I admit a dependency on jQuery, but you can replace that with document.write(). The obvious omission is a bundle of pre-defined assertions, but if you are developing a domain specific testing language then assertions should be part of that package, not part of the test running framework.

The obvious oddity is that the test suite demonstrates its test failure behaviour by failing a test; and you see that that is the correct result by inspecting the output.

This has led me to think that the TDD way to learn a new language is … to write a TDD framework for it.