Mr Andersson

Posts Tagged ‘testing

Integration testing HTTP service caller using PostBin.org

leave a comment »

Problem: Post-Receive service hooks in GitHub are great. However, the lack all kind of flexibility chosing which commiters and/or branch of a repository the service hooks should be applied to. The other end, Hudson CI in this case, of the service hook call has the same problem.
Solution: As many other have, I started out to write a HTTP proxy which could provide some conditional logic for when to proceed with a Post-Receive call.

I started out driving the development of my proxy using TDD. My initial requirement was to NOT proceed with Post-Receive calls for certain commiters. So I started with the following failing end-to-end tests:

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsByIgnoredUserOnly_ThenTargetWillNotBeCalled() {}

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsWithAMixOfIgnoredAndNotIgnoredUsers_ThenTargetWillBeCalled() {}

After a while of hacking around I ended up with the following tests which tested my needs of the logic in my little service hook proxy:

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsByIgnoredUserOnly_ThenTargetWillNotBeCalled()
{
	var postReceiveHook = GivenATargetAndAnIgnoredCommiter();

	Payload payload = PayloadWithCommitsByIgnoredUserOnly();
	postReceiveHook.Post(payload);

	Assert.AreNotEqual( payload, _testTarget.Received );
}

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsWithAMixOfIgnoredAndNotIgnoredUsers_ThenTargetWillBeCalled()
{
	var postReceiveHook = GivenATargetAndAnIgnoredCommiter();

	Payload payload = PayloadWithCommitsWithAMixOfIgnoredAndNotIgnoredUsers();
	postReceiveHook.Post(payload);

	Assert.AreEqual( payload, _testTarget.Received );
}

But these tests, are they really end-to-end tests involving a Hudson CI instance?
Let’s take a look at the class behind the _testTarget field:

internal class TestTarget : IPostReceiveTarget
{
	public HttpStatusCode Call( Payload payload )
	{
		Received = payload;
		return HttpStatusCode.OK;
	}

	public Payload Received { get; private set; }
}

I don’t think so!
But do I want to bundle a complete Hudson CI instance with my NUnit test project? No! So where to draw the line?
Let’s try to find out what we actually want here.

  • Is it important to actually verify that the Hudson CI server can process the GitHub Post-Receive Service Hook JSON payload? Nah.
  • Is it important to verify that it can forward requests to different target URLs (Hudson jobs)? Nah.
  • Is it important to verify that it can receive a payload from GitHub and then resend the same payload to an arbitrary HTTP POST target? Yes!

Let’s take a look at the interface I wrote to separate the conditional logic from faking HTTP requests to Hudson CI in the previous tests:

public interface IPostReceiveTarget
{
	HttpStatusCode Call( Payload payload );
}

Right here I felt a bit bored because I started out writing the HTTP POST service call gateway, no tests involved.
Ended up with the following piece:

public class HttpPostTarget : IPostReceiveTarget 
{
	public HttpPostTarget( string targetUrl )
	{
		Url = targetUrl;
	}

	public HttpStatusCode Call( Payload payload )
	{
		var wc = new WebClient {Encoding = Encoding.UTF8};
		wc.Headers.Add( HttpRequestHeader.ContentType, "application/json" );
		try
		{
			wc.UploadString( Url, JsonConvert.SerializeObject( payload ) );
			return HttpStatusCode.OK;
		}
		catch ( WebException ex )
		{
			var response = ex.Response as HttpWebResponse;
			return response == null ? HttpStatusCode.BadRequest : response.StatusCode;
		}
	}

	public string Url { get; private set; }
}

So what do you say? This code can’t be tested? Well, I could go down the road and write a test which utilizes the HttpListener and verify the payload received by the listener in the test. Doable, yes indeed, but earlier today I was wasting some hour or two on getting the HttpListener to work with Windows Firewall (a.k.a. the “netsh” ceremony) which totally failed since I was hosting the code on a network share on my Mac and trying to fire up the listener on the Windows 7 Parallels VM….Aaarghh! Frustration! I just want the simplest possible thing that could work out of the box!

Suddenly I remembered a web hook debug thingy I stumpled upon a couple of months earlier:

  1. Go to www.postbin.org and make a PostBin
  2. Your created PostBin’s URL is shown
  3. Start making HTTP POST requests to that URL
  4. See the result at your PostBin’s URL or access the Atom feed for it (append /feed to the URL)

Excellent!
So what is the most LOC effective and least ugly integration test I can make out of this which tests my HTTP POST code?
Here is what I ended up with:

[TestFixture]
public class HttpPostTargetPostBinTests
{
	[Test]
	public void GivenAPayloadAndAPostBinBucket_WhenCallingTheTarget_ThenPostBinReturnsThePostedPayload()
	{
		const string postBinBucket = "http://www.postbin.org/93483c01";
		var target = new HttpPostTarget( postBinBucket );
		var commitId = new Random().Next().ToString();
		Payload samplePayload = SamplePayload( commitId );
		var status = target.Call( samplePayload );

		var content = GetFirstEntryContentOfAtomFeed( postBinBucket + "/feed" );

		Assert.AreEqual( HttpStatusCode.OK, status );
		StringAssert.Contains( """ + commitId + """, content );
	}

	static string GetFirstEntryContentOfAtomFeed( string atomUrl )
	{
		XNamespace atom = "http://www.w3.org/2005/Atom"; 
		return XDocument.Load( atomUrl ).Descendants( atom + "content" ).First().Value;
	}

	private Payload SamplePayload( string commitId )
	{
		return new Payload { Commits = new[] { new Commit { Id = commitId, Author = new Author { Email = "foo@bar.tm", Name = "Foo Bar" } } } };
	}
}

Someone else can probably make it prettier and more effective but this just works out of the box, without adding any external library references except for NUnit.

Conclusion:
It is very easy to get into trouble when writing tests for problems which are closely tied up with or depending on some technical concepts. In my case I was trying to get away as far as possible from a data format (JSON) and a network protocol (HTTP). A good start seems to be to make the abstractions where you hit these techie stuff. Think of that before you find yourself extending the problem context; Do you really need that infrastructure or external library to deal with your current design issues? Stay in context. When time comes and you need to hit the outer limit of the context, proceed as always: Make the simplest possible thing that could work.

Note:
PostBin is a web site made by Jeff Lindsay. You can fork his work on GitHub.

Written by anderssonjohan

July 19, 2011 at 09:20

Posted in programming

Tagged with , , , ,

Swedish Alt.Net UG Coding Dojo at Avega, Stockholm

with 2 comments

Yesterday I was at my very first Coding Dojo with my fellow collegues Morten and Sebastian. I was somewhat nervous before this event. Coding in group is not an every day occasion for me :). But it went very well and I can only concur on what Morten blogged about.

Adding to that I must say it were so popular that we needed to split up into two conference rooms. Each room was equipped with a projector, a decent table with room enough for about 15 people.
However, even though we split up into two teams (that focused on the same Kata, see link in Morten’s post), we were just too many people around the table. I’m not sure about the exact head count but I guess we ended up with about 9 or 10 people. The most disappointing thing was that one guy was “rolling his own private solution” on his machine during the session. No hard feelings though; I believe this is a sign that we had too many participants in each group.
Worth to say is that each rotation was time limited to 4 minutes with a break of 15 seconds. That means each 4 minutes you rotate around the table and the next person takes the keyboard and continues where the last person left off. The person to the left of the person at the keyboard is the navigator in the pair.
Really nice experience and very nice and bright people in the user group! Thanks Avega (especially Joakim Sundén and his collegues) for hosting the meeting at their office here in Stockholm.
Today I couldn’t resist thinking of how we could do this internally at work (we’re 6 persons writing code at RemoteX so it seems to be just too perfect). Oh, so much cool code to write, so little time. 🙂

Written by anderssonjohan

November 11, 2009 at 23:01

Posted in events, programming

Tagged with , , ,

Pimping PsUnit with constraint based assertions

leave a comment »

Using PsUnit one can set up scripted “unit tests” for PowerShell scripts. I have sort of just started using PsUnit at work and right now I feel that a lot of the basic features of other test frameworks are missing.

PsUnit has one function for doing constraint based assertions. This may be familar to those who are using NUnit. Basically the syntax for this is:
<![CDATA[
$result = DoSomething
Assert-That $result { $ActualValue -eq “expected value” }
]]>

Using NUnit and this constraint based model, a simple assertion can look like this:
<![CDATA[
Assert.That( myString, Is.EqualTo(“Hello”) );
]]>

I quite don’t like the “barebone” scriptblock you need to write to use Assert-That in PsUnit, so here is what I came up with (using my example of above):
<![CDATA[
$result = DoSomething
Assert-That $result (IsEqualTo “expected value”)
]]>

I also added another constraint which allows you to assert that some code throws something:
<![CDATA[
# The following assertion will always pass
Assert-That { throw “blah!” } (ThrowsException)
]]>
This complements the already built-in feature of PsUnit which allows you to add a parameter, $ExpectedException, to the test. If you have been using MSTest you would use the method attribute ExpectedException for this.

Here is another example which tweaks the constraint to a somewhat more specific type of exception:
<![CDATA[
# The following assertion will fail since throw in PowerShell creates a System.Management.Automation.ErrorRecord
Assert-That { throw “blah!” } (ThrowsException -Exception $([System.ArgumentException]))
]]>

A more usable test using the ThrowsException constraint could be:
<![CDATA[
Assert-That {
# The following web resource does not exist, thus an error should be the result
Get-Http “http://blogger.com/asdfg&#8221;
} (ThrowsException -Exception $([System.Net.WebException]))
]]>

The biggest improvement using these new constraints is the output in the test result.
While using PsUnit out-of-the-box…
<![CDATA[
$foo = “bar”
Assert-That $foo { $ActualValue -eq “foo” }
]]>
…results with “Assert-That returned false!” in the test result.
Using the constraint in this blog post…:
<![CDATA[
$foo = “bar”
Assert-That $foo (IsEqualTo “foo”)
]]>
…results with “ActualValue ‘bar’ is not equal to expected value ‘foo’“.

The same level of detail is provided by the ThrowsException constraint, which may result in “No exception was thrown, expected exception of type ‘System.ArgumentException’“.

Any ideas, suggestions etc.? Please don’t hesitate to drop a comment!

Written by anderssonjohan

November 11, 2009 at 20:41

Posted in scripting

Tagged with ,

Integrating PsUnit and MSBuild

leave a comment »

The last month I have been digging into, adding new and refactoring the Powershell scripts we use in RemoteX Applications. We primarily use PowerShell to configure and deploy our product.

More often or too many times – with our latest releases – our fellow collegues at Customer Operations have been telling us “The xyz powershell script doesn’t work anymore, what have you done to it???”
I could have answered: “- We don’t have automated tests on those scripts, so what do you expect? Manual testing? Pfffff…”, but for obvious reasons I didn’t :).
So it was time to add some automated tests for those scripts so I could get this guy at Customer Operations back to work.
All compiled code (C#) is automatically tested in our Continuous Integration builds when someone checks in, but until now we haven’t had automated tests for our PowerShell scripts. With “automated” I mean that a test runner executes a test suite during a build.
In order to test our PowerShell scripts I have started using PsUnit. Since we are using Team Foundation Server and MSBuild for our CI builds, I needed to integrate PsUnit with MSBuild somehow.
There are several ways to invoke Powershell from an MSBuild script – we have chosen the Exec task which invokes powershell.exe.
First of all I created a separate build target which invokes PsUnit:

<![CDATA[
<PropertyGroup>
<CoreTestDependsOn>$(CoreTestDependsOn);RunPsUnit</CoreTestDependsOn>
<RunPsUnitDependsOn>CoreRunPsUnit</RunPsUnitDependsOn>
</PropertyGroup>

<Target Name=”RunPsUnit” DependsOnTargets=”$(RunPsUnitDependsOn)”/>
<Target Name=”CoreRunPsUnit” Condition=” ‘$(RunTest)’!=’false’ “>
<Exec Command=”$(PowerShell) -Command &quot;&amp;{ .\runpsunit.ps1 -PsUnitTestFile %(PsUnitTest.FullPath) }&quot;”/>
</Target>
]]>
Using this batch target, all that needs to be done is to add PsUnitTest items to the build script. Just like the way we set up TestContainer items for use with MSTest:
<![CDATA[
<ItemGroup>
<PsUnitTest Include=”$(SolutionRoot)\Scripts\Tests\*.Test.ps1″/>
<PsUnitTest Include=”$(SolutionRoot)\References\PsUnit\*.Test.ps1″/>
</ItemGroup>
]]>
So far so good, but when tests are failing, the build is successful. This is because the Exec task only reports a build error when the executed command returns an exit code other than zero (0x0). So I started looking into the test runner of PsUnit (PsUnit.Run.ps1) and discovered that it didn’t send any output nor produced some other exit code than 0x0. The only output PsUnit will send you is some nice colorful information written to the PowerShell host.

So I ended up with the following:
1. this tiny tweak at the end of PsUnit.Run.ps1:

<![CDATA[
$TestResults
]]>
This makes PsUnit.Run.ps1 send the test results along the pipeline.

2. A small wrapper for the test runner which returns some exit code other than 0x0 to the Exec task in MSBuild:

<![CDATA[
$results = Invoke-Expression “$psUnitRun -PsUnitTestFile $PsUnitTestFile”
$results | % {
if( $_.Result -eq “FAIL” ) {
throw “Test run failed: $($_.Test) – $($_.Reason)”
}
}
]]>
This makes the Exec task report a build error if we find any failing test in the test result from PsUnit – and last but not least – the name of the failing test and the reason ends up nicely in the MSBuild log file.

So now when somebody checks in a change to compiled code, which affects the result of our setup scripts, we will be able to detect errors ASAP! Yay!

Written by anderssonjohan

November 11, 2009 at 19:50

Posted in scripting

Tagged with , ,

Switching component implementations in Castle Windsor

leave a comment »

Problem: Test suite for a system for which I need to replace (inject) the behavior of a certain component. The system is using Castle Windsor with an XML configuration file.

In this system integration test I want to use the real container configuration except that I want to switch out a component that does authorization using a database.

The replacement for the real component is as simple as:


public bool IUsernamePasswordVerifier.VerifyPasswordFor( string username, string password )
{
return true;
}
Solution: Tried this, which failed on me. But with a little help from a tiny facility used by the test suite I can now “re-wire” service implementations in the container.
There are probably several other solutions on how one could accomplish the same thing, but this one worked out very well for me.

ComponentRewriterFacility.cs:


using System;
using System.Collections.Generic;
using Castle.Core;
using Castle.Core.Configuration;
using Castle.MicroKernel;

namespace ServiceTests.Service
{
public class ComponentRewriteFacility : IFacility
{
readonly IDictionary<Type, Type> _rewrites;

public ComponentRewriteFacility()
{
_rewrites = new Dictionary<Type, Type>();
}

public void AddRewrite<I, T>() where T : I
{
_rewrites.Add( typeof(I), typeof(T) );
}

public virtual void Init( IKernel kernel, IConfiguration facilityConfig )
{
if( kernel == null )
throw new ArgumentNullException( "kernel" );
kernel.ComponentRegistered += ComponentRegistered;
}

public void Terminate()
{
}

void ComponentRegistered( string key, IHandler handler )
{
if( !ShouldRewrite( handler.ComponentModel ) )
return;
handler.ComponentModel.Implementation = _rewrites[handler.ComponentModel.Service];
}

protected virtual bool ShouldRewrite( ComponentModel componentModel )
{
return _rewrites.ContainsKey( componentModel.Service );
}
}
}

And in our test suite initializer we add the “re-write rule” like this:


IWindsorContainer container = new WindsorContainer();
container.AddFacility<ComponentRewriteFacility>( f => f.AddRewrite<IUsernamePasswordVerifier, AnythingGoesUsernamePasswordVerifier>() );
container.AddFacility<WcfFacility>();
container.Install( Castle.Windsor.Installer.Configuration.FromXmlFile( "Windsor.xml" ) );

Like it? Not? Have a better solution? Please let me know!

Written by anderssonjohan

October 28, 2009 at 19:34

Posted in programming

Tagged with , , , ,

Helping young hackers understand the importance of software testing?

leave a comment »

I recently saw this fun picture over here:

(and I thought it would be fun for others as well – yes, I know this would probably be classified as nerd humor)

Written by anderssonjohan

October 14, 2008 at 21:07

Posted in misc

Tagged with , ,