Mr Andersson

Posts Tagged ‘http

Integration testing HTTP service caller using PostBin.org

leave a comment »

Problem: Post-Receive service hooks in GitHub are great. However, the lack all kind of flexibility chosing which commiters and/or branch of a repository the service hooks should be applied to. The other end, Hudson CI in this case, of the service hook call has the same problem.
Solution: As many other have, I started out to write a HTTP proxy which could provide some conditional logic for when to proceed with a Post-Receive call.

I started out driving the development of my proxy using TDD. My initial requirement was to NOT proceed with Post-Receive calls for certain commiters. So I started with the following failing end-to-end tests:

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsByIgnoredUserOnly_ThenTargetWillNotBeCalled() {}

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsWithAMixOfIgnoredAndNotIgnoredUsers_ThenTargetWillBeCalled() {}

After a while of hacking around I ended up with the following tests which tested my needs of the logic in my little service hook proxy:

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsByIgnoredUserOnly_ThenTargetWillNotBeCalled()
{
	var postReceiveHook = GivenATargetAndAnIgnoredCommiter();

	Payload payload = PayloadWithCommitsByIgnoredUserOnly();
	postReceiveHook.Post(payload);

	Assert.AreNotEqual( payload, _testTarget.Received );
}

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsWithAMixOfIgnoredAndNotIgnoredUsers_ThenTargetWillBeCalled()
{
	var postReceiveHook = GivenATargetAndAnIgnoredCommiter();

	Payload payload = PayloadWithCommitsWithAMixOfIgnoredAndNotIgnoredUsers();
	postReceiveHook.Post(payload);

	Assert.AreEqual( payload, _testTarget.Received );
}

But these tests, are they really end-to-end tests involving a Hudson CI instance?
Let’s take a look at the class behind the _testTarget field:

internal class TestTarget : IPostReceiveTarget
{
	public HttpStatusCode Call( Payload payload )
	{
		Received = payload;
		return HttpStatusCode.OK;
	}

	public Payload Received { get; private set; }
}

I don’t think so!
But do I want to bundle a complete Hudson CI instance with my NUnit test project? No! So where to draw the line?
Let’s try to find out what we actually want here.

  • Is it important to actually verify that the Hudson CI server can process the GitHub Post-Receive Service Hook JSON payload? Nah.
  • Is it important to verify that it can forward requests to different target URLs (Hudson jobs)? Nah.
  • Is it important to verify that it can receive a payload from GitHub and then resend the same payload to an arbitrary HTTP POST target? Yes!

Let’s take a look at the interface I wrote to separate the conditional logic from faking HTTP requests to Hudson CI in the previous tests:

public interface IPostReceiveTarget
{
	HttpStatusCode Call( Payload payload );
}

Right here I felt a bit bored because I started out writing the HTTP POST service call gateway, no tests involved.
Ended up with the following piece:

public class HttpPostTarget : IPostReceiveTarget 
{
	public HttpPostTarget( string targetUrl )
	{
		Url = targetUrl;
	}

	public HttpStatusCode Call( Payload payload )
	{
		var wc = new WebClient {Encoding = Encoding.UTF8};
		wc.Headers.Add( HttpRequestHeader.ContentType, "application/json" );
		try
		{
			wc.UploadString( Url, JsonConvert.SerializeObject( payload ) );
			return HttpStatusCode.OK;
		}
		catch ( WebException ex )
		{
			var response = ex.Response as HttpWebResponse;
			return response == null ? HttpStatusCode.BadRequest : response.StatusCode;
		}
	}

	public string Url { get; private set; }
}

So what do you say? This code can’t be tested? Well, I could go down the road and write a test which utilizes the HttpListener and verify the payload received by the listener in the test. Doable, yes indeed, but earlier today I was wasting some hour or two on getting the HttpListener to work with Windows Firewall (a.k.a. the “netsh” ceremony) which totally failed since I was hosting the code on a network share on my Mac and trying to fire up the listener on the Windows 7 Parallels VM….Aaarghh! Frustration! I just want the simplest possible thing that could work out of the box!

Suddenly I remembered a web hook debug thingy I stumpled upon a couple of months earlier:

  1. Go to www.postbin.org and make a PostBin
  2. Your created PostBin’s URL is shown
  3. Start making HTTP POST requests to that URL
  4. See the result at your PostBin’s URL or access the Atom feed for it (append /feed to the URL)

Excellent!
So what is the most LOC effective and least ugly integration test I can make out of this which tests my HTTP POST code?
Here is what I ended up with:

[TestFixture]
public class HttpPostTargetPostBinTests
{
	[Test]
	public void GivenAPayloadAndAPostBinBucket_WhenCallingTheTarget_ThenPostBinReturnsThePostedPayload()
	{
		const string postBinBucket = "http://www.postbin.org/93483c01";
		var target = new HttpPostTarget( postBinBucket );
		var commitId = new Random().Next().ToString();
		Payload samplePayload = SamplePayload( commitId );
		var status = target.Call( samplePayload );

		var content = GetFirstEntryContentOfAtomFeed( postBinBucket + "/feed" );

		Assert.AreEqual( HttpStatusCode.OK, status );
		StringAssert.Contains( """ + commitId + """, content );
	}

	static string GetFirstEntryContentOfAtomFeed( string atomUrl )
	{
		XNamespace atom = "http://www.w3.org/2005/Atom"; 
		return XDocument.Load( atomUrl ).Descendants( atom + "content" ).First().Value;
	}

	private Payload SamplePayload( string commitId )
	{
		return new Payload { Commits = new[] { new Commit { Id = commitId, Author = new Author { Email = "foo@bar.tm", Name = "Foo Bar" } } } };
	}
}

Someone else can probably make it prettier and more effective but this just works out of the box, without adding any external library references except for NUnit.

Conclusion:
It is very easy to get into trouble when writing tests for problems which are closely tied up with or depending on some technical concepts. In my case I was trying to get away as far as possible from a data format (JSON) and a network protocol (HTTP). A good start seems to be to make the abstractions where you hit these techie stuff. Think of that before you find yourself extending the problem context; Do you really need that infrastructure or external library to deal with your current design issues? Stay in context. When time comes and you need to hit the outer limit of the context, proceed as always: Make the simplest possible thing that could work.

Note:
PostBin is a web site made by Jeff Lindsay. You can fork his work on GitHub.

Advertisements

Written by anderssonjohan

July 19, 2011 at 09:20

Posted in programming

Tagged with , , , ,

Note to self: If you get error code 12250 from ISA server 2006…

leave a comment »

“Error Code: 403 Forbidden. ISA Server is configured to block HTTP requests that require authentication. (12250)”

Everything works from and to the protected networks but traffic from the external network is blocked with the error response above.
Solution: Stop using the darn “Publish web site” wizard and copy already existing stuff 🙂
-or- make sure that the “Allow client authentication over HTTP” check box is checked.
Found at the HTTP Listener properties – Authentication tab, click Advanced.

Written by anderssonjohan

November 17, 2009 at 07:30

Posted in troubleshooting

Tagged with ,