Mr Andersson

Posts Tagged ‘c#

Integration testing HTTP service caller using PostBin.org

leave a comment »

Problem: Post-Receive service hooks in GitHub are great. However, the lack all kind of flexibility chosing which commiters and/or branch of a repository the service hooks should be applied to. The other end, Hudson CI in this case, of the service hook call has the same problem.
Solution: As many other have, I started out to write a HTTP proxy which could provide some conditional logic for when to proceed with a Post-Receive call.

I started out driving the development of my proxy using TDD. My initial requirement was to NOT proceed with Post-Receive calls for certain commiters. So I started with the following failing end-to-end tests:

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsByIgnoredUserOnly_ThenTargetWillNotBeCalled() {}

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsWithAMixOfIgnoredAndNotIgnoredUsers_ThenTargetWillBeCalled() {}

After a while of hacking around I ended up with the following tests which tested my needs of the logic in my little service hook proxy:

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsByIgnoredUserOnly_ThenTargetWillNotBeCalled()
{
	var postReceiveHook = GivenATargetAndAnIgnoredCommiter();

	Payload payload = PayloadWithCommitsByIgnoredUserOnly();
	postReceiveHook.Post(payload);

	Assert.AreNotEqual( payload, _testTarget.Received );
}

[Test]
public void GivenATargetAndAnIgnoredCommiter_WhenPostingPayloadWithCommitsWithAMixOfIgnoredAndNotIgnoredUsers_ThenTargetWillBeCalled()
{
	var postReceiveHook = GivenATargetAndAnIgnoredCommiter();

	Payload payload = PayloadWithCommitsWithAMixOfIgnoredAndNotIgnoredUsers();
	postReceiveHook.Post(payload);

	Assert.AreEqual( payload, _testTarget.Received );
}

But these tests, are they really end-to-end tests involving a Hudson CI instance?
Let’s take a look at the class behind the _testTarget field:

internal class TestTarget : IPostReceiveTarget
{
	public HttpStatusCode Call( Payload payload )
	{
		Received = payload;
		return HttpStatusCode.OK;
	}

	public Payload Received { get; private set; }
}

I don’t think so!
But do I want to bundle a complete Hudson CI instance with my NUnit test project? No! So where to draw the line?
Let’s try to find out what we actually want here.

  • Is it important to actually verify that the Hudson CI server can process the GitHub Post-Receive Service Hook JSON payload? Nah.
  • Is it important to verify that it can forward requests to different target URLs (Hudson jobs)? Nah.
  • Is it important to verify that it can receive a payload from GitHub and then resend the same payload to an arbitrary HTTP POST target? Yes!

Let’s take a look at the interface I wrote to separate the conditional logic from faking HTTP requests to Hudson CI in the previous tests:

public interface IPostReceiveTarget
{
	HttpStatusCode Call( Payload payload );
}

Right here I felt a bit bored because I started out writing the HTTP POST service call gateway, no tests involved.
Ended up with the following piece:

public class HttpPostTarget : IPostReceiveTarget 
{
	public HttpPostTarget( string targetUrl )
	{
		Url = targetUrl;
	}

	public HttpStatusCode Call( Payload payload )
	{
		var wc = new WebClient {Encoding = Encoding.UTF8};
		wc.Headers.Add( HttpRequestHeader.ContentType, "application/json" );
		try
		{
			wc.UploadString( Url, JsonConvert.SerializeObject( payload ) );
			return HttpStatusCode.OK;
		}
		catch ( WebException ex )
		{
			var response = ex.Response as HttpWebResponse;
			return response == null ? HttpStatusCode.BadRequest : response.StatusCode;
		}
	}

	public string Url { get; private set; }
}

So what do you say? This code can’t be tested? Well, I could go down the road and write a test which utilizes the HttpListener and verify the payload received by the listener in the test. Doable, yes indeed, but earlier today I was wasting some hour or two on getting the HttpListener to work with Windows Firewall (a.k.a. the “netsh” ceremony) which totally failed since I was hosting the code on a network share on my Mac and trying to fire up the listener on the Windows 7 Parallels VM….Aaarghh! Frustration! I just want the simplest possible thing that could work out of the box!

Suddenly I remembered a web hook debug thingy I stumpled upon a couple of months earlier:

  1. Go to www.postbin.org and make a PostBin
  2. Your created PostBin’s URL is shown
  3. Start making HTTP POST requests to that URL
  4. See the result at your PostBin’s URL or access the Atom feed for it (append /feed to the URL)

Excellent!
So what is the most LOC effective and least ugly integration test I can make out of this which tests my HTTP POST code?
Here is what I ended up with:

[TestFixture]
public class HttpPostTargetPostBinTests
{
	[Test]
	public void GivenAPayloadAndAPostBinBucket_WhenCallingTheTarget_ThenPostBinReturnsThePostedPayload()
	{
		const string postBinBucket = "http://www.postbin.org/93483c01";
		var target = new HttpPostTarget( postBinBucket );
		var commitId = new Random().Next().ToString();
		Payload samplePayload = SamplePayload( commitId );
		var status = target.Call( samplePayload );

		var content = GetFirstEntryContentOfAtomFeed( postBinBucket + "/feed" );

		Assert.AreEqual( HttpStatusCode.OK, status );
		StringAssert.Contains( """ + commitId + """, content );
	}

	static string GetFirstEntryContentOfAtomFeed( string atomUrl )
	{
		XNamespace atom = "http://www.w3.org/2005/Atom"; 
		return XDocument.Load( atomUrl ).Descendants( atom + "content" ).First().Value;
	}

	private Payload SamplePayload( string commitId )
	{
		return new Payload { Commits = new[] { new Commit { Id = commitId, Author = new Author { Email = "foo@bar.tm", Name = "Foo Bar" } } } };
	}
}

Someone else can probably make it prettier and more effective but this just works out of the box, without adding any external library references except for NUnit.

Conclusion:
It is very easy to get into trouble when writing tests for problems which are closely tied up with or depending on some technical concepts. In my case I was trying to get away as far as possible from a data format (JSON) and a network protocol (HTTP). A good start seems to be to make the abstractions where you hit these techie stuff. Think of that before you find yourself extending the problem context; Do you really need that infrastructure or external library to deal with your current design issues? Stay in context. When time comes and you need to hit the outer limit of the context, proceed as always: Make the simplest possible thing that could work.

Note:
PostBin is a web site made by Jeff Lindsay. You can fork his work on GitHub.

Advertisements

Written by anderssonjohan

July 19, 2011 at 09:20

Posted in programming

Tagged with , , , ,

WordWrap function in c#

leave a comment »

Today I worked on a piece of code at RemoteX and needed a pretty simple word wrapping function in C# which:
1) breaks between words, if possible
2) breaks words, if they’re too long to fit on one line

I used the almighty code snippet resource but couldn’t find a quick fix for my need without digging further into the different solutions, so here is my contribution to the long list of word-wrapping functions posted around the web.
Just like I tried to use several of the code snippets I found on the web, you are free to use mine.

I take no responsibility for any bugs you may find in this code snippet may include. Please let the tests guide you!

https://github.com/anderssonjohan/snippets/blob/master/wordwrap/WordWrapTests.cs

Written by anderssonjohan

November 3, 2010 at 12:51

Posted in programming

Tagged with ,

Switching component implementations in Castle Windsor

leave a comment »

Problem: Test suite for a system for which I need to replace (inject) the behavior of a certain component. The system is using Castle Windsor with an XML configuration file.

In this system integration test I want to use the real container configuration except that I want to switch out a component that does authorization using a database.

The replacement for the real component is as simple as:


public bool IUsernamePasswordVerifier.VerifyPasswordFor( string username, string password )
{
return true;
}
Solution: Tried this, which failed on me. But with a little help from a tiny facility used by the test suite I can now “re-wire” service implementations in the container.
There are probably several other solutions on how one could accomplish the same thing, but this one worked out very well for me.

ComponentRewriterFacility.cs:


using System;
using System.Collections.Generic;
using Castle.Core;
using Castle.Core.Configuration;
using Castle.MicroKernel;

namespace ServiceTests.Service
{
public class ComponentRewriteFacility : IFacility
{
readonly IDictionary<Type, Type> _rewrites;

public ComponentRewriteFacility()
{
_rewrites = new Dictionary<Type, Type>();
}

public void AddRewrite<I, T>() where T : I
{
_rewrites.Add( typeof(I), typeof(T) );
}

public virtual void Init( IKernel kernel, IConfiguration facilityConfig )
{
if( kernel == null )
throw new ArgumentNullException( "kernel" );
kernel.ComponentRegistered += ComponentRegistered;
}

public void Terminate()
{
}

void ComponentRegistered( string key, IHandler handler )
{
if( !ShouldRewrite( handler.ComponentModel ) )
return;
handler.ComponentModel.Implementation = _rewrites[handler.ComponentModel.Service];
}

protected virtual bool ShouldRewrite( ComponentModel componentModel )
{
return _rewrites.ContainsKey( componentModel.Service );
}
}
}

And in our test suite initializer we add the “re-write rule” like this:


IWindsorContainer container = new WindsorContainer();
container.AddFacility<ComponentRewriteFacility>( f => f.AddRewrite<IUsernamePasswordVerifier, AnythingGoesUsernamePasswordVerifier>() );
container.AddFacility<WcfFacility>();
container.Install( Castle.Windsor.Installer.Configuration.FromXmlFile( "Windsor.xml" ) );

Like it? Not? Have a better solution? Please let me know!

Written by anderssonjohan

October 28, 2009 at 19:34

Posted in programming

Tagged with , , , ,

A fluent interface and the Builder pattern

leave a comment »

My former collegue Thomas Lundström blogged about fluent interfaces quite a while ago, which he followed up in a comment to Johan Lindfors’ post “A small DSL to order coffee” (in Swedish).

The blog post and its comments discusses two different approaches to fluent interfaces. The first approach is based on getters which mutates the instance when they are called. Some people think this is a very bad design – getters should not mutate the instance!
I agree on that. I strongly believe that a get operation should be idempotent and not mutate the object instance. This for the same reason a HTTP GET request should not mutate a resource and a HTTP POST/PUT should mutate a collection/the resource.
(That’s why I prefer RESTful web services more than SOAP over HTTP, but that’s a totally different blog post. 🙂

The second approach is the one that Thomas points out. It’s the classical Builder Pattern where you have a totally separate class which is used to build object instances. It differentiates itself among the creational OOD design patterns by targeting primarily mutable objects. It’s like a bunch of Factory methods which is used like a Factory, but instead of having thousands of different methods in one Factory (overloads for example), the Builder is used in several calls to create/configure the final object instance.

An approach which was recently used at work is a mix of the two approaches mentioned above. Instead of having a separate builder we have builder methods on the object instance itself, either directly or via extension methods. Separate from the builder methods we got some Factory Methods. These are the starting point to create a basic instance. The builder methods are then used to further shape the object instance to our needs.

Approach #1 – getters which creates/mutate the object instace:


Coffee myLatte = Coffee.OrderLatte.Double.FatFree.Vanilla;

Approach #2 – a separate builder


Coffee myLatte = CoffeeBuilder.Latte().Double().FatFree().Vanilla().Build();

Our approach – a mix of factory methods and builder methods on the object instance itself


Coffee myLatte = Coffee.Latte().Double().FatFree().Vanilla();

So what’s the pros and cons with the last two approaches?
I guess they are quite similar, but here is some pros:
– The separate builder seems a bit more aligned with the Separation Of Concerns principle
– The built-in builder methods via extension methods provides a lightweight class structure, even for classes in a third-party class library, such as the BCL.

(Please post a comment if you can think of more pros and cons or if you just totally disagree with me 🙂

So of course, the $10.000 question answer is… It depends on your context!
Anyhow, here is a sample how we tend to use this mix of Factory Methods and Builder methods at work.

The following code is used to build construct the collection of DataGridViewColumns used in a Windows Forms DataGridView control.
The CreateColumns() method is used to create an enumeration of the columns to use and TextColumn() is a sample Factory Method.


protected override IEnumerable CreateColumns()
{
yield return TextColumn()
.For( WorkOrderPresenter.Properties.Title )
.WeightedWidth( 100 );

yield return TextColumn()
.For( WorkOrderPresenter.Properties.Description )
.WeightedWidth( 200 );

yield return ComboBoxColumn( _activityBindingSource )
.For( WorkOrderPresenter.Properties.Activity )
.WeightedWidth( 100 )
.Width( 90, 130 );
....
}

///
/// Creates a textbox column with default behavior.
///

protected virtual DataGridViewTextBoxColumn TextColumn()
{
return new DataGridViewTextBoxColumn().WrapContents().WithDefaultBehavior();
}

The methods For(), WeightedWidth(), Width(), WrapContents() and WithDefaultBehavior() are Builder (extension) methods on DataGridViewColumn.

Written by anderssonjohan

March 12, 2009 at 18:03

Tail call recursion with C#

with one comment

An e-mail landed in my mailbox at work. It was a discussion which ended up in the following statement by my collegue Morten:
– In theory you could also write that method using tail call recursion and in that way avoid eating up space on the call stack“.”

One of the brilliant guys I have the opportunity to work with is Morten. If you would put him in a movie, he would definitely play the role of the crazy scientist! He’s a guy which I love to discuss computer science with, when time allows for it.
So this time he made me dig deeper into how one could practially make our little recursive method “tail call recursive”.

I started out with two questions:
1) What the h-ck is tail call recursion? (no, I have no degree in CS)
2) Why is the temperature minus 26 degress celsius here in Rättvik, Sweden which makes me dig deeper into question 1 and prevents me and the kids go out dig deeper/play in the snow?

IF you now would try to do tail call recursion with our recursive method, would it be possible with C#?
Well, sure it is POSSIBLE. But it requires a bit more than Ctrl-Shift-B in Visual Studio. As we all know, in C#, we have little control of the native code being executed. For example we have the JIT compilator, which depends on which platform you are on. Code can also be inlined, something which now also applies to value types on the x86 platform (introduced in .NET 3.5 SP1). But except the jitter and the inlining we also have the seems-to-be-spot-on-target-at-first-look MSIL tail prefix

Looking at a stack trace convering a call to a recursive method we can find the typical characteristics of a non-tail call recursive method:


at WaitForRunningThreads()
at WaitForRunningThreads()
at WaitForRunningThreads()
at WaitForRunningThreads()
at WaitForRunningThreads()
at WaitForRunningThreads()

By using tail call recursion it would look something like:


at WaitForRunningThreads()

Since there is no way I can make a tail call recursive method in C# produce such a stack trace by modifying the C# code, I need to patch the Release-compiled (using Visual Studio) IL code to make it use the MSIL tail prefix.

Suppose we have the following program:


class Program
{
static void Main()
{
WaitForRunningThreads( 42 );
}

static void WaitForRunningThreads( int n )
{
System.Console.WriteLine( n );
System.Threading.Thread.Sleep( 100 );
if ( n < 2 )
throw new System.Exception();
WaitForRunningThreads( n / 2 );
}
}

With the following IL code output generated by csc.exe:


.class private auto ansi beforefieldinit Program
extends [mscorlib]System.Object
{
.method private hidebysig static void Main() cil managed
{
.entrypoint
// Code size 8 (0x8)
.maxstack 8
IL_0000: ldc.i4.s 42
IL_0002: call void Program::WaitForRunningThreads(int32)
IL_0007: ret
} // end of method Program::Main

.method private hidebysig static void WaitForRunningThreads(int32 n) cil managed
{
// Code size 32 (0x20)
.maxstack 8
IL_0000: ldarg.0
IL_0001: call void [mscorlib]System.Console::WriteLine(int32)
IL_0006: ldc.i4.s 100
IL_0008: call void [mscorlib]System.Threading.Thread::Sleep(int32)
IL_000d: ldarg.0
IL_000e: ldc.i4.2
IL_000f: bge.s IL_0017

IL_0011: newobj instance void [mscorlib]System.Exception::.ctor()
IL_0016: throw

IL_0017: ldarg.0
IL_0018: ldc.i4.2
IL_0019: div
IL_001a: call void Program::WaitForRunningThreads(int32)
IL_001f: ret
} // end of method Program::WaitForRunningThreads

.method public hidebysig specialname rtspecialname
instance void .ctor() cil managed
{
// Code size 7 (0x7)
.maxstack 8
IL_0000: ldarg.0
IL_0001: call instance void [mscorlib]System.Object::.ctor()
IL_0006: ret
} // end of method Program::.ctor
}

When we execute the program it produces the following output:


42
21
10
5
2
1

Unhandled Exception: System.Exception: Exception of type 'System.Exception' was thrown.
at Program.WaitForRunningThreads(Int32 n) in Program.cs:line 13
at Program.WaitForRunningThreads(Int32 n) in Program.cs:line 14
at Program.WaitForRunningThreads(Int32 n) in Program.cs:line 14
at Program.WaitForRunningThreads(Int32 n) in Program.cs:line 14
at Program.WaitForRunningThreads(Int32 n) in Program.cs:line 14
at Program.WaitForRunningThreads(Int32 n) in Program.cs:line 14
at Program.Main() in Program.cs:line 5

As mentioned before, we have no option giving the C# compiler a hint nor optimization option which will make it emit the MSIL tail prefix. What we can do for sure is patching the IL code using ILDASM/ILASM to include the tail prefix in WaitForRunningThreads():


.method private hidebysig static void WaitForRunningThreads(int32 n) cil managed
{
// Code size 33 (0x21)
.maxstack 8
IL_0000: ldarg.0
IL_0001: call void [mscorlib]System.Console::WriteLine(int32)
IL_0006: ldc.i4.s 100
IL_0008: call void [mscorlib]System.Threading.Thread::Sleep(int32)
IL_000d: ldarg.0
IL_000e: ldc.i4.2
IL_000f: bge.s IL_0017

IL_0011: newobj instance void [mscorlib]System.Exception::.ctor()
IL_0016: throw

IL_0017: ldarg.0
IL_0018: ldc.i4.2
IL_0019: div
IL_001a: tail.
IL_001b: call void Program::WaitForRunningThreads(int32)
IL_0020: ret
}

The following is executed from a Visual Studio Command Prompt in the output directory:

> ildasm RTail.exe /out=rtail.il
> copy rtail.il rtail_modified.il
> notepad rtail_modified.il

(add the tail prefix as illustrated above)

> ilasm rtail_modified.il /out:rtail_modified.exe

..and the results are:

> rtail_modified.exe
42
21
10
5
2
1

Unhandled Exception: System.Exception: Exception of type 'System.Exception' was thrown.
at Program.WaitForRunningThreads(Int32 n)
at Program.Main()

Voila!
The MSIL tail prefix pops the call stack BEFORE the call instruction and makes our stack trace look like we are using tail call recursion. Nice, huh?

But, if we dig deeper into the differencies of the platform dependent JIT compilators we can find that this is done by the jitter on 64-bit platforms:

In fact, on 64-bit platforms, the CLR tries to do tail calls even if the tail. prefix is not specified. This is because platform-independent IL programs will get the same amount of stack space reserved for them by the OS, even though the 64-bit process will consume stack at a faster rate. Hence, we try to reduce stack usage by doing tail calls.

Which part do I not get here?

Why can’t the C# compilator do this optimization?

Why rely on the JIT compilator to do such an optimization?

Luke Hoban (nowadays on the F# team) gives me some answers in a feedback response:

1) There is actually a non-trivial overhead cost to using the .tail instruction in the CLR (it is not just a jump instruction as tail calls ultimately become in many less strict environments such as functional language runtime environments where tail calls are heavily optimized).

2) There are few real C# methods where it would be legal to emit tail calls (other languages encourage coding patterns which have more tail recursion, and many that rely heavily on tail call optimization actually do global re-writing (such as Continuation Passing transformations) to increase the amount of tail recursion).

3) Partly because of 2), cases where C# methods stack overflow due to deep recursion that should have succeeded are fairly rare.“

Also this statement (and the code in context) makes me think this is something closer to functional programming languages like F# rather than C#:

“I would have to assume the C# compiler team is using every trick in the book to make highly performant IL”

Are you kidding? The C# compiler does far less stuff, as you just saw right here. The F# compiler determined the code was a loop, and compiled it so. (It didn’t need to use the tail prefix ’cause it did even better.) I’m not saying the C# compiler doesn’t optimize, but it tries to keep a direct mapping from source to IL. (I think this is even sort of a goal for the C# compiler).“

More knowledge is brought to me by Jomo Fishers (also member of the F# team) with his post Adventures in F# – Recursion in Three Languages:

This tradeoff frees up the C# team to focus on things like LINQ.

=)

After reading the web pages linked from this post and a numerous of other about this topic, I believe I can stop bother about tail call recursion in my C# code.

Unfortunately I could not solve the problem of avoid having a recursive C# method eat up stack space, but at least I learned something about jitters, compiler optimization, the CLR and I definitely can say I now know the answer to my first question!

Written by anderssonjohan

January 5, 2009 at 15:40

Posted in programming

Tagged with ,