The release of Moq (and the subsequent updating of Rhino Mocks in response) has made mock frameworks much more approachable to me, but I still find situations where hand-rolled stubs seem to work best.
Wednesday, Jim and I were trying to see what it might look like if we replaced our stubs with a mocking framework. We have two interfaces which we are hand stubbing today:
public interface ITestCommand
{
string Name { get; }
bool ShouldCreateInstance { get; }
MethodResult Execute(object testClass);
}
public interface ITestClassCommand
{
object ObjectUnderTest { get; }
ITypeInfo TypeUnderTest { get; set; }
int ChooseNextTest(ICollection<MethodInfo> testsLeftToRun);
Exception ClassFinish();
Exception ClassStart();
IEnumerable<ITestCommand> EnumerateTestCommands(IMethodInfo testMethod);
IEnumerable<IMethodInfo> EnumerateTestMethods();
bool IsTestMethod(IMethodInfo testMethod);
}
A quick background on some of the types involved here:
- ITestClassCommand represents a “class with tests”. It’s used by the execution engine when running the tests. Most of what’s used there isn’t important, but what is important for this test is EnumerateTestCommands().
- ITestCommand represents a “single execution of a test”. Each test method may yield multiple executions (think data-driven testing), which is why ITestClassCommand.EnumerateTestCommands() returns a collection of these rather than just one.
- IMethodInfo is an abstraction of the CLR’s MethodInfo class, which is necessary because sometimes when you’re being asked to enumerate these things, you don’t actually have a real MethodInfo (f.e., when ReSharper is asking you to tell it what tests exist in the source file that user is looking at).
The actual class under test is TestCommandFactory. Its single responsibility is: given a ITestClassCommand instance and a MethodInfo, generate the ITestCommand instances to run all the tests. It does more than just get the instances from the class command: it wraps them using the chain of responsibility pattern to provide all the extra work that goes into running and reporting on a test command, which includes:
- Running all the BeforeAfterAttributes found decorating the test method
- Creating a new instance of the type under test (if the inner ITestCommand.ShouldCreateInstance is true)
- Turning Trace.Fail and Debug.Fail statements into thrown exceptions
- Catching exceptions thrown by the test and turning them into FailResult objects
- Timing how long it takes to run the class
As you can see, the presence and absence of these things is important, and order is also important. Our hand-rolled stubs record the values that were passed in, and allow you to control values that were returned. Our test, in pseudo-code, is approximately:
Arrange:
Create a stub of ITestCommand (stubTestCmd) - In response to ShouldCreateInstance, return true
Create a stub of ITestClassCommand (stubClassCmd) - In response to EnumerateTestCommands, return StubTestCmd
Get the MethodInfo for an empty, void-returning method (methInfo)
Act:
Call TestCommandFactory.Make, passing stubClassCmd and methInfo
Assert:
Ensure that the IMethodInfo passed into stubClassCmd.EnumerateTestMethod wrapped methInfo
Ensure that we returned a wrapped ITestCommand (test all the levels of wrapping described above)
Because our stubs record the passed-in values and we can inspect them, our first assertion is easy to write:
Assert.Same(methInfo, stubClassCmd.EnumerateTestCommands_TestMethod.MethodInfo);
Note that we’re not just testing the value (that would be stubClassCmd.EnumerateTestCommands_TestMethod) but we’re actually diving into it and testing a specific value (.MethodInfo). This is because ITestClassCommand takes an IMethodInfo, but we actually pass a MethodInfo to the method under test.
When we tried to convert this to Moq, we were stuck. We could make a Mock<ITestClassCommand>, but we could neither specify (via .Expect) nor verify (via .Verify) against the data item it recorded. We actually need the actual value; the wrapper isn’t important, but what it wrapped was.
We found a way to write the test, which was to cascade a .Callback after the .Expect and do the assertion. However, this put assert logic up inside our arrange. Alternately, we could’ve stashed the value into a temporary and did the assertion afterwards, but that made things start to feel a little unwieldy compared to the hand-rolled stubs.
We also found the use of “Expect” to be unfortunate, particularly the syntax that we ended up with (approximately):
var mockClassCommand = new Mock<ITestClassCommand>;
mockClassCommand.Expect(x => x.EnumerateTestCommands(It.IsAny<IMethodInfo>()))
.Returns(new List<ITestCommand>(mockTestCmd));
Our problem is with the monstrosity of “It.IsAny<IMethodInfo>()”. It looks bad enough with a single parameter, but would look positively awful with several parameters. The truth is that I rarely want to say “if given A, return B”; I almost always say “return B, and I’ll verify in the Assert that I was passed A”. This would be a much better syntax:
var mockClassCommand = new Mock<ITestClassCommand>;
mockClassCommand.Stub(x => x.EnumerateTestCommands)
.Returns(new List<ITestCommand>(mockTestCmd));
The compiler error I get from this tells me that the Lambda system in C# 3 probably won’t ever let me get to this kind of a syntax. I’m also not even sure about what the syntax would look like that lets me inspect the executions of each call to EnumerateTestCommands, so that I could ensure it was only called once and that the wrapper wrapped the right thing.
I think Wednesday we came to the realization that a code generator was probably what we were going to end up with. I’ve come around several times to the problem and, even after seeing Lambdas in C# 3, concluded that a generator which could creates stubs of any interface/abstract class is still the ideal situation.
