Brad Wilson

xUnit.net Moving to Git

Just a quick note: the xUnit.net source control repository is switching from Mercurial to Git on Friday the 13th of April. If you have outstanding forks or pull requests, please read this page for how this change will affect you.

http://xunit.codeplex.com/discussions/351482

Shipped: xUnit.net 1.9

On January 2nd, Jim and I shipped xUnit.net 1.9. We updated NuGet with the 1.9 build binaries, and for the first time, we're including the MSBuild runner inside the "xunit" NuGet package.

There are a few big new features that are worth calling out.

Prototype xUnit.net Visual Studio 11 Unit Testing Plugin

Update: Visual Studio 11 Beta is now available to the public. We've released the runner for xUnit.net for VS11 Beta in the Extension Gallery.

Update: Visual Studio 11 Developer Preview is now available to the public.

Earlier this week at the //build/ windows conference, Jason Zander announced the availability of a new "developer-focused" unit test runner in Visual Studio 11. They shipped the first Developer Preview of Visual Studio 11, and it includes this new unit test runner. Peter Provost demonstrated the pluggability of this new unit test runner by showing (and writing) tests written in xUnit.net. We collaborated with Peter and wrote the prototype runner that he used on stage during his demos.

Full Throttle TDD

I got a chance to sit down (virtually) with Rob Conery and do my TekPub production, which aimed to show what a real-world TDD session might look like. Rob gave me some vague requirements and then set me off to coding. I know the title says "Full Throttle", but it feels a little three-quarters-throttle to me, mostly because the business area was new to me, and I (purposefully) didn't spend much time thinking ahead about the problem. We wanted this to just be a fresh stab in the dark at implementing something that I knew next to nothing about ahead of time. So, if you're hoping for one of those super-speed katas, then this isn't quite the video for you.

Testing Routing and URL Generation in ASP.NET MVC

In ASP.NET MVC, getting your routes setup properly can be tricky if you have a lot of routes. Whenever something is tricky, and might be touched often, having a set of unit tests around it can very helpful to ensure you aren't breaking it when you make changes. Lots of routes can definitely qualify; your users rely on unbroken links and stable URLs for a good experience.

The techniques shown here not only work for unit testing, but can also be used outside of the ASP.NET pipeline when you need to interact with the routing system (for example, to generate URLs from a service). The routing system used by ASP.NET MVC uses the abstraction classes, so the actual running ASP.NET pipline isn't required.

What's New in xUnit.net 1.6

Yesterday we pushed the shiny red button to send you all the bits for xUnit.net 1.6. It turns out there were some interesting technical challenges behind the scenes, so after I do the obligatory feature list, I'll talk about all the work we did (and sometimes undid) along the way.

Working Around Build Error MSB3454

We recently moved the xUnit.net source over to Visual Studio 2010 (still targeting .NET 2.0, though). We do automated builds with TeamCity (the server is generously provided by CodeBetter and JetBrains for open source projects).

When we switched to VS 2010, we noticed that our automated build was failing with the following error:

C:\WINDOWS\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(1835, 9):
error MSB3454: Tracker.exe is required to correctly incrementally generate resources
in some circumstances, such as when building on a 64-bit OS using 32-bit MSBuild.
This build requires Tracker.exe, but it could not be found. The task is looking for
Tracker.exe beneath the InstallationFolder value of the registry key
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\Windows\v7.0A. To solve the
problem, either: 1) Install the Microsoft Windows SDK v7.0A or later. 2) Install
Microsoft Visual Studio 2010. 3) Manually set the above registry key to the correct
location. Alternatively, you can turn off incremental resource generation by setting
the "TrackFileAccess" property to "false".

This error didn't occur on our development machines, because we have Visual Studio 2010 installed, but the CI machine only has the .NET Framework installed. While some people disagree, we are of the opinion (as are the CodeBetter guys) that Visual Studio is not something you should install on your CI machines.

We use an MSBuild file for our CI builds. We needed to make a simple change to the MSBuild file, where it calls the MSBuild task for our solution, to pass this new flag:

<MSBuild
    Projects="xunit.sln"
    Targets="Build"
    Properties="Configuration=$(Configuration);TrackFileAccess=false"/>

This made our build succeed (which solved our immediate problem), but I was curious what was going on. Luckily, some fellow Microsofties were able to clue me into exactly what was going on.

The resource compiler performance is dramatically improved by tracking files changes to your .resx files between builds; it's like incremental compilation from the C++ days, but for your resources. Getting this to work requires a tracker application to watch the file access, which is normally only part of the Windows SDK or Visual Studio.

Unfortunately, the Windows SDK isn't available for download yet for .NET 4. We are willing to accept builds that take a little longer than normal on the CI machine, until the CI machine can get the Windows SDK installed.

Hopefully this can help you get your .NET 4/VS2010 projects up and running on CI for the time being!

xUnit.net 1.5 Shipped

Jim and I released xUnit.net 1.5 a couple weeks ago.

This was the first full RTM release in about 9 months, an unfortunate side-effect of our busy schedules. :) The major new feature of 1.5 is the new GUI runner, which has evolved over several CTP and beta builds.

Here is a screenshot of the new GUI runner in 1.5:

If you're used to using the GUI runners in other testing tools, you'll notice that our GUI runner is significantly different. Rather than the traditional "tree" view, we've opted for a flat, integrated list of tests (the large box in the upper right of the window). Whatever is visible in this list will be run if you click "Run All"; you can also select specific tests, which turns the button into "Run Selected".

These test methods can be filtered in several ways, using the Refinements section on the left:

• The search box will search and filter methods whose method name, namespace name, or display name match the provided text
• The assemblies list allows you to filter down to just the tests in one or more of your selected assemblies
• The traits list shows all the [Trait] attribute name/value combinations in your tests.
• The count buttons at the top of the method list also work as filters, so you can quickly just see the tests that have passed, failed, or were skipped.

Assemblies are not locked, so you can leave the GUI runner open while working on your test code, and when test assemblies are recompiled, the GUI will automatically reload the DLL. Tests are run on a background thread so that the UI remains responsive while running tests.

There are lots of other small new features, plus a bunch of bug fixes, which we cover in the release notes (shown below):

xUnit.net release 1.5

Build #1479
Change set #42400

Important notes for Resharper users:

Resharper support has been moved to the xUnit.net Contrib project.

Important note for TestDriven.net users:

If you are having issues running xUnit.net tests in TestDriven.net, especially on 64-bit Windows, we strongly recommend you upgrade to TD.NET version 2.20 or later. It is also important to uninstall and reinstall TD.NET support using xunit.installer.exe.

This release adds the following new features:

• Better support for 64-bit machines, including forced 32-bit runners
• Introduced a new GUI runner, which allows:
  • Multiple assemblies
  • Filters on text, assembly, traits, and last run state
  • Project (.xunit) files
  • Multithreaded UI so that tests aren't slowed down by UI updates
  • Assemblies are not locked and are automatically reloaded so that you can continue to work on your tests in Visual Studio while leaving the GUI runner running
  • Automatically remembers UI window placement and recent assemblies/projects
• Additional MSBuild task which consumes project (.xunit) files
• Support for MVC 1.0 and MVC 2 Preview 1, including both C# and VB.NET
• A new test runner object model for runner authors (supporting multiple assemblies)
• IEnumerable/IEnumerable<T> support for Assert.Equal
• Better support for Assert.Throws for expressions which return values
• Support for installation-free running with TestDriven.net (2.23 or later)

This release fixes several issues, including:

• Added an output property named ExitCode to the MSBuild runner (0=success, -1=fail)
• Expected/actual messages are improved when values are the same but types don't match
• Fixed an issue where assemblies without config files run from MSBuild were using MSBuild.exe.config as their config file.
• Fixed several broken samples.
• Fixed an issue where long-running tests would be aborted in the console runner.
• Fixed an issue with CDATA handling in test results.

For a complete list of fixed issues, see the Issue Tracker.

This release includes:

• Core library (xunit.dll)
• Support for ASP.NET MVC 1.0 and 2.0 Preview 1
• Support for CruiseControl.net (see the wiki for more information)
• Support for TeamCity (see the wiki for more information)
• Version-resilient runner support (xunit.runner.utility.dll)
• Installer utility for runners and ASP.NET MVC (xunit.installer.exe)
• Runners:
  • Console runner (xunit.console.exe and xunit.console.x86.exe)
  • GUI runner (xunit.gui.exe and xunit.gui.x86.exe)
  • MSBuild runner (xunit.runner.msbuild.dll)
  • TestDriven.net 2.x runner (xunit.runner.tdnet.dll) see note above
• Extensions for .NET 2.0+ (xunit.extensions.dll):
  • Data theories using [Theory] with support for data from inline values, properties, Excel spreadsheets, SQL Server, and OLE DB data sources
  • [AssumeIdentity] extension
  • [AutoRollback] extension
  • [FreezeClock] extension
  • [Trace] extension
  • TestClass and Assertions base class for assertion extensibility

xUnit.net 1.5 Beta Released

Jim and I just shipped xUnit.net 1.5 Beta.

We're pretty much done with the features for 1.5 at this point. Other than a non-functional recent item list for projects in the GUI, we believe we are feature complete. We will probably be restructuring some of the GUI runner code so that it can have a long and maintainable life. :)

The big new feature for 1.5 Beta is the introduction of xUnit.net Test Project files (.xunit files). These files contain information about multiple assemblies and their run parameters, and can be consumed from the GUI and MSBuild runners (as well as created in the GUI). To accommodate this, the GUI got a small makeover in the menus, and now behaves like a single document application (where the document is your project file). We also support launching .xunit files from Windows Explorer (run the installer utility, xunit.installer.exe, to enable this support).

The other big feature is that we've included builds of the console and GUI runner which always run in 32-bit mode, even on 64-bit OSes. The default versions will run in whatever is native. We also included environment information (32- vs. 64-bit and the .NET version) in several places, including the native XML output files.

Finally, we added preliminary support for ASP.NET MVC 2 Preview 1. This support may need to evolve with later releases of MVC.

As always, we look forward to usage reports, especially with the new GUI. Your help lets us know that we're ready to ship!

xUnit.net 1.5 CTP 1

This morning, we published the first CTP of xUnit.net 1.5. This release includes a new GUI runner, support for ASP.NET MVC 1.0 RTM (for both C# and VB.NET), and several bug fixes.

A note for Resharper users: support for xUnit.net in Resharper has been moved to the xUnit.net Contrib project, and is no longer part of the core xUnit.net distribution.

New GUI Runner

Our goal with the new UI is to streamline the test running process for test-first developers. It is common for test-first development to result in several test assemblies and thousands of tests. The traditional thinking of test runners and tree representations of tests seems to fall apart when your test list becomes large.

We wanted to give developers a quick way to filter test lists that don’t involve navigating large trees. This first CTP allows users to filter by loaded assembly, as well as using a search box which live-filters the lists of tests. Additionally, the user can use the test list to select a subset of tests to run.

What’s Left To Do?

This runner is still in a very early state. We wanted to get it out and get user feedback. We have several features (and some significant code restructuring) left to implement before it will be ready. We are intending to re-add the automatic assembly reloading feature, as well as additional test list filtering (for example, filtering by traits). Mostly, though, we would like to see the community provide feedback on the general effectiveness of the design for their day to day runner needs.

We will be releasing at least one more CTP before we release the final 1.5 bits. Please take this opportunity to try out the new runner and let us know what you think!