Our version independent runner scheme has served us well so far: all of our runners have converted over to it except for the Resharper runner. Jim and I intended to tackle the Resharper runner today and make it version independent as well, but ran into some stumbling blocks that caused us to postpone the work indefinitely.
The API for the version independent runners is relatively simple, and everything is communicated in terms of simple strings and XML. I explained a bit about it in my previous post about RC2, so I won't re-cover old ground.
The problem with Resharper that's different from all the other runners is that it operates in a mode where there is no assembly file. You are asked to answer questions about whether classes contain tests based on the source code that's in the editor, not the bits that are on the disk. This makes perfect sense: they want to provide "run this" chicklets as soon as you add a test to your assembly.
Unfortunately, our API is based on assemblies on the disk. Most of the decision making is done on your behalf; you're just asking to run assemblies, classes, or tests based on filenames and fully qualified type and method names.
In order to support Resharper, we would essentially have to serialize (potentially incomplete) type information into XML and pass it across an app domain boundary to get it inspected in order to make decisions about whether something is a test or not. Aside from the obvious performance implications, we were also concerned with the tremendous complexity it would introduce for essentially one runner. There were also issues about how to properly locate the xunit.dll that they would eventually be linking against.
In the end, we decided it wasn't going to happen.
Unfortunately, this means that the Resharper runner is going to suffer from version dependence, so you'll only ever be able to test against one version of xunit.dll with Resharper (whichever version the Resharper support was installed with).
Users who want integrated Visual Studio test runner capabilities with version independence are encouraged to use TestDriven.net.
P.S. Only Resharper 3.1 is currently supported. We won't be supporting 4.0 until the final RTM. Sorry for the inconvenience.
