Simon Harriyott

Running mstest from msbuild

Having used team test for unit testing since .NET beta 2 came out, I've finally got some time to set the build machine up to run the tests. It sounds like it should be quite straight forward, as there is a command line version of the test runner, mstest.exe. This has the command line parameters that one would expect: the .vsmdi file containing the test list, the localtestrun.testrunconfig file that contains (amongst other things) the locations of the test data and dependencies, and the path to the assembly being tested.

This is fine, and works well from the command line, when you know where these files are. On our build, we currently have around 30 solutions, each containing (as a minimum) a class library project and a test project, and there's no way I want to write the command lines out by hand, especially as we're still quite near the beginning of the project.

The double star in MSBuild is really useful to recursively find files in a directory tree, so I'm using this to find all the .vsmdi files under the source root:

<testmetadata include="C:\Build\Root\Source\**\*.vsmdi">

I can't see an MSBuild task for running tests yet, so I'm using the Exec task. The localtestrun.testrunconfig file is found in the same directory as the .vsmdi file by default, so there's one less search to do. The assembly however, is four directories lower in the solution, e.g. SolutionDir\Tests\TestProject\bin\Debug.

As the TestProject directory name is different for each of the solutions, I can't hard-code the path relative from the .vsmdi directory. I've not found a way to use a combination of **, $, % and @ in MSBuild to find the assembly under the path either.

In the .vsmdi file, it is possible to specify the assembly that the tests are run against, by (strangely enough) creating a new test list, and moving the existing tests into this list. There's bound to be a better way, but this is the first thing I found that works. Doing this adds the list of tests to the file, and the assembly is specified in the node in the XML. The location of the localtestrun.testrunconfig is also stored in the file, and the assembly location is also stored in the localtestrun.testrunconfig file.

The problem with this approach (for me) is that absolute paths are used. The source root directory on the build machine is different to those used by the developers. This means that the build script will work on the machine where the test project was setup, but not on the build machine. The only way I've found to fix this so far is to write a utility that rewrites the paths in both the .vsmdi and the localtestrun.testrunconfig files.

This utility is run prior to mstest in the build process, and uses the directory where the .vsmdi or localtestrun.testrunconfig is located as the solution root, and replaces the paths accordingly.

So, the build script:

<testmetadata include="C:\Build\Root\Source\**\*.vsmdi">
<target name="RunTests">
<exec command="'TestConfigAbsolutePath.exe" workingdirectory="Root\Source\Tools\TestConfigAbsolutePath">
<exec command="TestConfigAbsolutePath.exe %(TestMetaData.RootDir)%(TestMetaData.Directory)localtestrun.testrunconfig" workingdirectory="Root\Source\Tools\TestConfigAbsolutePath">
<exec command="'mstest.exe">

Each <exec> will be run one per .vsmdi file found using **. %(TestMetaData.FullPath) will give the full file name of the .vsmdi, and %(TestMetaData.RootDir)%(TestMetaData.Directory) will give the directory that contains it, which can be used as the path to the localtestrun.testrunconfig. The paths are rewritten before the mstest is executed, so mstest can find the correct files and assemblies.

I'm sure there must be a better way, but with so many things in development, the first thing that works is good enough.

Update: I've posted some of the source code for the utility.

[Tags: ]
27 July 2005