Testing

What’s New in CompositionTests 1.3

CompositionTestsv.1.3.0 (640x640)I hope you are all as excited as I am about the latest ApprovalTests release, 2.0.  This release polishes some rough edges, and adds some exciting new test scenarios.  Check out Llewellyn’s release notes for details.  I’ve been experimenting with CompositionTests 1.3 for a month or so, and now that ApprovalTests has released, I’m releasing 1.3 to go with it.  Other than versioning compatibility, CompositionTests 1.3 doesn’t have any specific dependencies on ApprovalTests 2.0, but it seemed like a good time to release.  Enjoy.

Generic VerifyCatalog Implementation

CompositionTests 1.2 introduced customized Verify*Catalog convenience methods that automatically provide scrubbers that make sense with each catalog type. Although this worked fine, using the convenience methods meant that you needed to make an extra change to your test if you decided to change the catalog type… not very convenient!

The new VerifyCatalog<T> method addresses this shortcoming. The new method will take any catalog descended from ComposablePartCatalog and attempt to find an appropriate Verify*Catalog implementation. When the method cannot find an appropriate implementation, the default is to use VerifyCompositionInfo with no scrubbers.

ApprovalTests 2.0

This version of CompositionTests is built against ApprovalTests 2.0. The CompositionTests 1.2.0 NuGet package incorrectly specified the ApprovalTests dependency as x >= 1.9. Because the ApprovalTest assemblies are signed, dependent assemblies (like CompositionTests) will only load the ApprovalTests assembly which they are built against, so “greater than or equal to” should have been “exactly equal to”. The package for CompositionTests 1.3.0 correctly specifies the dependency as x == 2.0.

If CompostionTests stopped working for you after upgrading to ApprovalTests 2.0, this release should fix that for you.

Reminder

Remember DiscoverParts and Composition are obsolete and will be removed in a future release. Please migrate to Verify* and MefComposition.

Go Get It

As usual you can get the bits from github or NuGet!

http://www.compositiontests.com/

https://nuget.org/packages/CompositionTests

Advertisements
Coding, Presentations

How Hard Can It Be?

Unit Testing ASP.NET MVC Views with ApprovalTests

videothumbnail
How Hard Can It Be? | aspConf 2012 | 7/22/2012

If you missed it, aspConf happened July 17 -18, 2012.  I presented a demo of the MVC View testing technique I’ve written about on this blog. My session was July 18 from 1-2:15 PM PST in Room 3.  If you missed the live show, I hope you will watch the screencast now posted on channel9.  Don’t miss the next aspConf (I presume next year), it’s a virtual, on-line conference and it’s free!  You don’t even have to get out of your chair.

In addition to the screencast (linked above), here are a few resources for the talk:

Testing

What’s New in CompositionTests 1.2

After a great weekend at SoCalCodeCamp San Diego, I came home “super excited”.  After a good night’s sleep I got cracking first thing in the morning, the result is CompositionTests v1.2.0.

Thanks to github:pages the project has a pretty new home page at compositiontests.com.  You can head over there to check out the source.  If you just want the bits, they’re waiting for you on NuGet.org.

Read on to find out what’s new.

Built-in Scrubbers Handle Nulls Properly

Since scrubbers are called one after the other, its possible for one scrubber to nullify the entire string and pass it to the next.  The built in scrubbers all check for null and simply pass the value through without processing when one is encountered.  Custom scrubbers that don’t follow the same design risk a NullReferenceException at runtime.

Normalized Line Endings and Indentation

For some reason the (newer) exception messages generated by MEF use LF instead of CRLF when describing the contract name of the missing part.  This creates a usability issue when viewing the approved or received files in Visual Studio because Visual Studio displays a modal dialog that will offer to normalize the line endings for you.  But, if you normalize the line endings and approve, then you will never get a matching received file because the exceptions will not be normalized.  If you choose not to normalize the line endings in order to ensure consistency between the received and approved files, you will have to dismiss Visual Studio’s offer every time you open the file.

So, to make life easy, CompositionTests will normalize the line endings before sending the formatted text to ApprovalTests.

Likewise, the exception messages use tabs, while the CompositionInfoTextFormatter uses spaces.  Certain Visual Studio extensions will offer to normalize this for you.  On my system, this is not nearly annoying as Visual Studio, because the offer is not made in a modal dialog box and I can just ignore it if I like.  However, if someone were to normalize and approve the normalized file, they once again have broken consistency between the received and approved file.  So, CompositionTests will replace tabs with spaces before sending the formatted text to ApprovalTests.

New API (Backward Compatible for Now)

I had a chance to sit down with Llewellyn Falco at SoCalCodeCamp, and we did some pairing on ApprovalTests and CompositionTests.  We renamed the primary helper class from Composition to MefComposition since only MEF composition is supported.

We also created a specialized VerifyDirectoryCatalog method that automatically provides the scrubbers that make sense when working with a DirectoryCatalog.  From this starting point I created additional Verify* methods for each of the four supported catalog types.  The old DiscoverParts methods have been marked with the  ObsoleteAttribute, along with the old Composition class.

MefComposition still supports the old API, but the methods have been renamed.  You can find them by looking at the group of methods called VerifyCompositionInfo.  These methods can directly replace your calls to DiscoverParts and they still allow you to pass in a custom ExportProvider, etc.

These changes are meant to be backward compatible in this release, so if I’ve inadvertently broken compatibility, that’s a bug, let me know.  Please stop using DiscoverParts and Composition, they will be removed in a future release.

OrderedCompositionInfoTextFormatter.Format

I find it awkward to work with the Write method provided by CompositionInfoTextFormatter.  Since I’m now using OrderedCompositionInfoTextFormatter by default, I added a Format method that hides the parts I feel are awkward.

Semantic Versioning

I’m trying to adopt semantic versioning.  If I had adopted this system from the start, then I probably would not yet be at v1.  I’m already there, so I’m just going to try my best to follow this convention from now on.  Since I don’t want to go to v2 yet, I’ve tried to maintain backward compatibility with the old API, so this release puts us at v1.2.0.

To document the API, I’ve added an ApiApprover test.  Like CompositionTests, ApiApprover leverages ApprovalTests to introduce a cool new testing scenario.  The test discovers the public API, and will fail if the API changes.  Since it uses ApprovalTests, the public API ends up documented in the .approved file for the API Approver test.  So, you can find the public API listed out in “ApiTest.ApprovePublicApi.approved.txt” in the test folder.

Check It Out

Head over to the new homepage.  I’ve updated the Readme with some nice examples that should help get you up to speed and testing quickly.

Coding, Testing

Catching Email With Rnwood.SmtpServer

Not too long ago, Llewellyn Falco posted Using ApprovalTests in .Net 19 Email, where he describes a really easy way to test email messages using ApprovalTests.  The video describes a testing seam that separates message creation from message sending, and this makes testing email straightforward.  If you are currently working with .NET source, then you really should follow the simple instructions in that video and stop reading this post, it’s not for you.

This post is for you if:

  • You don’t control the source of the email you want to test.
  • You control the source, but it’s not .NET.
  • You control the source in theory, but you can’t change it (eg. boss says no)

Context

In my case, I’m moving a little reporting script from Perl to .NET.  Lets call the script “report.pl” for the sake of discussion.  We want to move from Perl to .NET using TDD and without any noticeable changes to the users (which happen to be the Executive Committee where I work).  I don’t want to create a system that works more or less the same, I want one that works exactly the same.  I need to lock down the current system, and use it as the gold standard for the new system.

Here’s what the current script looks like with the responsibilities color-coded:

image

The blue blocks are responsibilities that are implementation specific, when the script wakes up the first thing it does is read its config and validate its environment.  Its not likely that these tasks will transfer to a new system.

The script creates an Excel Workbook, this responsibility is identified by green blocks.

The script reads its data from a database, this responsibility is identified by red blocks.

The script manipulates the data before writing it to the Workbook, these blocks are purple.

The script sends the Workbook to its recipients over email, this block is yellow.  This is the first block I’ll work on, since it’s the only one that isn’t interwoven with other responsibilities.

Strategy

1. Identify a responsibility

image

2. Let’s assume that this responsibility was encapsulated in a subroutine.  Copy that subroutine to a new Perl script “sendReport.pl”.  If the responsibility was not encapsulated in a subroutine, make sure to wrap the lines of code you copy with a sub{} in sendReport.pl.

image

3. Use Perl’s “require” mechanism to include “SendReport.pl” in “Report.pl” and remove the local definition of the “sendReport” subroutine.  The script will call the imported definition instead.

image

4. Create a third Perl script “SendReportRunner.pl” which is just a thin shell around the extracted responsibility, which will let me execute the responsibility with any parameters I like.

image

5. Create a unit test in C# that uses a Process object to invoke SendReportRunner.pl.  Notice that Report.pl is no longer in the picture.

image

6. Capture the output in an ApprovalTest.  Because SendReport.pl actually wants to send email over SMTP, this is where the plan starts to go off the rails, but we can work through it.

image

7. Build C# implementation (Report.dll) that passes same ApprovalTest.  Once we have a C# implementation, we can create a seam that separates the message creation from the message sending, and use EmailApprovals, just like Llewellyn described, but getting past step 6 will be tricky.

image

8. Now we need to get Report.dll working with Report.pl.  I’ll create C# shell that invokes Report.dll from the command line.

image

9. Replace sendReport() call in Report.pl with a system() call to to ReportRunner.exe

image

10. Repeat until Report.pl is just a bunch of blue blocks containing system() calls to C# code runners.

11. Replace Report.pl with a C# executable, PowerShell script or whatever.

Moving the responsibility down to Step 4 was relatively easy.  I spent some tedious hours tracing variables to determine their scope and effect.  Then I spent half a day wresting with the COM object the Perl script uses to send the mail.  At the end of the day I could use SendReportRunner.pl to send emails and catch them with smtp4dev.  But to get past Step 6 I needed to answer the question: How do I get those messages into a .NET unit test so I can capture and approve them?

Catching Email

Smtp4dev is a nice little application that fills a similar niche to CassiniDev a webserver I wrote about previously.  Smtp4dev sits in your system tray and listens on the SMTP port for incoming mail.  When it gets a message, it logs the message arrival in it’s window and you you can double click the message to see it in your default email program:

image

Since smtp4dev lives on CodePlex, I figured there was a good chance that it was written in .NET and sure enough it was written in C#.  Thinking back to CassiniDev, I wondered if there was a way I could host smtp4dev in my unit test, catch the messages from the Perl process, and then hand them over to ApprovalTests.  I grabbed the source for the project and found an example named “SimpleServer’’ that looked like it could be used to create a test fixture similar to the CassiniDev fixture I used when testing MVC views.

I created an empty class library and a test project to go with it.  The test project will need ApprovalTests and a reference to Rnwood.SmtpServer, which is the server that powers smtp4dev.  The server wasn’t on NuGet yet, so I put it there and used NuGet to add both references.  The pattern for creating the test fixture was nearly the same as creating a fixture for CassiniDev:

using Microsoft.VisualStudio.TestTools.UnitTesting;
using Rnwood.SmtpServer;

namespace Report.Tests
{
  [TestClass]
  public class SmtpFixture : DefaultServer
  {
    public SmtpFixture(Ports port)
      : base(port)
    {
    }

    [TestInitialize]
    public void StartServer()
    {
      this.Start();
    }

    [TestCleanup]
    public void StopServer()
    {
      this.Stop();
    }
  }
}

To implement a test, I’ll extend this fixture.  In the broad strokes, we want something like this:

public SendReportTest()
  : base(Ports.SMTP)
{
}

[TestMethod]
public void SendReportOverEmail()
{
  try
  {
    this.MessageReceived += CatchMessage;
    GenerateMessage();
    ApprovalTests.Email.EmailApprovals.Verify("??");
  }
  finally
  {
    this.MessageReceived -= CatchMessage;
  }
}

I specify the default SMTP port in the constructor, I could have used “Ports.AssignAutomatically” and the server would pick an empty port.  That’s nice functionality, but the Perl script wants to use the default port.  I’ve declared some methods but not implemented them, and I’m still not sure what I’m going to give to ApprovalTests.

When we get a MessageReceivedEvent, it will come with a MessageEventArgs and we need to figure out if we can somehow get a MailMessage from that, which is what EmailApprovals is expecting from us.  CatchMessage needs to do that for us.

We also need to generate a message. I’ll do that first, since once I can generate and catch messages I’ll be able to look at a live instance of MessageEventArgs and see what its guts look like.  Our goal is for the message to come from Perl, but for the moment, we’ll just use an SmtpClient to stand in for the script.  Notice that we pass the fixture’s port number to the SmtpClient, if we were using a random port, this would ensure that we actually send it to the right place.

private void GenerateMessage()
{
  using (var client = new SmtpClient("localhost", this.PortNumber))
  {
    using (var message = new MailMessage(
      "noreply@localhost", 
      "jim@localhost", 
      "Hello World", 
      "Well, you caught me."))
    {
      client.Send(message);
    }
  }
}

Implementing CatchMessage gave me some pause.  If I use a lambda, I can’t easily unsubscribe from the event.  Maybe that doesn’t matter in the context of a test, but it’s a bad idea to leave events attached, and I don’t want to be in the habit.  I could unsubscribe safely I had a regular method, but then I need some plumbing to get the data back to the test method. I thought about it for a minute or two and decided to create a class to handle the event.  Later on this turned out to be a pretty good decision, because I was able to substitute some special logic to handle the Perl message in the catcher class without obscuring the test intention.

The basic MessageCatcher just needs to handle the event and store the message data.  Then we can create one of these in our test and use it there.

public class MessageCatcher
{
  public IMessage Message { get; private set; }
  public void CatchMessage(object sender, MessageEventArgs e)
  {
    this.Message = e.Message;
  }
}
[TestMethod]
public void SendReportOverEmail()
{
  var catcher = new MessageCatcher();
  try
  {
    this.MessageReceived += catcher.CatchMessage;
    GenerateMessage();
    ApprovalTests.Email.EmailApprovals.Verify(catcher.Message);
  }
  finally
  {
    this.MessageReceived -= catcher.CatchMessage;
  }
}

But it turns out that the IMessage interface is not what we want, because it’s not what EmailApprovals wants, and its not convertible into a MailMessage.  At the moment it doesn’t look like we can use EmailApprovals, but that doesn’t mean we can’t use ApprovalTests.  The SimpleServer example code shows how to dump the IMessage to a eml file:

// If you wanted to write the message out to a file, then could do this...
File.WriteAllBytes("myfile.eml", e.Message.GetData());

It turns out that *.eml is just a fancy name for “text file”.  I don’t want to dump it to the file system if I can avoid it.  Since GetData() returns a Stream, I should be able to read it directly.

public class MessageCatcher
{
  public string Message { get; private set; }

  public void CatchMessage(object sender, MessageEventArgs e)
  {
    using (var reader = new StreamReader(e.Message.GetData()))
    {
      this.Message = reader.ReadToEnd();
    }
  }
}

Then I can update my test to be an ordinary Approval instead of an EmailApproval.  Since I think I’m about ready to run this, I add a FileLauncherReporter.

[TestMethod]
[UseReporter(typeof(FileLauncherReporter))]
public void SendReportOverEmail()
{
  var catcher = new MessageCatcher();
  try
  {
    this.MessageReceived += catcher.CatchMessage;
    GenerateMessage();
    ApprovalTests.Approvals.Verify(catcher.Message);
  }
  finally
  {
    this.MessageReceived -= catcher.CatchMessage;
  }
}

The test run completes and notepad launches:

image

This is both really cool, and kind of a bummer.  Its really cool because I can (in theory) catch the Perl script’s messages and use them as a baseline for developing my C# implementation.  Although, before moving on, I see one thing I need to take care of, and that is the timestamp in the middle of the message.  A little regex should take care of that:

public void CatchMessage(object sender, MessageEventArgs e)
{
  using (var reader = new StreamReader(e.Message.GetData()))
  {
    this.Message = Regex.Replace(
      reader.ReadToEnd(),
      @"Date:\s[\d\s\w,:-]+\d+\r\n",
      string.Empty);
  }
}

The bigger disappointment is that notepad launched at all.  When Llewellyn used a FileLauncherReporter in his video, Thunderbird launched.  That was cool.  I’m jealous.  Luckily ApprovalTests is open source so I can go see how de did that.  Turns out to be pretty simple, we just need to make sure that when ApprovalTests saves the received file, it uses the .eml extension.  To do this, I make a small change to the way I call Verify().

[TestMethod]
[UseReporter(typeof(FileLauncherReporter))]
public void SendReportOverEmail()
{
  var catcher = new MessageCatcher();
  try
  {
    this.MessageReceived += catcher.CatchMessage;
    GenerateMessage();
    Approvals.Verify(new ApprovalTextWriter(catcher.Message, "eml"));
  }
  finally
  {
    this.MessageReceived -= catcher.CatchMessage;
  }
}

And now the file launches in my default mail client, which happens to be Outlook.

image

Catching Perl

Now that I understand how to catch mail using Rnwood.Smtpserver, and my childish need to see my message in an email client is satisfied, I can get this working with Perl.  I’m going to create a PerlMessageGenerator class for that.

public class PerlMessageGenerator : IMessageGenerator
{
  private const string MissingPerlMessage = "You must have a 32-bit perl at [{0}]. Please visit http://http://www.activestate.com/ to acquire Perl.";
  private const string PerlPath = @"C:\Perl\bin\perl.exe";

  public PerlMessageGenerator()
  {
    if (!File.Exists(PerlPath))
    {
      throw new InvalidOperationException(MissingPerlMessage.FormatWith(PerlPath));
    }
  }

  public void GenerateMessage(string host, string to, string attachementPath)
  {
    var binPath = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
    var arguments = "sendReportRunner.pl {0} {1} {2}".FormatWith(host, to, attachementPath);
    var pi = new ProcessStartInfo(PerlPath, arguments)
    {
      UseShellExecute = false,
      WorkingDirectory = binPath,
      CreateNoWindow = true
    };
    using (var p = new Process { StartInfo = pi })
    {
      p.Start();
      p.WaitForExit();
    }
  }
}

Now I just need to get my scripts into place by adding them as linked files to my test project, with “Copy To Output Directory” set to “Copy Always.”  This actually works, the test catches the Perl message, but as I mentioned the Perl output needs some additional scrubbing over and above the simple message.  SendReport.pl adds another timestamp in the subject line, a Message-ID field that varies on each run, and because it has an attachment, there are MIME boundaries that need to be ditched.  I’ll spare you the gory details.

The important part is that we caught the message.  After creating a separate PerlMessageCatcher to handle all the special cases, my test passes consistently in Visual Studio.  Just for kicks, and because this will eventually be production code, I turn on NCrunch.  And I’m very happy to see that the test passes under NCrunch as well.

Here’s the final test class:

[TestClass]
public class SendReportTest : SmtpFixture
{
  public SendReportTest()
    : base(Ports.SMTP)
  {
  }

  [TestMethod]
  public void SendReportOverEmail()
  {
    var catcher = new PerlMessageCatcher();
    try
    {
      this.MessageReceived += catcher.CatchMessage;
      new PerlMessageGenerator().GenerateMessage("localhost", "jim@contoso.com", "sendreport.pl");
      Approvals.Verify(new ApprovalTextWriter(catcher.Message, "eml"));
    }
    finally
    {
      this.MessageReceived -= catcher.CatchMessage;
    }
  }
}

That’s probably enough for one day.  I’ve made it past Step 6 in my porting list.  PerlMessageCatcher is pretty twisted code and could use some refactoring.  On the other hand, once I make it to Step 9, the (as yet non-existent) .NET implementation will be the canonical implementation, and I can simply use EmailApprovals directly.  The need for the PerlMessageCatcher will go away, so perhaps getting to Step 9 is a more worthy goal than refactoring the catcher.  We’ll see.

Coding, Testing

MEF Composition Tests, Redux

Photo Credit: FutUndBeidl

A while back I gave a talk at SoCalCodeCamp about testing MEF composition.  Because there are a few steps involved in setting up the test, I wrote Stop Guessing About MEF Composition And Start Testing.  I had hoped that the article, along with the source code from my talk would help more people cross the gap from having once heard that it was possible to test composition, to actually implementing tests.  Since there were only about five people at my talk, it didn’t take long for the number of blog readers to exceed the number of attendees, and yet I still wonder how many people decide setting up the test is too complicated and just decide to use MEFX or Visual MEFX “as needed”.

While MEFX tools are valuable (and these composition tests are just a more powerful way to automate MEFX), running tests on an ad-hoc basis is never ideal.  Bugs creep in when you don’t expect them to, and automatic tests are your defense against them.  I also find composition tests provide valuable design-time feedback.  While taking questions at the end of the talk, I was asked about my process for designing MEF parts.  I answered that I more or less disregarded MEF until the end of the process, and then just added the needed attributes.  That was a truthful answer, but over the intervening months, I’ve realized it was the wrong answer.  What finally put the nail in the coffin was this great article: How test-driven development works (and more!), by J. B. Rainsberger.  After reading, it hit me that that I had setup a little waterfall process.  First, design and build a unit.  Then, glue MEF onto it.  The problem is that even when “keeping MEF in mind” I would sometimes do foolish things like pass a bunch of primitives into a constructor, and this would lead me to rework at the end of the development cycle when I tried to shove my class into a container.

So, to avoid rework and break some bad habits, I decided to start testing composition to get feedback during the design phase as soon as I know I’m going to use MEF (which these days is most of the time).  Its already paying dividends on my current project, as soon as I see the need to pass in a simple piece of data like a string or an integer, my composition test will blow up if I pass it through the constructor.  A lot of times I still prefer to pass the data in through the constructor, but since my test is blowing up, I’m forced design and test the system for getting that data to the constructor immediately, rather than at the end of the cycle.

I started testing composition out of necessity, I had a broken composition that needed fixing.  Now, I do it to save time and effort, and it seems more valuable to me every day.  More people should do it, but I’m afraid some will be turned off by the number of steps and concepts involved.

Here are a few of them:

  1. Synchronizing catalogs between test and production
  2. For some, this could also be their first exposure to ApprovalTests
  3. Finding a copy of MEFX.
  4. Setting up the test.

So, does it have to be so complicated?

Reducing the Surface Area

Lets look at a sample project and implement the integration test I described in my last article about composition tests.  Some people have a one-track mind.  Mine has two tracks: cars and pizza.  This time we will go with cars and our sample project will be a simple program called CarDealership with some parts related to cars in it.  It doesn’t have to be very complex since our primary interest is the test.

Lets look at that test:

[TestClass]
public class IntegrationTest
{
  [TestMethod]
  public void DiscoverParts()
  {
    try
    {
      var catalog = new DirectoryCatalog(".");
      var host = new CompositionContainer(catalog);
      var compositionInfo = new CompositionInfo(catalog, host);
      using (var stringWriter = new StringWriter())
      {
        CompositionInfoTextFormatter.Write(
            compositionInfo,
            stringWriter);
        Approvals.Verify(stringWriter.ToString());
      }
    }
    catch (ReflectionTypeLoadException ex)
    {
      Array.ForEach(
          ex.LoaderExceptions,
          lex => Console.WriteLine(lex.ToString()));
      throw;
    }
  }
}

A few things are already different about the test.  There’s no UseReporterAttribute on the class anymore, because I usually define this in AssemblyInfo.cs now using an assembly scoped attribute:

// AssemblyInfo.cs
using ApprovalTests.Reporters;
[assembly: UseReporter(typeof(DiffReporter))]

I also moved a few more lines inside the test body because I don’t usually have a reason to run more than one test with the same CompositionInfo.  Also, since my intention is to break this test down into something cleaner, I want to see it all in one place.

Right now, our test wont compile because we don’t have references to ApprovalTests or MEFX.  ApprovalTests has been available on NuGet for some time, and lately the NuGet package has been kept up to date with releases on SourceForge.  So, go ahead and use NuGet to install ApprovalTests:

In my previous post I explained where to find MEFX, which includes the CompositionDiagnostics library that we need for our test.  But you had to download it or compile it, and then add the reference by hand… icky.  But, we had no choice because this library wasn’t available on NuGet.  Well, it is available now, because I put it there a few months ago.  Now its much easier to get your hands on a copy of the library:

image

Now our test compiles and runs.  While you weren’t looking I added some parts to the CarDealership program.  But it doesn’t matter because I haven’t setup a pre-build event to copy assemblies from production to test.

I never liked doing that.  I knew it would be brittle and it was.  Ideally, I’d just like to use the exact catalog from production in my test.  It turns out this is a lot easier to arrange if you don’t use a DirectoryCatalog.  AssemblyCatalogs or TypeCatalogs are easy to share programmatically.  So, ask yourself if you really need to use a DirectoryCatalog.  Lets see what the test looks like when we use an AssemblyCatalog:

[TestMethod]
public void DiscoverParts()
{
  try
  {      
    var compositionInfo = new CompositionInfo(
      Program.Catalog, 
      Program.Host);
    using (var stringWriter = new StringWriter())
    {
      CompositionInfoTextFormatter.Write(
          compositionInfo,
          stringWriter);
      Approvals.Verify(stringWriter.ToString());
    }
  }
  catch (ReflectionTypeLoadException ex)
  {
    Array.ForEach(
        ex.LoaderExceptions,
        lex => Console.WriteLine(lex.ToString()));
    throw;
  }
}

To facilitate sharing, I’ve added static properties on the Program class.  They are simple properties, initialized in Program’s class constructor:

using System.ComponentModel.Composition.Hosting;
using System.ComponentModel.Composition.Primitives;
using System.Reflection;

namespace CarDealership
{
  public class Program
  {
    static Program()
    {
      Catalog = new AssemblyCatalog(Assembly.GetAssembly(typeof(Program)));
      Host = new CompositionContainer(Catalog);
    }

    public static ComposablePartCatalog Catalog { get; private set; }

    public static ExportProvider Host { get; private set; }

    private static void Main(string[] args)
    {
    }
  }
}

Looking back at the test, we can see the effect of sharing the catalog and host from Program.  We can now get the CompositionInfo with just one statement:

var compositionInfo = new CompositionInfo(Program.Catalog, Program.Host);

Another interesting development, the catalog and host are only referenced on this line.  Could we create a method that takes “compositionInfo” as a parameter without changing this test’s behavior?  The answer is no, but lets try it and see why.

[TestMethod]
public void DiscoverParts()
{
  var compositionInfo = new CompositionInfo(
    Program.Catalog,
    Program.Host);
  DiscoverParts(compositionInfo);
}

private static void DiscoverParts(CompositionInfo compositionInfo)
{
  try
  {
    using (var stringWriter = new StringWriter())
    {
      CompositionInfoTextFormatter.Write(
          compositionInfo,
          stringWriter);
      Approvals.Verify(stringWriter.ToString());
    }
  }
  catch (ReflectionTypeLoadException ex)
  {
    Array.ForEach(
        ex.LoaderExceptions,
        lex => Console.WriteLine(lex.ToString()));
    throw;
  }
}

To make the CompositionInfo a parameter, we pulled it out of the try-catch block.  Since the CompositionInfo constructor can throw the ReflectionTypeLoadException we’re trying to catch, moving it out of the try-catch block defeats the purpose of the exception handler.  Moving the exception handler back to the test method requires us to implement the handler (which never changes) every time we write a test.  We need to control when “new” is called on CompositionInfo.  We can do so by changing our parameter into a Func<CompositionInfo>:

[TestMethod]
public void DiscoverParts()
{
  DiscoverParts(() => new CompositionInfo(Program.Catalog, Program.Host));
}

private static void DiscoverParts(Func<CompositionInfo> getCompositionInfo)
{
  try
  {
    using (var stringWriter = new StringWriter())
    {
      CompositionInfoTextFormatter.Write(
          getCompositionInfo(),
          stringWriter);
      Approvals.Verify(stringWriter.ToString());
    }
  }
  catch (ReflectionTypeLoadException ex)
  {
    Array.ForEach(
        ex.LoaderExceptions,
        lex => Console.WriteLine(lex.ToString()));
    throw;
  }
}

Now, I could still create a CompositionInfo instance in the test method, then use the delegate to pass back that instance.  Doing so, I would not be protected by the exception handler, but that would be a minor problem.  The exception handler only exists as a time saver, when a ReflectionTypeLoadException occurs, it will print additional information not included in the default dump.  Without the handler, you would need to rerun the test in the debugger in order to see the loader exceptions.  So, its not the end of the world, but at least now its possible to reuse the exception handler as long as we remember how to build the delegate correctly.

Can we make DiscoverParts easier to use by removing the caveat that the constructor must be part of the delegate?  We could provide an overload that creates the delegate for the caller:

[TestMethod]
public void DiscoverParts()
{
  DiscoverParts(Program.Catalog, Program.Host);
}

private static void DiscoverParts(ComposablePartCatalog catalog, ExportProvider host)
{
  DiscoverParts(() => new CompositionInfo(catalog, host));
}

private static void DiscoverParts(Func<CompositionInfo> getCompositionInfo)
{
   // ...
}

That’s better, and maybe we don’t need that delegate after all.  For the time being, I’m going to leave it there because I think the more interesting question is: Do we need the host?  If you look again at the construction of the host in Program, you’ll see that it’s the simplest possible CompositionContainer, and the only interesting thing about it is the catalog it consumes.  So why not focus our test on the catalog?

[TestMethod]
public void DiscoverParts()
{
  DiscoverParts(Program.Catalog);
}

private static void DiscoverParts(ComposablePartCatalog catalog)
{
  DiscoverParts(catalog, new CompositionContainer(catalog));
}

private static void DiscoverParts(ComposablePartCatalog catalog, ExportProvider host)
{
  DiscoverParts(() => new CompositionInfo(catalog, host));
}

In a great many cases, the container itself is very boring, but there certainly are plenty of cases where some other, more interesting, ExportProvider comes into play. So, both overloads are useful.  In the case where only the catalog is interesting, the overload without the host should be used.  This sends the signal that the host is not important.  On the other hand, if you choose to use the overload with the host specified, then you’re sending the signal that the host is interesting and important.  Who are you sending the signal too?  Probably your future self.

So we’ve accomplished a lot.  Our test method has been reduced to a simple method call that can take as little as one parameter.  The best thing about it is that this parameter is part of the “normal’ set of Composition types, it does not require the caller to know anything about CompositionInfo, CompositionInfoTextFormatter, or ReflectionTypeLoadException.  All that code is still there though, and its not conveniently packaged for distribution.  Can we make this test even easier by completely hiding all this code?

Introducing CompositionTests

One could take everything we’ve looked at here, bundle it up into a library, and put it on NuGet and GitHub.  I’ve done just that:

image

It’s just a little library but it should make composition tests dead simple:

using CompositionTests;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace CarDealership.Tests
{
  [TestClass]
  public class IntegrationTest
  {
    [TestMethod]
    public void DiscoverParts()
    {
      Composition.DiscoverParts(Program.Catalog);
    }
  }
}

That’s the whole test.

You can also scrub the formatted text before sending it to ApprovalTests.

using ApprovalUtilities.Utilities;
using CompositionTests;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace CarDealership.Tests
{
  [TestClass]
  public class IntegrationTest
  {
    [TestMethod]
    public void DiscoverParts()
    {
      Composition.DiscoverParts(
        Program.Catalog,
        StringExtensions.ScrubVersionNumber,
        s => s.ScrubPath(@"C:\Test"));
    }
  }
}

You can use any number of scrubbers to transform your text into a consistent state.  For example, scrubbing version numbers is useful when you use an AssemblyCatalog with an assembly that has an auto incrementing build number.  I’ve also included an extension to scrub the public key token, which is useful when using NCrunch (which disables assembly signing in its builds) alongside other test runners (which do sign the assemblies).  Or, you can use any Func<string, string> that you can think of.

The library also includes (and uses by default) an text formatter that orders the parts by name.  The default text formatter does not guarantee that the parts come out in any particular order.  I have seen them printed in different orders in different build environments, so enforcing order is a must if we want our tests to play nice with VS, NCrunch and a build server.  I’m still not sure that I’ve covered all the scenarios that can lead to things appearing out of order, because the behavior is not easy to trigger.  Luckily I had an example in production which I was able to recreate in the library’s unit tests.  If I encounter any more ordering problems, they will be addressed in future versions of the library.

Check It Out

Coding, Testing

ApprovalTests and MVC Views: Tackling the Build Server

In part one of ApprovalTests and MVC Views, Getting Started, I covered the basic mechanics of getting an MVC View under test.  In part two, Working with Data, I showed how you can create seams in your MVC project that allow you to avoid your repository and inject test data into the View.  These tests were more interesting that the tests in part one, and once you know where and how to create your seams for testing MVC views, you can really get into the groove covering your view with tests.  In this article we’ll look at running our tests on a build server.  Trying to run the tests on the build server will reveal one of the drawbacks of our dependency on the webserver.  But, we’ll see how to overcome it.

To review, we started this example with File->New Project and our goal is to create a set of tests that run consistently under Visual Studio, NCrunch, and CC.NET.  All of the source code is available as a mercurial repository on bitbucket.org.  If you haven’t already watched it, take 15 minutes to check out Llewellyn Falco’s video: Using ApprovalTests in .Net 20 Asp.Mvc Views.

Automated View Testing

You can build upon the techniques described in the previous two articles to build tests appropriate for your project. And you toddle along happily for a while writing your tests and code, until you hit the next speed bump when you try to send your code up to the build server.  After reading this article, you’ll know what to do to get your tests running on the server, and I hope it won’t seem like a speed bump to you at all. For now, lets just charge ahead like we don’t know any better and send our code to the build server.

I’m using CC.NET and other than the fact that I’m pretty sure its raising my electric bill my only complaint is that the configuration system is a little clunky. Maybe there’s better stuff out there but for now CC.NET is meeting my needs.  I configured the build server to keep an eye on MVCTestSite’s repository on bitbucket.org and fetched the test results once it was done. All the tests passed except for the view tests. Here’s our error:

Test method MVCTestSite.Tests.Views.HomeViewsTest.TestAboutView threw exception:

System.Exception: The following error occurred while connecting to:

http://localhost:61586/Home/About

Error:

Unable to connect to the remote server —> System.Net.WebException: Unable to connect to the remote server —> System.Net.Sockets.SocketException: No connection could be made because the target machine actively refused it 127.0.0.1:61586

Looks familiar. It’s the second gotcha all over again, the web server is not running.  On the build server we don’t have the option of clicking “Start Without Debugging” in Visual Studio.  Even if we could, we want our build server to be completely automated.  If the build process requires us to do something before we can test, it’s broken.

Lucky for us, CassiniDev is a fantastic open source project well suited for solving this problem (emphasis original):

The goal of the CassiniDev project is to provide an open platform for developing a robust ASP.Net web server implementation that addresses many of the limitations and difficulties encountered when using Cassini and/or Visual Studio Development Server.
CassiniDev is packaged as a standalone WinForms GUI application, a console application and library assembly suitable for self hosting and in automated testing scenarios including continuous integration and as a 100% compatible drop-in replacement for the Visual Studio 2008/2010 development server.

Sounds like just the thing we need. CassiniDev can take the place of IIS Express or Cassini, but since we want to run this server from a continuous integration system, we want to use the self-hosting option. We’ll do this by taking a reference to CassiniDev in our test project, starting the server during test setup, and stopping the server during teardown. With CassiniDev to serve our pages, we should be able to get our tests working in all three build environments.

Install CassiniDev using NuGet.

mvcviewtest7

Once we have a reference to CassiniDev, test classes that contain web requests need to extend the CassiniDevServer class. All of our view tests (some real and some imagined) already extend our own MvcTest base class to perform the PortFactory setup, so by extending CassiniDev server with MvcTest, we should be able to add CassiniDev support to all of our View tests at once.

Import the CassiniDev namespace and extend CassiniDevServer:

using CassiniDev;

namespace MVCTestSite.Tests.Views
{
  public class MvcTest : CassiniDevServer
  {
    public MvcTest()
    {
      PortFactory.MvcPort = 61586;
    }
  }
}

Next, add setup and teardown to MvcTest. In MSTest these methods are identified by the TestInitalizeAttribute and the TestCleanupAttribute. If we try to use these attributes in a class not marked with the the TestClassAttribute, we’ll get a warning, so we’ll add that attribute as well. You can make your life a little easier by writing the teardown method first. Remember, NCrunch is watching you, and if you write the setup method first, NCrunch will execute it and start the server. I found that this ended up locking some files in the bin directory and I had to restart Visual Studio. So, write the cleanup first.

[TestCleanup]
public void Cleanup()
{
  StopServer();
}

Now write the setup:

[TestInitialize]
public void Setup()
{
  StartServer(@"..\..\..\MVCTestSite", PortFactory.MvcPort, "/", "localhost");
}

The minimum information needed by CassiniDev is the path to the web application folder. With that info in hand, it can find an open port to host the site on and start. Since we are using ApprovalTests, we can’t allow CassiniDev to pick its own port, since changing the expected port would cause the tests to fail in a variety of ways. First, VerifyMvcPage wont be hitting the right port, so we’ll get a 500 error.  CassiniDev provides a method to generate paths with the correct port and we could use this method along with VerifyUrl to reach our MVC actions. However, this wont work for us because the port number is actually in the HTML we receive with ApprovalTests, so we need that port to remain consistent for comparison with the approved file.

CassiniDev allows us to choose the port using an overload to StartServer, so that’s what we do, and we pass in the port that we earlier assigned to the PortFactory. While we’re here, we should also change the port. This breaks our tests momentarily, but we don’t want to use the same port that IIS Express or Cassini are configured to use, we may have started one of those servers for debugging and CassiniDev cannot start if the port is already occupied.

Lets just pick the next port up:

public MvcTest()
{
  PortFactory.MvcPort = 61587;
}

Now CassiniDev can start and is able to serve our pages, our ApprovalTest fails as expected because we changed our port. File launcher reporter shows us some dramatic differences from what we have seen until now:

mvcviewtest8

All of our styles appear to be missing. By the time that the browser loads the received file, CassiniDev has already shut down and is not available to serve the request of the CSS data.  All that we see is raw HTML without any styles.

Although this isn’t the reason the test failed, it’s an important caveat when using CassiniDev and ApprovalTests together.  If you are using ApprovalTests to get feedback while making changes, you should probably use a local server like IIS Express, Cassini, or even the WinForms variation of CassiniDev.  This way you can see your changes with the styles applied and feel confident that your design is correct.  Once you have approved your changes, and you’re using ApprovalTests to detect regression, you can switch back to the self-hosted variant of CassiniDev.  You certainly need to switch back to self-hosted CassiniDev before sending your tests to the build server.

With that discussion in mind, lets switch to a DiffReporter and see what’s really changed in this file.

mvcviewtest9

First, we notice that the port is different. That change was expected so we can just use the diff utility to move the new line over to the approved file. But we also see that the welcome message changed because I switched computers after approving this file the first time. The greeting is part of the default MVC template that isn’t really important to me, so I simply delete this block from the approved file:

<div id="logindisplay">
  Welcome <strong>Starbuck\Jim Counts</strong>!
</div>

I also need to visit the master page and delete the same block. Now, if you require a personalized greeting then it is actually possible to make a strongly typed master page with a model, and you could introduce seams to control that model. I’ve also read that its possible to pass this data in using the ViewData dictionary, but I haven’t tried it. Check out SO2821488 if you’re interested in going that route.

After updating the master page to match the new expectations in my approved file, I run the tests again, expecting them to pass. Index works now, but About is impacted by the change I made to the master page, so I need to reapprove its output. I run the tests again in Visual Studio and they all pass, but NCrunch still seems unhappy. Looking at the output from NCrunch I can see that it’s getting 500 errors when it tries to run the View tests. It turns out that this line is the culprit:

[TestInitialize]
public void Setup()
{
  StartServer(@"..\..\..\MVCTestSite", PortFactory.MvcPort, "/", "localhost");
}

This relative path to the web application folder is not valid from NCrunch’s workspace. Nasty annoyance, or valuable feedback? In my opinion, this is valuable feedback, NCrunch has uncovered an invalid assumption. Its possible that this assumption would also be invalid on the build server. Having it break in NCrunch gives me the opportunity to make changes now, while it’s a little more convenient because I don’t have to inspect the failure in the build server logs. The first time I encountered this problem I solved it using a brute force approach to locate the MVCTestSite folder, but I knew this had a bad code smell and when I showed it to Llewellyn, he immediately proposed a more elegant solution.

So let’s skip the brute force and look at the elegant solution. Here is what we would like our test setup to look like:

[TestInitialize]
public void Setup()
{
  StartServer(MvcApplication.Directory, PortFactory.MvcPort, "/", "localhost");
}

It would be nice if MvcApplication would just tell us where it is on the disk. We can make it do so by leveraging ApprovalUtilities once more. MvcApplication lives in Global.asax.cs, the “code behind’ for Global.asax. Go there import this namespace:

using ApprovalUtilities.Utilities;

Then declare a read-only property called Directory with this implementation:

public static string Directory
{
  get
  {
    return PathUtilities.GetDirectoryForCaller();
  }
}

As soon as I finish implementing this property, NCrunch turns green. Looks like my tests are working again. I run them in Visual Studio and confirm.

Another Note for NCrunch Users

When using NCrunch and CassiniDev with a statically defined port, your tests may fail with an error like this:

Initialization method MVCTestSite.Tests.Views.HomeViewsTest.Setup threw exception. System.Exception: System.Exception: Port 61587 is in use..

This is caused by a race condition and will only happen if NCrunch and the Visual Studio test runner are executing the tests concurrently. Sometimes NCrunch wins the race and occupies the port, and other times Visual Studio wins the race. Once the winner finishes testing, you can run the loser again and the tests should pass.

For the time being this is just a minor inconvenience if you know what’s going on. Because our port number is part of the approved file, we can’t just use a compiler condition to give a different port to NCrunch, we have to let NCrunch try to use the same port. On the bright side, this is really only a problem on the developer’s workstation. NCrunch does not run on the build server, so the test agent running in that environment should be able to get exclusive access to the port whenever it runs.

Does It Work Now?

Yes, it works now on the build server without any further changes. When I commit the changes, the build server fetches the new code, and turns green.  So now we have three green lights: Visual Studio, NCrunch, and CC.NET.  We have a mix of tests on the Models, the Views, and the Controllers, some go through the web server and some are Plain-Old-Tests.  When we want to test Views with fake data, we need to use seams inside the MVC project.  When we want fully automated regression tests, we can use CassiniDev, and when we want to design with feedback, we can use a persistent server, but we have to start it ourselves.

If you’ve read this far, I congratulate you. This was a long read, but hopefully it gets you up to a plateau where you’re no longer confused by MvcApprovals and it all seems simple. If you do have any more questions leave a comment, or tweet with the hash-tag #ApprovalTests. Llewellyn watches that feed closely, and I’m usually lurking on it as well.  Better yet, if you get it working, please consider helping other developers by writing about your experiences and sharing online.

Thanks for reading and good luck.

Coding, Testing

ApprovalTests and MVC Views: Working With Data

In part one of ApprovalTests and MVC Views, Getting Started, I covered the basic mechanics of getting an MVC View under test, but the pages I tested weren’t very interesting.  Those pages were dynamic only in the slimmest sense (they displayed the user identity).  In this article we will introduce some data models into our testing scenario so that we can see what testing more realistic pages look like.

Just a reminder, we started this example at File->New Project and our goal is to create a set of tests that run consistently under Visual Studio, NCrunch, and CC.NET.  All of the source code is available as a mercurial repository on bitbucket.org.  Finally, if you are interested in testing MVC Views, a great place to get started (other than this blog series) is by watching Llewellyn Falco’s video on the subject: Using ApprovalTests in .Net 20 Asp.Mvc Views.

Introducing Models

Designing and testing Models are not that different than testing any other .NET class, and smarter people than me have already covered the subject. I won’t go into details about the models for this example site, but if you are interested you can see them in the code repository that accompanies the article. The only feature of interest in this example’s model is the repository implementation. Our repository is never available, any attempt to read data from the default repository implementation will throw a NotImplementedException. This behavior is meant to simulate an unavailable SQL server, but NotImplementedExceptions are easier to generate than SQLExceptions.

After creating the models, I deleted the default Index view and replaced it with a strongly typed list view using the default tools in Visual Studio. The test immediately fails in NCrunch with a NullReferenceException because current controller implementation does not pass a Model to the new View. We have broken our Index View test, and also broken one of the original tests provided by Visual Studio.

Lets replace the broken Controller test with something that works, then move on to our View test. In the interest of readability, this test has been formatted to fit your screen:

[TestMethod]
public void Index()
{
  // Arrange
  Mock<ICarRepository> repositoryMock 
  repositoryMock = new Mock<ICarRepository>();
  repositoryMock.DefaultValue = DefaultValue.Mock;
  HomeController controller;
  controller = new HomeController(repositoryMock.Object);

  // Act
  controller.Index();

  // Assert
  repositoryMock.Verify(repo => repo.GetCars());
}

Since this is a Controller test we have the option to create an instance and manipulate its state. So I create a mock repository and pass it to the HomeController, call Index, and use the mock to verify that Index method is trying to get its data from the repository. Whether or not that data access call should be locked down by a test, or simply kept hidden as an implementation detail is up to you. The point of showing this example is to remind you that this is exactly the kind of thing you can’t do with a View test. Any changes to the Controller instance created in the test method will not be reflected in the View when it’s compiled on the web server and delivered to MvcApprovals.

In this example, I do want the data access to remain locked down so that this test will ensure me that I don’t break the default behavior as I create seams to get my View test to work. In “production” we want the call to Index to hit the repository and bring down the cars. Naturally, I had to make some changes to the HomeController after writing this test, and after overloading the constructor, adding a field for the repository, and putting the necessary method call into the Index method, the test passes, but our View test still fails.

We know that we have an uncooperative repository that always throws exceptions, and that might be a reason its failing. Sure enough, it is.

mvcviewtest4_thumb2

We need to resolve that issue before we can approve the new version of the index page. We can’t resolve the issue by creating Mocks in our test method. Remember that the controller instance we create in the test is just a convenient way to pass metadata to MvcApprovals. The real code that we’re testing is the code on the web server. The web server has its own instance of the HomeController, which it instantiated using the default constructor.  In our case, the default constructor happens to create an instance of the unusable repository. The key to testing this new Index view is to create a seam that lets us avoid the repository entirely.

Once we create seams, we can add MVC actions that pass fake data into the seams for us. These new actions have to be inside the production site, not the test project.  The web server needs to see and execute the test actions for us.  After we create the test actions, we update our View tests to use the test actions instead of the default actions.  Maybe now you’re screaming, “Dear God, first you try to convince me that taking a dependency on the web server isn’t an integration test, and now you want to put test code in production!” Well, you shouldn’t scream like that, you’re scaring the neighbors. Its just code, and we’ll take steps later to isolate this code so it can be easily excluded during Release builds.

Now that you’re done screaming, lets work our way inward until we’ve implemented the scenario described above. First, we’ll update our test so that it points to our unwritten test action in the MVC project.

[TestMethod]
public void TestIndexView()
{
  MvcApprovals.VerifyMvcPage(new HomeController().TestIndex);
}

Now we can create this method on our HomeController using our favorite refactoring tool. Before we can implement this action, we need to create a seam inside HomeController. Lets think about what the default Index method does:

public ActionResult Index()
{
  var cars = repository.GetCars();
  return View(cars);
}

Looks to me like TestIndex needs to do two things:

public ActionResult TestIndex()
{
  // Get the data
  // Pass data to the view.
}

Of these two parts, one part is the same between test and production, and one part varies. We need to separate the part that varies from the part that stays the same. In our simple scenario, that means moving the part that stays the same “Pass data to the view” into its own method so that both the production method and the test method can do what they want with the part that changes “Get the data”.

Here is the new Index method and the seam:

public ActionResult Index()
{
  var cars = repository.GetCars();
  return Index(cars);
} 
private ActionResult Index(IEnumerable<ICar> cars)
{
  return View(cars);
}

For reasons that will become apparent, the private seam must have the same name as the MVC action. We make the seam private to avoid ambiguity when MVC tries to resolve the route to the Index method.

We can create any model that we want in TestIndex, and pass it to the view using the private seam. The repository is not required inside the seam so we don’t care if it throws exceptions. In the test action we could use fakes or mocks, but we don’t even need to. We have access to our concrete models, and we can just insatiate them with any data we want. In this example, our Car implementation is just an ordinary class, but you can just as easily create EntityFramework or LINQ to SQL objects.

Before getting into that, lets isolate our test code by turning HomeController into a partial class and putting TestIndex into another file. Create HomeController.Tests.cs in your Controllers folder and put TestIndex in there:

public partial class HomeController
{
  public ActionResult TestIndex()
  {
    // Create data
    // Pass to view
    return null;
  }
}

Now lets create some test data and pass it to the seam.

public ActionResult TestIndex()
{
    // Create data
    IEnumerable<ICar> cars = new Car[] {
        new Car() { Make = "VW", Model = "Passat", Year = 2002 },
        new Car() { Make = "Honda", Model = "Fit", Year = 2007 },
    };

    // Pass to view
    return Index(cars);
}

That should be it. Our test should pass. But now its failing with a new error:

mvcviewtest5_thumb8

MVC is looking for a View called “TestIndex” even though our seam is called Index. The view name inference depends on the view engine implementation.  In the WebFormsViewEngine, the view name seems to be inferred from the virtual path as near as I can tell.  After reading the source code for awhile my brain started to hurt.  Anyway, whether MVC is inferring the data from the path, or getting it from the route data, we can override the behavior.  We’ll give MVC the view name using an overload that allows us to specify the name explicitly:

private ActionResult Index(IEnumerable<ICar> cars)
{
  return View("Index", cars);
}

This introduces a magic string into our code and we’d like to avoid that.  We can do so by taking advantage of a convenience method provided by the ApprovalTests companion library, ApprovalUtilities. We need to add a reference to ApprovalUtilities to our MVC project.  The easiest way to do that is to use NuGet.  If we use NuGet we will also get a reference to ApprovalTests.  If having a reference to ApprovalTests in your MVC project bothers you, don’t use NuGet.  You can always add the reference by hand if you like.   I’m going to use NuGet.

After adding the reference, import this namespace:

using ApprovalUtilities.Asp.Mvc;

Then update the private Index method again:

private ActionResult Index(IEnumerable<ICar> cars)
{
  return View(cars).Explicit();
}

This extension method is functionally equivalent to the magic-string version.

Note for NCrunch Users

Update 2012/06/18: You can skip this!  NCrunch v1.40b resolved this issue.  Cheers!

There is an issue using this extension method in the current version of NCrunch. On two of my machines, NCrunch will fail the Controller test (not the View test) when run with the Explicit method. I get this exception:

System.MissingMethodException: Method not found: ‘System.Web.Mvc.ViewResult ApprovalUtilities.Asp.Mvc.MvcUtilites.Explicit(System.Web.Mvc.ViewResult)’.

The documentation for MissingMethodException indicates that the exception is thrown when you try to dynamically access a method that does not exist.  For example, you use reflection to access a type, but then call InvokeMember with the name of a method that doesn’t exist.  I’ve looked at the assemblies generated by NCrunch, both with and without instrumentation, and its not trying to do this.  So I don’t know why we get this exception.  This occurs whether called as an extension method or as a regular static method call. This occurs in both Debug and Release configurations. This occurs when compiled for Any CPU and when compiled for x86. This occurs under MSTest and under xUnit.  It happens in 1.38b and 1.39b.

Researching this, I could only find one other person who looked like they had an issue that was remotely similar, but the similarity was very remote indeed and he hadn’t found a resolution yet. So for the time being I’m working around this issue by checking for NCrunch at compile time:

  private ActionResult Index(IEnumerable<ICar> cars)
  {
#if NCRUNCH
    return View("Index", cars);
#else
    return View(cars).Explicit();
#endif
  }

There are two other alternatives. First we can give up on using the extension method, and second we can give up on NCrunch. I don’t think we need to give up either. NCrunch is a great tool, but its still in beta. This is the first real issue I’ve had with it since the March 2012 release. I’m hopeful that this issue can be resolved with further research, or in a future version of the tool . This compile time condition will remind me to test this with NCrunch the next time an update is released, and to keep looking for an answer online.  When this issue is resolved I can get rid of the branch that uses the magic string.

Update 2012-05-18: 

I decided it might be a good idea to report this issue to the NCrunch developer, Remco Mulder, over on the NCrunch Forums.  We we were able to isolate the cause of the exception.  Our example site is built against MVC 3 and ApprovalUtilities is built against MVC 2.  The current version of NCrunch seems to be incorrectly loading both MVC assemblies into its test environment, creating confusion over what version of ViewResult is valid for use with Explicit().  The good news is that the problem only occurs if you have MVC 2 in your GAC.  I still need it because I have some sites that are built against MVC 2, but if you don’t have MVC 2 on your machine, then you probably wont see this problem at all, and you won’t need to use this workaround.

Back to Our Story…

Now that we’ve explicitly specified the view’s name (one way or another) we can run the tests again. This time we get an approval failure and ApprovalTests launches the browser to show us the received file. Great Success!

mvcviewtest6_thumb2

This page may not be the epitome of good design or usability, and maybe we want to customize it further, but at the moment we can see our test data in the page, and that’s awesome. If we do want to customize, we can do so with feedback. When we finally get the page looking the way we would like, we can approve the output and achieve protection from regressions.

At this point we have a few tests. We have an original test generated by Visual Studio, and we replaced one generated test with one that now exercises the default Index method with a mock. We have our two View tests which use MvcApprovals, and one of these uses seams to test the Index with fake data. We also have a couple of tests on the Models which I haven’t mentioned until now, because they are not very interesting. The only failing test is the Index view test, and it passes after I approve the output.

The tests are green in two build environments, Visual Studio and NCrunch. Seems like a good time for a commit, followed by some housekeeping.

Housekeeping

This housekeeping round is relatively minor. We want to make sure that our test actions are not available when we compile in Release mode. This is optional, but probably a good idea.

We can just eliminate half of our partial class using a compiler condition:

#if DEBUG
namespace MVCTestSite.Controllers
{
  public partial class HomeController
  {
    public ActionResult TestIndex()
    {
      // Create data
      IEnumerable<ICar> cars = new Car[] {
        new Car() { Make = "VW", Model = "Passat", Year = 2002 },
        new Car() { Make = "Honda", Model = "Fit", Year = 2007 },
      };

      // Pass to view
      return Index(cars);
    }
  }
}
#endif

Since our test action will no longer be available in release mode, we should also eliminate our “test pointer” when compiling in release mode:

namespace MVCTestSite.Tests.Views
{
  [TestClass]
  [UseReporter(typeof(FileLauncherReporter), typeof(ClipboardReporter))]
  public class HomeViewsTest : MvcTest
  {
#if DEBUG
    [TestMethod]
    public void TestIndexView()
    {
      MvcApprovals.VerifyMvcPage(new HomeController().TestIndex);
    }
#endif

    [TestMethod]
    public void TestAboutView()
    {
      MvcApprovals.VerifyMvcPage(new HomeController().About);
    }
  }
}

We can leave the test on the About view outside the conditional, since it does not require any non-production actions.

Since we are still in Debug mode, these conditions remain true and our tests still run and pass. There is another factor to consider when our project is compiled in Release mode and that is optimization. Our friend the Explicit() method infers the view name by examining the name of the method that calls it. This is why we needed to make sure that our private Index method had the same name as the public method. However, the default configuration for Release mode includes optimizations which could cause all of our action methods to be in-lined out of existence. This changes the method that calls Explicit. In my case, an anonymous delegate ended up calling Explicit and MVC complained that there was no view called “lambda_method”. Oops.

Llewellyn’s video shows how to work around this issue by disabling optimization in the Release configuration. If that idea bothers you, there is another way to do it on a method-by-method basis using an attribute:

[MethodImpl(MethodImplOptions.NoInlining)]
private ActionResult Index(IEnumerable<ICar> cars)
{
  return View(cars).Explicit();
}

Both work, but you need to pick one of these or you risk the “View Not Found” error.

Now that we have seen how to work with data, we can use these techniques continue to develop our site.  We can use ApprovalTests or traditional Asserts to work with our Models and Controllers, and we now have the ability to use MvcApprovals with our views.

Up until now, we’ve been working in two build environments, Visual Studio and NCrunch.  Although these environments are different, they share a lot of environmental factors because they both run on the same host.  To help ensure that our tests are robust, we should get them running on a different host with a different set of rules.  In the last part of this series, ApprovalTests and MVC Views: Tackling the Build Server, I’ll try to get these tests running under CC.NET.