What’s New in CompositionTests 2.0

Download the latest version of CompositionTests from nuget.org!

ApprovalTests 3.0

Updated the dependency on ApprovalTests to 3.0.01. Thanks to the new version updating policy for ApprovalTests, CompositionTests should remain forward compatible with future versions of ApprovalTests, unless there are breaking changes in the API.

New version policy

Following LLewellyn’s lead with ApprovalTests, I am adopting a JSON.NET-style version update policy. Adopting this policy will enable me to sign CompositionTests in the future without creating forward-compatibility problems for anyone else. For now, the package remains unsigned because its other dependency, the MEFX Diagnostic Library is unsigned. I’ll have to decide if I’m willing to do anything about that before I can consider a signed version of CompositionTests.

The impact is that the CompositionTests AssemblyVersion will stay at 2.0.0 from now on. The real version can be found by looking at AssemblyFileVersion, or by looking at the nuget package version, which will be 2.0.1 for this release.

Common Language Specification Compliance

The CompositionTests library now declares itself CLS compliant. However, MEFX.Core does not make the same declaration, so certain methods that interact with the core are individually marked non-compliant. I don’t think that MEFX.Core uses anything non-compliant, the library is simply missing the declaration of compliance. I don’t think Microsoft has plans to provide any more updates to this piece, so I’ll have to decide that I’m willing to modify and maintain a fork of MEFX.Core before I can do anything about that missing attribute.

Removed Obsolete Methods

Methods and types marked with the ObsoleteAttribute in the 1.0 time-frame have been removed in order to clean up the interface in 2.0. You must now migrate to Verify* and MefComposition if you wish to use new versions of the library.

From "MEF, WTF?!" to "MEF, FTW!"

In addition to my talk on CoffeeScript at SoCalCodeCamp I also reprised my talk on testing and debugging MEF. I didn’t record this session at UCSD but if you missed it, you can watch a recording of the “premiere” at CSU Fullerton. While I covered a lot of the same ground at UCSD, I did revise the slides and rewrote the sample code from scratch. The biggest difference is that at UCSD I introduced CompositionTests at the end of the talk, which I hope lowers the barrier of entry for beginners.

This post should be short because I’ve already written a few posts on this topic. However, if you attended the session and are here looking for additional information, here’s a list of resources:

Since I don’t have a video, here’s a picture of me standing in front of a Pizza Factory.  See, it’s more that just a lame example from a design patterns book.

WP_000324

A Few Comments on the Sample Code

As I mentioned I rewrote the code. Mostly because the old code was way too complicated. For some reason I was obsessed with using DirectoryCatalogs for everything during the first six months of my career with MEF. In the last six months I’ve come to appreciate the other catalogs, especially while refactoring and certainly for demos. This helped simplify the code a great deal.

Having the benefit of giving the talk before, I knew that I didn’t have time to cover all the failure scenarios I programmed into the original demo. I think I had about 8 different scenarios in the original PizzaStore, and in reality there is only time to demo about two. So, while rewriting the code, I only setup two scenarios, one failure that doesn’t throw an exception, and one that does.

To demonstrate the difference between .NET 4 and 4.5, I used a separate solution. The first time around I copied the solution folder, renamed it, changed references and I was rockin’ and rollin’. This worked well enough to Demo, but the truth was that only one line of code needed to change between .NET 4 and 4.5. This time around the projects share the same files using the “Add As Link” option in Visual Studio.

If you want to poke around the code, you should know that the “Program” project does almost nothing. It holds part definitions that the “Demo” project can play with. The Program class, while it does nothing useful, does show an example of my preferred way to share ExportProviders and ComposablePartCatalogs with test code.

Over in the Demo project we use the TypeCatalog instances to control whether the composition succeeds or fails. We also have an example of implementing a composition test by hand, which includes some really ugly brute force formatting necessary to ensure that the parts are listed in the same order every time. Finally, we have an example of composition test using my own CompositionTests library, which not only removes the burden of writing broiler plate code, but handles the ordering for you.

When is the ExportProvider Interesting?

Recently, when writing about the CompositionTests API (here) I wrote that our test should focus on the interesting part of the composition and that most of the time the catalog is the only interesting item.  Later I wrote that there were other scenarios where the ExportProvider was also interesting and when I wrote that I was thinking of custom ExportProvider implementations.  Just Google “mef custom exportprovider” and you will find a wealth of articles written by people who created their own ExportProviders to adapt MEF for their specific needs.  I think I even built one back when I first started out with MEF.

But lets say you haven’t needed or wanted to roll your own ExportProvider.  Does trusty old CompositionContainer ever effect composition?  Can it ever be interesting enough to motivate you to use VerifyCompositionInfo?

A Simple Mailer

Here’s an example of when you might want to use VerifyCompositionInfo because the CompositionContainer is interesting.  Say you are working on an application that wants to send emails. You might start with a little mail sender like this:

[Export(typeof(ISendMail))]
public class BasicMailSender : ISendMail
{
    [ImportingConstructor]
    public BasicMailSender(
        SmtpHost host,
        Port port)
    {
        this.port = port;
        this.host = host;
    }

    public void SendMessage(MailMessage message)
    {
        // ...
    }
}

You have read the CompositionTests readme, and written a simple Program class that allows you to share your ComposablePartCatalog (in this case a TypeCatalog) with your test.

public class Program
{
    static Program()
    {
        Catalog = new TypeCatalog(typeof(BasicMailSender));
        Host = new CompositionContainer(Catalog);
    }

    public static TypeCatalog Catalog { get; private set; }

    public static CompositionContainer Host { get; private set; }

    public static void Main(string[] args)
    {
    }
}

And written a test.

using ApprovalTests.Reporters;
using CompositionTests;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace MailApp.Tests
{
    [TestClass]
    [UseReporter(typeof(DiffReporter))]
    public class CompositionTest
    {
        [TestMethod]
        public void VerifyComposition()
        {
            MefComposition.VerifyTypeCatalog(Program.Catalog);
        }
    }
}

Given our current setup, we would expect an unhappy composition, shown below.

[Part] MailApp.BasicMailSender from: TypeCatalog (Types='MailApp.BasicMailSender').
  [Primary Rejection]
  [Export] MailApp.BasicMailSender (ContractName="MailApp.BasicMailSender")
  [Import] MailApp.BasicMailSender..ctor (Parameter="host", ContractName="MailApp.SmtpHost")
    [Exception] System.ComponentModel.Composition.ImportCardinalityMismatchException: No exports were found that match the constraint: 
    ContractName    MailApp.SmtpHost
    RequiredTypeIdentity    MailApp.SmtpHost
   at System.ComponentModel.Composition.Hosting.ExportProvider.GetExports(ImportDefinition definition, AtomicComposition atomicComposition)
   at System.ComponentModel.Composition.Hosting.ExportProvider.GetExports(ImportDefinition definition)
   at Microsoft.ComponentModel.Composition.Diagnostics.CompositionInfo.AnalyzeImportDefinition(ExportProvider host, IEnumerable`1 availableParts, ImportDefinition id)
  [Import] MailApp.BasicMailSender..ctor (Parameter="port", ContractName="MailApp.Port")
    [Exception] System.ComponentModel.Composition.ImportCardinalityMismatchException: No exports were found that match the constraint: 
    ContractName    MailApp.Port
    RequiredTypeIdentity    MailApp.Port
   at System.ComponentModel.Composition.Hosting.ExportProvider.GetExports(ImportDefinition definition, AtomicComposition atomicComposition)
   at System.ComponentModel.Composition.Hosting.ExportProvider.GetExports(ImportDefinition definition)
   at Microsoft.ComponentModel.Composition.Diagnostics.CompositionInfo.AnalyzeImportDefinition(ExportProvider host, IEnumerable`1 availableParts, ImportDefinition id)

So we need to provide SmtpHost and Port implementations. The decision on the Port implementation is easy. While we might want to use a non-default Port during testing, we feel pretty safe determining that we want to use the default SMTP port (25) in production. So we can whip up a DefaultPort implementation and export it.

[Export(typeof(Port))]
public class DefaultPort : Port
{
    public DefaultPort() :
        base(25)
    {
    }
}

Everything works as expected if we update our TypeCatalog with this new type.

static Program()
{
    Catalog = new TypeCatalog(typeof(BasicMailSender), typeof(DefaultPort));
    Host = new CompositionContainer(Catalog);
}

And now the test produces this result:

[Part] MailApp.BasicMailSender from: TypeCatalog (Types='MailApp.BasicMailSender, MailApp.DefaultPort').
  [Primary Rejection]
  [Export] MailApp.BasicMailSender (ContractName="MailApp.BasicMailSender")
  [Import] MailApp.BasicMailSender..ctor (Parameter="host", ContractName="MailApp.SmtpHost")
    [Exception] System.ComponentModel.Composition.ImportCardinalityMismatchException: No exports were found that match the constraint: 
    ContractName    MailApp.SmtpHost
    RequiredTypeIdentity    MailApp.SmtpHost
   at System.ComponentModel.Composition.Hosting.ExportProvider.GetExports(ImportDefinition definition, AtomicComposition atomicComposition)
   at System.ComponentModel.Composition.Hosting.ExportProvider.GetExports(ImportDefinition definition)
   at Microsoft.ComponentModel.Composition.Diagnostics.CompositionInfo.AnalyzeImportDefinition(ExportProvider host, IEnumerable`1 availableParts, ImportDefinition id)
  [Import] MailApp.BasicMailSender..ctor (Parameter="port", ContractName="MailApp.Port")
    [SatisfiedBy] MailApp.DefaultPort (ContractName="MailApp.Port") from: MailApp.DefaultPort from: TypeCatalog (Types='MailApp.BasicMailSender, MailApp.DefaultPort').

[Part] MailApp.DefaultPort from: TypeCatalog (Types='MailApp.BasicMailSender, MailApp.DefaultPort').
  [Export] MailApp.DefaultPort (ContractName="MailApp.Port")

The Port is now satisfied, but SmtpHost is trickier. While the default port for SMTP is defined by standard, there is no such thing as a default server. Now, I would note that one option is to simply approve this result. There is no rule that says you can only approve a CompositionTest when it has no exceptions. The fact that SmtpHost can’t be satisfied could just mean that this part must be provided by a third-party at deployment time. However, there are two reasons why I’ll try to satisfy all of the parts in this test.

  • Ignoring exceptions is like ignoring warnings. When there are only a few you can say to yourself, “Well, that one is there for a reason, and I understand why.” But when the number of warnings/exceptions grow, the signal to noise ratio rises and eventually obscures real errors.
  • If I don’t try to satisfy the part, I won’t be able to show you a scenario where the ExportProvider is interesting.

Lets say that we have a SmtpHost in our test project that we use for development, it points at the localhost. When making these kind of test collaborators I usually nest the Testing class inside the test it supports.

[TestClass]
[UseReporter(typeof(DiffReporter))]
public class CompositionTest
{
    [TestMethod]
    public void VerifyComposition()
    {
        MefComposition.VerifyTypeCatalog(Program.Catalog);
    }

    [Export(typeof(SmtpHost))]
    public class TestingSmtpHost : SmtpHost
    {
        public TestingSmtpHost()
            : base("localhost")
        {
        }
    }
}

It would be nice to use this part when verifying composition, but since it is in our test class, our production code can’t see it. One way around this is to add an instance to our CompositionContainer using a CompositionBatch. We’ll need to add a method, Program.AddInstance, to allow this.

public class Program
{
    static Program()
    {
        Catalog = new TypeCatalog(typeof(BasicMailSender), typeof(DefaultPort));
        Host = new CompositionContainer(Catalog);
    }

    public static TypeCatalog Catalog { get; private set; }

    public static CompositionContainer Host { get; private set; }

    public static void AddInstance(object part)
    {
        var batch = new CompositionBatch();
        batch.AddPart(part);
        Host.Compose(batch);
    }

    // ...
}

Now we can add a class constructor to our test, so that Program is only manipulated once during the course of the test.

[TestClass]
[UseReporter(typeof(DiffReporter))]
public class CompositionTest
{
    static CompositionTest()
    {
        Program.AddInstance(new TestingSmtpHost());
    }

Note that you can’t just throw any object into the method and expect it to compose, it has to be an attributed MEF part. Ok, now we might expect our test result to show a happy composition, but the BasicMailSender still fails to compose.

[Import] MailApp.BasicMailSender..ctor (Parameter="host", ContractName="MailApp.SmtpHost")
  [Exception] System.ComponentModel.Composition.ImportCardinalityMismatchException: No exports were found that match the constraint: 
    ContractName    MailApp.SmtpHost
    RequiredTypeIdentity    MailApp.SmtpHost

What’s up with that? Look closely at the AddInstance method. Nowhere in that method do we manipulate the Catalog.  When we use CompositionBatch, we are adding part directly to the CompositionContainer–outside the catalog. So, our ExportProvider (in this case just a plain old CompositionContainer) is suddenly interesting. Without it, there is no way for the CompositionInfo used by CompositionTests to know about the instance of TestingSmtpHost that we added.

Lets rewrite our test to use an overload of VerifyCompositionInfo so that we can pass in the ExportProvider.

[TestMethod]
public void VerifyComposition()
{
    MefComposition.VerifyCompositionInfo(Program.Catalog, Program.Host);
}

And now our composition is happy.

[Part] MailApp.BasicMailSender from: TypeCatalog (Types='MailApp.BasicMailSender, MailApp.DefaultPort').
  [Export] MailApp.BasicMailSender (ContractName="MailApp.BasicMailSender")
  [Import] MailApp.BasicMailSender..ctor (Parameter="host", ContractName="MailApp.SmtpHost")
    [SatisfiedBy] MailApp.Tests.CompositionTest+TestingSmtpHost (ContractName="MailApp.SmtpHost") from: MailApp.Tests.CompositionTest+TestingSmtpHost
  [Import] MailApp.BasicMailSender..ctor (Parameter="port", ContractName="MailApp.Port")
    [SatisfiedBy] MailApp.DefaultPort (ContractName="MailApp.Port") from: MailApp.DefaultPort from: TypeCatalog (Types='MailApp.BasicMailSender, MailApp.DefaultPort').

[Part] MailApp.DefaultPort from: TypeCatalog (Types='MailApp.BasicMailSender, MailApp.DefaultPort').
  [Export] MailApp.DefaultPort (ContractName="MailApp.Port")

Notice that while the TestingSmtpHost satisfies the host parameter on the BasicMailSender constructor, its “from” tag doesn’t indicate that it comes from a catalog (in contrast, you can see that DefaultPort came from the TypeCatalog). Likewise, TestingSmtpHost has no [Part] listing and a [Part] listing for DefaultPort does exist.

Most of the time, you can analyze your composition using just the catalog, but the container can have parts too. When your container has parts, that’s when you’ll need to use VerifyCompositionInfo directly to get the full analysis.

You can download and play with the MailApp code from the Sample directory in the CompositionTests github repository.

MEF Composition Tests, Redux

Photo Credit: FutUndBeidl

A while back I gave a talk at SoCalCodeCamp about testing MEF composition.  Because there are a few steps involved in setting up the test, I wrote Stop Guessing About MEF Composition And Start Testing.  I had hoped that the article, along with the source code from my talk would help more people cross the gap from having once heard that it was possible to test composition, to actually implementing tests.  Since there were only about five people at my talk, it didn’t take long for the number of blog readers to exceed the number of attendees, and yet I still wonder how many people decide setting up the test is too complicated and just decide to use MEFX or Visual MEFX “as needed”.

While MEFX tools are valuable (and these composition tests are just a more powerful way to automate MEFX), running tests on an ad-hoc basis is never ideal.  Bugs creep in when you don’t expect them to, and automatic tests are your defense against them.  I also find composition tests provide valuable design-time feedback.  While taking questions at the end of the talk, I was asked about my process for designing MEF parts.  I answered that I more or less disregarded MEF until the end of the process, and then just added the needed attributes.  That was a truthful answer, but over the intervening months, I’ve realized it was the wrong answer.  What finally put the nail in the coffin was this great article: How test-driven development works (and more!), by J. B. Rainsberger.  After reading, it hit me that that I had setup a little waterfall process.  First, design and build a unit.  Then, glue MEF onto it.  The problem is that even when “keeping MEF in mind” I would sometimes do foolish things like pass a bunch of primitives into a constructor, and this would lead me to rework at the end of the development cycle when I tried to shove my class into a container.

So, to avoid rework and break some bad habits, I decided to start testing composition to get feedback during the design phase as soon as I know I’m going to use MEF (which these days is most of the time).  Its already paying dividends on my current project, as soon as I see the need to pass in a simple piece of data like a string or an integer, my composition test will blow up if I pass it through the constructor.  A lot of times I still prefer to pass the data in through the constructor, but since my test is blowing up, I’m forced design and test the system for getting that data to the constructor immediately, rather than at the end of the cycle.

I started testing composition out of necessity, I had a broken composition that needed fixing.  Now, I do it to save time and effort, and it seems more valuable to me every day.  More people should do it, but I’m afraid some will be turned off by the number of steps and concepts involved.

Here are a few of them:

  1. Synchronizing catalogs between test and production
  2. For some, this could also be their first exposure to ApprovalTests
  3. Finding a copy of MEFX.
  4. Setting up the test.

So, does it have to be so complicated?

Reducing the Surface Area

Lets look at a sample project and implement the integration test I described in my last article about composition tests.  Some people have a one-track mind.  Mine has two tracks: cars and pizza.  This time we will go with cars and our sample project will be a simple program called CarDealership with some parts related to cars in it.  It doesn’t have to be very complex since our primary interest is the test.

Lets look at that test:

[TestClass]
public class IntegrationTest
{
  [TestMethod]
  public void DiscoverParts()
  {
    try
    {
      var catalog = new DirectoryCatalog(".");
      var host = new CompositionContainer(catalog);
      var compositionInfo = new CompositionInfo(catalog, host);
      using (var stringWriter = new StringWriter())
      {
        CompositionInfoTextFormatter.Write(
            compositionInfo,
            stringWriter);
        Approvals.Verify(stringWriter.ToString());
      }
    }
    catch (ReflectionTypeLoadException ex)
    {
      Array.ForEach(
          ex.LoaderExceptions,
          lex => Console.WriteLine(lex.ToString()));
      throw;
    }
  }
}

A few things are already different about the test.  There’s no UseReporterAttribute on the class anymore, because I usually define this in AssemblyInfo.cs now using an assembly scoped attribute:

// AssemblyInfo.cs
using ApprovalTests.Reporters;
[assembly: UseReporter(typeof(DiffReporter))]

I also moved a few more lines inside the test body because I don’t usually have a reason to run more than one test with the same CompositionInfo.  Also, since my intention is to break this test down into something cleaner, I want to see it all in one place.

Right now, our test wont compile because we don’t have references to ApprovalTests or MEFX.  ApprovalTests has been available on NuGet for some time, and lately the NuGet package has been kept up to date with releases on SourceForge.  So, go ahead and use NuGet to install ApprovalTests:

In my previous post I explained where to find MEFX, which includes the CompositionDiagnostics library that we need for our test.  But you had to download it or compile it, and then add the reference by hand… icky.  But, we had no choice because this library wasn’t available on NuGet.  Well, it is available now, because I put it there a few months ago.  Now its much easier to get your hands on a copy of the library:

image

Now our test compiles and runs.  While you weren’t looking I added some parts to the CarDealership program.  But it doesn’t matter because I haven’t setup a pre-build event to copy assemblies from production to test.

I never liked doing that.  I knew it would be brittle and it was.  Ideally, I’d just like to use the exact catalog from production in my test.  It turns out this is a lot easier to arrange if you don’t use a DirectoryCatalog.  AssemblyCatalogs or TypeCatalogs are easy to share programmatically.  So, ask yourself if you really need to use a DirectoryCatalog.  Lets see what the test looks like when we use an AssemblyCatalog:

[TestMethod]
public void DiscoverParts()
{
  try
  {      
    var compositionInfo = new CompositionInfo(
      Program.Catalog, 
      Program.Host);
    using (var stringWriter = new StringWriter())
    {
      CompositionInfoTextFormatter.Write(
          compositionInfo,
          stringWriter);
      Approvals.Verify(stringWriter.ToString());
    }
  }
  catch (ReflectionTypeLoadException ex)
  {
    Array.ForEach(
        ex.LoaderExceptions,
        lex => Console.WriteLine(lex.ToString()));
    throw;
  }
}

To facilitate sharing, I’ve added static properties on the Program class.  They are simple properties, initialized in Program’s class constructor:

using System.ComponentModel.Composition.Hosting;
using System.ComponentModel.Composition.Primitives;
using System.Reflection;

namespace CarDealership
{
  public class Program
  {
    static Program()
    {
      Catalog = new AssemblyCatalog(Assembly.GetAssembly(typeof(Program)));
      Host = new CompositionContainer(Catalog);
    }

    public static ComposablePartCatalog Catalog { get; private set; }

    public static ExportProvider Host { get; private set; }

    private static void Main(string[] args)
    {
    }
  }
}

Looking back at the test, we can see the effect of sharing the catalog and host from Program.  We can now get the CompositionInfo with just one statement:

var compositionInfo = new CompositionInfo(Program.Catalog, Program.Host);

Another interesting development, the catalog and host are only referenced on this line.  Could we create a method that takes “compositionInfo” as a parameter without changing this test’s behavior?  The answer is no, but lets try it and see why.

[TestMethod]
public void DiscoverParts()
{
  var compositionInfo = new CompositionInfo(
    Program.Catalog,
    Program.Host);
  DiscoverParts(compositionInfo);
}

private static void DiscoverParts(CompositionInfo compositionInfo)
{
  try
  {
    using (var stringWriter = new StringWriter())
    {
      CompositionInfoTextFormatter.Write(
          compositionInfo,
          stringWriter);
      Approvals.Verify(stringWriter.ToString());
    }
  }
  catch (ReflectionTypeLoadException ex)
  {
    Array.ForEach(
        ex.LoaderExceptions,
        lex => Console.WriteLine(lex.ToString()));
    throw;
  }
}

To make the CompositionInfo a parameter, we pulled it out of the try-catch block.  Since the CompositionInfo constructor can throw the ReflectionTypeLoadException we’re trying to catch, moving it out of the try-catch block defeats the purpose of the exception handler.  Moving the exception handler back to the test method requires us to implement the handler (which never changes) every time we write a test.  We need to control when “new” is called on CompositionInfo.  We can do so by changing our parameter into a Func<CompositionInfo>:

[TestMethod]
public void DiscoverParts()
{
  DiscoverParts(() => new CompositionInfo(Program.Catalog, Program.Host));
}

private static void DiscoverParts(Func<CompositionInfo> getCompositionInfo)
{
  try
  {
    using (var stringWriter = new StringWriter())
    {
      CompositionInfoTextFormatter.Write(
          getCompositionInfo(),
          stringWriter);
      Approvals.Verify(stringWriter.ToString());
    }
  }
  catch (ReflectionTypeLoadException ex)
  {
    Array.ForEach(
        ex.LoaderExceptions,
        lex => Console.WriteLine(lex.ToString()));
    throw;
  }
}

Now, I could still create a CompositionInfo instance in the test method, then use the delegate to pass back that instance.  Doing so, I would not be protected by the exception handler, but that would be a minor problem.  The exception handler only exists as a time saver, when a ReflectionTypeLoadException occurs, it will print additional information not included in the default dump.  Without the handler, you would need to rerun the test in the debugger in order to see the loader exceptions.  So, its not the end of the world, but at least now its possible to reuse the exception handler as long as we remember how to build the delegate correctly.

Can we make DiscoverParts easier to use by removing the caveat that the constructor must be part of the delegate?  We could provide an overload that creates the delegate for the caller:

[TestMethod]
public void DiscoverParts()
{
  DiscoverParts(Program.Catalog, Program.Host);
}

private static void DiscoverParts(ComposablePartCatalog catalog, ExportProvider host)
{
  DiscoverParts(() => new CompositionInfo(catalog, host));
}

private static void DiscoverParts(Func<CompositionInfo> getCompositionInfo)
{
   // ...
}

That’s better, and maybe we don’t need that delegate after all.  For the time being, I’m going to leave it there because I think the more interesting question is: Do we need the host?  If you look again at the construction of the host in Program, you’ll see that it’s the simplest possible CompositionContainer, and the only interesting thing about it is the catalog it consumes.  So why not focus our test on the catalog?

[TestMethod]
public void DiscoverParts()
{
  DiscoverParts(Program.Catalog);
}

private static void DiscoverParts(ComposablePartCatalog catalog)
{
  DiscoverParts(catalog, new CompositionContainer(catalog));
}

private static void DiscoverParts(ComposablePartCatalog catalog, ExportProvider host)
{
  DiscoverParts(() => new CompositionInfo(catalog, host));
}

In a great many cases, the container itself is very boring, but there certainly are plenty of cases where some other, more interesting, ExportProvider comes into play. So, both overloads are useful.  In the case where only the catalog is interesting, the overload without the host should be used.  This sends the signal that the host is not important.  On the other hand, if you choose to use the overload with the host specified, then you’re sending the signal that the host is interesting and important.  Who are you sending the signal too?  Probably your future self.

So we’ve accomplished a lot.  Our test method has been reduced to a simple method call that can take as little as one parameter.  The best thing about it is that this parameter is part of the “normal’ set of Composition types, it does not require the caller to know anything about CompositionInfo, CompositionInfoTextFormatter, or ReflectionTypeLoadException.  All that code is still there though, and its not conveniently packaged for distribution.  Can we make this test even easier by completely hiding all this code?

Introducing CompositionTests

One could take everything we’ve looked at here, bundle it up into a library, and put it on NuGet and GitHub.  I’ve done just that:

image

It’s just a little library but it should make composition tests dead simple:

using CompositionTests;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace CarDealership.Tests
{
  [TestClass]
  public class IntegrationTest
  {
    [TestMethod]
    public void DiscoverParts()
    {
      Composition.DiscoverParts(Program.Catalog);
    }
  }
}

That’s the whole test.

You can also scrub the formatted text before sending it to ApprovalTests.

using ApprovalUtilities.Utilities;
using CompositionTests;
using Microsoft.VisualStudio.TestTools.UnitTesting;

namespace CarDealership.Tests
{
  [TestClass]
  public class IntegrationTest
  {
    [TestMethod]
    public void DiscoverParts()
    {
      Composition.DiscoverParts(
        Program.Catalog,
        StringExtensions.ScrubVersionNumber,
        s => s.ScrubPath(@"C:\Test"));
    }
  }
}

You can use any number of scrubbers to transform your text into a consistent state.  For example, scrubbing version numbers is useful when you use an AssemblyCatalog with an assembly that has an auto incrementing build number.  I’ve also included an extension to scrub the public key token, which is useful when using NCrunch (which disables assembly signing in its builds) alongside other test runners (which do sign the assemblies).  Or, you can use any Func<string, string> that you can think of.

The library also includes (and uses by default) an text formatter that orders the parts by name.  The default text formatter does not guarantee that the parts come out in any particular order.  I have seen them printed in different orders in different build environments, so enforcing order is a must if we want our tests to play nice with VS, NCrunch and a build server.  I’m still not sure that I’ve covered all the scenarios that can lead to things appearing out of order, because the behavior is not easy to trigger.  Luckily I had an example in production which I was able to recreate in the library’s unit tests.  If I encounter any more ordering problems, they will be addressed in future versions of the library.

Check It Out

Manage Your MEF Parts with NuGet

Why?

With MEF you have the flexibility to compose applications without taking a reference to every required assembly. MEF’s flexibility is one of its most seductive features. But I see a dark side. Maybe I’m paranoid, but I miss the security of taking references. Before MEF, I never worried about getting my assemblies into the bin directory because MSBuild took care of that for me. Now I do worry about it, a little, and this is one of the reasons I’ve written about a method to test MEF composition.

I also really like NuGet. Most people compare NuGet to apt, but to me it’s always been “CPAN for .NET”. I guess that tells you a little bit about where I’m coming form. But whatever your background, you have to love a system that lets you say “Here is the thing I’m interested in using, you go get the rest.” But at first glance, MEF and NuGet don’t play well together (even though NuGet uses MEF!) NuGet makes the fundamental assumption that when a library exists in a NuGet package, then the intent when installing that package is to reference that library. I’m not saying that’s a bad assumption in general, it’s just not a very good assumption to make when dealing with libraries of MEF parts.

The good news is that NuGet can be used to create MEF friendly packages that don’t create references when installed, but do ensure that the parts are copied to the bin directory. You get runtime-only references, with all the NuGet package management goodness. I’ll be the first to admit that some of the techniques used to make this happen are hacky, but I’ve tested it and it works for me. So strap yourself in, and read on.

A Little Background

I’m going to start by showing the structure of a nuspec file which NuGet can use to make a MEF friendly package.

After showing the nuspec, I’ll show how to automate spec and package creation using MSBuild.

  • It will be helpful to know a little MSBuild. I wish I had a good resource, but I don’t. Somehow, I’ve just learned it though osmosis. This one looks nice. I mostly look at the reference when I have questions.
  • To create the MEF-friendly package, I’ll extend the recipe I wrote for creating a “Standard” NuGet package with MSBuild.
  • You will need the nightly build of the MSBuild Community Tasks.
  • There will be some XSLT. I can vouch for w3schools, since I used it as my primary reference for creating the XSLT used in this post.

Anatomy of a MEF Friendly Package

Here is the nuspec which builds a MEF friendly package for the CheesePizzaMaker assembly from my MEF talk.

<?xml version="1.0" encoding="utf-8"?>
<package>
  <metadata>
    <id>CheesePizzaMaker.MEF</id>
    <version>1.0.4441.28470</version>
    <authors>Jim Counts</authors>
    <owners>Jim Counts</owners>
    <requireLicenseAcceptance>false</requireLicenseAcceptance>
    <description>Makes Cheese Pizza</description>
    <copyright>Copyright 2012</copyright>
    <title>CheesePizzaMaker (MEF)</title>
    <dependencies>
      <dependency id="PizzaStore" version="" />
    </dependencies>
    <frameworkAssemblies>
      <frameworkAssembly assemblyName="System" targetFramework="" />
      <frameworkAssembly assemblyName="System.ComponentModel.Composition" targetFramework="" />
    </frameworkAssemblies>
  </metadata>
  <files>
    <file src="bin\x86\Release\CheesePizzaMaker.dll" target="content\Parts" />
    <file src="bin\x86\Release\CodeContracts\CheesePizzaMaker.Contracts.dll" target="content\Parts" />
    <file src="bin\x86\Release\CheesePizzaMaker.pdb" target="content\Parts" />
    <file src="NuGet\install.ps1" target="tools" />
  </files>
</package>

This nuspec file is meant to be the target of the “nuget pack” command. Since the target will not be a project file, NuGet will not fill any of the fields for us at runtime, so there are no replacement tokens like there were in the “standard” nuspec I wrote about before. But, you can avoid filling out most of the data by hand if you give NuGet an assembly to examine when it runs the “nuget spec” command. I’ll show you how to do that later in this post, or you can consult the NuGet docs. After generating the spec, I customized some fields:

  • Id – to distinguish the MEF friendly package, I added .MEF to the Id.
  • Title – I added the title field, and used a title that made it easy to distinguish in the Package Manager
  • Dependencies – I added the PizzaStore package as a dependency. This package contains the interface definitions shared between parts and their host. When the version field is blank, NuGet will look for the highest available version, and that’s the behavior I want, but you can specify a version if you like. Also, note that the PizzaStore package is a standard package; it will create a reference in the project where it is installed. Since it only contains interfaces, and not parts this is what we want. However, you could (in theory, I haven’t tried this yet) have MEF packages that depend on other MEF packages.
  • Framework Assemblies – Since this is a MEF friendly package, we know that the target project will need MEF. So, System.ComponentModel.Composition and System are both listed as dependencies. NuGet will add them to the project if they aren’t already there. You could leave this off, but then you would have to add the references by hand. Automation is smoother, take advantage of it.
  • Files – Finally we have the files section. We have the same three files that I included in the standard package, and one more file, which is the install.ps1 script which runs when the package is installed. Other than the PowerShell script, which we will look at in a moment, the big difference is where the files are located in the package. Instead of placing the files in the lib folder, as is standard, I’ve put them into the content folder. Furthermore, I’ve placed them in a subdirectory called “Parts”.

Because the libraries are in the content folder, NuGet will not add a reference to the files when the package is installed. However, it won’t copy them to the bin directory either, so placing the files into the content folder only solves half the problem. Install.ps1 solves the other half:

param($installPath, $toolsPath, $package, $project)

foreach ($runtimelib in $project.ProjectItems.Item("parts").ProjectItems)
{
    if(-not $runtimeLib.Name.EndsWith(".Contracts.dll"))
    {
        # Set to "Copy Always"
        $runtimelib.Properties.Item("CopyToOutputDirectory").Value = 1
    }
    else
    {
        # Change to "None"
        $runtimelib.Properties.Item("BuildAction").Value = 0;
    }
}

To make the MEF friendly package work, I’ve adopted the convention that the Parts folder contains MEF parts which should be copied to the bin directory. The only exception is the Contracts library, which also contains the MEF attributes for the assemblies they describe, and can cause cardinality mismatches if it is deployed to bin. By default, content files are placed directly in the project folder, the build action is set to “Content”, and the copy to output directory property is set to “Do not copy”. Because we specified a subfolder in the nuspec file, NuGet will create a “Parts” directory and place the files in there. To get the files copied to the bin directory, the PowerShell script enumerates through the contents of the Parts folder (which is a “ProjectItem” from MSBuild’s point of view) and change the value of the “CopyToOutputDirectory” property to 1 as long as the file doesn’t end with “.Contracts.dll.” In the enumeration for “CopyToOutputDirectory”, 1 means “Copy Always”, good luck on finding that documented anywhere (send me a link if you find it.)

In theory, the top half of the if-statement is all you need to make the package MEF friendly. But, I discovered that MSI files will include the Contracts library if you leave the Build Action equal to “Content.” Again, this will cause cardinality mismatch exceptions if you then use the MSI to install your application. What’s worse is that you will only see the problem in production. So, to save us from this headache, the PowerShell script will set the Build Action equal to “None” when it finds the Contracts library.

So that’s a MEF friendly NuGet package. When you install the package, your libraries will appear in the Parts folder. On build, your assembly and its pdb will be copied to a “Parts” folder in the bin directory. The only adjustment you need to make in your application is that you must include a DirectoryCatalog for this new subfolder in your composition. You might also have noted the hardcoded value for the package version. In the next section I’ll show how this number can be updated at runtime, but if you don’t want to read that, just know that this value can be overridden from the command line when calling “nuget pack”.

Automating with MSBuild

I’m sensitive to the needs of those who like to cut and paste, so here is a link to the NuGet folder for the CheesePizzaMaker on bitbucket.org. Do come back though, because automating the MEF friendly package is a little more complex than the standard package.

Consuming the Build Script in the Project

This step is just like consuming the standard build script.

  1. Unload your project.
  2. Edit your project.
  3. Update your project file’s “AfterBuild” target.

If you are unfamiliar with editing a project file, visit my previous article on creating NuGet packages with MSBuild for step-by-step instructions. If you are planning to build both types of packages (standard and MEF-friendly) then your AfterBuild target should look like this when you are done:

  <Target Name="AfterBuild">
    <MSBuild Condition="'$(Configuration)|$(Platform)' == 'Release|x86'"
       Projects="NuGet\NuGet.msbuild" />
    <MSBuild Condition="'$(Configuration)|$(Platform)' == 'Release|x86'"
       Projects="NuGet\NuGet.MEF.msbuild" />
  </Target>

Setup Support Files

We need to create several supporting files, two that enable the build script to customize the nuspec file for us, and one which we include in the nuget package. Finally, we include NuGet itself, for convenience (as discussed previously, this is my personal preference, but the build script will need to be adjusted if your preference differs.) You don’t need to add these files to the solution, but if you do, make sure that you set the Build Action to “None”.

  • NuGet.exe – Go ahead and create a folder called NuGet in your project, and save a copy of NuGet.exe in this folder.
  • Install.ps1 – This file is included in the MEF-friendly NuGet package tools folder. The generated nuspec will expect to find this file in the NuGet subfolder. So create a text file called install.ps1 and copy the contents of the PowerShell script into it (shown above and available on bitbucket.org).
  • Mefdata.xml – This file is a source file for an XmlMassUpdate task in the MEF-friendly build script. It contains all the custom elements except the dependencies. Some of the data is dummy data, which the script will update in a later step. Create this file in the NuGet folder.
<?xml version="1.0" encoding="utf-8"?>
<package xmlns:xmu="urn:msbuildcommunitytasks-xmlmassupdate">
  <metadata>
    <title>title</title>
    <frameworkAssemblies>
      <frameworkAssembly xmu:key="assemblyName"
         assemblyName="System" targetFramework="" />
      <frameworkAssembly xmu:key="assemblyName"
         assemblyName="System.ComponentModel.Composition" targetFramework="" />
    </frameworkAssemblies>
  </metadata>
  <files>
    <file xmu:key="src" src="AssemblyPath" target="content\Parts" />
    <file xmu:key="src" src="ContractPath" target="content\Parts" />
    <file xmu:key="src" src="SymbolPath" target="content\Parts" />
    <file xmu:key="src" src="NuGet\install.ps1" target="tools" />
  </files>
</package>
  • dependencydata.xslt – If the project has a “packages.config” file then it means that there are some NuGet packages installed in the project. The script uses this transform file to convert packages.config into an XmlMassUpdate source which contains the contents of the <dependencies /> node in the nuspec file. Create this file in the NuGet folder.
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0"
                xmlns:xsl="http://www.w3.org/1999/XSL/Transform"
                xmlns:msxsl="urn:schemas-microsoft-com:xslt"
                exclude-result-prefixes="msxsl"
>
  <xsl:output indent="yes"/>
  <xsl:template match="/">
    <package xmlns:xmu="urn:msbuildcommunitytasks-xmlmassupdate">
      <metadata>
        <dependencies>
          <xsl:for-each select="packages/package">
            <dependency xmu:key="id">
              <xsl:attribute name="id">
                <xsl:value-of select="@id"/>
              </xsl:attribute>
              <xsl:attribute name="version"/>
            </dependency>
          </xsl:for-each>
        </dependencies>
      </metadata>
    </package>
  </xsl:template>
</xsl:stylesheet>

Now we’re ready to create the NuGet.MEF.msbuild script.

Create the Build Script

Create an XML file in the NuGet folder and call it NuGet.MEF.msbuild. You can download the complete file from bitbucket.org, but I will go through each section step by step here.

<?xml version="1.0" encoding="utf-8" ?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
         ToolsVersion="4.0"
         DefaultTargets="default">
  <Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />
  <PropertyGroup>
    <NuGetApp>NuGet.exe</NuGetApp>
    <ProjectDir>$(MSBuildProjectDirectory)\..</ProjectDir>
    <PackageDir>$(ProjectDir)\..\..\..\PackageSource</PackageDir>
    <ReleasePath>bin\x86\Release</ReleasePath>
    <ProjectName>CheesePizzaMaker</ProjectName>
    <PackagesFile>$(ProjectDir)\packages.config</PackagesFile>
    <DependencyFile>$(MSBuildProjectDirectory)\dependencydata.xml</DependencyFile>
    <MefDataFile>$(MSBuildProjectDirectory)\mefdata.xml</MefDataFile>
    <XsltFile>$(MSBuildProjectDirectory)\dependencydata.xslt</XsltFile>
    <PackageId>$(ProjectName).MEF</PackageId>
    <MefNuSpecFile>$(ProjectDir)\$(PackageId).nuspec</MefNuSpecFile>
    <ReferenceLib>$(ReleasePath)\$(ProjectName).dll</ReferenceLib>
    <ContractLib>$(ReleasePath)\CodeContracts\$(ProjectName).Contracts.dll</ContractLib>
    <SymbolLib>$(ReleasePath)\$(ProjectName).pdb</SymbolLib>
  </PropertyGroup>

We start off by creating our project and importing the MSBuild Community Tasks. Next we setup some data used throughout the script. Some of these properties aren’t as clean as I’d like them to be, so there might be some refactoring in my future. Here are the highlights:

  • PackageDir – the script will move completed packages to this folder at the end of the process.
  • ReleasePath – relative path to the bin directory to pull files from.
  • ProjectName – if you don’t make any customizations to this process, then this is the only field you need to worry about updating.
  • PackageId – my convention is to suffix “.MEF” to the project name. You can setup your own convention here.
  • MefNuSpecFile – this is where the build script will put the customized nuspec file when it is done writing it.
  • ReferenceLib – this is the part library you want to package.
  • ContractLib – the Code Contracts reference assembly is optional, but you need to make some adjustments to this recipe if you don’t want to include it.
  • SymbolLib – the debugging symbols, also optional, but it’s useful to me in my packages.

The first targets follow the property section:

  <Target Name="clean">
    <Delete Files="$(PackageId).nuspec; ..\$(PackageId).nuspec; $(ProjectName).nuspec; $(DependencyFile)" />
  </Target>
  <Target Name="default" DependsOnTargets="MefSpec; MefPackage; MovePackages"/>

During script development, it’s nice to be able to reset everything, so the clean target will delete intermediate and output files. You can invoke it using the /t switch on the command line.

C:\PizzaStore\CheesePizzaMaker\NuGet>c:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe NuGet.MEF.msbuild /t:clean

The default target specifies which targets must be executed to build and deploy the package. There is nothing special about the name “default”. It is only the default target because it is specified in the DefaultTargets attribute in the project element. We could specify “clean” or “MefSpec” as the default, but that wouldn’t make much sense.

  <Target Name="MefSpec"
          Condition="!Exists($(MefNuSpecFile))"
          DependsOnTargets="CreateDependencyFile; CreateMefSpec; CleanMefSpec; MergeMefSpec; UpdateMefSpec">
  </Target>

Like “default”, the MefSpec target is a meta-target which specifies other targets to run. It also includes a condition which will skip the spec creation if a spec already exists. This means that you can further customize the generated spec by hand without worrying about overwriting your changes on the next build.

  <Target Name="CreateDependencyFile" Condition="Exists($(PackagesFile))">
    <XslTransformation
      OutputPaths="$(DependencyFile)"
      XmlInputPaths="$(PackagesFile)"
      XslInputPath="$(XsltFile)"
      />
  </Target>

CreateDependencyFile executes an XslTransformation task when “packages.config” exists. It applies dependencydata.xslt to packages.config and creates dependencydata.xml, which is an XmlMassUpdate source file.

  <Target Name="CreateMefSpec">
    <Exec WorkingDirectory="$(MSBuildProjectDirectory)"
          Command="$(NuGetApp) spec -AssemblyPath &quot;$(ProjectDir)\$(ReferenceLib)&quot;" />
    <Move SourceFiles="$(ProjectName).nuspec"
          DestinationFiles="$(MefNuSpecFile)" />
  </Target>

CreateMefSpec executes “nuget spec” to create a default spec file. We give NuGet the path to the parts assembly so that it can fill in some of the standard data for us. Note that this command is executed from the “NuGet” subfolder for two reasons. First, we want to avoid overwriting the standard nuspec file if it exists, I couldn’t see a way to specify the output file name from the NuGet command line. Second, NuGet will automatically use the csproj file to generate a tokenized spec file for us if we run the spec command from a directory which contains a csproj (or any other project) file. We don’t want a tokenized spec file, so we need to run from a folder where NuGet can’t see the project file. Ultimately, we want the MEF-friendly spec file in the project folder, so we move it there before finishing CreateMefSpec.

  <Target Name="CleanMefSpec">
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
               XPath="/package/metadata/licenseUrl"
               Delete="true" />
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
               XPath="/package/metadata/projectUrl"
               Delete="true" />
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
               XPath="/package/metadata/iconUrl"
               Delete="true" />
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
               XPath="/package/metadata/releaseNotes"
               Delete="true" />
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
               XPath="/package/metadata/tags"
               Delete="true" />
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
               XPath="/package/metadata/dependencies"
               Delete="true" />
  </Target>

In CleanMefSpec, I delete fields I don’t care about, and one field I intend to replace with XmlMassUpdate. If you have use of the different URLs, notes, etc., then you should not delete these fields, but update them in the UpdateMefSpec target.

  <Target Name="MergeMefSpec">
    <XmlMassUpdate Condition="Exists($(DependencyFile))"
       ContentFile="$(MefNuSpecFile)"
                   ContentRoot="/"
                   SubstitutionsFile="$(DependencyFile)"
                   SubstitutionsRoot="/" />
    <XmlMassUpdate Condition="Exists($(MefDataFile))"
                   ContentFile="$(MefNuSpecFile)"
                   ContentRoot="/"
                   SubstitutionsFile="$(MefDataFile)"
                   SubstitutionsRoot="/" />
  </Target>

MergeMefSpec performs one or two XmlMassUpdate tasks. If there was a packages.config file, then there should now be a dependencydata.xml file created by the “CreateDependencyFile” target. If so, MergeMefSpec will update the MEF-friendly nuspec with the dependencies. You might want to review the dependencies after the spec is written. This technique does no interdependency analysis of the packages in packages.config. It just takes a dependency on all of them. So if you are writing package A, and A depends on B, and B depends on C, then your packages.config will list B and C as installed packages. However, package A may not need a direct dependency on C. You can imagine a scenario where B is updated and no longer requires C, in which case package A doesn’t need to include it any more either, but your spec doesn’t know that. If you are worried about this scenario, your current option is to edit the spec by hand to remove the indirect dependencies. This is one of the hacky areas I would like to revisit in the future, but for now just be aware of it.

Next, MergeMefSpec uses XmlMassUpdate to merge in the “static” customizations from mefdata.xml that don’t depend on whether or not there are other NuGet packages installed in the project. At this point we have a template nuspec which needs to be updated with relevant data from the project.

  <Target Name="UpdateMefSpec">
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
               XPath="/package/metadata/id"
               Value="$(PackageId)" />
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
               XPath="/package/metadata/title"
               Value="$(ProjectName) (MEF)" />
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
              XPath="/package/files/file[@src='AssemblyPath']/@src"
               Value="$(ReferenceLib)" />
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
               XPath="/package/files/file[@src='ContractPath']/@src"
               Value="$(ContractLib)" />
    <XmlUpdate XmlFileName="$(MefNuSpecFile)"
               XPath="/package/files/file[@src='SymbolPath']/@src"
               Value="$(SymbolLib)" />
  </Target>

UpdateMefSpec replaces some defaults and dummy data with the real values. Here we update the id, the title, and the paths to the package files. The custom nuspec is now complete.

  <Target Name="MefPackage"
          Condition="Exists($(MefNuSpecFile))">
    <!-- Get the version number of the main assembly to pass to nuget cli -->
    <GetAssemblyIdentity AssemblyFiles="$(ProjectDir)\$(ReferenceLib)">
      <Output TaskParameter="Assemblies"
              ItemName="AsmInfo" />
    </GetAssemblyIdentity>
    <Exec WorkingDirectory="$(ProjectDir)"
          Command="&quot;$(MSBuildProjectDirectory)\$(NuGetApp)&quot; pack &quot;$(MefNuSpecFile)&quot; -Verbose -NoPackageAnalysis -Version %(AsmInfo.Version)" />
  </Target>

Next up is MefPackage, where we build the MEF-friendly package. MefPackage first examines the parts assembly using a GetAssemblyIdentity task. This task places the assembly information into the property specified by the ItemName attribute; in our case it is called “AsmInfo”. Next we execute “nuget pack”. Our target is the nuspec file. I use Verbose because I like it. NuGet will not be happy that we have put dll files in the content folder and issues warnings suggesting that we might want to put them in a lib folder. Since the whole point of this exercise is to distribute libraries as content rather than references, I’ve suppressed the analysis step using NoPackageAnalysis. Finally, we specify the Version flag and give it the version data supplied by the GetAssemblyIdentity task.

  <Target Name="MovePackages"
          Condition="Exists($(PackageDir))">
    <!-- Using command line because I want to be sure to get the most up to date list of *.nupkg -->
    <Exec WorkingDirectory="$(ProjectDir)"
          Command="move /y *.nupkg &quot;$(PackageDir)&quot;" />
  </Target>
</Project>

As with the standard nuget build script, MovePackages checks that the destination directory exists then uses the shell to move the packages to the destination. I found that it was much easier to use the shell because it will get the most recent list of *.nupkg files in the source folder, and will happily overwrite packages in the destination folder. Finally, we close the project element and the MSBuild script is done.

Test the Build

Switch to release mode and build your project. If everything is setup correctly, your packages should appear in the folder specified by the PackageDir property. If not, see my previous post for troubleshooting tips.

Test the Package

In the PizzaStore example, there is a dummy program that does nothing but serves as a place to simulate different composition scenarios. To test the package I removed the project reference to PizzaStore. There was no reference to CheesePizzaMaker since I used .bat files to move the dll files around during the MEF talk.

After removing the PizzaStore reference I setup Visual Studio to use PackageDir as a package source, and then used the NuGet package manager in GUI mode to install the MEF package. As expected, NuGet created the Parts folder, set the Build Action and other properties, and also pulled in the dependency to PizzaStore and re-created the reference (this time as an assembly reference rather than a project reference).

Uninstalling the CheesePizzaMaker.MEF package also removed all the content files and the assembly reference.

Thanks for Reading

There you have it: MEF, now with NuGet. I’m sure there is a candy bar pun in there somewhere.

Getting Started with Visual MEFX

I noticed that someone found their way to my blog by searching for “visual mefx how to open”.  This person probably went away disappointed, because although I mention Visual MEFX on this site, I never explained how to use it.  I ran the same search on Google and looked at the results, apparently no one who talks about Visual MEFX explains how to get it set up. Since its not obvious to at least one person out there, here’s how to do it.

Get MefContrib-Tools

From what I can tell, it’s a source only distribution.  So here’s a direct link to the source.  After downloading the zip file, be sure to right click on it, access the properties, and click “Unblock”.

This step will ensure that Visual Studio will not try to warn you about untrustworthy sources.  Unzip the source in a location of your choosing.

Build the Tools

There are two flavors of Visual MEFX, one for Silverlight, and one for the desktop.  Each has it’s own solution in the the “src” folder of the source you just downloaded.  Choose the one you are interested in, and open the solution. Choose Debug or Release mode according to your preference and build the project.

Run the Tools

Lets assume for the moment that you wanted Release versions of the tools.  Minimize Visual Studio and navigate to the bin\Release folder for the project you just built.  There are a few dlls, pdbs, xml manifests, and config files in the folder.  There are only two executable files.  The Visual MEFX desktop executable is “MefContrib.Tools.Visualizer.exe”.  Double click.

Silverlight

If you are interested in the Silverlight version, follow the same steps to build the project, but you will need to load the application in a browser.  Lucky for you, there will be an html file “TestPage.html” in the Release directory that you can use to get the application started.

Here it is running in Firefox and examining its own xap file. Hope this helps.

Stop Guessing About MEF Composition And Start Testing

Diagnosing MEF Failures | SoCalCodeCamp @ CSU Fullerton | 1/29/2012

I gave my first talk at a Code Camp last weekend. The topic was “Diagnosing MEF Failures” at it was well received by the 5 attendees.  A recording of the talk is embedded at the top of this post, and the direct link is just below, along with the slides and demo code.  This blog post compliments the talk by providing a step by step guide about how to test your MEF composition using ApprovalTests and Microsoft’s CompositionDiagnostics library.

Finally, I revisted this subject in June 2012.  If you only want to know the easiest way to setup composition tests, read MEF Composition Tests, Redux instead.  If you’re interested in knowing how everything works under the hood, read this article first, then read the June article.

Why?

Diagnosing MEF composition failure is tricky, especially when parts have two or more nested dependencies.  In this scenario, MEF’s error messages aren’t very helpful.  MEF was unable to satisfy one or more parts in the chain, but the error message points you to the part at the top of the chain, to really fix the problem you need to know about the bottom of the chain.   If you start by looking where the error message sends you, you can end up feeling like you’re on a wild goose chase because nothing appears to be wrong with that part.

Recognizing this problem, the MEF team created the a static diagnostics library as a “Sample”, but when MEF was integrated into the BCL with .NET 4, this useful library never made it in.  You can still get the code from the MEF CodePlex site, along with a shell that allows you to define directory or assembly catalogs from the command line and analyze the parts within.  This shell is useful and flexible (for example, you could use it to examine deployed parts in production without changing any code) but sometimes tedious to use.  An alternate shell (Visual MEFx) brings some nice GUI touches to the concept.  The GUI makes it much easier to play with what if scenarios while examining the collection of parts.

Either of these tools can be used to analyze real, deployed compositions, even when dealing with parts authored by a third party.  That’s a really powerful concept.  But for some MEF users (myself included), third parties are not a big concern.  I’m using MEF to compose a collection of known parts, so I can say with confidence what the composition should look like.  Sometimes I might want to play out a what-if scenario, but most of the time I just want some assurance that all my parts are there, and therefore all my application’s expected behaviors will be there.

Ideally, I’d like to automate this assurance without relying on a GUI, or trying to spin up a separate process during integration testing.  Finally, it would be great if I could be assured that my composition looked correct without having to write one test for every part that should be in the container.  How I achieve these goals is the subject of this post.

Background

If you don’t know about ApprovalTests you really should take a moment to learn about them.  Read about this library at http://approvaltests.sourceforge.net/, and be sure to listen to the Herding Code podcast.  You don’t need to understand much about ApprovalTests to follow the recipe I plan to lay out below, but even if you aren’t interested in MEF at all, you should still check out ApprovalTests.

Presumably if you are reading this you are already familiar with MEF, but if not, get started here: “What is MEF?”.  You don’t have to read the whole thing to start getting the point of what MEF is for, maybe just the first few sections.  Follow that with the MEF Programming Guide on CodePlex.

If you would rather read than watch my video, I primarily drew from two references while putting together my talk.  Part 13 of the MEF programming guide is about Debugging and Diagnostics.  Finally, this entry on Daniel Plaisted’s Blog is a great reference which includes background on why MEF fails in general, quite a few corner cases, and an overview of live and static debugging tools.  My talk focuses on using the tools, so read Daniel’s blog if you’re really interested in studying MEF failure.

Since Daniel covers MEFx and Visual MEFx, I wont discuss them here, instead I’ll focus on presenting the recipe for testing MEF composition with ApprovalTests.

What You Will Need

  • For starters, you’ll need a project that uses MEF, but since you are here I’ll assume you already have that.
  • You’ll need to download ApprovalTests and reference it in your test project.  There is a version on NuGet, but its not always up to date with the latest release from the SourceForge site.  The recipe should work with 1.0.11 or greater, so its up to you if you want the latest and greatest.
  • You’ll need a version of the CompositionDiagnostics library.  A quick way to get your hands on a binary copy is to download MEFx.  The MEFx zip contains two files, one is a command line shell, and the other is the library.  You can obtain the source by downloading MEF Preview 9.  Note that there are MEF 2 previews also in the release list, you want the preview from MEF 1.

The CompositionDiagnostics source may be useful if you run into any issues with imports showing up out of order on different machines.  If you have this problem, add this method to the CompositionInfoTextFormatter:

private static string GetPartDefinitionKey(object definition)
{
    if (definition == null)
        return null;
    var compositionElement = definition as ICompositionElement;
    return compositionElement != null ?
        compositionElement.DisplayName :
        definition.ToString();
}

Then update the Write() method to order the PartDefinitionInfos by this “key”.

Test Setup

I’m using MSTest, it works fine for me.  If you have a favorite test framework, this shouldn’t be too hard to adapt.

If you don’t have a test project for your solution, set one up.  It will be important to keep your test and production catalogs in sync.  You don’t want to fix a problem in test and still have your composition fail in production.  I use directory catalogs, so I keep my catalogs in sync using a simple pre-build event in the test project properties:

copy /y “$(SolutionDir)$(SolutionName)\bin\x86\Debug\*.dll” “$(TargetDir)”

Add references to ApprovalTests.dll and ApprovalUtilities.dll.

Unzip the MEFx zip file, or compile the code yourself.

Grab the Microsoft.ComponentModel.Composition.Diagnostics.dll file, and place it near your solution (a lib folder would be a good option) then take a reference to the library.

Add a test class to the solution and call it whatever makes sense to you, I’ll use the name “IntegrationTest” in my example.

Import these namespaces into your IntegrationTest file.

using System;
using System.ComponentModel.Composition;
using System.ComponentModel.Composition.Hosting;
using System.IO;
using System.Reflection;
using ApprovalTests;
using ApprovalTests.Reporters;
using Microsoft.ComponentModel.Composition.Diagnostics;

Add a UseReporterAttribute to your test class.   DiffReporter is my favorite:

[TestClass]
[UseReporter(typeof(DiffReporter))]
public class IntegrationTest

The CompositionDiagnostics library provides us with a few new types.  The most interesting to me is the CompositionInfo class.  This is where the magic happens.  We create a catalog and a container as usual, then instantiate an instance of CompositionInfo, passing the catalog and container into the constructor.  I find it useful to put this into its own method, since I usually have at least two integration tests.

private static CompositionInfo GetCompositionInfo()
{
    var catalog = new DirectoryCatalog(".");
    var host = new CompositionContainer(catalog);
    var compositionInfo = new CompositionInfo(catalog, host);
    return compositionInfo;
}

Now you can add a test method.  Here’s the code, followed by an explanation:

[TestMethod]
public void DiscoverParts()
{
    try
    {
        using (var stringWriter = new StringWriter())
        {
            CompositionInfoTextFormatter.Write(
                GetCompositionInfo(),
                stringWriter);
            Approvals.Verify(stringWriter.ToString());
        }
    }
    catch (ReflectionTypeLoadException ex)
    {
        Array.ForEach(
            ex.LoaderExceptions,
            lex => Console.WriteLine(lex.ToString()));
        throw;
    }
}

The CompositionDiagnostics library provides another useful class, the CompositionInfoTextFormatter, which we can use to create a text representation of the data in the CompositionInfo.  The CompositionInfo class does all the heavy lifting.  We use the method Approvals.Approve to verify the text representation, more on this later, for now its enough to know that this line takes the place of an “Assert” call in an ordinary unit test.

This test also catches the ReflectionTypeLoadException. Type loading problems can cause your test to fail when the CompositionInfo is under construction. Therefore, you would never reach the ApprovalTest. MSTest will catch this and dump the exception in the test result, but the really interesting information on that exception is in the LoaderExceptions property which isn’t shown by default. So, in the catch block each loader exception is dumped to the console before allowing the exception to go on it’s merry way. This saves the extra step of rerunning the test under the debugger just to drill into the LoaderExceptions.

That’s it for the first test, if you’ve been following along you should be able to run it.

The Results

When you run the test for the first time, it will fail.  This is not unusual for ApprovalTests.  The ApprovalTest library needs “approved” output to verify the test output against.  Since we used the DiffReporter, you are notified of the failure when a diff tool launches showing output on one side and a blank slate on the other.  (If nothing happens, you probably need TortiseSVN, which provides the default diff tool.  You can switch to NotepadLauncher if you decide you don’t like the idea of installing TortiseSVN just for this purpose.  Other reporters are supported, but that’s outside the scope of this entry.)

MEF1

You can flip the left and right sides to make it easier to read.

So what is this?  Briefly, the diagnosis begins by looking at the Parts property on the catalog, examining the metadata for each, and eventually tries to compose each part defined in the catalog.  This will uncover any problems with the composition by throwing exceptions.  The analyzer catches the exceptions, and stores them in a data structure along with the other metadata.  Any parts that fail for reasons not related to importing other parts have the potential to be the root cause of the composition failure.  Each will be marked as a [Primary Rejection] by the text formatter.  Exceptions and the other metadata will also be printed with each part.

That’s the sad path.  On the happy path, your composition succeeds and each part is still listed, along with some metadata and information about which parts satisfied nested imports, if any.  So, in our test, we get the CompositionInfo, and format the data using the CompositionInfoTextFormatter.  The output is captured by a StringWriter, and passed to Approvals.Approve() for comparison.  If the output is correct, we should see no exceptions listed, no rejections, and each part [SatisfiedBy] the exports we expected to be imported.  If so, we can right click on the “received” file and choose “Use this whole file”.  Save.  It is now approved.  Rerunning the test should result in a pass.

If the composition has problems, we need to work on those problems until the output looks right, just as we would in any other TDD scenario.  Once it is right, approve it so it can be checked in the future.  Whenever any change is made to the composition (expected or not) this test will fail and let you know that you either need to approve the new composition, or resolve the unexpected failure.

Since the ReflectionTypeLoadException causes the test to fail before the approval stage, you should look at the trace output or test results when these occur.  Your loader exceptions will likely be MissingMethodExceptions with messages explaining that certain methods have no implementation.  Whichever part is meant to provide those implementations cannot load, and you may have to do some detective work to figure out why.  Often I find that these problems are related to parts expecting different versions of the same assemblies, or that the parts refer to assemblies that are unavailable.

Another Test

I usually have this next test in my IntegrationTest as well.

[TestMethod]
public void InstantiateService()
{
  try
  {
    Approvals.Verify(
      GetCompositionInfo().Host.GetExportedValues<IPizzaMaker>(),
      s => s.GetType().FullName);
  }
  catch (ReflectionTypeLoadException ex)
  {
    Array.ForEach(
      ex.LoaderExceptions,
      lex => Console.WriteLine(lex.ToString()));
    throw;
  }
}

I look at it now and I wonder why.  This test will grab all instances of a certain type of part, compose them, and return them.  ApprovalTests will approve the list.  This doesn’t really do anything new compared to the full dump of the CompositionInfo in DiscoverParts.

I think that over time I must have become over zealous with the Don’t Repeat Yourself principle and used the GetCompositionInfo() method when I shouldn’t have.  The original form of this test was probably this:

[TestMethod]
public void InstantiateService()
{
  try
  {
    var catalog = new DirectoryCatalog(".");
    var host = new CompositionContainer(catalog);
    Approvals.Verify(
      host.GetExportedValues<IPizzaMaker>(),
      s => s.GetType().FullName);
  }
  catch (ReflectionTypeLoadException ex)
  {
    Array.ForEach(
      ex.LoaderExceptions,
      lex => Console.WriteLine(lex.ToString()));
    throw;
  }
}

So, what’s the difference/use of this second version?  In the version that uses GetCompositionInfo, the container and catalog construction happen inside the method, then are passed to a CompositionInfo.  Later, the test just used the container (accessed through the Host property) to ask for the IPizzaMaker instances.  So the construction of the CompositionInfo was a waste, but you might live with that for the sake of staying DRY.  However, since we know that the CompositionInfo can die during construction, its more than just a waste, it’s a potential source of test failure that has nothing to do with the ability to instantiate IPizzaMakers.

In the second version we construct the host without bothering with the CompositionInfo, and we avoid ReflectionTypeLoadExceptions unless they have something to do with IPizzaMaker.  So this test might tell you something interesting if making IPizzaMakers was your primary concern, but it wont give you the type of root cause analysis provided in the CompositionDiagnostics output.  So using both can potentially give you a smidge more insight into your composition.

If you reuse GetCompositionInfo to access the host, it tends to fail in concert with the DiscoverParts test, and so tells you nothing new.

MEF 2

In MEF 2, the InstantiateService test has the potential to take center stage, while the DiscoverParts test will be more of a niche player.  This is because the MEF team has made big improvements to the baked in diagnostics and exception messages.  First of all, MEF always knew what the root cause was, it just failed to tell you about it in the error message.  In MEF 2 Preview 5, the composition exception indicates the primary rejection at the top of a chain of failures leading all the way down to the part you asked for.

By default, the container will still wait for you to ask for a part that requires the rejected part (or the rejected part itself) before any exception is thrown.  This is part of MEFs “Stable Composition” feature.  You can disable this feature by passing the CompositionOptions.DisableSilentRejection option to the container’s constructor.  The result will be that MEF will throw an exception as soon as it rejects the part.  You will definitely want to use this option on the container used for integration tests, and if you aren’t working with third parties, disabling silent rejection is recommended on your production container as well.

When you run InstantiateService with silent rejection disabled, you should get an exception if any part in the container would be rejected, not just the IPizzaMakers.  This allows InstantiateService to detect a larger class of failures in MEF 2 compared to MEF 1.

The DiscoverParts test will still have its niche in MEF 2.  This test will remain useful in detecting missing optional parts.  A part that is optional and missing will not throw an exception even when silent rejection is disabled.  Although other parts want to import the optional parts, they don’t “depend” on them to operate, since classes that import optional parts should all be able handle the case where the parts aren’t available.  Since its missing (meaning—not in the catalog) there’s no way it could have a problem itself and throw an exception.

So if the part is optional should your test pass or fail?  Its up to you to decide, but remember that the whole premise of this test strategy is that you already know what the composition should look like.  You could have a collection of three services you want hosted that work together in an assembly line fashion. You might want to reserve the right to increase the number of steps in the assembly line in the future so you use [ImportMany] to get the whole collection.  Now you’ve just made your imports optional, but do you really want to deploy your application if service 2 of 3 is missing?  The DiscoverParts test can detect this for you, but “noisy” rejection can’t.  It’s a niche, but it’s a niche I find myself in, so I think the DiscoverParts test will remain useful even after MEF 2 arrives.

Conclusion

Really, I just hope that someone will find this idea useful.  Feel free to run with it and let me know if you can think of any other strategies for testing composition.  In particular I’d be interested to hear ways that people have found for testing scenarios with third parties or in production.  If you are interested in additional thoughts from me on the subject, please read: MEF Composition Tests, Redux.

Thanks for reading.