Coding

Creating Packages with NuGet the MSBuild Way

Authors Note:

Hello!  Thanks for visiting this page.  Unfortunately it is very out of date.  Unless you have unusual needs you don’t need to jump through all the steps described in this post.  I don’t make packages this way anymore, and neither should you. Instead, check out my updated post: NuGet Like A Pro, the MSBuild Way.

The new post provides a greatly streamlined process, and this post remains here for historical purposes, or extreme edge cases.

Which ever post you choose, thanks for reading!

There are quite a few ways to integrate NuGet into your build process:

  • Check out this article on NuGetter if you use TFS.
  • Here is an article on integrating with TeamCity by Scott Hanselman. This one has many great resources, including a video from his TechEd talk that goes with the article.
  • Here’s a github:gist that shows how to use a windows command shell batch file to build a package, also shows a ruby script.
  • Finally, here is the reference at docs.nuget.org that explains several scenarios that NuGet.exe understands.

I use CruiseControl.NET and reading the reference at docs.nuget.org was enough info for me to figure out how to setup NuGet as a post build event, but the solution felt hacky. I searched the web and found a few pages like those referenced above. In the end, I found that my solution was pretty much state of the art.

Even so, I wasn’t quite satisfied:

  1. Creating the nuspec file wasn’t very fun. When there was no nuspec, the script used NuGet.exe to create the file and launched notepad so that I could edit it. That’s hacky, and it didn’t take long to get tired of making the same edits to every nuspec. After editing the file, I had to run the build again to make the package.
  2. I found the default packages unsatisfying. The nuspec file does not include everything I wanted. For example, I wanted to include the .pdb file. I looked at the “symbols” option in NuGet.exe but it was both more and less than I wanted. Although symbolsource.org looks nice, I don’t want to involve a third party just so I can step through my own code.  I also use Code Contracts. I wanted the Contract reference assemblies included in the package too.
  3. I wanted more control over the versioning. As I mentioned before, I have a scenario where I need more than one package configuration for the same assembly. The quick and dirty package building process gathers version numbers and other tokens from the assembly and combines these with the statically defined data in the nuspec when writing a package. This works great when the target is a csproj, vbproj, fsproj or even nproj (nemerle project!), but does not happen when building from a nuspec. NuGet lets you specify the version on the command line, but you have to figure out how to get the version number to the command line so that NuGet can use it. You can use PowerShell to figure out the assembly version number, but then you have to worry about execution policy and/or signing.

Some of these issues can be resolved by updating the nuspec file. NuGet won’t stomp on an existing nuspec, so that’s nice. Resolving problems in this way means I will have to repeat the process for every assembly I want to package, which makes problem #1 worse. PowerShell did a decent job but I didn’t want to write about it because I don’t want to go into the PowerShell security briar patch. (Side rant: I can build a .NET CLI application that will perform the exact same task as the PS script, and run without altering the execution policy or signing the assembly. I considered writing one and publishing it in a future blog entry, but it turns out there is another way to do this. So in the end, the security policy around PowerShell did not do much to encourage me to start signing PowerShell scripts, it just encouraged me to use something other than PowerShell.)

Integrating NuGet and MSBuild

I am not the first person to do this. Over at Technical Jargon, Jeremy Skinner has an article that shows a scenario you might want to use if your packages are public and you intend to publish them. My solution would not have been possible without his example.  My goal is to create a separate MSBuild file for NuGet and chain it off the end of the Visual Studio project file. I’m going to work my way in, and start by showing how to consume the NuGet.msbuild file from the project file.

Edit Project File

I’m going to use a C# project (one of the PizzaStore projects from my MEF talk), but the same process should work for any Visual Studio project file.

You will need to edit your project file by hand. Once you have opened your project in Visual Studio, right-click on the project node in the Solution Explorer and choose “Unload Project,” then right click again on the unloaded project and select “Edit <projectname>.csproj.”

image

Once the project opens, you will see that it is an XML file. Scroll all the way to the bottom of the file and you should see lines commented lines like these:

  <!-- To modify your build process, add your task inside 
 one of the targets below and uncomment it. Other 
 similar extension points exist, see 
 Microsoft.Common.targets.
 <Target Name="BeforeBuild">
 </Target>
 <Target Name="AfterBuild">
 </Target>
 -->

We are going to make use of the “AfterBuild” target. Move it out of the commented block and add the following lines to the element:

<Target Name="AfterBuild">
    <MSBuild Condition="'$(Configuration)|$(Platform)' == 'Release|x86'"
             Projects="NuGet\NuGet.msbuild" />
</Target>

This snippet tells MSBuild (and Visual Studio) to execute a second MSBuild script once the targets in the project file finish building. I have set this up the way I like it, but you might want to do things differently.

  1. I setup a condition. This script will only execute when the build is configured for an x86 Release. I like building x86 assemblies, I like controlling package creating by switching to release mode. You can adjust this to your liking by consulting this MSDN page, or remove the condition entirely if you want.
  2. I plan to place the NuGet.msbuild file in a subfolder of the project. You can put it wherever you like, but you will need to make other adjustments to the paths in NuGet.msbuild if you choose a different location.

That’s it for the project file. Save it. Right click on the project node in Solution Explorer again, and choose “Reload Project.” Visual Studio will ask you if you want to close the XML representation of the project. Click OK and the project should load.

Setup MSBuild Community Tasks

To automate the nuspec file creation, we will need to supplement MSBuild with some better XML rewriting tools. The MSBuild Community Tasks nightly build has everything we need. Do not get the last “release” it seems that has not been updated in a while.

Download the nightly build and install.

Setup NuGet.msbuild

Add a folder called “NuGet” under your project folder. I like to have this folder as part of the project for easy access, so I use the “New Folder” item on the Project menu in Visual Studio to create the folder. You will put two things into this folder.

  1. NuGet.exe. Putting this in the local folder is not 100% necessary. You could put NuGet somewhere in your path, and share it between all your projects. I like to keep a copy with each project, that way I don’t have to remember to setup every build machine with NuGet. I trade disk space for convenience, but it is up to you. My example assumes NuGet is in the folder with the build file. Make sure you set the exe Build Action to “None” if you add the file using Visual Studio.
  2. NuGet.msbuild. Use Visual Studio to add a new XML file to the NuGet folder and call it NuGet.msbuild.

Next we’ll look at the contents of the MSBuild file for the PizzaStore project, section by section.  If you want to cut and paste, you can find the whole file on bitbucket.org.

<?xml version="1.0" encoding="utf-8" ?>
<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"
               ToolsVersion="4.0"
               DefaultTargets="default">
  <Import Project="$(MSBuildExtensionsPath)\MSBuildCommunityTasks\MSBuild.Community.Tasks.Targets" />

We start by declaring an MSBuild project and setting the default build target, next we import the community tasks. Immediately after the import element, we declare some properties that we will use later:

  <PropertyGroup>
    <ProjectName>PizzaStore</ProjectName>
    <BaseDir>$(MSBuildProjectDirectory)\..</BaseDir>
    <BuildDir>bin\x86\Release</BuildDir>
    <PackageDir>$(BaseDir)\..\..\..\PackageSource</PackageDir>
    <NuGetApp>NuGet\NuGet.exe</NuGetApp>
    <NuSpecFile>$(BaseDir)\$(ProjectName).nuspec</NuSpecFile>
    <ProjectFile>$(ProjectName).csproj</ProjectFile>
    <ReferenceLib>$(ProjectName).dll</ReferenceLib>
    <ContractLib>
      $(BuildDir)\CodeContracts\$(ProjectName).Contracts.dll
    </ContractLib>
    <SymbolDB>$(BuildDir)\$(ProjectName).pdb</SymbolDB>
  </PropertyGroup>

If you follow this article like a recipe, then the ProjectName property should be the only thing you need to change.  Of course, you are free to tweak things if you know what’s going on. The path to the BuildDir will need updating if you do not use x86 builds. You might want to put the PackageDir somewhere else.

Next, come some XML snippets that the XmlMassUpdate task will merge into the default nuspec file:

<ProjectExtensions>
  <NuGetReferences>
    <references>
      <reference file="$(ReferenceLib)" />
    </references>
  </NuGetReferences>
  <NuGetFiles>
    <files xmlns:xmu="urn:msbuildcommunitytasks-xmlmassupdate">
      <file xmu:key="src"
              src="Contracts"
              target="lib\net40" />
      <file xmu:key="src"
              src="Symbols"
              target="lib\net40"/>
    </files>
  </NuGetFiles>
</ProjectExtensions>

The first section will add an explicit reference to the assembly file. By default, NuGet would also add a reference to the Code Contracts assembly, which we do not want. The <reference /> element prevents that behavior. The second section will add two files to the package, one for the Code Contracts assembly, and one for the pdb file. You can do a lot with XmlMassUpdate, but finding the documentation isn’t obvious. You need to download the source and look at the compiled html help file in the documentation folder.

Now we move on to the build targets. First, we declare the default target, which just tells MSBuild which other targets it needs to execute:

<Target Name="default"
   DependsOnTargets="StdSpec; StdPackage; MovePackages"/>
  • StdSpec creates the “standard” (meaning my standard) nuspec file.
  • StdPackage creates the “standard” (again my standard) nupkg file.
  • MovePackages delivers any packages created during the build to the path specified by the PackageDir property.

Here’s the target to create the nuspec file, the condition ensures that it only runs if the nuspec is missing. It is also the longest:

<Target Condition="!Exists($(NuSpecFile))"
        Name="StdSpec">
  <Exec Condition="!Exists($(NuSpecFile))"
        WorkingDirectory="$(BaseDir)"
        Command="$(NuGetApp) spec" />
  <XmlMassUpdate Condition="Exists($(NuSpecFile))"
    ContentFile="$(NuSpecFile)"
    ContentRoot="/package/metadata"
    NamespaceDefinitions="msb=http://schemas.microsoft.com/developer/msbuild/2003"
    SubstitutionsFile="$(MSBuildProjectFullPath)"
    SubstitutionsRoot="/msb:Project/msb:ProjectExtensions/msb:NuGetReferences" />
  <XmlMassUpdate Condition="Exists($(NuSpecFile))"
    ContentFile="$(NuSpecFile)"
    ContentRoot="/package"
    NamespaceDefinitions="msb=http://schemas.microsoft.com/developer/msbuild/2003"
    SubstitutionsFile="$(MSBuildProjectFullPath)"
    SubstitutionsRoot="/msb:Project/msb:ProjectExtensions/msb:NuGetFiles" />
  <XmlUpdate Condition="Exists($(NuSpecFile))"
    XmlFileName="$(NuSpecFile)"
    XPath="/package/metadata/licenseUrl"
    Delete="true" />
  <XmlUpdate Condition="Exists($(NuSpecFile))"
    XmlFileName="$(NuSpecFile)"
    XPath="/package/metadata/projectUrl"
    Delete="true" />
  <XmlUpdate Condition="Exists($(NuSpecFile))"
    XmlFileName="$(NuSpecFile)"
    XPath="/package/metadata/iconUrl"
    Delete="true" />
  <XmlUpdate Condition="Exists($(NuSpecFile))"
    XmlFileName="$(NuSpecFile)"
    XPath="/package/metadata/releaseNotes"
    Delete="true" />
  <XmlUpdate Condition="Exists($(NuSpecFile))"
    XmlFileName="$(NuSpecFile)"
    XPath="/package/metadata/tags"
    Delete="true" />
  <XmlUpdate Condition="Exists($(NuSpecFile))"
    XmlFileName="$(NuSpecFile)"
    XPath="/package/metadata/references/reference/@file"
    Value="$(ReferenceLib)" />
  <XmlUpdate Condition="Exists($(NuSpecFile))"
    XmlFileName="$(NuSpecFile)"
    XPath="/package/files/file[@src='Symbols']/@src"
    Value="$(SymbolDB)" />
  <XmlUpdate Condition="Exists($(NuSpecFile))"
    XmlFileName="$(NuSpecFile)"
    XPath="/package/files/file[@src='Contracts']/@src"
    Value="$(ContractLib)" />
</Target>

First, MSBuild invokes NuGet.exe to generate a nuspec file for the project. XmlMassUpdate tasks merge the two XML snippets we previously specified into the nuspec. Next, several XmlUpdate tasks execute to delete default values from the nuspec file. You need to delete these elements or update them with meaningful information. If you ignore these fields, NuGet will complain when generating the package. I delete them, but if your project needs these fields, you can use a Value attribute to set the value instead of using the Delete attribute to remove the element. Finally, the last three XmlUpdate tasks update the merged XML snippets with meaningful information. When this task finishes, I have a nuspec setup just the way I like, and you can customize the process to your liking as well.

Next, MSBuild creates the package file:

<Target Name="StdPackage">
  <Exec WorkingDirectory="$(BaseDir)"
        Command="$(NuGetApp) pack $(ProjectFile) -Verbose -Prop Configuration=Release" />
</Target>

We invoke Nuget.exe again, and pass it the pack command, targeting the project file. I target the release configuration, and I use the verbose output because I like it.

Finally, the last target and the end of the file:

  <Target Name="MovePackages"
          Condition="Exists($(PackageDir))">
    <!-- Using command line because I want to be sure to get the most up to date list of *.nupkg -->
    <Exec WorkingDirectory="$(BaseDir)"
          Command="move /y *.nupkg &quot;$(PackageDir)&quot;" />
  </Target>
</Project>

I tried using the <Move /> task from MSBuild Community Tasks, but this task uses System.IO.File.Move() to move the files, and this method wont overwrite existing files. I tried <Copy /> followed by <Delete /> tasks, but this did not work out for me either because MSBuild scanned the directory for nupkg files before it had actually built any. The good old DOS move command worked best for me in the end, but you have options.

Test It

If everything is setup right, then you should just be able to switch to “Release” and hit F6. When the build finishes, you should find your package in the PackageDir.

If not, there are several places to look. View the Build Output by selecting “Output” from the “View” menu, and run the build again. Usually this output is enough to find any problems with the build. You should check your AssemblyInfo.cs and make sure you filled out the assembly properties. If NuGet cannot find the tokens it wants for the package, then it might cause the build to fail.

If you want more information, open the command line and run MSBuild by hand:

C:\PizzaStore\PizzaStore\NuGet>C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe NuGet.msbuild /p:Configuration=Release /verbosity:d

You can also run MSBuild against your project if you need even more information:

C:\PizzaStore\PizzaStore>C:\Windows\Microsoft.NET\Framework\v4.0.30319\MSBuild.exe PizzaStore.csproj /t:Clean,Build /p:Configuration=Release /verbosity:d

One More Thing

The last problem I mentioned was retrieving the version number to use with NuGet on the command line, and I did not show that scenario because it was not necessary for this example. Since we are targeting a project file, NuGet automatically gathers the version number for you. However, if you need to get that information you would use a <GetAssemblyIdentity /> task inside the StdPackage target to get the version, then pass that to NuGet on the command line during the <Exec /> task. Jeremy has an example of the syntax on Technical Jargon, but he uses it with an XmlUpdate task to update the nuspec. Either method should work.

Update 3/2/2012: I posted an example using GetAssemblyIdentity in my post on creating MEF-friendly NuGet packages.

Advertisements
Tips, Troubleshooting

How NuGet Chooses the Nuspec File When Building From a Project File

If you have a class library project, it’s a good idea to learn how to use NuGet to package the resulting assembly. It’s a good idea even if you do not plan to share your assembly with the world, because you can still take advantage of the smarts built into NuGet to make managing your own packages easier. It’s not very hard at all to get up and running with your first package, and then you can start customizing from there.

Once you start customizing, you begin to realize there are some scenarios where you might want some slightly different packages. For example, maybe you have some code contract assemblies, but you don’t always want them included. Perhaps you have a library and a shell, and you want to give people the option of downloading just the library in one package, but to download both in another.

I have a scenario like that in at least a couple of my projects. I copied my nuspec file, customized it, added it back to my project then setup a script in the post-build event to create both packages. Worked great the first time I tried it, then failed when I tried to replicate the process in the next project. In the both projects, I kept the “easy” path (described below) which builds against the project file and finds the nuspec on its own. In both projects, I used the same scripts and naming conventions. Yet when it came time to create the project-based package the first project correctly selected the nuspec file that belongs with the project-based package, while the second project consistently selected the nuspec file that belongs with the customized package.

I searched around for an answer to how NuGet was making its decision, but could not find anything. Maybe I wasn’t using the right search terms, but I decided it would be best to stop being lazy and just go read the source. I have written a few CLI applications over the years, but I have to say I have never designed one as nice as NuGet. Next time I have a CLI to write, I’m going to study NuGet more carefully before writing any code.

As you may have guessed by the title of this post, I did find the answer to my question. You can go read the source, or you can read this post.

(Go ahead and read the source anyway, it is great.)

The Easy Way

The easiest way to get your assembly into a NuGet package is to follow the outline here: Creating and Publishing a Package. First download NuGet.exe then skip down to the section under the heading “From a Project.” Read the docs.nuget.org page for full details, but it boils down to this once you are used to the process:

  • Open command window and navigate to your project folder. I will assume your project is called “foo.csproj”
  • nuget spec
  • notepad foo.nuspec
  • Delete a few lines of junk from the nuspec then save.
  • nuget pack foo.csproj –Prop Configuration=Release

Done, now you can copy your package to your local package source or upload it to NuGet.org.

The Question

Now you want to create a custom nuspec to go with the one that you generated with NuGet.exe. Let’s call it “foo.custom.nuspec.” Now when you run “nuget pack” the process may or may not pick the correct nuspec. If it picks the custom nuspec file, it will build the incorrect package. After all, the premise is that you want both the default package and a custom package. If NuGet does pick the wrong metadata file, how do you get it to change its mind? You could be like me and try a bunch of things.

  • Maybe it’s picking the newest file?
  • Maybe it’s picking the oldest?
  • Shortest file name?
  • Longest?
  • Blah, blah blah, blah blah blah

Actually, I didn’t try all of those. I did try a couple, and then I went to the code.

The Answer

Here is the code that selects the nuspec file in version 1.6:

private string GetNuspec()
{
  return GetNuspecPaths().FirstOrDefault(File.Exists);
}

private IEnumerable<string> GetNuspecPaths()
{
  // Check for a nuspec in the project file
  yield return GetContentOrNone(file => Path.GetExtension(file).Equals(
      Constants.ManifestExtension, 
      StringComparison.OrdinalIgnoreCase));

  // Check for a nuspec named after the project
  yield return Path.Combine(
    _project.DirectoryPath, 
    Path.GetFileNameWithoutExtension(_project.FullPath) + Constants.ManifestExtension);
}

We can see right away that the search ends as soon as it finds an existing file that ends with “nuspec.” We can also see that the process prefers to get that file from the project file before it goes looking in the project directory. The “GetContentOrNone” method looks at the project file and chooses files marked with build action “Content” or “None” and yields back the matches. That was the key for me to figuring out why NuGet picked out the wrong file.

I added my nuspec files to the project for convenience. In one project, I had decided from the start to keep both Nuspec files, so I left the original alone and created a copy, renamed and added the copy to the project. This caused the new file’s ProjectItem to appear after the original in the csproj xml file.

In the other project, I had only thought that I wanted to customize the nuspec, then realized later that I wanted both. In the second project, I had customized the nuspec that was already there, then generated a new “default” nuspec, then added the new default to the project. This caused the default nuspec file’s ProjectItem to appear after the customized nuspec in the project xml. The order of operations mattered. I opened the misbehaving csproj in a text editor, rearranged the order of the ProjectItems, and tried again, problem solved.

Summary

For easy reference, here is the search path NuGet takes to choosing the file when the pack target is a project file:

  1. Project Members are preferred
    • NuGet checks items that have “None” or “Content” build actions for the .nuspec extension.
    • The first of these found is used.
  2. If there are no nuspec files in the project, NuGet tries to find a .nuspec file in the project file’s directory with the same name as the file.

Maybe this information is already out there somewhere, but if it is, I had a hard time finding it. If you have this problem, I hope this helps you.