Coding, Presentations, Tools

Video: Introduction to Hadoop Using Syncfusion Big Data Platform

Some of my latest work was just published on WintellectNOW as a free video.  Check it out!

Screenshot of Introduction to Hadoop
Introduction to Hadoop Using Syncfusion Big Data Platform @ WintellectNOW
Tips, Tools

Use ProGet to Host Your Private Packages

I’ve been using private NuGet packages for a while. The easiest way to get up and running with private packages is to drop them in a folder and setup the NuGet Package Manager to use that folder as a package feed. Like many simple solutions, this works great until you outgrow it. I create NuGet packages whenever I build in release mode, and the package-build automation copies the packages to the feed folder. I use Windows Live Mesh to keep the folder in sync across development workstations.

The drawback with this approach is that the relative path to the feed folder must be consistent (I’m too lazy to specify a custom path in each MSBuild file). This means that I’m not free to put my projects anywhere I would like. For example, GitHub for Windows clones into Documents\GitHub\<project> by default, this won’t work with my hacked together system. Also, Visual Studio 2012 is on the horizon, and by default it will create its own Documents\Visual Studio 2012\Projects folder, this is also incompatible with the relative path I use to find my feed folder during build.

What I need is some kind of “universal” way to “locate” a “resource”. Hmm, where have I heard those words before? Oh yes, the internet is based on this idea. The NuGet documentation explains how to set up an ASP.NET MVC site as a “read-only” remote feed. This is read-only from the perspective of the web in that there is no way to push new packages to the site over HTTP. You can update the packages by placing them into the site’s Packages folder. Essentially, this simple remote feed takes the concept of the local feed and wraps a web server around it. This option is not half bad, I could store the packages on my build server and configure my build automation to use a UNC path pointing at that folder, but it would be really sweet to have a more complete style experience. There are a couple ways to do this.  Inspired by Damian Hickey’s post on his NuGet workflow, I decided today was the day to get it going.

Damian has a lot of cool tools in his pipeline, but today I’m going focus on one part and look at a package called ProGet by Inedo

Download the Bits

Here’s the ProGet overview page. You can read a little about the software and download. They have a free (beer) version as well as enterprisey offerings. The download page has a dead link to an installation guide (as of this writing anyway, hopefully that gets fixed soon.) The installation package comes in two flavors, a 60MB version that comes with SQL Server Express, and a 4MB version without the database. Since I already have a database available, I downloaded the smaller package, unblocked it, and copied it over the the build server.


Install the Bits

After starting the installer, I accept the EULA (which is surprisingly short and simple), then select an edition to install.


I choose “ProGet Free” and the installer asks for my email address. I give it my junk mail address, and wait a moment while the installer talks to the mothership. Next, I accept the default installation directory, then its on to database configuration.


I click “Test” but the connection fails. Although there is definitely a server installed, I’m not even sure if its running, so now its time to do some SQL Server troubleshooting.  I open the SQL Server Configuration Manager and see that the server is running and that TCP/IP connections are allowed. But I notice that its actually a SQL Express server, and that the ProGet installer is trying to connect to a (default) instance. I update the instance to SQLExpress and the connectivity test passes.


Next, the installer asks whether I would like to use its integrated web server, or use IIS. Since I already have IIS up and hosting CruiseControl.NET’s web dashboard, I choose to use IIS. I can also choose the port to host ProGet on, but I leave the default 81.


Looks like that’s all the installer wants to know, so I click the Install button. Installation takes about 2 minutes.


I click “Launch ProGet” and the ProGet site launches in IE.


Adding Packages

I think there’s no point in clicking the giant “Browse Packages” button since I don’t have any yet.  Later on I learn that ProGet will act as a type of proxy for, and would have shown me a bunch of remote packages if I had clicked there.  But I was more interested in adding my own packages anyway, so I clicked on “Add Package”. The site wants to authenticate, but also warns me that I haven’t changed the password for the default Admin user.


I like how they say that they “took the liberty” of creating Admin for me.  It gives a nice “this is not your fault” feeling to the text.  I login with the default credentials, then click the “Administration” tab, then “Users and Groups” and finally “Edit” on the Admin user. I change Admin’s password and click “Save User”. Then I create a second administrator using my own name.

Update 2012/7/11: Now that you have created an admin, stay on the Administration tab, then go to “Licensing and Activation”.  Click the button that says “Activate ProGet”.  It’s still free, but if you don’t do this step then your server will stop working in about a week.

Back to the “Add Package” tab. I have the option of Uploading an existing package, using NuGet.exe to push to the repository (sweet), create a package right on the server, or pulling from a remote feed. I’ll try all of these except for “Create New Package”.


Pull From Another Repository


Just as an experiment, I’ll pull CompositionTests down from I fill out the requested information and check the “Download Dependencies” option. For “Feed URL:” I presume they mean the over all feed, not the specific path to CompositionTests and I copy the URL out of Visual Studio. After clicking “Install Package” I browse over to the “Packages” tab.


The first three packages are CompositionTests and its dependencies. The rest of the packages are on (I presume). I didn’t expect to see remote packages, but the page indicates that what I’m seeing is not just local packages, but also “additional packages from any connectors”. The additional packages have a little plug overlay on their icon, indicating they are available via the connector. I click on “Administration” then “Feeds” and I can see that I do indeed have a connection to by default.


I can also see the “Feed Path” where ProGet stores the local packages, and we can see CompositionTests and friends stored there.


Upload From Disk

Under my old system, my local package feeds are stored on my development workstations. So, I’ll open a browser on one of them and see if I can upload a package from the local feed to the remote feed. I hit a snag trying to login, but it just turns out that I need to allow JavaScript (using the correct password also helps).


On the “Add Package” tab, I click “Upload From Disk”. Other than the target feed, all ProGet wants to know is where the file is. I choose one of my private packages: “TemporaryFile” (a little class I use with testing that wraps a temp file in an IDisposable container so that the file is deleted even if an exception is thrown). After I click “Upload File”. We can see that the file is stored on the server.


And now appears in the feed.


Push via NuGet.exe

After I click on “Push via NuGet Command Line Utility” ProGet displays instructions to push via the command line.


Since I haven’t created any additional feeds, my <Feed Name> will be Default. Of course, <package path> will depend on the package I want to upload, but what about [API key]? Browsing around I don’t see it anywhere obvious in ProGet. Pushing without it won’t work, NuGet.exe requires an API key for the server. Using a fake value like “foo” also fails.

I poked around and one of the screenshots clued me in to the fact that the API key in ProGet is associated with the feed, not the user. By default it is blank.


The ProGet documentation gives some insight into why the key goes with the feed and not the user, but no guidelines on the format of the key. I head over to and create a GUID and setup the feed to use it, but I still get a 500 error every time I try to push a package.

After poking around IIS, and eventually busting out Fiddler I still couldn’t get it to work so I used the “Live Help” button to open a chat with Inedo support.


They were able to reproduce the problem on their side, and indicated it seemed to be a bug when using ProGet with the latest build of NuGet. They’re working on a fix. I found older versions of NuGet rattling around my hard drive (1.5 and 1.7) and still couldn’t get it to work. So I guess I’m stuck waiting for the update for the time being, that’s no fun.  I decided to go have lunch.

A Few Hours Later…

I get an email on my junk account explaining a feature of Buildmaster--one of Inedo’s other products that I never downloaded. I mistook it for a release announcement because I didn’t really read it, and even though ProGet isn’t mentioned in the email I decided to visit the ProGet download page. Sure enough, ProGet 1.0.6 was released today.

As an aside, it makes me sad that they sent me an email about something I haven’t expressed interest in, while failing to send me an email about something I am interested in. But I can’t fault them too much, that’s just the price you pay for free beer. And if this new release fixes my issue, then I’ll gladly receive their marketing emails in exchange for such good service.  (Not promising to read, just receive.)

Upgrading ProGet

Once again I download the smaller package. This time the installer asks if I want to update.


I click the update button and ProGet strongly recommends that I backup the database. I let it do so and it fails. There is a button to submit an error report, so I click it. I fill out a little report and submit the issue.


Being impatient, I decide to try again. Now, the installer acts like a new install and offers me the EULA again. Not good. Maybe my old install got borked. Yep, same old installation now. I follow the steps outlined above and wait while ProGet installs. It seems to have fewer steps this time, so I think it recognized that the database already existed. I click “Launch ProGet” at the end and the site launches. I click on “Browse Packages” and all the packages I added previously are still there. So, the install only suffered minor borking.  (As an aside, before I could even finish writing up this post, I got an email from their support asking if I had been able to resolve the issue, which included a suggestion on an alternate way to backup the database.)

Over on the admin tab, the credentials I set up previously still work and the API key I created is still associated with the default feed.

Push via NuGet.exe, Redux

Ok, do over time. Lets see if I can push packages with NuGet now.

I open a command window and navigate to my local package folder. This time I’ll try pushing my “Strings” package (a library of helpers for System.String… c’mon, you know you have one too.)

C:\...\PackageSource>NuGet.exe push Strings.1.0.4360.34765.nupkg API_KEY -Source http://dee:81/nuget/default 
Pushing Strings 1.0.4360.34765 to 'http://dee:81/nuget/default'... Failed to process request.
'You are not authorized to add a package to this feed.'. 
The remote server returned an error: (403) Forbidden.. 

So it still fails, but now it fails with a 403 instead of a 500. Note that replaced the API key with a placeholder. That’s more for the sake of brevity than security. Because I control the key, I can reveal it as long as I’m willing to replace it later.

Thinking I had an idea about fixing the 403 error, I go to “Administration” then “Privileges and Roles”. I create a new Role called “Publisher” and give it the “Feeds_AddPackage” privilege.


Then I click “Add Privilege” and I grant “Publisher” to “Anonymous”. Lets see if that works.

C:\...\PackageSource>NuGet.exe push Strings.1.0.4360.34765.nupkg API_KEY -Source http://dee:81/nuget/default 
Pushing Strings 1.0.4360.34765 to 'http://dee:81/nuget/default'... 
Your package was pushed. 

Bam! It worked.  I love it when a plan comes together.



Final Thoughts

It would have been nice if the official install guide was online. However, it wasn’t to hard to figure out the basics. Even, if it hadn’t been my bad luck to hit a bug on day one, then I still would have needed to figure out how to let anonymous users upload packages. The only reason I was able to figure it out quickly was because I had gotten pretty familiar with the configuration options while debugging 1.0.5.

Lucky for you, you won’t have to deal with that bug, and I hope this guide will make your setup experience smoother than mine was. I’m looking forward to playing with this tool and seeing how I can integrate it with my build system. Up until now, I haven’t configured to create NuGet packages because it seemed like a pain when my system was based on local feeds. Also, I haven’t been using Package Restore with the build server since the server didn’t have access to the private packages. With ProGet in the mix, I’m hope to have that up and running soon.