Damian Hickey

Mostly software and .NET related. Mostly.

Our Open Source policy at eVision

There is a problem within our industry, particularly around .NET, known as the "no-contrib culture". That is, companies take and benefit from OSS but don't given anything back and also prevent their employees from doing so. A while ago I blogged about establishing an Open Source Software policy at a proprietary software company that happens to consume and depend on a significant amount of open source libraries. eVision was a typical company in that the standard employment contract had that catch-all clause that they own everything one does on a computer.

With the proliferation of OSS, the fact the platform we predominately build on is going OSS, and the fact that OSS is a viable business strategy (even within proprietary businesses), this was a somewhat short-sighted. Indeed, several employees were simply ignoring this; a situation which was in neither party's interest.

Thus I am delighted to announce that we have established a policy that we believe strikes the right balance and have made it available on our github organisation site

In layman's terms, this means that our employees are free to create any sort of open source outside of business hours (as long as it doesn't compete with our business), are free to contribute to open source we depend on at any time, and they own the copyright to that work (or whatever the terms are of the project they contribute to). The only real stipulation is that the project's licence must allow us to use it in our commercial software.

We hope this will have the effect of encouraging contribution to the platform we depend on strengthening it to our mutual benefit. We also hope that engagement with the open source community will have a positive learning effect for our engineers.

I personally hope that more organisations and companies adopt our policy and make it known publicly. Thus, we've released this policy under a creative commons licence. Disclaimer: you should have your lawyer check it over!

To all developers out there, when considering whom to work for I would strongly suggest that you should only look for organisations that have this or a similar policy in place. These are mature organisations that care for the platform and ecosystem they and their staff build upon.

Any feedback or questions, please contact me dhickey at gmail.com. Thanks to Adam RalphFrans Bouma and Christian Horsdal for their feedback.

On that note, eVision are looking for talented engineers (and more) to work on some challenging, distributed and large scale problems. :)


Establishing OSS policies in a proprietary software company

My company produces proprietary software but consumes a lot of FOSS libraries and components. Until recently, our standard contracts were of the 'we own everything you do' type. Obviously that created some difficulties for someone like myself given that I'm active in the .NET OSS community. The irony being that if I signed it, I wouldn't be able to support the very OSS libraries I maintain and that the company uses.

We'd like to establish a corporate policy where developers can contribute back and, possibly, create new OSS. I believe this is strategically a good move for several reasons, including:

 A) It supports the software that we've built our business on, and without which, we might not even exist
 B) It exposes our developers to more code and ways of working, thereby helping to improve their craft.
 C) It's an attractive policy to the type of developer that we are interested in attracting as we grow.
 
At the same time though, it is critically important for us to protect the proprietary aspect of our software.
 
Our legal department and I are not experienced in establishing such policy and procedures. I'd like to get in touch with someone from a company who has enacted something like this, so if you know anyone, I'd be very grateful if you could introduce me (dhickey@gmail.com, @randompunter).

I hope to publicly share what we come up with such that other companies can learn and implement similar policies.

Gulliver's Travels Tests

Like writing lots and lots of fine-grained "unit" tests, mocking out every teeny-weeny interaction between every single object?

This is your application:


Now try to change something. 

(Note: someone else stated something something similar with this image and I can't for the life of me find it.)

Bubbling exceptions in Nancy up the owin pipeline

In the application I am building I have a requirement to do common exception handling within my owin pipeline via custom middleware:
private class CustomExceptionMiddleware : OwinMiddleware
{
	public CustomExceptionMiddleware(OwinMiddleware next) : base(next)
	{}

	public override async Task Invoke(IOwinContext context)
	{
		try
		{
			await Next.Invoke(context);
		}
		catch(Exception ex)
		{
			// Custom stuff here
		}
	}
}
Where the startup looked something like this:
public class Startup
{
	public void Configuration(IAppBuilder app)
	{
		app
			.Use()
			.UseNancy()
			.UseOtherStuff();
	}
}
The "problem" is that Nancy is very safe by default - it will always try to capture the exception and set the status code to 500, even if you have cleared all the status code handlers from the bootstrapper. Thus, any exceptions thrown in Nancy will never reach the CustomExceptionMiddleware. 

Taking the PassThroughStatusCodeHandler found in Nancy.Testing and tweaking the handle method, we can capture the exception and rethrow it using .NET 4.5's ExceptionDispatchInfo:
public class RethrowStatusCodeHandler : IStatusCodeHandler
{
	public bool HandlesStatusCode(Nancy.HttpStatusCode statusCode, NancyContext context)
	{
                // This is unchanged
	}

	public void Handle(Nancy.HttpStatusCode statusCode, NancyContext context)
	{
		Exception innerException = ((Exception) context.Items[NancyEngine.ERROR_EXCEPTION]).InnerException;
		ExceptionDispatchInfo
			.Capture(innerException)
			.Throw();
	}
}
Finally we need to tell Nancy to use this. In your bootstrapper, override InternalConfiguration property:
protected override NancyInternalConfiguration InternalConfiguration
{
	get
	{
		return NancyInternalConfiguration.WithOverrides(config =>
			config.StatusCodeHandlers = new[] {typeof (RethrowStatusCodeHandler)});
	}
}
Now the exception, with proper stack trace will be thrown and captured by the middleware.



Protecting a Self-Hosted Nancy API with Microsoft.Owin.Security.ActiveDirectory

This post is the Nancy version of Vittorio's 'Protecting a Self-Hosted API with Microsoft.Owin.Security.ActiveDirectory'. Since I'm lazy, I may even be lifting some parts of it verbatim, if you don't mind, Vittorrio ;)

I'm going to skip the intro to Owin etc, if you are reading this, you probably know all about it by now. (Except for you Phillip; Get yout s**t together man!). This tutorial will be using Nancy.MSOwinSecurity that I introduced in the previous post.

What I am going to show you is how you can set up a super simple Nancy HTTP API in a console app and how you can easily secure it with Windows Azure AD with the exact same code you use when programming against IIS Express/full IIS.

Here's what we are going to do:

  • Create a minimal self-hosted Nancy Http API in a console app.
  • Add middleware for validating JWT tokens from AAD.
  • Create a console app client to poke our API and challenge the user for credentials.

Create a Minimal Self-Hosted Nancy Http API

Let's start by creating blank solution. I'm a bit unimaginative so I am calling this 'NancyAAD'. Note: this entire solution will be targeting .NET 4.5. .NET 4.0 is not supported. Next, we add a console application, 'NancyAAD.Server', that is going to host our Nancy HTTP API. To this project, we're going to add a number of nuget packages, in this order:

  1. Nancy.Owin - This will bring in Nancy and the adapter that allows Nancy to be hosted in an OWIN pipeline.
  2. Microsoft.Owin.Hosting - Provides default infrastructure types for hosting and running OWIN-based applications. There are other hosting implementations out there, if you feel the need to explore.
  3. Microsoft.Owin.Host.HttpListener - OWIN server built on the .NET Framework's HttpListener class.
  4. Microsoft.Owin.Security.ActiveDirectory - The middleware through which we will secure our Nancy HTTP API. This package will pull in a bunch of other packages. We'll be using this later.
  5. Nancy.MSOwinSecurity - This will provide integration between Nancy's context and modules and Microsoft.Owin.Security.*.

Your NancyAAD.Server packages.config should look similar to this:

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.Owin" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Host.HttpListener" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Hosting" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Security" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.ActiveDirectory" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.Jwt" version="2.0.2" targetFramework="net45" />
  <package id="Microsoft.Owin.Security.OAuth" version="2.0.2" targetFramework="net45" />
  <package id="Nancy" version="0.21.1" targetFramework="net45" />
  <package id="Nancy.MSOwinSecurity" version="1.0.0" targetFramework="net45" />
  <package id="Nancy.Owin" version="0.21.1" targetFramework="net45" />
  <package id="Newtonsoft.Json" version="4.5.11" targetFramework="net45" />
  <package id="Owin" version="1.0" targetFramework="net45" />
  <package id="System.IdentityModel.Tokens.Jwt" version="1.0.0" targetFramework="net45" />
</packages>

Next we'll define our Nancy module and a very simple GET endpoint that we'll want to secure later:

namespace NancyAAD.Server
{
    using Nancy;

    public class ValuesModule : NancyModule
    {
        public ValuesModule()
        {
            Get["/values"] = _ => new[] { "value1", "value2" };
        }
    }
}

That done, let's add the Startup class for hosting our Nancy application in OWIN:

namespace NancyAAD.Server
{
    using Owin;

    public class Startup
    {
        public void Configuration(IAppBuilder app)
        {
            app.UseNancy();
        }
    }
}

This says that Nancy is handling all requests from the root. Nice and simple. 

Lastly, we need to host the OWIN application itself. In Program.cs, we write the following:

namespace NancyAAD.Server
{
    using System;
    using System.Net.Http;
    using System.Net.Http.Headers;
    using Microsoft.Owin.Hosting;

    internal class Program
    {
        public static void Main(string[] args)
        {
            using (WebApp.Start<Startup>("http://localhost:9000/"))
            {
                Console.ForegroundColor = ConsoleColor.Blue;
                Console.WriteLine("Nancy listening at http://localhost:9000/");

                // Test call
                var client = new HttpClient();
                client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
                var response = client.GetAsync("http://localhost:9000/values").Result;
                Console.ForegroundColor = ConsoleColor.Red;
                Console.WriteLine(response.Content.ReadAsStringAsync().Result);

                Console.ReadLine();
            }
        }
    }
}

The call to WebApp.Start initializes a new server, which listens at the specified address. The rest of the method calls the Nancy HTTP API to double check that we did everything correctly.

There is one subtle difference here compared to Vittorio's post - we are defining an Accept header "application/json". The reason is that because Nancy supports view engines where it's default behaviour when returning a model for a request with no accept header, is to try to resolve a view. This would result in an exception for us as we haven't defined any views.

F5'ing the app should result in:

Exactly the same as Vittorio's screen shot. Nice.

Before moving on to the next task, remove the test call from the Main method and change it to the following:

public static void Main(string[] args)
{
	using (WebApp.Start<Startup>("http://localhost:9000/"))
	{
		Console.ForegroundColor = ConsoleColor.Blue;
		Console.WriteLine("Nancy listening at http://localhost:9000/");
		Console.WriteLine("Press ENTER to terminate");
		Console.ReadLine(); 
	}
}

Secure the Nancy HTTP API with Windows Azure AD

Here comes the raison d’être of the entire post. We have already added the Microsoft.Owin.Security.ActiveDirectory nuget package, now let's add it in the right place into the OWIN pipeline. Note, the order things are defined in the pipeline are important, so we must add this before Nancy so requests are piped through it before hitting the Nancy handlers. 

Our OWIN pipeline with the middleware added now looks like:

public void Configuration(IAppBuilder app)
{
	app.UseWindowsAzureActiveDirectoryBearerAuthentication(
		    new WindowsAzureActiveDirectoryBearerAuthenticationOptions
		    {
		        Audience = "https://contoso7.onmicrosoft.com/RichAPI",
		        Tenant = "contoso7.onmicrosoft.com"
		    })
	      .UseNancy();
}

The new code is the "UserWindowsAzure....". You’ll notice that instead of relying on app.config settings, values are passed directly to the middleware. This is another beauty of OWIN, you can now use can keep such settings wherever you deem most appropriate.

Now our pipeline includes the right middleware: if we receive a JWT, we’ll validate it and if it checks out we’ll pass the corresponding ClaimsPrincipal to the handler delegate. Very good, but not good enough. Let’s modify the module class to mandate that all callers must present a valid token from the tenant of choice: 

public class ValuesModule : NancyModule
{
	public ValuesModule()
	{
		this.RequiresMSOwinAuthentication();
		Get["/values"] = _ =>
		{
			ClaimsPrincipal claimsPrincipal = Context.GetMSOwinUser();
			Console.WriteLine("==>I have been called by {0}", claimsPrincipal.FindFirst(ClaimTypes.Upn));
			return new[] {"value1", "value2"};
		};
	}
}

The key statement here is "this.RequiresMSOwinAuthentication" - this ensures that a request hitting this module must have a valid authenticated user, otherwise an unauthorized HTTP status code is returned.

The second key statement, "Context.GetAuthenticationManager().User", retrieves the user from IAuthenticationManager where we write a claim to the console, just for demonstration purposes. The UPN claim is among the ones that Windows Azure AD sends in JWTs.

Believe it or not, that’s all we had to do to secure the Nancy HTTP API: we just had to add the code in bold :)

Create a Client App and Test the Nancy HTTP API

Add a new console app project called NancyAAD.Client (see, I'm not really imaginative!). To this project, install the 'Microsoft.WindowsAzure.ActiveDirectory.Authentication' nuget package. This package contains the main assembly for the Windows Azure Authentication Library (AAL) and provides easy to use authentication functionality for .NET client apps. Your packages.config should look like;

<?xml version="1.0" encoding="utf-8"?>
<packages>
  <package id="Microsoft.WindowsAzure.ActiveDirectory.Authentication" version="0.7.0" targetFramework="net45" />
</packages>

Next up, the Main method should looks like this:

namespace NancyAAD.Client
{
    using System;
    using System.Net.Http;
    using System.Net.Http.Headers;
    using Microsoft.WindowsAzure.ActiveDirectory.Authentication;

    internal class Program
    {
        [STAThread]
        public static void Main(string[] args)
        {
            Console.ForegroundColor = ConsoleColor.Green;
            Console.WriteLine("Client ready.");
            Console.WriteLine("Press any key to invoke the service");
            Console.WriteLine("Press ESC to terminate");
            ConsoleKeyInfo consoleKeyInfo;

            var authenticationContext = new AuthenticationContext("https://login.windows.net/contoso7.onmicrosoft.com");

            do
            {
                consoleKeyInfo = Console.ReadKey(true);
                // get the access token
                AuthenticationResult authenticationResult = authenticationContext.AcquireToken(
                    "https://contoso7.onmicrosoft.com/RichAPI",
                    "be182811-9d0b-45b2-9ffa-52ede2a12230",
                    "http://whatevah");
                // invoke the Nancy API
                var httpClient = new HttpClient();
                client.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
                httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", authenticationResult.AccessToken);
                HttpResponseMessage response = httpClient.GetAsync("http://localhost:9000/values").Result;
                // display the result
                if (response.IsSuccessStatusCode)
                {
                    string result = response.Content.ReadAsStringAsync().Result;
                    Console.WriteLine("==> Successfully invoked the service");
                    Console.WriteLine(result);
                }
            } while (consoleKeyInfo.Key != ConsoleKey.Escape);
        }
    }
}

This is a very classic ADAL client and is virtually identical to the one in Vittorio's post. It simply acquires the token, attaches it to the Authorization header before making the call. The one difference is the Accept header is defined, as explained previously.

Starting both projects and triggering a call from the client should result in:

Sign in as any user in your tenant, and you’ll get to something like the screen below:

Ta-dah!

Now go forth and secure your enteprise(!) Nancy Apps!

Full code for this is up on Github

Introducing Nancy.MSOwinSecurity

Under the Katana Project, Microsoft has released a suite of security related middleware including cookie and oauth based providers such as Google, Twitter etc. Nancy.MSOwinSecurity allows your Nancy application to work and interact with the Microsoft.Owin.Security.* middleware by giving you access to Microsoft.Owin.Security.IAuthenticationManager. From there you can access the CurrentUser as a ClaimsPrincipal, as well as perform various operations such as SignIn(), SignOut(), Challenge() etc.

How to use

Install the nuget package:

install-package Nancy.MSOwinSecurity

Getting the authentication manager and current user from the context:

public class MyModule : NancyModule
{
    public MyModule()
    {
        Get["/"] = _ =>
        {
            IAuthenticationManager authenticationManager = Context.GetAuthenticationManager();
            //ClaimsPrincipal user = authenticationManager.User;
            //authenticationManager.SignIn(..);
            //authenticationManager.SignOut(..);
            //authenticationManager.AuthenticateAsync(..);
            //authenticationManager.Challenge(..);
        };
    }
}

Securing a nancy module:

public class MyModule : NancyModule
{
    public MyModule()
    {
        this.RequiresMSOwinAuthentication();
        Get["/"] = _ => {...}
    }
}

Securing an individual route:

public class MyModule : NancyModule
{
    public MyModule()
    {

        Get["/"] = _ => 
        {
            this.RequiresMSOwinAuthentication();
            ....
        }
    }
}

Getting the current user (just a helper extension around IAuthenticationManager.User):

public class MyModule : NancyModule
{
    public MyModule()
    {

        Get["/"] = _ => 
        {
            ClaimsPrincipal = Context.GetMSOwinUser();
            ....
        }
    }
}

Authorizing the user at module level:

public class MyModule : NancyModule
{
    public MyModule()
    {
        this.RequiresSecurityClaims(claims => claims.Any(
            claim.ClaimType = ClaimTypes.Country &&
            claim.Value.Equals("IE", StringComparision.Ordinal)));
        Get["/"] = _ => 
        {
           ....
        }
    }
}

Authorizing the user at route level:

public class MyModule : NancyModule
{
    public MyModule()
    {

        Get["/"] = _ => 
        {
            this.RequiresSecurityClaims(claims => claims.Any(
                claim.ClaimType = ClaimTypes.Country &&
                claim.Value.Equals("IE", StringComparision.Ordinal)));
            ...
        }
    }
}

Personal note: this nancy extension package would integrate much better if we had extenstion properties in c# :(

Note: This package doesn't replace nor integrate with the exisiting nancy authentication and authorization infrasctructure (IUserIdentity). We're currently considering the best way to proceed on this.

Uniqueness in a DDD-CQRS-ES application

A while back I blogged that set based based concerns, i.e. duplicates, that when analyzed aren't really a problem. An example of this could a be a duplicate product in a catalog. The worst that can happen is the user sees the item twice when browsing / searching. Everything still works - the customer can still purchase it, you can still ship it. It's low risk, low impact and life goes on.

There are situations though where global uniqueness is a real requirement, for example, choosing a username during a new user registration process. The problem here is that if a duplicate occurs you may have a security issue, or maybe neither user can log in. It depends on your application of course, but this may be considered high risk.

The solution to this is to use the reservation pattern where we 'reserve' the username from some fully consistent store before creating the user. When the user is successfully created, we then confirm the reservation, via a process manager. Reservations have an expiry so if for some reason the user is not created, i.e. they abandoned the process, the username is eventually released for someone else to use. The worst thing that can happen is that a reserved/unconfirmed username is unavailable to other users for whatever duration you decide to set as an expiry.

A word of caution though - this pattern introduces a single point of failure. If the reservation system is not available, new users won't be able to sign up. Therefore this should only be considered if you *absolutely* need to have uniqueness (which is far less often than you think). Of course, you'll keep this component separate from your authentication and application so existing users can still log in, if it does go down :)

From a disaster recovery perspective, a username reservation system should be rebuildable from domain events.

In the end, it's all about risk analysis.

RavenDB and OWIN Webinar

Yesterday I did a webinar for RavenDB 3's Mystery Feature #2 - OWIN support. I'd like to thank Oren, Daniel and the HR team for giving me the opportunity to work on this. 

If you notice any typos, or things aren't as smooth as they should be, it's because it was my first webinar and I only starting preparing 45 mins before hand.

( It's tough to listen to my own voice :s )

NDC-London Talk on Advanced Nancy

On wednesday, 4th of December, I'll be delivering a talk at NDC London on some of the more advanced aspects of Nancy FX. The outline of the talk (to be finalized) looks something like this:

  • Dependency injection
  • Authentication and authorization
  • Content Negotiation
  • Validation
  • Extending and overriding
  • Logging and Diagnostics
  • Error handling and Custom Status Code Handling
  • Combining Nancy with Signalr via OWIN

It appears I am competing with Jon Skeet, Dan North and few other heavy hitters (as well as beer o'clock) so I expect this to be a more intimate affair.

Hope to see you there.

Testing Nancy with Owin.Testing

I always like a challenge :)

Nancy has a very nice way of testing, including a mechanism to configure the application as required for each test (swap in a mock etc). It does this with the Browser class:

[Fact]
public void NancyTest()
{
	// Arrange
	var browser = new Browser(with => with.Module<ConfigBootTestModule>());

	// Act
	BrowserResponse response = browser.Get("/config");

	// Assert
	Assert.Equal("Hello configured fellow", response.Body.AsString());
}

(The 'with => ' lambda here is altering the configurable boostrapper)

If you are hosting Nancy with OWIN, and you are using other middleware, you may prefer to test with Owin.Testing for broader test scope. I should state that Owin.Testing was indeed inspired by Nancy.Testing in the first place - I wanted to do the same type of testing with my OWIN application that was possible with Nancy.

So first let's create our OWIN startup class. Here, we are injecting the INancyBootstrapper which will allow us to supply a configurable one in a test:

private class Startup
{
	private readonly INancyBootstrapper _nancyBootstrapper;

	public Startup(INancyBootstrapper nancyBootstrapper)
	{
		_nancyBootstrapper = nancyBootstrapper;
	}

	public void Configuration(IAppBuilder builder)
	{
		builder.UseNancy(_nancyBootstrapper);
	}
}

We're going to add a helper method to configure the bootstraper, to make our test read more like the Nancy Browser example above:

private static HttpClient CreateHttpClient(Action<ConfigurableBootstrapper.ConfigurableBootstrapperConfigurator> configureBootstrapperAction)
{
	return OwinTestServer.Create(builder => 
		new Startup(new ConfigurableBootstrapper(configureBootstrapperAction)).Configuration(builder))
		.CreateHttpClient();
}

Here we are creating our test server, configuring the boostrapper via supplied action and returning an HttpClient.

And the Owin.Testing equivalent test:

public async Task OwinTest()
{
	// Arrange
	var httpClient = CreateHttpClient(with => with.Module<ConfigBootTestModule>());

	// Act
	var response = await httpClient.GetAsync("http://localhost/config");
	var body = await response.Content.ReadAsStringAsync();

	// Assert
	Assert.Equal("Hello configured fellow", body);
}

Looks familiar, doesn't it? ;)

Full gist.