Aside from a better configuration language, what I really like about coding fluently is that it’s a steady stream of left-to-right operators, like a pipeline. You think of the next operation and just chain it along, never having to back up. Domain specific sublanguages like LINQ (which I adore, incidentally) can make this even more compact, but I’m talking about the function-based fluent operators.

For enumerables, it’s easy:

var names = list.Where(i => i.Foo > 5).Cast<IBar>().Select(j => j.Name);

Take the list, filter it, cast to the type we want, and pull out an element. Operation by operation, left to right. But what about for a single object? That fluency hits a brick wall when you run into C#’s old C-style casting on an individual object.

var name = ((IBar)list.First()).Name;

Yuck. Not fluent, and hard to read. As you’re typing, you have to back up, stick the casted type on the front in parens, and, oh yeah, have to also go wrap the whole thing in parens to be able to continue the method chain. I’ve always hated having to do this and in the past would usually pull out the casting into a separate local.

It’s also easy to fix. We just need a little extension method:

public static partial class Extensions
{
    [DebuggerStepThrough]
    public static T To<T>(this object o)
        { return (T)o; }
}

Now it’s nice and fluent:

var name = list.First().To<IBar>().Name;

So we now have the system Cast<T> operator for casting elements within an enumerable, and To<T> for casting the overall type. Solid.

Edit absolute_privacy.php. Easiest way is through the admin interface under Edit Plugins. Change this line:

if ( !$user || ($user->user_login != $username) ) {

To this:

if ( !$user || (strtolower($user->user_login) != strtolower($username)) ) {

Now you can log in with whatever case you like.

Make sure you do the following:

• You must have the 64 bit compilers installed or you won’t be able to add 64-bit configurations to p4dn. Most likely you didn’t check the box for this when originally installing Visual Studio. Just go to add/remove programs, “change”, and check 64-bit under the C++ area.
• The p4dn library requires Perforce’s p4api native library, which comes in many flavors. It is included in the distribution for P4.Net, but it is an old version. I recommend getting the latest API from their FTP site.
• I used the p4api_vs2008_dyn and p4api_vs2008_dyn_vsdebug configurations from the ntx64 and ntx86 builds.
• Why dynamic instead of static? .NET is all about DLL’s – using static just causes problems with the linker with conflicting runtimes.
• Update the include and lib paths in the p4dn C++ project settings so that the library can find your p4api. If you don’t do this you’ll get #include errors about missing paths.

Say you’re using Shawn Hladky’s great P4.Net project (source) so your C# can speak Perforce – two great tastes that taste great together. P4.Net talks to the server using a native Perforce API. Now, unlike a C# EXE, which is usually AnyCPU (i.e. “let the jitter decide”), the native code that talks to the Perforce server must be compiled as either x86 or x64. This causes two big problems.

The first problem is that your app will crash with a confusing error if run on the wrong version of the .Net Framework! Say you’re using the x86 P4.Net and you run your app on Win7-x64. The jitter will compile the app in 64 bit, but on the first reference to P4.Net, it tries and fails to load the x86 DLL, and puts up an unhelpful error about a bad image format.

The easy workaround to this, of course, is to mark your EXE as x86 instead of AnyCPU. Solved. Unfortunately, you have to remember to do this with every single app that references P4.Net. Or references a DLL that indirectly references P4.Net. It’s like a virus in that way, but we can handle it.

Well, no. That leads to the second problem. What happens if you reference P4.Net (directly or indirectly) into an app that really does need 64 bit? Like, say, some memory-hoovering game build related tool that runs on the server farm? Well now you need a 64-bit version, not only of P4.Net, and not only of your EXE, but of every single DLL that is referenced on the path down to your new P4.Net_x64.DLL. Now we have a real problem. The pain in the ass to maintain kind of problem. Do we really want to have our tool chain output 32 and 64 bit versions of everything, just in case?

Had Microsoft supported fat binaries like NeXTSTEP did back in the early 90’s, we’d just have P4.Net with 32 and 64 bit in the same DLL, and go on with our lives. But no, we have to jump through hoops. This article is the story of how to jump through those hoops to make your bits go.

I’m actually a bit shocked that Microsoft hasn’t extended PE and their OS loaders to support fat binaries. There would be zero perf cost, and it’s been a long time since we had to worry about the size of binaries on disk (content overwhelms executable code size in nearly every app today).

About P4.Net

If I’m going to continue to use P4.Net for my example, I need to give a little more background. P4.Net is built from three components:

1. p4api.lib: A native C++ API (headers and libs) provided by Perforce to talk to their server directly through sockets, without running p4.exe or using the COM object.
2. p4dn.dll: A bridge assembly that statically links in p4api, and uses Managed C++ to export p4api as a low level set of .Net types.
3. p4api.dll: A managed C# API that wraps up the low level p4dn and adds functionality to make it easier to work with. This is what everybody does an “add reference” on to talk to P4 from C#.

Note that p4api.dll is not strictly necessary as a separate assembly. The low level types exported by p4dn.dll could instead be kept internal, and all of that C# code from p4api.dll be written in Managed C++ and moved into p4dn.dll, entirely eliminating the need for p4api.dll.

Personally I was a fan of Managed C++, up until C# 3.0 where we started getting all kinds of nice language syntax to write better code more compactly. Today, I suppose I’d keep the extra DLL just for easier maintenance.

The Solution

In a nutshell, the solution is to trick the loader! Reference a p4dn.dll that does not exist, and use the AssemblyResolve event to intercept the load and reroute it to the correct bit size assembly.

It’s simple in concept but has a lot of details that took me a whole bus ride to figure out all the way (happily, there was a lot of traffic). Here is what I ended up doing to make it work how I wanted:

1. Rename the x86 output of p4dn.dll to p4dn.proxy.dll.
2. Update the x86 linker input settings to add __DllMainCRTStartup@12 to Force Symbol References.
3. Build a new x64 configuration for p4dn, using the x86 configuration (well, ‘Win32’) as a template. Have it output to p4dn.x64.dll.
4. Update the x64 linker input settings to add _DllMainCRTStartup to Force Symbol References.
5. Add a post-build event to p4dn’s x86 configuration that deletes p4dn.x86.* and copies the p4dn.proxy.* to p4dn.x86.*.
6. Update p4api to reference p4dn.proxy.dll. Not the csproj, but the actual DLL.
7. Update the SLN settings to make p4api dependent on p4dn.
8. Add a post-build step to p4api to delete p4dn.proxy.dll.
9. Set all p4api and p4dn configurations to output to the same bin folder.
10. Add a static constructor to P4API.P4Connection that registers an event handler on AppDomain.CurrentDomain.AssemblyResolve to pick the right DLL when the proxy is requested. I’ve pasted my code at the bottom of this post.

Once this is done, you’ll be able to have p4api as well as any assemblies that reference it set to AnyCPU. It will, upon first usage of the P4Connection class, fail to resolve the proxy and reroute to the correct bit width DLL.

A few notes on the above:

• The _DllMainCRTStartup is required because, without it, I got a crash from uninitialized memory systems in the CRT DLL’s that p4dn was linked to. This happened regardless of static vs. dynamic linking. I didn’t bother to find out the real reason for it. The different symbol names for 32 bit vs. 64 bit are because the convention changed when Microsoft went to 64 bit.
• The name of the DLL being referenced must match the original name of the DLL being built. That is, if you were to have p4dn outputting to p4dn.x86.dll then renaming it to proxy, and then referencing that, then it will actually look for the referenced DLL’s “true” name of p4dn.x86.dll and never call your hook.
• In projects that reference p4api it’s best to set the references to non-private (clear the “Copy Local” flag) and have a post-build step that just copies whatever is in the p4api bin folder. That makes sure you get the exact files that you need. None of this will work if you accidentally end up with the proxy file existing.

Here’s the code for my hook function. Note that it attempts to catch problems with the post-build scripts.

static P4Connection()
{
	string assemblyDir = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);
	if (	File.Exists(Path.Combine(assemblyDir, "p4dn.proxy.dll"))
		|| !File.Exists(Path.Combine(assemblyDir, "p4dn.x86.dll"))
		|| !File.Exists(Path.Combine(assemblyDir, "p4dn.x64.dll")))
	{
		throw new InvalidOperationException("Found p4dn.proxy.dll which cannot exist. "
			+ "Must instead have p4dn.x86.dll and p4dn.x64.dll. Check your build settings.");
	}

	AppDomain.CurrentDomain.AssemblyResolve += (_, e) =>
	{
		if (e.Name.StartsWith("p4dn.proxy,", StringComparison.OrdinalIgnoreCase))
		{
			string fileName = Path.Combine(assemblyDir,
				string.Format("p4dn.{0}.dll", (IntPtr.Size == 4) ? "x86" : "x64"));
			return Assembly.LoadFile(fileName);
		}
		return null;
	};
}

Microsoft, if you’re listening: FAT BINARIES.

I have two things in mind when I say “rich base class”. First is for mixing in object functionality. I’ve spoken about this before. There really is no way to do this nicely in C#. However, today’s post is about the other thing C++ gives you with multiple inheritance: utility methods.

Consider this interface and implementing class, paying attention to the overloads:

interface ILogger
{
	void Log(string message);
	void LogLine(string message);
	void Log(string format, params object[] args);
	void LogLine(string format, params object[] args);
}

class DebugLogger : ILogger
{
	void ILogger.Log(string message)
		{ Debug.Write(message); }
	void ILogger.LogLine(string message)
		{ Debug.WriteLine(message); }
	void ILogger.Log(string format, params object[] args)
		{ Debug.Write(string.Format(format, args)); }
	void ILogger.LogLine(string format, params object[] args)
		{ Debug.WriteLine(string.Format(format, args)); }
}

The more versions of the log function you want, the more work you have to do in any class implementing the interfaces. You could create a LoggerBase that does all of this, but that is severely limiting.

In C++ it’s easy of course. Make a mixin class.

class Logger
{
public:
	void Log(const char* message)
		{ OnLog(message); }
	void LogLine(const char* message)
		{ OnLog(message); OnLog("\n"); }

	void LogF(const char* format, ...)
	{
		char buffer[2000];
		va_list args;
		va_start(args, format);
		vsprintf_s(buffer, format, args);
		va_end(args);
		Log(buffer);
	}

	void LogLineF(const char* format, ...)
	{
		char buffer[2000];
		va_list args;
		va_start(args, format);
		vsprintf_s(buffer, format, args);
		va_end(args);
		LogLine(buffer);
	}

protected:
	virtual void OnLog(const char* message) = 0;
};

class DebugLogger : public Logger
{
	virtual void OnLog(const char* message)
	{
		OutputDebugString(message);
	}
};

Exactly one virtual method is required in the derived class. To add new overloads, you just add them to the base, and have them call the virtual. All end up with dynamic behavior. With interfaces in C#, you’re forced to implement each overload. This ends up with a lot of duplication everywhere you implement this same interface. Worse, if you want to add more utility functions to the interface, you break every implementing class, which must now implement that function as well.

The other day, it hit me that there’s an easy and perhaps obvious solution to this in C#: extension methods.

interface ILogger
{
	void Log(string message);
}

static partial class Extensions
{
	public static void LogLine(this ILogger logger, string message)
	{
		logger.Log(message);
		logger.Log("\n");
	}

	public static void LogFormat(this ILogger logger, string format, params object[] args)
	{
		logger.Log(string.Format(format, args));
	}

	public static void LogLineFormat(this ILogger logger, string format, params object[] args)
	{
		logger.LogFormat(format, args);
		logger.Log("\n");
	}
}

class DebugLogger : ILogger
{
	void ILogger.Log(string message)
	{
		Debug.Write(message);
	}
}

This will do what I want. I can implement a minimal interface, and easily add new utility functions. It’s not quite as good as C++, because I can’t store any data in my extension methods, and I must work solely through the published interface, but it’s good enough for 80%.

In one very small way it’s actually better than C++. In C++, that base class isn’t always something that can be changed. Perhaps it was provided by a standard library or a third party. Yet in C#, anybody can create an extension class to add functionality to any other class. So you can add all the overloads you like.

In my previous post, I talked about why I have stopped using finalizers for unmanaged resource collection. I want this to be done through the disposable pattern instead, forcing the programmer to manage resources manually.

Ironically, finalizers are a great way to verify this.

Sander van Rossen quickly figured out where I was going with this and proposed in a comment that we can just assert in a finalizer. We just need a couple more things:

1. We need to track the source of the object to figure out where the leak is.
2. We need to ensure that finalizers run on shutdown, or our assert will never get hit.

This is familiar territory to C++ programmers. Most of us use memory management libraries that provide leak detection and reporting. Let’s do something similar in C#.

DisposableBase

If you look around online, you’ll find some full-featured IDisposable base classes intended to deal with the full finalization model. We can eliminate most of that as either too complicated or unnecessary. We just need a few things:

• A stack trace grabbed at construction time
  This will be used for the error report. Because this is expensive to gather, we need to control it with an #if that is off by default, only turned on if needed to help an investigation. Most of the time a leaked disposable will be easy to find by inspection, but in the 5% case we will need a lot more context.
• A finalizer that reports the problem
  It could throw an exception, fire an assert, or route to an error reporter. Depends on the application. In my example I just have it output to the debug window for a demo.
• Disposal helper methods
  These wrap up disposal a bit, so inheritors only need to implement OnDispose. The most important feature, though, is that Dispose will call GC.SuppressFinalize when our object is disposed. This eliminates the performance cost of having a finalizer in the normal case when clients are disposing this class properly. This is why the finalizer has no “if” in it – if it ever gets called, then we have a bug.

This is what I am currently using as my base class for handling unmanaged resources:

// comment out unless diagnosing a leak
#define DEBUG_DISPOSE

public abstract class DisposableBase : IDisposable
{
    // store stack at point of construction for possible later use
#	if DEBUG_DISPOSE
    StackTrace _trace = new StackTrace(true);
#	endif

    // finalizer will not be called if object was properly disposed
    ~DisposableBase()
    {
        string message = "!! Forgot to dispose a " + GetType().FullName;
#		if DEBUG_DISPOSE
        message += "\n\nStack at construction:\n\n" + _trace + "!!";
#		endif
        Debug.WriteLine(message);
    }

    public bool IsDisposed { get; private set; }

    public void Dispose()
    {
        ThrowIfDisposed();
        IsDisposed = true;

        try { OnDispose(); }
        finally { GC.SuppressFinalize(this); }
    }

    protected abstract void OnDispose();

    protected void ThrowIfDisposed()
    {
        if (IsDisposed)
            throw new ObjectDisposedException(GetType().FullName);
    }
}

Demonstration

Here is a simple test app that shows what happens if we forget to dispose an instance.

public class DatabaseConnection : DisposableBase
{
    protected override void OnDispose()
        { Debug.WriteLine("Disposing"); }
}

public class Program
{
    static void Main(string[] args)
    {
        using (var remembered0 = new DatabaseConnection())
        using (var remembered1 = new DatabaseConnection())
        {
        }

        var forgotten = new DatabaseConnection();

        GC.Collect();
        GC.WaitForPendingFinalizers();
    }
}

The first two instances dispose fine, but the third is leaked and so we get a log to the output window:

Disposing
Disposing
!! Forgot to dispose a DatabaseConnection

Stack at construction:

   at DisposableBase..ctor() in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 18
   at DatabaseConnection..ctor()
   at Program.Main(String[] args) in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 89
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
!!

We can easily zero in on the exact spot where the leaked resource was allocated and wrap it in a ‘using’ to resolve.

Note the use of GC.Collect and GC.WaitForPendingFinalizers right before the test application exits. This is necessary in order to force all finalizing objects to be collected and reported before process shutdown, otherwise they are simply dropped by the system when the process’s memory is released.

These two can even be called during normal app run as well for leak testing without needing to wait for shutdown. This would be useful in a live service with a periodic leak for which we want an in-session nonfatal log of leaks.

What About External Classes?

This takes care of our own classes, where we have full control. But what about system or third party classes that are using finalizers? With those it’s back to square one.

Well, I suppose would write a helper class..

public class SafeDisposer<T> : DisposableBase where T : IDisposable
{
    public SafeDisposer(T disposable) { Obj = disposable; }

    public T Obj { get; private set; }

    protected override void OnDispose()
    {
        Obj.Dispose();
        Obj = default(T);
    }
}

public static class SafeDisposer
{
    public static SafeDisposer<T> Wrap<T>(T disposable) where T : IDisposable
        { return new SafeDisposer<T>(disposable); }
}

And equivalent updates in the demo code:

public class Program
{
    static void Main(string[] args)
    {
        using (var remembered2 = SafeDisposer.Wrap(new StringReader("foo")))
        using (var remembered3 = SafeDisposer.Wrap(new StringReader("poo")))
        {
            remembered2.Obj.Peek();
        }

        var forgotten2 = SafeDisposer.Wrap(new StringReader("boo"));

        GC.Collect();
        GC.WaitForPendingFinalizers();
    }
}

And output:

!! Forgot to dispose a SafeDisposer`1[[System.IO.StringReader, mscorlib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089]]

Stack at construction:

   at DisposableBase..ctor() in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 18
   at SafeDisposer`1..ctor(T disposable) in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 53
   at SafeDisposer.Wrap[T](T disposable) in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 67
   at Program.Main(String[] args) in C:\Users\Scott\Cloud\Proj\tests\BlogSamples\ConsoleApplication1\Finalizers.cs:line 98
   at System.AppDomain._nExecuteAssembly(RuntimeAssembly assembly, String[] args)
   at System.AppDomain.ExecuteAssembly(String assemblyFile, Evidence assemblySecurity, String[] args)
   at Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly()
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean ignoreSyncCtx)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()
!!

So to use, we just wrap a disposable at the point of creation in SafeDisposer.Wrap() and access it through “.Obj”. Not bad, but not great either. I’m ok with the wrapper function, but the required access via the Obj member is a pain, and also means that converting an unsafe disposable into a safe one requires updating a lot of code.

Another option is (maybe) to use a dynamic proxy system to inject the functionality we need, letting client code remain unchanged except at the point of creation. Or perhaps a run-time system that patches system assemblies to do the injection.

I’ll leave this as an exercise for the reader because I think we’re probably getting seriously diminishing returns at this point. Most disposable objects in a large system will be classes that we have full control over. The few objects not under our control will likely be low level primitives that will be wrapped up by our own foundation classes anyway.

I learned .NET when the beta of 1.0 was out, and at the time Microsoft was putting a lot of effort into educating programmers the difference between native and managed programming to make the switch. “In C++ you’d do this, but in .NET you do this” kind of thing. So practically every article on the CLR talked about finalizers. They often made analogies to C++ destructors. C# even added some ill-advised sugar that converts ~Class() to Finalize() to make us grognards feel more at home.

Finalizers were obviously very important, and so we learned about them. We evolved base classes to help us do the boilerplate work and wished that the .NET languages supported mixins. But it never felt right to me.

The Problem With Finalizers

Recently, after a couple conversations at work, I figured out what’s been bothering me about finalizers. The problem is that we’re using a commodity manager to manage non-commodity resources.

In .NET, raw memory can be treated as a commodity, like a gas tank. We can use it fluidly and at any granularity, and treat it all roughly the same. This is a simplification of course, and there are performance considerations, but they closely map onto the underlying OS and are familiar to everyone.

Yet non-memory resources are not commodities and cannot be treated with a one-size-fits-all pattern. They have semantics and effects far beyond incrementally draining and refilling that gas tank. Each situation, each class, is different, and has different implications that we have to remember. Because of this, we can’t treat these resources in a nondeterministic way without potential hazards.

For example, take a file handle managed by a finalizer. Off the top of my head, there are three adverse effects of an unused handle not finalizing in time:

1. If there are pending writes to the file, then they are lost if the app exits without forcing finalizers to be called.
2. If the file was opened with restrictive sharing, then other applications cannot manipulate it until the GC gets around to finalizing the file object.
3. The OS isn’t designed to treat file handles as a commodity, and using them that way can negatively affect performance. We could even run out of file handles because the GC has no idea when there’s pressure in this space (all finalizers are equal).

Worse yet, in diagnosing any of the above issues, we also must roll our own tools. System tools such as LockHunter and Process Explorer have no way of distinguishing which handles are actually in use and will just give us a noisy, useless mess.

And that’s just a simple file handle example. The situation is obviously worse as more limited and complicated resources are involved like DirectX surfaces or database connections.

Dispose: Only A Partial Solution

You might wonder why I’m making a fuss. There’s an obvious answer to the non-determinism, right? Microsoft recognized this problem early in in .NET 1.0 and gave us the disposable pattern. It is a standard way of managing resources the old-fashioned way: ‘new’ allocates the resource, and Dispose() frees it. We even have some special syntax that helps automate this:

using (var textureMgr = new TextureManager())
using (var texture = textureMgr.AllocTexture())
{
    texture.Fill(Color.White);
}

The above is roughly equivalent to:

var textureMgr = new TextureManager();
try
{
	texture = textureMgr.AllocTexture();
	try
	{
		texture.Fill(Color.White);
	}
	finally
	{
		texture.Dispose();
	}
}
finally
{
	textureMgr.Dispose();
}

This gives us a rough, though somewhat tedious approximation of the RAII pattern used universally in C++.

In Microsoft’s .NET classes that wrap unmanaged resources, both patterns are typically used. There is a Dispose system that closes the handle, and a safety Finalize that cleans up if not already done already via Dispose.

So what’s the big deal? Well, the problem is in (a) knowing when it is necessary to call Dispose, (b) remembering to call it, and (c) updating dependent code when an existing type adds a new IDisposable implementation. It’s easy to miss something and let bugs creep in. For every single ‘new’ call, we must check the type to see if it or one of its parent classes implements IDisposable. If it does, then we must manage the instance directly. This means wrapping allocation in the ‘using’ construct for local temporaries, or implementing IDisposable and forwarding Dispose if contained as a member in another class.

Forgetting to dispose an unmanaged resource can lead to some of the most frustrating and difficult to track down bugs. And the nondeterminism in the underlying system pretty much squares that problem. It’s guaranteed to behave differently in the field than on a development machine.

Every new .NET programmer figures this out quickly when they write their first command line app that opens a file, does something to it, and writes out a new file. They find that, apparently randomly, sometimes the end of the new file is cut off. Forgot to call Dispose to flush and close it eh?

Most Strategies Not 100% Guaranteed

It’s not hopeless. We’ve come up with strategies to deal with this. First is knowledge base. Over time, we get a sense for what types of objects tend to implement IDisposable. File handles, database connections and so on, those are easy. Though what about systems written by someone else on your team? Does the source control connection have to get disposed when you’re done with it because it wraps a COM object? For those cases, it’s better to be safe and check for IDisposable until the foreign classes become familiar.

There are also tools that help out. FxCop can find problems that are discoverable via static analysis. CodeRush will draw graphical markup for types it detects as IDisposable.

But these strategies still aren’t enough for me. I want absolute certainty that Dispose is called on all unmanaged resources, and not left to the nondeterministic finalizer system to cause latent timing problems.

So I want to propose an additional strategy. It’s pretty simple, actually: consider the finalizer to be a last, and illegal resort. Microsoft’s guidelines say that you should not assume Dispose has been called. I propose the reverse: if the finalizer is called, then there is a bug in the code.

More on this in the next post.

The Problem

Say you have a pair of functions that bracket other operations you want to do. For example, you must first call BeginRender() before you call RenderModel(), and after you’re all done, you need to call EndRender(). Another example would be a EnterLock() on a critical section, that must be accompanied by a LeaveLock(). I’ll call these bracketing functions simply “BeginX” and “EndX” for the purposes of this article.

The problem is that the caller must not only remember to call EndX, but must do so even in case of an exception, to avoid blocking the system.

The Solution: IDisposable

What we’re doing here, really, is acquiring and releasing a resource. The .NET Framework already has a pattern for that, and it’s called IDisposable. Ok, that’s not much of a trick in itself. What I learned today was a little twist on the concept.

The trick is simple:

1. Create a private inner class that implements IDisposable.
2. Return a new’d instance of this class in BeginX.
3. In the instance’s Dispose(), have it call the private EndX.
4. Callers can simply wrap the ‘handle’ in a using declaration for cleanup.

It’s obvious in retrospect, and it’s clean. The ‘using’ guarantees calling EndX even in case of an exception. And in leveraging IDisposable we can build on top of the C# programmer’s instinct to wrap usages of IDisposable objects in ‘using’ constructs. As a fallback, we can rely on tools like CodeRush to catch it as we type (or FxCop as a fallback).

Implementation

First, we need a fun little helper base class.

public abstract class DisposerBase<T> : IDisposable
{
    bool _disposed;
    T _owner;

    public DisposerBase(T owner)
        { _owner = owner; }

    public void Dispose()
    {
        Debug.Assert(!_disposed);
        _disposed = true;
        OnDispose(_owner);
    }

    protected abstract void OnDispose(T owner);
}

Here’s a sample class that demonstrates the IDisposable-bracket pattern. Note the inner class that has access to private methods, which we use to call EndX. This method is private so that the only way to close the bracket is to call Dispose() on the handle. Simple, straightforward, consistent.

public class TransactionManager
{
    bool _inTransaction;

    public IDisposable BeginTransaction()
    {
        Debug.Assert(!_inTransaction);
        Console.WriteLine("Begin Transaction");

        _inTransaction = true;

        // ... do other transaction setup here ...

        // caller must dispose this when done
        return new TransactionDisposer(this);
    }

    private void EndTransaction(TransactionDisposer state)
    {
        Debug.Assert(_inTransaction);
        Console.WriteLine("End Transaction");

        // ... do transaction cleanup here ...
        // (state could provide more context if needed)

        _inTransaction = false;
    }

    #region TransactionDisposer

    class TransactionDisposer : DisposerBase<TransactionManager>
    {
        // obviously some state could go here
        // (say, for more transactions in flight at once)

        public TransactionDisposer(TransactionManager owner)
            : base(owner) { }

        protected override void OnDispose(TransactionManager owner)
            { owner.EndTransaction(this); }
    }

    #endregion

    public void DoSomeAction(string actionName)
    {
        Debug.Assert(_inTransaction);

        // ... do the action ...
        Console.WriteLine("  Action performed: " + actionName);
    }
}

Note that this is a very simple example that only permits one transaction in flight at once. If we wanted to do more, we could have a Dictionary that maps outstanding transactions to pending state, or we could just put the state directly into the TransactionDisposer. I’ve noted this in the comments.

Now, it’s a bit of typing to implement this disposer, but that’s what snippets are for, right? Bracketed code like this isn’t overwhelmingly common either, so it shouldn’t be a big deal. Still, I have to say I miss my old friend #define from the C world sometimes. Not often. I’m looking forward to a future version of the C# compiler that lets us plug into the AST directly and codegen as we go.

Anyway, here is an example usage of this pattern:

var transactionManager = new TransactionManager();
using (var handle = transactionManager.BeginTransaction())
{
    transactionManager.DoSomeAction("action 1");
}

Implicit Chaining

There is one interesting benefit that comes from this: implicit chaining.

Say you have a higher level class that provides additional functionality within its own BeginX. Perhaps it calls BeginRender() and then immediately sets some default states. Rather than having to create a new inner class to manage the bracketing, you can simply return the result of BeginRender, and the caller will not know the difference.

Let’s extend the above example in this way.

public class RequestManager
{
    TransactionManager _transactionManager;

    public RequestManager(TransactionManager transactionManager)
        { _transactionManager = transactionManager; }

    public IDisposable BeginRequest(string initialActionName)
    {
        var handle = _transactionManager.BeginTransaction();
        _transactionManager.DoSomeAction(initialActionName);
        return handle;
    }
}

See how it chains the TransactionManager’s bracketed calls through itself? We could have gone through the trouble of making an inner RequestDisposer class and a private EndRequest and all that, using the IDisposable from TransactionManager as the state, but why bother? Just chain it along. When the caller disposes the handle, it routes straight through to the TransactionManager, which is all we needed.

Some sample usage:

var requestManager = new RequestManager(transactionManager);
using (var handle = requestManager.BeginRequest("action 2"))
{
    transactionManager.DoSomeAction("action 3");
}

Of course, if RequestManager has to do anything special in its EndX, then we’ll have to go and add that inner class after all. It’s a nice shortcut otherwise.

The output from both examples looks like this:

Begin Transaction
Action performed: action 1
End Transaction
Begin Transaction
Action performed: action 2
Action performed: action 3
End Transaction

Easy huh?

Performance Implications

It’s worth asking what the costs of this method are. How does this compare to simply calling BeginX and EndX?

Let’s consider this sample usage again:

using (var handle = transactionManager.BeginTransaction())
{
    transactionManager.DoSomeAction("action 1");
}

This is syntactic sugar (I loathe this term but it is unfortunately apt) that the compiler translates into something like this:

IDisposable handle = transactionManager.BeginTransaction();
try
{
    transactionManager.DoSomeAction("action 1");
}
finally
{
    if (handle != null)
        handle.Dispose();
}

Note that the runtime does not do anything special with IDisposable types. They are created and collected exactly the same as any other type. And as a result, this is almost identical to what we’d have to do for a manual BeginX/EndX:

transactionManager.BeginTransaction();
try
{
    transactionManager.DoSomeAction("action 1");
}
finally
{
    transactionmanager.EndTransaction();
}

Now, this is assuming no state needs to be maintained in the handle. If it were, then we’d have to store it in a local and pass it into EndTransaction, and the BeginX/EndX pattern would almost exactly match the disposer pattern.

So: the cost of the disposer pattern over BeginX/EndX is that an extra object gets new’d up, with a bit of glue logic that runs. Insignificant, not worth worrying about.

Even if it did have a real cost, I’d still not worry about it. This is intended to be used in bracket patterns where resources are being allocated and freed. The cost of the underlying resource management is going to mask whatever minor cost the disposer adds.

The cleaner, more maintainable, and statically-analyzable code that results is an easy win.

Until now, this stuff has been scattered all over four different defunct websites in a mess of broken links. This new page fixes all of that for good.

I worked at Gas Powered Games between November 1999 and October 2003, back when it was a fresh startup. In that time, I lectured and wrote about some of the technology we used to build Dungeon Siege and later projects. I also released specifications for some of the internals of the engine to help the mod community.

This page collects all of my published Dungeon Siege work.

…continued here

It’s pretty exciting, even if I am joining this game very late.

As I work in the WPF world (and maybe WCF and other techs as I start messing around with them) I’ll be posting odds and ends here about research I do and dead-end paths I go down.

Not WinForms++

The first thing I learned is that WPF isn’t WinForms++ or even “WinForms in XML”. It lives in an entirely different universe. Though, for folks like me that have been stuck in that old world too long, WPF is initially going to feel like a simple upgrade of an old framework. There are a lot of similar-sounding classes and concepts in the new framework. These are merely false cognates.

I’ve been learning that it’s better for me to enter into new WPF concepts like a 101 student: by reading books and MSDN docs. I haven’t done that in a while. Normally I just bang on code and refine, using Google and StackOverflow to guide my way (and surfclarity to filter out spam sites like experts-exchange.com). I’m still doing that as I pick up WPF, but adding on top of it some serious reading.

Pro WPF in C# 2008 (available at Safari where I’m reading it) is my current favorite book for learning WPF concepts. Most books I come across are lame reference manuals disguised as tutorials, or sensationalized drooling over new tech with no real experience to back it up. The author does great things like quickly explaining a cool-sounding WPF concept, then telling you not to use it and why. Not all of WPF turned out to be a good idea, after all.

So, here are some of the key benefits I’ve noticed in the short time I’ve worked in WPF:

Layout and Composition

The problem of control layout (wrap, dock, table, stretch, fill, and so on) is solved. It’s nice of WPF to catch up with where HTML has been for a while, though in a much cleaner and more internally consistent way. Later versions of WinForms started to get somewhere with TableLayoutPanel and friends, but it’s still the same old underlying framework, where you run into limitations very quickly.

WPF lets you easily nest any control within nearly any other, and has a sensible system for guiding layout containers (attached properties rather than Control-derived least-common-denominator properties). It’s simple, for example, to put a pair of icons and a few checkboxes into each element of a ListView, and make it look good. Virtualization is automatic as well, given certain constraints. Trying to make any of that work in old WinForms requires a lot of custom code.

In fact, it was enough work in WinForms that I tended to avoid doing anything complicated. This led to UI’s that were easier to program but harder to use due to the need to break out elements or hide things behind the ubiquitous “more” or “advanced” buttons. Consider your average Tools|Options dialog and the custom toolwork required to make that work in a Form. This is why “field of checkboxes” settings dialogs proliferate in so many apps (see pic at right).

With WPF I’ve found it easy to not only wire up the initial draft of the UI, but also to go back and do a big reorg without any real trouble, and usually without significantly changing any code. I Just did it three times today, in fact!

A Special Note on XAML

Layout and composition are now typically done in XML, using Microsoft’s XAML. At first I was tentative on XAML compared to MXML due to the total lack of support for writing code inline in the attributes. Well, it turns out that viewmodels eliminate the need for much of that inline code (with the glaring exception of value converters – more on this in a future post). It’s also probably a good idea to keep code out of the XAML if you’re splitting the work with a designer. Having code in those attributes would not only get in the way, but would be brittle and hard to maintain.

The bottom line is that WinForms’ crazy InitializeComponent is gone. Bye bye merge problems and periodic, inexplicable total reorganization of that function by Visual Studio. Instead, we lay things out hierarchically in nice, simple, consistent, and nearly unreadable XML. Well, that’s what the visual designer is for.

Data Binding

I’ll admit I didn’t mess around with data binding much in the WinForms world. It just felt like too much work for what was a simple thing to do by hand. And WPF is vastly more complicated and capable than Forms, so it was intimidating at first.

But in WPF you really need to fully embrace data binding. I suppose it’s possible to do old UI-driven Visual Basic style state management in WPF, by putting everything in the codebehind, but you’re not going to get much support for this. All samples and tutorials you’ll find online use data binding. It really is how the entire system was designed to be used. Data binding is about a lot more than simply tying a textbox to a text string member. It’s about separation of concerns.

Thankfully, it’s not difficult to pick up. There’s some crazy complicated 20-levels-deep-stack logic that makes it all work under the hood, but the end result is that you just cover your eyes and type {Binding SomeName} on whatever field you want bound to your viewmodel and it magically works. “Except”, as they say, “when it doesn’t”. So far it’s been rare for me when something fails to bind, or binds to the wrong place, and I’m developing a bag of tricks to debug these issues. I’ll try to post some of these in the future.

Now, about that viewmodel thing. It’s also practically required to embrace the MVVM pattern in building up nontrivial WPF-based UI’s. This is the “separation” I mentioned previously. Now, this takes a lot of practice to figure out and to start applying correctly. It took me about two months to really grok WPF+MVVM. So be sure to budget time to learn and practice this. Read the article I just linked for a good introduction.

Styling

The problem of styling controls and windows is hugely improved. It simply isn’t possible to modify controls in WinForms without writing a totally custom solution. WPF takes a big step forward in enabling simple as well as complex styling of the standard controls. You can completely alter the look and visual behavior of any WPF control without a lot of work.

And then there’s Expression Blend, my new love interest. Blend is a tool for the design and styling of WPF UI’s, and permits dividing responsibilities between engineers writing code and designers building the look & feel. We have this exact split at work, in fact. Engineers will poke out a rough UI, and later on one of the game UI designers will take some time to upgrade elements to look cool and be consistent with the rest of the tool.

Blend has a goofy, nerdy interface, but it’s really growing on me as I use it every day. It’s far from intuitive, but is powerful in terms of designing controls and styles, previewing it all live, live live! in its awesome designer. It can even build your full Visual Studio solution from within itself. I do almost all of my design work within Blend, partially because VS2008’s designer is so awful as to be almost useless (haven’t used 2010’s enough yet to comment) but mostly because it’s more familiar to the Flash programmer within me. What can I say, I like arty tools.

Rendering

One of the reasons styling can be done so easily is that WPF doesn’t rely on the underlying OS window manager to draw checkboxes and so on. This removes all limitations on what you can change. If you use a tool like Snoop or Mole to take apart a UI control, you can see that every little graphical piece and state of it resolves down to primitive WPF objects that you can inspect.

For fun, I edited the template of the ResizeGrip control in Blend to see what makes it tick. This is the little triangular dotted thingy that sits in the lower right corner of most sizable windows. Blend showed me that it’s rendered using a simple brush that has a repeated pattern of gradient squares clipped to a little stair-step box. All vector so it scales nicely.

How does this new rendering model affect Remote Desktop, by the way? Custom-rendered windows typically perform poorly through RDP. Try running Chrome or iTunes through RDP, or nearly any app through VNC, on a low bandwidth VPN connection to see what I mean. In earlier versions of .NET, if a DWM existed on both sides, the WPF graphics primitives would get remoted directly. However, it seems that in .NET 3.5SP1 and newer, WPF is remoted completely as bitmaps.

Microsoft is saying that this should actually improve performance in many cases. Well, we can only expect a future of more complex, denser GUI construction (I mean complex in the UI tree sense, not in the end user’s perception of complexity). So the 100%-bitmap-remoting solution may be needed in the long term anyway. But I’m not sure I agree with the timing of this decision. Bitmap remoting so far performs an order of magnitude slower for me than graphic primitive remoting, at least in the everyday apps I use through RDP.

Bye Bye WinForms

It’s clear to me that anyone working in WinForms should ditch it as soon as possible in favor of WPF. Rewriting old tools is always a cost-benefit analysis, but for new tools, this is clearly the way to go. Especially if you’re in control of your user environment, such as within the enterprise, where you can force install the framework on every machine.

If you’re still hanging out in the WinForms world, or considering buying an expensive Forms-based UI package to pick up the pace on building UI’s, you really ought to take a hard look at WPF.

Don’t expect to be productive immediately, or even within a month. It will take a while to get used to it, learn the framework, and develop patterns to use in building UI. It’s time well spent. This is clearly the future.