Nanny tests

The more time and code that I write, the more I learn about testing. The project that I work on now introduced me to the idea of a nanny test. A loose definition is a test that doesn’t test functionality, but tests some aspect of your code.

---

Forbidden code

I answered a SO question a while back making some suggestions as to how you could prevent anyone from calling a particular method. The method was Directory.Delete. Forget about “you need better developers” or “just make sure they don’t do it”. Trying hard is a noble effort, but rarely works. Better to have a test to guard against it.

The key is to read the IL of your code. I like Cecil, but there are other options like CCI Metadata and loading the raw IL. The one downside to Cecil (and CCI) is that there is a gap between normal reflection and the library. As a result, you often need to using strings to go between them.

Let’s say you are looking for File.Delete. What you want is a way to catch this:

 

This will do the job.

   1: var resolver = new DefaultAssemblyResolver();

   2: var assembly = resolver.Resolve(typeof(ClassWithForbiddenCode).Assembly.FullName);

   3:  

   4: // the string compare is to bridge the gap between Cecil and the Fx reflection types

   5: var typeToTest = assembly.Modules.SelectMany(x => x.GetTypes())

   6:     .First(x => x.FullName == typeof (ClassWithForbiddenCode).FullName);

   7:  

   8: foreach (var method in typeToTest.Methods)

   9: {

  10:     foreach(var instruction in method.Body.Instructions)

  11:     {

  12:         var opCode = instruction.OpCode;

  13:         if (opCode.Code == Code.Call 

  14:             || opCode.Code == Code.Calli 

  15:             || opCode.Code == Code.Callvirt)

  16:         {

  17:             if (instruction.Operand.ToString() == "System.Void System.IO.File::Delete(System.String)")

  18:             {

  19:                 Console.WriteLine(

  20:                     string.Format("Forbidden code found in class {0} method {1}",

  21:                                                 typeToTest.FullName, method.FullName));

  22:             }

  23:         }

  24:     }

  25: }

The first bit is getting the assembly reference. In order to load the correct assembly, I am using the full name of the assembly that I am testing against. This is an example of using strings to get the correct Cecil object.

For this example, I am loading the type to test explicitly, but this could easily be done against all types.

Line 10 is where it gets interesting. It grabs the IL inside of the method body, and looks for any IL that is a method call. Note that I am checking for all three ways to call a method. This isn't needed for this example, but shows how to catch all calls. At this point, I am using a string compare to check the method. There are typed ways to do this, but this is nice and easy.

And you're done. Now just add this test to your unit test suite, and you can sleep soundly in the knowledge that a file won't be deleted. Unless they pinvoke…

What is this blog, and why does it exist?

This is now my third technology related blog. The first one is long gone - lost in the dustbin of the internet. It was active in the late 90s, and dealt with the trials and tribulations of working on classic ASP and server side coding for the web.

The second was an MSDN blog from when I was at Microsoft, but I found that it was better to post to the official blogs (e.g., XML blog, ADO.NET blog, data blog, etc).

This will be my third. Over the years I have worked with some fantastic people, and learned a great deal. I will try to pass some of it along. If nothing else, I can use it to compensate for my memory. The internet will never forget, right?

Erick