Creating Nullable<T> When You Don't Know T

I ran into a very strange problem tonight. Here's my scenario:

• I have a value which comes in as a string (from a URL parameter)
• I need to convert this string into a Nullable<T> where I don't know T at compile time (but I do at runtime).

This first thing I tried was the direct approach: call Convert.ChangeType() to convert directly into the Nullable<T>. That fails, with convert throwing an exception that it doesn't know how to convert from string to Nullable<T>. This felt kinda strange, since Convert.ChangeType() works just fine when I pass T; you'd think that it would be easy to convert T into Nullable<T>.

Strike one.

Okay, I think, let's leverage Convert.ChangeType() to get it into T, and then use Activator.CreateInstance(). The constructor for Nullable<T> takes a T for its value. Here's what the code essentially looks like, in a nicely compact system:

using System;

public class MainClass {
    public static void Main() {
        string original = "1";
        PrintValueAndType("original", original);
        
        object converted = Convert.ChangeType(original, typeof(int));
        PrintValueAndType("converted", converted);
        
        Type type = typeof(Nullable<>).MakeGenericType(typeof(int));
        object result = Activator.CreateInstance(type, converted);
        PrintValueAndType("result", result);
    }
    
    static void PrintValueAndType(string name, object obj) {
        Console.WriteLine("{0} = {1} ({2})", name, obj, obj.GetType().FullName);
    }
}

This code prints out the values at all three stages, including the type. What do you expect to see when you run this? This is what you get:

original = 1 (System.String)
converted = 1 (System.Int32)
result = 1 (System.Int32)

I'd expected the result type to be Nullable<T>, not T.

Strike two.

Thinking maybe Activator was the problem, I resorted to reflection. I called GetConstructor() on the nullable type, and then called Invoke on the resulting ConstructorInfo. Same result.

Strike three. Time for a blog post.

What exactly is going on here?

I'm going to have to make some educated guesses, but this was the clue. This is my updated code:

using System;

public class MainClass {
    public static void Main() {
        string original = "1";
        PrintValueAndType("original", original);

        object converted = Convert.ChangeType(original, typeof(int));
        PrintValueAndType("converted", converted);

        Type type = typeof(Nullable<>).MakeGenericType(typeof(int));
        object result = Activator.CreateInstance(type, converted);
        PrintValueAndType("result", result);

        Nullable<int> x = 1;
        PrintValueAndType("x", x);
    }

    static void PrintValueAndType<T>(string name, T obj) {
        Console.WriteLine("{0} = {1} ({2}, {3})", 
                          name, 
                          obj, 
                          obj.GetType().FullName, 
                          typeof(T).FullName);
    }
}

Now when I run it, this is what I get:

original = 1 (System.String, System.String)
converted = 1 (System.Int32, System.Object)
result = 1 (System.Int32, System.Object)
x = 1 (System.Int32, System.Nullable`1[[System.Int32, ...

With the generic version of the method, you'd expect obj.GetType() and typeof(T) would be the same thing, but they're not. I suspect the reason is that obj gets boxed for the call to GetType(), and the CLR's behavior when boxing a Nullable<T> is to box the value as type T, throwing away its nullability.

What makes this maddening is that, if you don't know the T ahead of time, there's essentially no way to actually store Nullable<T>. Since the result of both Activator.CreateInstance() and the ConstructorInfo.InvokeCall is object, even though you actually had a Nullable<T> for a very short time, it immediately got boxed and the nullability was thrown away.

Oops, strike four. Now what!?

I found a work-around for my specific case, but I think I'm still stuck with the notion that programmatically creating Nullable<T> for an arbitrary T is just not possible.

In my case, I was programmatically modifying a LINQ expression, adding a Where clause. In essence, I'm writing code to do this:

query = query.Where<T>(x => x.PropertyName == value);

when I don't know T, PropertyName or value until runtime. (For what it's worth, modifying LINQ queries like this feels a lot like using CodeDOM to generate code at runtime, but less brain bending.)

When you are putting together the where expression programmatically, you use System.Linq.Expression.Constant() to generate the "value" part of the expression. I had been using this:

var valueExpr = Expression.Constant(value);

Luckily, there is an overload which takes the type that you want the constant to be:

var valueExpr = Expression.Constant(value, myNullableType);

Even though Convert.ChangeType() can't change T into Nullable<T>, LINQ constant expressions don't appear to have any problems doing it.

I like to have blog posts that end in answers rather than questions, but unfortunately I'm still a little bit stumped by this behavior and how to overcome it. I was lucky to have found an appropriate work-around for LINQ expressions, but I could still be stuck if I hadn't.

Is it possible that you simply can't create Nullable<T> programmatically for an arbitrary T? Is Nullable<T> a Heisenberg-Type, where observing it changes it fundamentally?

Comments

[Jason Bock]

I'm wondering if you could create a new method at runtime via DynamicMethod to create the Nullable. It's not a trivial way to solve it (and there may be an easier way) but this should work.

[wekempf]

int? t = null;
Type type = t.GetType();

The result is unexpected. http://msdn.microsoft.com/en-us/library/ms366789(VS.80).aspx

[Jb Evain]

Still using ETs, you can use something like that:

static Func NullableNew () where T : struct
{
var p = Expression.Parameter (typeof (T), "t");
return Expression.Lambda> (
Expression.New (typeof (T?).GetConstructor (new [] { typeof (T) }), p), p).Compile ();
}

That will give you a delegate you can call to create arbitrary nullable instances.

[Jb Evain]

Hrm, the code undergone some kind of generic erasure :) Code is here:

http://monoport.com/34372

[wekempf]

Jb Evain,

That doesn't really help. Your NullableNew function relies on static typing. Try to use it when given a Type and an object instance. Once you go through the pain of using reflection to call both NullableNew and the created delegate, the returned value will still be boxed, erasing the Nullable type wrapper.

[Brad Wilson]

Sorry, TypePad appears to removes things that resemble HTML elements rather than just escaping them. Stupid and lame. :-p

[Brad Wilson]

@Jason,

The problem isn't getting the thing created; the problem is, what do you stick it in afterwards?

You don't know the type T, so you can't declare the variable of type Nullable, and as soon as you put it into an Object, the nullable gets boxed away.

[Greg Beech]

Interesting post; it looks like you're right, that Nullable gets boxed to T. I'm sure I read about that in the past but had completely forgotten, so thanks for reminding me!

Just for fun, here's a dynamic method that creates an arbitrary nullable, but it still gives the same result as you found because the return type is boxed as object when returned from the Invoke method.

var t = typeof(int);
var nullableT = typeof(Nullable<>).MakeGenericType(t);
var dynamicMethod = new DynamicMethod("CreateNullable", nullableT, new[] { t });
var il = dynamicMethod.GetILGenerator();
il.Emit(OpCodes.Ldarg_0);
il.Emit(OpCodes.Newobj, nullableT.GetConstructor(new[] { t }));
il.Emit(OpCodes.Ret);

var result = dynamicMethod.Invoke(null, new object[] { 123 });
PrintValueAndType("result", result);

[Brad Wilson]

@Greg,

I'm sure it's being properly created and returned from IL, but since Invoke returns object, casting away happens again.

Yeah, I'm convinced you simply can't do this.

[Curt Hagenlocher]

Ah, but the nullable doesn't exactly get boxed away. Instead, if the nullable had a value, it's boxed as its value. And if the nullable didn't have a value, it's boxed as "null".

The good news is that this operation is 100% symmetrical. When you unbox into a Nullable, a null becomes a valid Nullable with HasValue = false and an int becomes a valid Nullable with value = the originally boxed value.

As you say, the real issue is one of representation. There's no way to represent a value type whose actual type is not known until runtime -- unless it's boxed.

[Curt Hagenlocher]

Bah, and now I too have been bitten by the "angle bracket" problem despite being forewarned... Each "Nullable" in my second paragraph should really have been "Nullable[int]".

[Jeff Brown]

I actually find it rather convenient that that Nullable wrapper type gets discarded when the value is boxed and automatically created when unboxed via a conversion to nullable.

This tremendously improves the interoperability of code that uses nullables because you only have to worry about one kind of null being passed in an object array - a null reference. There's no need to do any weird things with checking whether a type is nullable and calling HasValue.

As a result you can just think of the "?" on the nullable type as a modifier of sorts whereas the Nullable struct is just an implementation detail with somewhat curious semantics.

[configurator]

Excuse me, but I can't think of a case where you'd want to do that; that operation seems meaningless to me.
What exactly do you need this for?

[David Nelson]

Since any value type T is implicitly convertible to Nullable[T], I am having trouble envisioning a scenario where you would need to do this. Can you elaborate?

[Mark]

I've noticed a similar thing when using the non generic version of nullable. GetType always returned the non-nullable type rather than the nullable version, but I could pass the instance into a method that expected a nullable version of that type no problem. So it looked like a 'normal' type, but it was actually a nullable.

I was in a situation where I sometimes needed a nullable version of a type and other times I did not, so in the end I just set up a factory that instantiated nullable types and converted them back to non-nullables if needed (vice-versa wouldn't work -- I couldn't get it to convert from normal to nullable).

I actually knew how to instantiate my types at runtime (as there weren't that many of them) via factory; that's the only reason it worked. In the end it was... hackish.

My head hurt that day. I'm never going near this stuff again unless I feel a bit masochistic.

It made me feel good to see other people have experienced the nullable madness, too!

[carpptn]

static void Main(string[] args)
{
Nullable n = 0;
int i = 0;

Console.WriteLine(IsGeneric(n));
Console.WriteLine(IsGeneric(i));

Console.ReadLine();
}

public static bool IsGeneric (Nullable n)
{
return true;
}
public static bool IsGeneric (int i)
{
return false;
}