xUnit: using non-public types as test cases for parameterized tests
One of the most useful features of xUnit test framework is the [Theory] attribute, which allows us to write parameterized tests. This way, we can inject multiple test cases into a single test method and test our SUT (system under test) with a variety of test data representing multiple use cases – both typical and edge-cases. There are plenty of tutorials on various ways to inject test cases into a Theory, but I haven’t yet found one on how to use test cases that use data marked with internal or other non-public access modifiers.
The problem
Let’s consider the following scenario: we need to consume some input data (e.g. coming from an external API) and translate it into some internal representation optimized for doing our work. For the purpose of data encapsulation, that translated representation needs to be truly internal – that is, marked with an internal or other non-public access modifier.
For the rest of this article, we’ll work on the following data model:
public record SourceData
{
public string CommaSeparatedValues { get; set; }
}
internal record TranslatedData
{
public IReadOnlyCollection<string> Values { get; set; }
}And our SUT (system under test) will look like this:
internal class Translator
{
internal TranslatedData Translate(SourceData source)
{
// doing some complicated work here
return new TranslatedData { Values = source.CommaSeparatedValues.Split(",") };
}
}Of course we need to mark the assembly containing the TranslatedData and Translator classes with [assembly: InternalsVisibleTo("MyTestProject")] attribute, so that our tests are able to discover and call the Translate() method. There are multiple tutorials on how to do it, so I won’t cover it here.
As for injecting test cases into our test method, xUnit gives us three useful attributes: InlineData, MemberData and ClassData.
Attempt 1: InlineData
The most straightforward way to inject test cases into a Theory is to inject them directly with an [InlineData] attribute. This works well with simple CLR types, like int or string:
[Theory]
[InlineData("foo", "bar", "foobar")]
internal void Test_concatenation_using_InlineData(string first, string second, string expected)
{
var actual = string.Concat(first, second);
Assert.Equal(expected, actual);
}However, the compiler doesn’t allow us to use complex types with InlineData, as that attribute’s parameters all need to be constant values. So, the following code produces a compiler error CS0182: An attribute argument must be a constant expression, typeof expression or array creation expression of an attribute parameter type:
[Theory]
[InlineData(
new SourceData { CommaSeparatedValues = "foo" },
new TranslatedData { Values = new[] { "foo" } }
)] // error CS0182
[InlineData(
new SourceData { CommaSeparatedValues = "foo,bar" },
new TranslatedData { Values = new[] { "foo", "bar" } }
)] // error CS0182
[InlineData(
new SourceData { CommaSeparatedValues = "foo,bar,baz" },
new TranslatedData { Values = new[] { "foo", "bar", "baz" } }
)] // error CS0182
internal void Test_using_InlineData(SourceData source, TranslatedData expected)
{
var actual = new Translator().Translate(source);
Assert.Equivalent(expected, actual);
}So using InlineData is unfortunately out of the picture.
Attempt 2: MemberData
Another way is to use [MemberData] attribute. With this attribute, we need to specify a public static property or method returning IEnumerable<object[]> – a collection of arrays, with each array representing arguments for a single test case:
[Theory]
[MemberData(nameof(TestCases_MemberData))]
public void Test_using_MemberData(SourceData source, TranslatedData expected)
{
var actual = new Translator().Translate(source);
Assert.Equivalent(expected, actual);
}
public static IEnumerable<object[]> TheoryTestCases_MemberData()
{
return new List<object[]>()
{
new object[] {
new SourceData { CommaSeparatedValues = "foo" },
new TranslatedData { Values = new[] {"foo" } }
},
new object[] {
new SourceData { CommaSeparatedValues = "foo,bar" },
new TranslatedData { Values = new[] {"foo", "bar" } }
},
new object[] {
new SourceData { CommaSeparatedValues = "foo,bar,baz" },
new TranslatedData { Values = new[] {"foo", "bar", "baz" } }
}
};
}This time the compiler gives us the following error: CS0051: Inconsistent accessibility: parameter type 'TranslatedData' is less accessible than method 'Test_using_MemberData(SourceData, TranslatedData)'.
Fortunately, the error is pretty simple to fix: xUnit allows the test methods to be not only public, but also internal, protected and even private, so we just have modify our test method to use internal access modifier.
Attempt 3: ClassData
Finally we have the [ClassData] attribute. With this attribute, wee need to specify a class that implements IEnumerable<object[]> interface and returns our test cases as a list of arrays of object. The simplest way to achieve this is to create a List<object[]> containing our test cases and defer the IEnumerable implementation to that list.
As we learned from the previous point, our test method needs to be internal. Additionally, the test data class also needs to be internal.
Let’s see it in action:
[Theory]
[ClassData(typeof(TestCases_ClassData))]
internal void Test_using_ClassData(SourceData source, TranslatedData expected)
{
var actual = new Translator().Translate(source);
Assert.Equivalent(expected, actual);
}
internal class TestCases_ClassData : IEnumerable<object[]>
{
private readonly List<object[]> _testCases = new List<object[]>
{
new object[]
{
new SourceData { CommaSeparatedValues = "foo" },
new TranslatedData { Values = new[] { "foo" } }
},
new object[]
{
new SourceData { CommaSeparatedValues = "foo,bar" },
new TranslatedData { Values = new[] { "foo", "bar" } }
},
new object[]
{
new SourceData { CommaSeparatedValues = "foo,bar,baz" },
new TranslatedData { Values = new[] { "foo", "bar", "baz" } }
}
};
public IEnumerator<object[]> GetEnumerator() => _testCases.GetEnumerator();
IEnumerator IEnumerable.GetEnumerator() => _testCases.GetEnumerator();
}Some problems with previous attempts
In the attempts #2 and #3 we finally managed to write code that allows our tests to run and report meaningful results, but there are still problems with that approach.
First of all, in the case of ClassData we need to write a bunch of boilerplate code – specifically, we need to manually implement IEnumerable interface to point to our list of test cases. This makes writing test cases less pleasant and intuitive than it could be, as we have to remember to use the proper boilerplate – which more often than not leads to searching through StackOverflow to remind us of how exactly to write the test data class.
Our test cases, whether implemented via MemberData or ClassData, also lack type safety: as they are simply arrays of objects, the compiler doesn’t check for things like whether the array length matches the number of parameters for our test method, or whether the types of objects in the array match the types of parameters of our test method.
So if for some reason we had to change either to singature of our test method or the contents of our test cases, we may end up with the following code:
[Theory]
[ClassData(typeof(TestCases_ClassDataWithTypeSafetyErrors))]
internal void Test_using_ClassDataWithTypeSafetyErrors(SourceData source, TranslatedData expectation)
{
var translated = new Translator().Translate(source);
Assert.Equivalent(expectation, translated);
}
public static IEnumerable<object[]> TheoryTestCases_MemberData()
{
return new List<object[]>()
{
new object[] {
new string("foo"),
new string("foo")
},
new object[] {
new string("foo,bar"),
new string("foo"),
new string("bar")
},
new object[] {
new string("foo,bar,baz"),
new string("foo"),
new string("bar"),
new string("baz")
}
};
}The above code will compile without any errors or warnings, even from xUnit’s own code analyzer, but it will crash when we try to run our tests: Object of type 'System.String' cannot be converted to type 'SourceData'. So, can we do better?
Attempt #4: TheoryData
The answer is: yes, we can. xUnit gives us a generic TheoryData<> class to use in conjunction with either MemberData or ClassData attribute. TheoryData<> is just a type-safe wrapper around IEnumerable<object[]> with all the required boilerplate already written for us. Not only does this solution reduce boilerplate and thus automate much of our work, but is also enforces type-safety. All we need to do is to implement the parameterless constructor and declare our test cases there using the built-in Add() method.
Again, let’s see it in action:
[Theory]
[ClassData(typeof(TestCases_TheoryData))]
internal void Test_using_TheoryData(SourceData source, TranslatedData expectation)
{
var translated = new Translator().Translate(source);
Assert.Equivalent(expectation, translated);
}
internal class TestCases_TheoryData : TheoryData<SourceData, TranslatedData>
{
public TestCases_TheoryData()
{
Add(
new SourceData { CommaSeparatedValues = "foo" },
new TranslatedData { Values = new[] { "foo" } }
);
Add(
new SourceData { CommaSeparatedValues = "foo,bar" },
new TranslatedData { Values = new[] { "foo", "bar" } }
);
Add(
new SourceData { CommaSeparatedValues = "foo,bar,baz" },
new TranslatedData { Values = new[] { "foo", "bar", "baz" } }
);
}
}And now we have finally checked all of our boxes: we are able to use non-public types as input for our test cases, the code compiles without errors or warnings, our tests report appropriate results, and also we’ve reduced the amount of boilerplate code to a minimum and have the benefit of compiler-enforced type safety.
Note that TheoryData<> can generally be used as a target for both MemberData and ClassData attributes; however in our case we need to use ClassData, as MemberData requires the test cases to be marked with public static modifiers, and our test data are non-public.
Conclusion
When using xUnit’s Theories to write parameterized tests that take non-public types as parameters, remember these things:
- mark your test methods as
internal, as opposed to traditionalpublicaccess modifier - use either
MemberDataorClassDatato specify test cases… - … but I recommend using
ClassDatain conjunction withTheoryData<>for maximum convenience and type safety, as shown in the Attempt 4: TheoryData section of this article.
