Those of us that practice TDD daily already know how important good error messages in tests are. After all writing a failing test that clearly states what functionality the program is missing is the first step in TDD cycle. The rest of us that either can’t or simply don’t want to practice TDD must put extra effort to ensure that tests always fail with meaningful error messages. Unfortunately, according to what I have learned from my personal experience, the most devs either don’t have enough time or simply don’t bother to check if their tests fail with something meaningful. For average Joe developer writing tests and making them green is already a lot of work. Things like good test names and proper error messages are often forgotten.
But the developers are not the only one here to blame. Too often tools and libraries that supposedly should make unit testing simpler and easier, generate horrible and often cryptic error messages.
In this post we will take a close look at NSubstitute, a modern and popular mocking libary for .NET and see how we can improve messages generated by its argument matchers.
Let’s start by looking at a simple test. It demonstrates how NSubstitute is often used to assert that a method was called with an argument in a certain state:
When the argument passed to the checked method was in an unexpected state (e.g. first name was not “jan” but “john”), we get an error message similar to (formatting added):
This error message is terrible. It contains a lot of informations that are easily obtainable by looking at the test method’s source code. Yet it does not contain the most important piece of information that we need: which properties have unexpected values and what these values are.
We can slightly improve this error message by overloading
Let’s call this new class
Now the error message looks similar to (formatting added):
This is better than before. Now we can see both expected and actual values of the matched argument’s properties.
One drawback of this approach is that
the quality of the error message depends on the quality of
If we are using AOP solution like
Fody to generate
for most of our classes, then this solution may be good enough.
On the other hand if we are generating and updating our
manually (even if this means pressing a shortcut in our IDE)
then I would prefer to look for a better solution that is totally automatic.
There is also another problem that we were ignoring so far.
Consider what will happen if we add a new field to our
Because we are using property access syntax inside of a lambda expression,
our existing matchers will not only compile without any problems when
we add a new field, they will also pass!
In order to ensure that our matchers and tests remain valid,
we must go through all argument matchers
StringableArgument class and make sure that they use
the newly added field.
The above problem may be solved by moving equality checking
StringableArgument class itself.
Let’s call this new class
With this solution it is impossible to forget to update our matchers when we add a new field. We also get a slightly better error message (formatting added):
So far so good. But what if our argument has ten or more properties.
With complex arguments looking for a one property with
unexpected value may quickly change into “Where’s Wally?” game.
The only way to further improve error messages is to stop relaying on
and instead to use specialized assertion library like
Here is how our test would look like if we decide to use FluentAssertions:
The error message is:
Not bad, I must say. We get a list of only these properties that have
unexpected values. Certain messages seem a little bit too verbose for me
Expected member FirstName to be "jan" with a length of 3,
but "john" has a length of 4, differs near "ohn" (index 1).
Expected FirstName to be "jan" but was "john". would be
Still it is the best solution that we have so far.
The only downside that I see is that the test code is now a little more verbose and less readable. Mainly because we are now responsible for manually capturing argument’s value:
With a bit of C# magic we may make argument capturing less painful:
To see how it works please check ArgCapture.cs file.
Catching argument’s value manually is cumbersome and makes tests less readable. On the other hand using some “magical” syntactic sugar also does not looks like a good idea. After all our code should be simple. If we can avoid using “magic” we should do it!
Our final solution is to create a custom NSubstitute argument matcher. The matcher uses undercover FluentAssertions library to perform the check. Here is how the test code looks like with this approach:
The error message generated for an argument that does
ToString looks like this (formatting added):
It is clear that the problem occurred at the first argument (
Also we can see the actual and expected values of the argument’s
fields and properties.
And the test code is simple and clean.
If you are interested how it is implemented please see
As we can see there exists no perfect solution. Still with a little effort
we can make our error messages much more readable and pleasurable to work with.
I personally suggest to use either the last solution or
the solution presented in
Source code and examples: GitHub