As I was mentioning in my previous post, the way we are doing unit testing today has many issues. One of the biggest problems is to know for sure when we have enough tests to be confident to release in production. Why is that? Well, we mostly base our tests on examples, so it's tough to know for sure if we covered all the edge cases and that we have all the required samples.

Tale of a developer

Let's use a story to demonstrate what I mean.

Notes:

Here, the gray circle will be defining what the client wants, the Xs will visually represent the tests, and the red lines will represent the actual output of the algorithm.

Once upon a time

Client: Could you create an algorithm that draws a circle given a radius? Something like that.

Developer: Of course! Easy peasy.

After some time

Developer: Alright I'm done, it's working. Client: How do you know if it's working properly? Did you test it? Developer: Well... I made it run on my machine. If you want, I can add a unit test to prove it. Client: That would be great. Developer: Here you go. I added a unit test that makes sure that the point (1,0) is part of the circle.

Client: How do you know that you are not only drawing a horizontal line that passes through that point? Developer: Ok, here's another test that makes sure that the point (0,1) is part of the circle.

Client: What about coordinates that are not int? Developer: Sure. I added another test for this.

Client: I don't want just a quarter of a circle. Can you make sure other quadrants are also supported? Developer: Arrrrg... Sure. I added another test for this. I'm pretty sure we covered everything. We should ship it.

Client: Alright, after all, you are the expert. Let's ship it.

Once in production

Client: What the heck. It looks nothing like the circle I wanted. Developer: I don't understand. All my tests are passing, which means this should work just fine according to your requirements.

Developer: Maybe if I add another test, I'll find the issue.

The truth is

In fact, with this kind of problem, you'll never be done adding examples. It's because a circle is composed of an infinity of points, which makes it almost impossible to validate with example-based tests.

What do we do then

At this point, I'm pretty sure we all agree that we need a better strategy to test this. That's precisely the kind of problem where Property-Based testing shines. Instead of validating points along the circle, let's try to understand and model the relationship between the input (the radius) and the output (the circle). It turns out that this relationship is well known and defined in the math world.

A circle can be defined as the locus of all points that satisfy the equation x^2 + y^2 = r^2

It's an excellent property for a test. With only one test, we can validate any point of a circle given a radius. Voilà!

Conclusion

Stay tuned for my next blog post, where we'll solve a real problem with C# and property-based testing.