Quit writing individual tests and let the computer write thousands of tests for you
Here's a sad fact: even with unit tests, your code can still have bugs.
It turns out that we are bad at writing tests that really exercise the code. Did you find all of the corner cases? Did you test a huge range of inputs?
What if you could enlist the help of your computer to write tests for you, finding more bugs and gaining more confidence in your code?
That's what Property-Based Testing lets you do.
Unfortunately, there haven't been good tutorials on the topic. That's why I created this course.
Introducing Beginning Property-Based Testing with test.check
Clojure's library for Property-Based Testing is called
test.check. It's a powerful, full-featured library for getting your computer to write tests for you and finding hard-to-reproduce bugs.
Here's how Property-Based Testing works:
- Define the random values the computer will generate—these are called your generators.
- Define a boolean expression where
trueindicates passing, and
falseindicates failing—this is your property.
- Tell the computer how many tests to generate.
test.check will generate random inputs, run them through your code, and see if they pass. It will generate way crazier inputs than you'd ever think to generate, and as many as you ask for. This is why
test.check can find bugs that you would have missed.
However, it's not done there. Seeing a crazy, random value that happens to fail is not helpful. Why did it fail? What, in particular, is wrong with it? That's where the magic happens.
test.check will figure out a minimum failing value—through a process called shrinkage—that is so small and clear that you can know very easily why it failed. Shrinkage is one of the keys to making Property-Based Testing work for you.
What you will learn in this course
Property-Based Testing is a powerful technique and there's a lot to learn. This beginner's introduction to the topic will give you everything you need to get started.
- How to use built-in generators and how to build your own
- 3 strategies for defining properties so you'll never get stuck
- Testing before you implement, like in TDD
- Testing after you implement, which is only possible with property-based testing
And, of course, plenty of examples for each of them. You'll also get an in-depth understanding of how the whole process works.
In this course, you will be watching me test code. You'll see how I read error messages and get out of trouble. You'll see how I develop the tests interactively using the REPL. You'll see how to use the REPL to better understand your generators and properties.
Beginning Property-Based Testing with test.check
Want this course?Advanced Clojure: An Eric Normand Signature Course
Love it or leave it guarantee
If you don't learn as much from this course as you thought, just ask for a refund within 30 days and I'll give you your money back.
1. What is Property-Based Testing?
Property-Based Testing is a way to generates lots and lots of tests so you don't have the write them. It finds bugs and helps you design your software. In this introduction to the course, we learn what Property-Based Testing is.
2. What is test.check?
test.check is the Property-Based Testing library we'll be using in this course. In this lesson, we'll take a look at the origin of this library, how it integrates with Clojure, and how it's getting a lot of attention from the Clojure core team.
3. Advantages of Property-Based Testing
Property-Based Testing has several advantages over regular Example-Based Testing. It's worth calling them out.
4. When can you use Property-Based Testing
Property-Based Testing can be used at four different times, from design-time to after your users have found a bug, to help you reproduce it.
5. How does Property-Based Testing work?
In this lesson, we go through the basic algorithm used by test.check to test our code and show us useful failures. The algorithm gives us a map of the things we will have to learn.
6. An overview of property-based testing with an example test
In this lesson, we will see an example of the process of testing an existing function using Property-Based Testing.
7. A tour of the built-in generators
test.check comes with many built-in generators that we can use to create properties and build new generators from. We take a tour of the generators provided so we have a feeling of what's available.
8. Building your own simple generators
We can put together generators that we need from the existing generators. There are four tools provided by test.check that help us do this. In this lesson, we go over such-that, fmap, bind, and let.
9. Strategies for properties: functionality
Coming up with properties is hard. So we need some strategies to try out when we need to come up with them. In this lesson, we go over two ways to directly test the functionality of a function.
10. Strategies for properties: invariants
Coming up with properties can be a challenge. One strategy is to look at what doesn't change. In this lesson, we take a look at testing invariants.
11. Strategies for properties: algebraic properties
I often run through a short list of algebraic properties, the ones you learn in Algebra class, to find properties to test. In this lesson, we go over six properties that you can use and modify to your needs.
12. Testing pure functions
Pure functions are the easiest things to test. You can run them as many times as you want without fear of side effects. In this long episode, watch over my shoulder as I test five different functions. Three of the functions are fairly mathematical, so they're pretty straightforward. But two are copied from real-world code. You'll get to see the raw process, right here.
13. When to test: before implementation
In this lesson, we implement a function using TDD. Property-based testing is great for doing TDD since the tests are constantly giving you small examples that fail, one at a time. Make it pass, and get another. Make that pass, get another. Eventually, you'll have the whole thing tested and implemented.
14. When to test: after implementation
Testing after an implementation exists and is working is often frowned upon. Your brain is biased toward your implementation, so it's hard to find good counterexamples. However, in Property-based testing, the system is generating tests, and it doesn't have a bias.
Although PBT is very valuable for testing after implementing, it still has two challenges. You still need to think of how to trick the tests without having the benefit of modifying the implementation. The second challenge is to avoid duplicating the implementation. A test that uses the same code as the function it tests is useless.
In this lesson we go through the process of testing an existing function. We write many tests, and settle on just a couple that seem to cover the entire functionality.