PurelyFunctional.tv

Gain more confidence in your stateful code

Stateful code is hard to test. Each test defines a sequence of operations on the state, and then you check if it's in the right state at the end. It takes a lot of time to write those, so we don't tend to write a lot of them. And we rarely test the corner cases.

Property-Based Testing (PBT) gets the computer to generate those tests for us. It can generate thousands of tests, each with a different sequence of operations. The fact is that PBT finds bugs in databases, queues, and other stateful systems.

test.check is the Clojure library that implements Property-Based Testing. test.check is deeply integrated into Clojure Spec.

In this course, we build off what we learned in the Beginner's course to take our tests even further. We learn:

  • How to build sophisticated generators for your complex, structured data
  • A powerful strategy for writing properties when the input is hard to generate
  • The inner-workings of generator size and the shrinkage process
  • How to test stateful systems—code with mutable state
  • To integrate with Spec when Spec's automatic generators aren't good enough
  • How to use Spec's fspec and when you have to use test.check directly

Property-Based Testing is a powerful way to test your software. This course starts where the Beginning course left off. We look at more complex generators, interesting strategies for properties, and integration with Clojure Spec.

Intermediate Property-Based Testing with test.check

Updated July 30, 2021

Want this course?

Advanced Clojure: An Eric Normand Signature Course

Love it or leave it guarantee

If you don't learn as much from this course as you thought, just ask for a refund within 30 days and I'll give you your money back.

Video time: 06h10m
0 / 9
0 / 9

Lessons

41 min

1. Building complex generators: email address

We use Strings a lot in programming, but often our Strings are assumed to have very specific formats. We need to be able to generate Strings that are still random, yet fit the format. In this lesson, we see how we can build a new generator by generating the pieces of the String and putting them back together.

42 min

2. Building complex generators: matrices

Sometimes the data we need to generate is complex and can't easily be done directly. In that case, we can often do well by generating instructions to build the value. In this lesson, we see how that can be applied to generating random matrices of different sizes.

24 min

3. Strategies for properties: generate the output

Sometimes it is not possible to generate the input directly. And sometimes it is not possible to test the output without reimplementing the function you are testing. In either of these cases, you could always do the opposite of what's normal: generate the OUTPUT and convert it to an input.

21 min

4. Behind the scenes: size

It's important to know about size. Size controls the range of values of a generator. It changes over the length of a run of a property. The built-in generators have an intuitive notion of size, and you can control the size yourself using three functions from the generators namespace.

35 min

5. Behind the scenes: shrinkage

When a test case fails, the shrinkage process begins. It takes the failing test case and tries to make it smaller so that it's easier for the programmer to isolate the problem. In this lesson, we look inside the shrinkage process. It's not something you have to think about all the time, but sometimes you do.

1 hr

6. Testing stateful systems

Testing mutable systems that keep state is slightly harder than testing pure functions, but PBT is up to the challenge. In this lesson, we test two different systems and encounter the challenges we face with gusto!

1 hr

7. When to test: during system design

One of the secrets benefits of Property-Based Testing is that it helps you design a system. Think about it: you've got a bunch of constraints you know you need to operate within, there are certain desired properties, etc. You can encode those as Property-Based Tests and develop a model (not a real implementation) and see what it would take to meet those requirements. Then, you can use your model to test the real system. In this episode, we design an email sending system that needs a very specific kind of idempotency.

1 hr

8. Testing with Spec: custom generators

Clojure Spec integrates with test.check and uses it for testing. When you create a spec, Clojure Spec will try to construct a generator for you. Sometimes it can't do that, so you need to know how to coerce it into making one. And sometimes the generator isn't satisfying, so you need to replace it with another one. In this lesson, we go over the best practices for customizing generators with Clojure Spec.

1 hr

9. Testing with Spec: functions

Spec lets you define function specs, which give you a great place to spec the arguments and return values of functions. In addition, you can define invariants that need to be true between the arguments and return values. A lot of your test.check tests will be able to go right in there. Algebraic properties, though, will not. In this second lesson about Spec integration, we go over how Spec can run automated tests of your functions, how to integrate it in a simple way with clojure.test, and the limits of the Spec tests.