PurelyFunctional.tv Newsletter 428: CRUD Paging

Written by Eric Normand. Published: May 24, 2021.

Issue 428 - May 24, 2021 · Archives · Subscribe

Follow-up 🙃

Last week, we developed an adapter from Ring to CRUD. It converted Ring requests to CRUD requests. But we left one thing out: I forgot to convert CRUD responses to Ring responses. Basically, we need to translate the few CRUD response statuses we defined two weeks ago.

I imagine a simple way to do it: a lookup table. The table would translate the CRUD status keywords to HTTP status codes. Here it is:

(def status-table
  {:entity-not-found 404
   :entity-type-not-found 404
   :success 200 ;; or 201 if not GET
   :operation-not-allowed 400
   :request-malformed 400
   :server-error 500})

The :success status may have to be special cased. Either way, the translation should be straightforward.

Another thing I forgot to mention is that the adapter model is very flexible. The adapter presents an abstraction barrier. The internals of CRUD are hidden from the externals of messaging. This means that, although we've defined an adapter that translates HTTP<=>CRUD, we could swap out any messaging format instead of HTTP. For instance, we could write a new adapter that translates Slack messages or ZeroMQ messages into CRUD. The barrier means we can vary the inside and the outside independently.

Clojure Tip 💡

CRUD Paging

We built a simple, almost minimalist CRUD request representation, similar to how Ring is a minimalist translation of HTTP into Clojure maps. Many common web functionalities are left out of the basic Ring adapters. For instance, Ring does not include, by itself, parsing of the query string. Instead, it comes with middleware that does the parsing. You can include the middleware or not. I believe this is a smart choice. It makes Ring relevant if you want to parse the query string in a slightly non-standard way. The only cost is being explicit all the time, even if you want the standard parsing.

One very common feature of CRUD APIs is paging. When you list all entities of a given type, you probably don't want all of them to come back in the same request. If you have thousands of users, listing them would make for one big JSON. Paging solves this by limiting the number of items you get back per request. If you want more, you can ask for the next "page" of them.

Paging is an example of a cross-cutting concern. It's something you would probably want for all of your entities---and even all of your APIs. It should be built into a middleware---a way to augment a handler with new functionality. Building this functionality before we release CRUD as a standard is important. We want to know that our abstraction can be extended and this is a good test.

Paging shouldn't be part of the generic adapter. Although 90% of APIs would use the basic paging, the other 10% will want something custom. Following Ring's example of query string parsing, we will include it in the library but require being explicit that you want to use it.

Adding paging to the request is simple. A CRUD request for the :list operation can include a :page and :page-size key. The :page indicates the page number, and :page-size tells the maximum number of items to show per page.

(defn wrap-paging [handler {:keys [max-page-size default-page-size]}]
  (fn [req]
    (let [page (get-in req [:params :page] 0)
          page-size (min max-page-size
                         (get-in req [:params :page-size] default-page-size))
          req' (assoc req :page page :page-size page-size)
          resp (handler req')]
      resp)))

This middleware adds :page and :page-size to the CRUD request, with appropriate defaults. The middleware asks you to provide some options.

Notice we have to make a change to our adapter for this. We are now asking for some information from the HTTP request (query params) to be included in the CRUD request (under the :params key). That's an easy change and a good reason to try to build functionality in this way: It makes our CRUD spec more general.

Installing this middleware means our handlers will have to be changed to use the new information in the CRUD requests.

We will also want to change the response. We will want to add a few more pieces of data to the CRUD response that will get passed back through the HTTP request.

{:status :success
 :entity-type "user"
 :entities [...]
 :page 2
 :page-size 30
 :total-count 76   ;; how many of this entity type are available
 :has-more? true}

The HTTP response will look like this:

{:status 200
 :headers {...}
 :body {:page 2
        :page-size 30
        :total-count 76
        :has-more true
        :entities [...]}}

We will need to extend the adapter a bit to be able to include this information. We want something generic that can easily be extended. I think making the CRUD response a little more structured with an explicit section for the data that will be passed unchanged to the client, like so:

{:status :success
 :message {:entity-type "user"
           :entities [...]
           :page 2
           :page-size 30
           :total-count 76 ;; how many of this entity type are available
           :has-more? true}}

:status gets translated into the Ring :status and :message gets used as the Ring :body as-is. We will need to modify the CRUD spec we have build over the last two weeks and change the adapter accordingly. But this is a change for the better. It gives a little more control to the CRUD side and will allow more extension through middleware.

I think it's prudent to try one or two more extensions to stress the abstraction barrier of the adapter. We want to define it once and for all while allowing extension from the inside. We'll tackle another extension next week: authorization.

Podcast episode🎙

This week on the podcast, I talk about the onion architecture again. There are two ways to go, and one way leads to problematic semantic dep endencies.

Book update ✍️

Grokking Simplicity is now available on Amazon! People have already been receiving their copies.

Please, if you like the book and/or believe in its mission of starting a discussion about the practice of FP in the industry, please leave a 5-star review. Reviews will help people learn whether the book is good before they buy.

You can also get a copy of Grokking Simplicity at Manning's site. There you can use the coupon TSSIMPLICITY for 50% off.

Clojure Media 🍿

The REPL has just sent out a new issue after a year hiatus! The REPL is a Clojure-focused newsletter from Daniel Compton. It's always packed full of links to great stuff about Clojure. Go sign up!

Quarantine update 😷

I know a lot of people are going through tougher times than I am. If you, for any reason, can't afford my courses, and you think the courses will help you, please hit reply and I will set you up. It's a small gesture I can make, but it might help.

I don't want to shame you or anybody that we should be using this time to work on our skills. The number one priority is your health and safety. I know I haven't been able to work very much, let alone learn some new skill. But if learning Clojure is important to you, and you can't afford it, just hit reply and I'll set you up. Keeping busy can keep us sane.

Stay healthy. Wash your hands. Wear a mask. Take care of loved ones.

Clojure Challenge 🤔

Last issue's challenge

Issue 427

Friday the 13ths - Submissions

This week's challenge

Formatted prime factorization

Prime factorization means representing an integer as a product of primes. A function that factorizes a number w ill return a vector of primes, like so: [2 2 3 5]. Your job is to take such a vector and create a nice string that shows the mathematical notation of the product.

Examples

(format-product [2 2 3 5]) ;=> "2^2 x 3 x 5"
(format-product [2 3 3 3 11 11]) ;=> "2 x 3^2 x 11^2"
(format-product [7]) ;=> "7"

Use x to indicate multiplication and ^ to indicate exponentiation.

Thanks to this site for the problem idea, where it is rated Hard in Ruby. The problem has been modified.

Please submit your solutions as comments on this gist.

Rock on!
Eric Normand