PurelyFunctional.tv Newsletter 402: up-front vs incremental design

Written by Eric Normand. Published: November 3, 2020.

Issue 402 - November 03, 2020 · Archives · Subscribe

Hurricane update 🌬️

Sorry the newsletter is late today. I just got internet back after Hurricane Zeta. We're okay over here. Just getting back to it!

Clojure Tip 💡

up-front vs incremental design

There's a very important debate in the software industry. In fact, you could call it a primordial debate, out of which whole methodologies emerge. That debate is between up-front design and incremental design. It will do us some good to revisit just what we mean by those terms before continuing.

Up-front design, at the extreme, dictates a complete, coherent design of the system before you start programming. The motivation is good: don't waste programmer time if the design isn't finished yet. Work out the details of the design, perhaps on a whiteboard or in a document, so that the it can be a guide for the programmers who follow it.

Of course, we know this isn't practical. Most software is incredibly complex. We can't understand the second-order or third-order consequences of all of the design decisions. Those of us who have written software in this way know that the design is often incomplete and inconsistent. The design document does not contain all important decisions because there were decisions that weren't obvious until you had to write out the code. And the design document is often inconsistent because we can't think and write precisely enough to catch logical contradictions. Finally, because the design of a large system, followed by the programming of that system, can take many years, the design requirements often change in the meantime. All of these taken together indicate that maybe we should code sooner.

A rebellion against up-front design is to design incrementally. We assume the design is always incomplete. That is, someone designs a small chunk, then the chunk is programmed and integrated into the software. The chunk size is chosen to be small enough to be consistent and complete. It solves the problems of having to wait a long time for the design, and the troubles with inconsistencies are pushed off to the indefinite future.

Of course, this isn't practical either. It can deliver software quickly, but at the extreme it turns the dev team into a black box. Chunks come in, software comes out. It assumes someone has a global vision of the chunks or the software becomes a mess. Without the global design vision, programmers resort to "code cleanliness", which is like keeping your poorly designed house organized. It helps you make progress, but it's not a better house. In short, it's not a design methodology as much as a software delivery methodology. There is a lot practical wisdom in the incremental approach, but a design philosophy is still needed.

What I'm exploring as a possible philosophy is this notion of truth. When we first start coding a piece of software, we don't understand the domain very well. We don't understand the design. So the incremental approach makes sense. But we learn as we go. As we learn the contours of the domain, we need to represent those in our software. Too often, we keep those things in our heads. But until we incorporate them into our code, they can't contribute to truth.

Truth is the fit between your code and the relevant details of your domain. It requires you to discover the relevant details and encode them in your software. You can discover these incrementally, as you deliver software. But they need to be written into your code. I believe that by doing this, your software can improve over time as you discover the domain.

The big objection I foresee is that a notion of truth, with implications of timelessness, doesn't work well with the idea of changing requirements. What happens after you've encoded all of this truth into your code and the requirements change? Do you change the laws of physics when you build a new building? I think any conflict only comes from a misunderstanding of what timeless means. We'll talk more about that next time.

Quarantine update 😷

I know a lot of people are going through tougher times than I am. If you, for any reason, can't afford my courses, and you think the courses will help you, please hit reply and I will set you up. It's a small gesture I can make, but it might help.

I don't want to shame you or anybody that we should be using this time to work on our skills. The number one priority is your health and safety. I know I haven't been able to work very much, let alone learn some new skill. But if learning Clojure is important to you, and you can't afford it, just hit reply and I'll set you up. Keeping busy can keep us sane.

Also, if you just want to subscribe for a paid membership, I have opened them back up for the moment. Register here.

Stay healthy. Wash your hands. Stay at home. Wear a mask. Take care of loved ones.

Clojure Challenge 🤔

Last week's challenge

Issue 401

Sums of pairs - submissions

Please do participate in the discussion at the submission links above. It's active and it's a great way to get comments on your code.

This week's challenge

Most frequent element

Write a function that takes a collection and returns the most frequent element. But here's the thing: you can't use the built-in function clojure.core/frequencies. And if there are ties, just pick one.

Examples

(most-frequent [2 2 3 4 4 2 1 1 3 2]) ;=> 2
(most-frequent []) ;=> nil
(most-frequent [1 1 4 4 5]) ;=> 4

Notes

return nil for an empty collection
in the case of a tie, return one of the winners

Thanks to this site for the challenge idea where it is considered Very Hard level in JavaScript.

Please submit your solutions as comments to this gist. Discussion is welcome.

Rock on!
Eric Normand