Using your test suite to educate with guard rails

As a codebase grows and ages, writing comments on dangerous parts of your codebase stops scaling well — comments easily drift out of date if someone forgets to update them and you’re relying on others (who may be in a hurry) to pause and read your comment before making their changes. You won’t always have time to go back and refactor an area of the codebase that has become dangerous, but you can at least minimize future pain for others by putting a guard rail around it.

Here’s an example:

• Open pull request that changes constant value.
• Pass code review.
• Merge pull request, take down production.

By adding this test, the workflow we are trying to encourage is:

• Open pull request that changes constant value.
• Pass code review.
• Wait, CI is failing?
• Read and understand error message. Consider whether changing constant value is required. Don’t take down production!

Because these tests are a little different, there are a few rules that go into writing a good one compared to your average test:

• Write a clear, helpful failure message (ideally prose and multi-lined). You want the impact of the change someone is making to be SUPER CLEAR, so do what you have to in order to communicate the impact of the change. I’ve written versions of these tests that include markdown tables demonstrating what changing a constant will do — the sky’s the limit, as long as you can make the impact of someone’s change front and centre. Do whatever you need to in order to satisfy this, and spend a lot of time on making sure your failure message is clear.
• Be as self contained as possible; duplicate any value that you plan to check. If the test loads a value dynamically, you’ll lose the benefit of the test (making sure a value doesn’t change without intent).

By migrating things I’d normally write a comment on to instead be executed by the test suite, I’ve been able to build better guard rails into our codebase so that newcomers get surprised less often while taking their changes to production. That failing test acts as a self documenting guard rail, and it has the added benefit of only appearing when someone is interacting with the code that I wrote the documentation on which means they get feedback as close as possible to the change they are making.

What are the guard rails in your codebase?