The joys of a modern checklister system


The elevator speech

Checklists are great for QA, though a bad checklist can be more hassle than good. I think everyone should be using a checklist to save themselves trouble in the future. I'm designing a scheme for dynamic checklists that makes QA easier and less in the way of getting stuff done. The key idea is to create a good interface for the ugly and lengthy internals of a checklist to make it scale into hundreds of items, as opposed to ten items at most. It's the 21st century, let's update the checklist for the modern world!

Check, mate!

You may scoff at a checklist. You don't need such toys. A checklist, after all, is pretty much only rubber duck debugging, with the duck being replaced by a piece of paper and trying to be proactive rather than reactive. Silly. You are a Big (Girl/Boy/Person/Thing) and don't need to do highly formal QA.

I wish you'd give it a chance. Checklists save lives. Checklists keep your head cool when things are going crazy around you. Checklists make you not forget things that are easily missed. Finally, checklists give you comfort in knowing that you have done your job right.

I found five serious issues with this very (tiny) website, just by formalizing things into a checklist and then running it. All those steps I was doing anyway, or so I thought. With writing the steps out as a checklist and actually paying attention to the list I managed to improve the quality of the site for my awesome readers. Pretty cool, no?

So... Why aren't you using a checklist, silly? I have some ideas why not!

A checklist may be hard to start. What is comprehensive enough? Too many checks and people won't want to do it or will get sloppy. Skip too many steps and you may have left out something important. A balance is required and if you don't factor this into your design then you are doomed. You cannot just grow a checklist with time. This initial investment of time is a big deal.

Another problem is procedural. A checklist is something that you usually have to get around to doing, which will tend to make you forget to run it at all. Once you remember than you even had a checklist in the first place, it is too late and the damn thing is long obsolete and only useful for giving you a flare-up of your dust allergy.

To prevent this, a checklist must be an required part of your workflow. Checklisting ought to be an automatic and integral part of your processes. You should not be able to do anything major until a checklist program had a look at it. You still should be able to say "Shut up, I know what I'm doing, let me do it!", but you should at least know you are about to break the safety guidelines that you previously set for yourself.

You might think that having to always deal with a checklist every step of the way would become your own idea of a nightmare. Extra work! Horrid! Bad! No!

Well, you can relax. This does not have to be so bad. Let me tell you about three things that checklists can help you with. Each kind needs a different kind of checklist support.

I'm going to use my website as an example, but the techniques can be generalized easily enough to just about anything.

Just this once!

luisant.ca is a new website. I'm honestly not sure how long I'll keep running this playful experiment, but for now it exists. I'm pretty new to creating anything webby as well. I don't know JavaScript and barely know everything else. At the same time, quality of service is important to me. This means fast loading times.

Let's both take a look at my front page. We currently have three .png images there, to represent the logos of my projects. They make the front page the heaviest part of the whole site, but I really like having a logo for every project I do. To remedy the bloat I want those images to be as small as possible.

There are a few steps to making a .png image small. First, the file needs to be resized down. Secondly, we can run pngquant to cut down on the color count used and remove the metadata. Finally, pngzopfli will squeeze down the file a little bit more without losing quality. Three steps that I have to do once for every .png file. After I have done then, I never want to worry about them again.

I will create a checklist item for a broad category of files, all .png files. Excuse the clunky interface, this is just a very early and preliminary draft of a prototype of a pre-alpha version of a pre-release preview.

# First, create the checklist
# I'll name it website.
$ julia checklister/new_list.jl website
# Now I add in a regular expression to my checklist
# The checklist runs on the basis of file extensions
$ julia checklister/new_matcher.jl website
Enter Regex: [.]png$
# I run the blank checklist
$ julia checklister/run_list.jl website luisant/
Adding a new file: antipode.png
Adding a new file: guide.png
Adding a new file: remynifier.png
Running the [.]png$ checklist for antipode.png
Running the [.]png$ checklist for guide.png
Running the [.]png$ checklist for remynifier.png

Right! The .png files are now under checklisting. By default the software will add an entry to the checklist for every matching file it finds. There is no effort on your part there. As there are no questions to ask, there is nothing else to do. So, how about we add a question?

$ julia checklister/new_question.jl website
Select matcher
1: [.]png$
1
New item text: Is the file 300x300?
Always check? true/false: false
Only first check? true/false: true

Now I can run the checklist again:

$ julia checklister/run_list.jl website luisant/
Running the [.]png$ checklist for antipode.png
Is the file 300x300? y
Running the [.]png$ checklist for guide.png
Is the file 300x300? y
Running the [.]png$ checklist for remynifier.png
Is the file 300x300? y

So far this isn't so different than a checklist you may have known. The magic happens when I run the checklister again.

$ julia checklister/run_list.jl website luisant/
Running the [.]png$ checklist for antipode.png
[DONE] Is the file 300x300?
Running the [.]png$ checklist for guide.png
[DONE] Is the file 300x300?
Running the [.]png$ checklist for remynifier.png
[DONE] Is the file 300x300?

I don't need to answer those again. After all, the file isn't going to grow spontaneously.

I'm an absentminded academic, just ask my students. When I add a new project I will probably just throw in a new .png file into the website build process and forget about making it nice and small. My checklister will prevent me from doing that. Let's see what happens when I add in something that looks like a png file into my website directory and run the website checklist.

$ touch luisant/oh_look_new_file.png
$ julia checklister/run_list.jl website luisant/
Adding a new file: oh_look_new_file.png
Running the [.]png$ checklist for antipode.png
[DONE] Is the file 300x300?
Running the [.]png$ checklist for guide.png
[DONE] Is the file 300x300?
Running the [.]png$ checklist for remynifier.png
[DONE] Is the file 300x300?
Running the [.]png$ checklist for oh_look_new_file.png
Is the file 300x300? y

Only the important question got asked. The checklister knows about the other files and does not bother me about them. I get to stop worrying and not put in too much effort! My sweet sweet neuroses calmed and my laziness pleased! Yes!

I'm a fickle beast and I just changed my mind!

Inserting images is a one-time process, but what about things that may slightly change and require some level of QA after such a minor change? Let's say an article. A brand new article should give me several questions, to allow me to make sure that I have done all the QA steps to make it available for publication. If an article is unchanged from that clean state then I should not have to worry about it. In case of changes being made, I should be prompted to run less-extensive checks, such as a spellcheck. I should not have to worry about checking if there are any FIXME marks within, as I would not have introduced one of those in the course of a small change.

Let's take a look. I'm going to be checklisting articles. Let's do that in a separate checklist, through you could just register a new regular expression in the old one. (I'm trying to keep the visual clutter to a minimum for you guys.)

$ julia checklister/new_question.jl articles
Select matcher
1: ^blog[.]
1
New item text: Check for FIXME marks.
Always check? true/false: false
Only first check? true/false: true
$ julia checklister/new_question.jl articles
Select matcher
1: ^blog[.]
1
New item text: Spellcheck.
Always check? true/false: false
Only first check? true/false: false

Right. That was the setup. Now to run the thing. First run asks all questions.

$ julia checklister/run_list.jl articles luisant/
Adding a new file: blog.cr816
Adding a new file: blog.tracking-loops
Running the ^blog[.] checklist for blog.cr816
Check for FIXME marks. y
Spellcheck. y
Running the ^blog[.] checklist for blog.tracking-loops
Check for FIXME marks. y
Spellcheck. y

The second run shows that everything is done.

$ julia checklister/run_list.jl articles luisant/
Running the ^blog[.] checklist for blog.cr816
[DONE] Check for FIXME marks.
[UNCHANGED] Spellcheck.
Running the ^blog[.] checklist for blog.tracking-loops
[DONE] Check for FIXME marks.
[UNCHANGED] Spellcheck.

But should something change, as decided by modification date, the checklister will make sure that things don't get forgotten.

$ touch luisant/blog.cr816
$ julia checklister/run_list.jl articles luisant/
Running the ^blog[.] checklist for blog.cr816
[DONE] Check for FIXME marks.
Spellcheck. y
Running the ^blog[.] checklist for blog.tracking-loops
[DONE] Check for FIXME marks.
[UNCHANGED] Spellcheck.

The chance of typos due to small changes averted! It won't save the world, but it will let me sleep better.

Running "regular running" checklists.

Then there are the things that I need to worry about every time I run a process. The checklist is pretty much the whole process here. Every step of the checklist is something I need to run every single time so that my task completes successfully. Take updating the site. I want to have a rigorous sequence of steps so that I don't mess it up. I'm new to this, and a bit of an idiot, remember?

Currently all the checklisting runs from file matches, whereas the action of uploading a site does not really have a file associated to it. Thankfully, I do have a file that will be always present in each of my projects, so I can match from that one.

$ julia checklister/new_list.jl publish
$ julia checklister/new_matcher.jl publish
Enter Regex: ^site$
$ julia checklister/new_question.jl publish
Select matcher
1: ^site$
1
New item text: Publish to staging.
Always check? true/false: true
# Add a bunch more here!

Whenever I run the publication checklist, I will be asked those questions. It makes sense. This is my run list. Everything needs to be checked.

$ julia checklister/run_list.jl publish luisant/
Adding a new file: site
Running the ^site$ checklist for site
Publish to staging. y
Does it display nicely? y
Push it out to release! y
$ julia checklister/run_list.jl publish luisant/
Running the ^site$ checklist for site
Publish to staging. y
Does it display nicely? y
Push it out to release! y

This is the third kind done. A bit awkward, but still useful. I really need that "those questions don't have a file name pattern associated with them" matcher. One for the future.

Future directions

There's so much more here.

No one said that I have to answer the questions myself.

The checklist could run the spellcheck itself. If it comes clean, done. If it complains, it bounces it back to me to ask if everything is fine. Or the checklist could, instead of asking me, just go and find out the size of the images itself. Or run the regression tests. Using this would depend on how much you trust your checklister program or your our judgment.

You also could have more levels of checklists. Weekly. Daily. Per ten changes. Conditional checks, if a previous passes or fails.

Or maybe we could arrange so that a checklist is divided between team members? You could rotate the tasks, so that the members don't get fatigued of repeating the same check.

By keeping the overhead of having a checklist to a minimum, the checklists could grow much bigger than they currently are and still stay acceptable to the end users. The problem was never a large checklist. The problem was a checklist that does not automatically shrink when the jobs are done, or one that does not split the load between many people.

Closing thoughts

I like not being stressed quite so much.

The website is better. I'm happy. Yay!

The checklister is good enough for me, but I should polish this and release it some day. If I only had time.

I wonder if people will one day use checklisters like they use version control now. Maybe the two could be one?

Probably not. One could hope though.

Spare some coinage for a poor adjunct instructor who would like to spend more time writing and less time worrying about their wallet?

Donate

Past: The lesser known CPUs: CoolRISC 816