Write packages, not programs

From The Power of Go: Tools

How to climb a mountain from the top

All design decisions start and end with the package.

—Bill Kennedy, “Design Philosophy On Packaging”

This is the first of a two-part tutorial on designing Go packages, guided by tests:

1. Write packages, not programs
2. From packages to commands

This is fine

What’s wrong with this program?

package main

import (
    "fmt"
)

func main() {
    fmt.Println("Hello, world")
}

(Listing hello/1)

If you looked in vain, unable to spot the bug, I don’t blame you. The fact is, there’s nothing wrong with this program as such. It works, to the extent that it does what the author intended: print a message to the terminal and exit.

But there are some limitations on what we can do with the program. The most serious of these is that it’s not an importable package. Let’s talk about why this is a big deal.

The earliest computers were single-purpose machines: they could only execute the specific computation they were designed for. To compute something different meant physically re-wiring the machine.

It took a leap of insight to realise that a much more useful kind of computer would be one that could execute any computation, without being re-wired. That is to say, it would be programmable.

Packages are a force multiplier

A further significant advance was the idea that we don’t need to write every program from scratch every time. Instead, we could create re-usable “routines” For example, the code to calculate square roots only has to be written once, and we can then copy and use it in any program that needs to take a square root.

Nowadays, we would call such independent chunks of software components, or packages, and they’re fundamental to all modern programming.

Without packages we would always have to instruct the computer about every detail of what we want to do. The packages in the Go standard library, for example, would be very hard to manage without. Every time we wanted to write an HTTP server, for example, we’d have to implement the HTTP protocol ourselves, which is far from a trivial task (try it).

Packages, in other words, are an immensely powerful force multiplier. When we’re programming in a language like Go that has a rich ecosystem of importable packages, we never have to reinvent the wheel. If we can figure out how to break down our unsolved problems into a bunch of mini-problems that have already been solved by existing packages, we’re 90% done.

The universal Go library is huge

Go’s standard library, in particular, is an insanely great idea. The Go language is pretty small, in the sense that there aren’t many keywords and there’s not a vast amount of syntax to learn. And that’s great news for those of us learning it.

But since this little language ships with a big standard library, full of all sorts of useful and well-designed packages, we can use it to construct some really powerful programs right away. If we listen carefully enough, we can hear Go sending us a message: Packages are awesome. Let’s write more of them.

Fortunately, lots of Gophers have heard this message and acted on it, contributing their own packages to what we might call the universal library. These packages aren’t “built in” to Go in the way that the standard library packages are, but they’re nonetheless available to us. Providing their licence allows it, we can simply import them and use the functionality they provide in any way we want.

A quick search on GitHub reveals something close to half a million packages in this universal library (and there are more published in other places). If you can imagine it, in other words, there’s probably a Go package that provides it. So many, indeed, that simply finding the package you want can be a challenge. The pkg.go.dev site lets you search and browse the whole universal library, but a good place to look first is awesome-go, a carefully-curated list of a couple of thousand or so of the very best Go packages, by subject area.

This is one of the many reasons that Go is such a popular choice for developing software nowadays. In many cases, all we need to do to create a particular program is to figure out the right way to connect up the various packages that we need. We can then make our program a package that other people can use, and the process continues as a chain reaction.

Sharing our code benefits us all

No program is an island, in other words. But that’s what’s wrong with our “hello, world” program: it is isolated, breaking the chain of importability. The syntax rules of Go mean that all code has to be in some package, and it happens that ours is in package main. But there’s something special about the main package in Go: it can’t be imported.

That means no one else can benefit from our wonderful code. We’re taking from the universal package ecosystem, but not giving anything back. That’s just rude. If our program is worth writing, it’s worth sharing with the millions of other Gophers who also benefit from the universal library.

Good programmers, then, are always thinking in terms of writing importable packages, not mere dead-end programs. And since packages are components, this would still be a good design idea even if we can’t contribute them to the universal library for some reason.

So what do we do differently when we’re writing packages, not programs?

Writing packages, not programs

What Bill Kennedy has aptly called package-oriented design represents a fundamental shift of mindset:

All design decisions start and end with the package. The purpose of a package is to provide a solution to a specific problem domain. To be purposeful, packages must provide, not contain. The more focused each package’s purpose is, the more clear it should be what the package provides.

—Bill Kennedy, “Design Philosophy On Packaging”

In other words, we start with some problem that we need to solve. Instead of jumping straight to a program that solves the problem, we first of all design a well-focused package that solves the problem, and then we can use it in a program.

The biggest shift in our thinking, then, is from solving our very specific and parochial problem, to solving a general class of problems that includes ours. For example, instead of writing code to calculate the square root of 2, we write a package that calculates any square root, and then we apply it to the number 2.

Both approaches produce the square root of 2, but the package approach is much more valuable because it also solves the square root problem for all developers, for all time. We can then contribute our package back to the universal library, so that everybody else can benefit from our work in the same way that we benefit every day from theirs.

Command-line tools

We can write as many packages as we want, of course, but nothing will actually happen until we run some executable binary. That means there must be a main package, but we’ve already said that the substantive code in our program should be in some importable package, which means it can’t be in main.

It follows, then, that package main should do almost nothing except import our package and call some entrypoint function to start the real program. We don’t really want to write any non-trivial logic in the main package, since it can’t be (directly) tested. And whether it’s correct or not, it can’t be imported and used in other programs, so it’s a dead end as far as the open-source community is concerned.

Let’s see what would be left if we extracted all the substantive code out of main, then:

package main

import (
    "hello"
)

func main() {
    hello.Print()
}

(Listing hello/2)

This won’t work yet, because we haven’t written the hello package, but we can see how it would work. We import hello, so that we can use its machinery. To do that, we call some exported function hello.Print, which presumably does the actual printing of “Hello, world”.

Zen mountaineering

How can we call a function that doesn’t exist? Well, it’s an exercise in imagination. As we’re writing main, we can ask ourselves “What kind of function would we like to call here?” What name would make sense for it? Would it need to take any arguments? If so, what? Would it return any results? How many? What type? And so on.

In fact, this is a good way to design such a function—and, by extension, the whole public API of our package: by using it.

There’s a Zen saying that applies here:

If you want to climb a mountain, begin at the top.

In other words, if we want to design a package, we should begin by pretending it already exists, and writing code that uses it to solve our problem. When this code looks clear, simple, and readable, we probably have a nice design. The hard part is over: all we need to do now is actually implement that design.

We’ve done a little design work already on the hello package, even though we haven’t written a single line of code in that package. For example, we know there will be a Print function that takes no parameters, returns nothing, and whose behaviour is to print a message to the terminal.

Guided by tests

We also want a test for this behaviour. In fact, let’s try to build this package guided by tests. A good way to do this is often to write the test first, before implementing the behaviour it tests.

If that sounds strange, it shouldn’t. This is actually the way we test most things, including software engineers. We decide the problem for the coding challenge in advance, and then ask the candidates to write a program that solves it.

The other way round would be crazy: imagine asking people to submit random programs, then choosing a problem and hoping one of the programs happens to solve it. That probably wouldn’t be a helpful way of identifying good programmers.

Similarly, if you want to make good software components, the right way to do that is to decide in advance what you want the behaviour to be, express that requirement in code, and then write the component.

This not only avoids wasting time on solving the wrong problem, but it also helps us keep things focused, and avoid writing more code than we actually need. When the test starts passing, you know you’ve got it right.

Building a hello package

Let’s try this idea with our hello-printing example. We’ll write a test and then see if we can come up with the right code to pass it.

All Go tests need to be in a file whose name ends with _test.go, so we’ll create a new file named hello_test.go, and start there.

Here’s a first attempt:

package hello_test

import (
    "hello"
    "testing"
)

func TestPrintPrintsHelloMessageToTerminal(t *testing.T) {
    t.Parallel()
    hello.Print()
}

Let’s break this down, line by line. Every Go file must begin with a package clause defining what package its code belongs to. Since this code will be about testing the hello package, let’s put it in package hello_test.

We need to import the standard library testing package for tests, and we will also need our hello package.

The naming of tests

The test itself has a name, which can convey useful information if we choose it wisely. It’s a good idea to name each test function after the behaviour it tests. You don’t need to use words like Should, or Must; these are implicit. Just say what the function does when it’s working correctly. In this case, it prints a hello message to the terminal.

To help us think about what our test names are saying about the behaviour of the system, we can use the gotestdox tool. It simply rewrites the test names as space-separated words that we can read as a sentence:

 ✔ Print prints hello message to terminal

As you probably know, every Go test function takes a single parameter, conventionally named t, which is a pointer to a testing.T struct. This t value contains the state of the test during its execution, and we use its methods to control the outcome of the test (for example, calling t.Error to fail the test).

The structure of tests

If you’re not already familiar with writing tests in Go, I recommend you read For the Love of Go, which will give you some helpful background on what follows. (If you are pretty familiar with testing in Go, try The Power of Go: Tests for a more in-depth treatment.)

The call to t.Parallel() signals that the test should be run concurrently with other tests, and is a standard prelude to any test. Now here comes the substantive part, where we actually call the function under test:

hello.Print()

It doesn’t seem, from the way we’ve used it in our modified main package, that the Print function needs to take any parameters, and at the moment there don’t seem to be any useful results it could return.

And that’s the end of the test. But there’s a problem: when would this test fail?

I’ll let you think about this a bit, and we’ll talk about the solution in Part 2.