From packages to commands

From The Power of Go: Tools

Happiness is a one-line main function

This is the second part of a two-part tutorial on designing Go packages, guided by tests:

1. Write packages, not programs
2. From packages to commands

This test is fine?

In Part 1, we talked about what it means to develop software as importable packages, not just one-off programs, and we used the example of printing “Hello, world”. Pretty simple, right? But it presents us with some interesting problems when we try to package this behaviour as a reusable software component.

Right from the start, we’re thinking about our package as a contribution to the universal library of Go software, and that starts with testing. So here’s our first attempt at a test for the hello.Print function:

package hello_test

import (
    "hello"
    "testing"
)

func TestPrintPrintsHelloMessageToTerminal(t *testing.T) {
    t.Parallel()
    hello.Print()
}

This looks reasonable, and it does call our function. But there’s a problem: how could this test fail?

Tests are bug detectors

Go tests pass by default, so unless we take some specific action to make it fail (we’ll see how to do that in a moment), this test will always pass. That might sound great, at first: we like passing tests!

But how useful is a test that can never fail? What is the point of a test, actually? Why do we write them? Partly to help us design the API of the system under test, as we’ve seen, but the most important role of any test is as a bug detector.

To see how that works, let’s think about potential bugs that could be in our hello.Print function. There are many, of course: it could print the wrong message, print the right message in the wrong language, print nothing but delete all your files, and so on.

These would all be important bugs, and we’d like to be able to detect and fix them, but let’s strip it right back to basics. The most fundamental bug that could be in Print is that it doesn’t do anything at all.

For example, suppose the implementation looked like this:

package hello

func Print() {}

The question we need to ask ourselves here is “would our test detect this bug?” In other words, will the test fail when Print does nothing? Well, we already know the answer to that. The test can’t fail at all, so it can’t detect any bugs, including this one.

So when should a test fail?

Most tests have one or more ways of failing under specific circumstances. In other words, we have some code in the test like this:

if CONDITION {
    t.Errorf("test failed because %s", reasons)
}

So what would CONDITION be in our case? It’s not easy to work out at first. Let’s remind ourselves what the desired behaviour of Print is. Its job is to print a hello message to the terminal. Fine, but what does that mean specifically?

In the original version of the program, we called fmt.Println to do this job. So how does that work? Let’s take a look at the source code.

In most editors, such as Visual Studio Code, you can place the cursor on some function call like this, right-click, and select Go to definition to see how it’s implemented (or you can just hold down the Cmd key and click on the function name).

This is helpful for navigating our own projects, but it works with imported packages too, even packages like fmt that are part of the standard library. This code spelunking is fun and interesting, and I recommend you spend some time exploring the standard library, or even packages from the universal library that interest you.

Where does fmt.Println print to?

Here’s what we find when we spelunk into the definition of fmt.Println:

func Println(a ...any) (n int, err error) {
    return Fprintln(os.Stdout, a...)
}

It seems that there’s a more general function, fmt.Fprintln. Its first argument is, we find, an io.Writer, representing the destination to “print” to, and the other arguments are the data to be printed.

Where does the text go when we call fmt.Println, then? The answer is os.Stdout, which is a file handle representing the standard output. If our program is being run from a terminal, then, its standard output will be that terminal, which explains why we see the “Hello, world” message there.

In the real program, that is indeed where we want the message to go. But the problem with testing our Print function is that there’s no way to “inspect” what’s been printed to os.Stdout. Anything we print to it can be seen by the user, but not the program. What could we do?

Well, perhaps the answer already occurred to you. If we can’t test what got printed to os.Stdout, then don’t print there: print somewhere else! Since Fprintln accepts any object that implements io.Writer, could we construct our own?

Meet bytes.Buffer, an off-the-shelf io.Writer

Like all good interfaces, io.Writer is easy to implement, because it’s small. All we need is something with a Write method.

For example, we could create some struct type whose Write method stores the supplied text inside the struct, and we could add a String method so that we can retrieve that text and inspect it in the test.

That’s not necessary, though, because such a type already exists in the standard library: bytes.Buffer. It’s an all-purpose io.Writer that remembers what we write to it, so it’s ideal for testing. Because a buffer is such a helpful friend, we often affectionately name it just buf for short.

Let’s rewrite our test using a buffer as the print destination, then:

func TestPrintTo_PrintsHelloMessageToGivenWriter(t *testing.T) {
    t.Parallel()
    buf := new(bytes.Buffer)
    hello.PrintTo(buf)
    want := "Hello, world\n"
    got := buf.String()
    if want != got {
        t.Errorf("want %q, got %q", want, got)
    }
}

(Listing hello/3)

New behaviour, new name

The first thing to note here is that the test name has changed. That makes sense, because we’ve changed our definition of the way the program is supposed to behave. gotestdox reports:

 ✔ PrintTo prints hello message to given writer

We begin with the standard t.Parallel() prelude, and then, since we’ll need a bytes.Buffer as our printing destination, we create one and name it buf.

Now comes the call to the function under test. Since the behaviour of the function has changed, its name should change too. It now takes an io.Writer argument to tell it where to print its message to, so let’s rename it PrintTo. And that reads quite naturally in the code:

hello.PrintTo(buf) // prints hello message to buf

Indeed, this is probably the easiest way to find a good name for things: write the code that uses it, and see what makes sense in context. Then use that name. Again, this usually works better than deciding in advance what the name should be, and then trying to jam it awkwardly into code where it doesn’t fit.

Unlike the first version of this test, we can now write down a condition under which the test should fail. First, we know what message we want to see printed to buf; it’s “Hello, world”. Let’s name that variable want.

What we actually got, though, is the result of calling buf.String(), which returns all the text written to buf since it was created. We’ll name this value got.

And now we know the failure condition: if got is not equal to want, then we should fail. Since that can now happen, we have a useful test at last.

Implementing hello, guided by tests

So, how close are we to having this test pass? Well, that would obviously require PrintTo to exist, and it doesn’t yet, even though we’ve done a lot of useful thinking about it. And we’re still not quite ready to write it, because there’s one thing we need first. We need to see our test fail.

If you think that sounds weird, I don’t blame you. A failing test is usually bad news—and it would be, if we were under the impression that we’d successfully implemented the PrintTo function.

But we know we haven’t, so if we can get the test to run at all, it should fail, shouldn’t it? The alternative would be that it passes, even though the PrintTo function does nothing whatsoever. That would be weird.

We agree, I hope, that if PrintTo does nothing, then that’s a bug. Would we detect that bug with this test? Yes, I think we would. Running this test against an empty PrintTo function should fail. Let’s see why.

If PrintTo doesn’t write anything to the buffer, then when we call buf.String() we’ll get the empty string. That won’t be equal to want, which is the string “Hello, world”.

That mismatch should fail the test, and it should also give us some confidence in our bug detector. No doubt there could be other bugs in PrintTo that we won’t detect at the moment. That’s nearly always true. But we feel at least we should detect this one, so let’s find out.

Creating a module

We can’t actually run the test just yet, though. At the moment, running go test gives us a compile error:

go: cannot find main module

This makes sense, since we haven’t yet created a go.mod file to identify our module. A module in Go is a collection of (usually related) packages. While a module can contain any number of packages, it’s a good idea to keep things simple and put each of our packages in its own module.

We’ll do that with the hello package, so we’ll name its containing module hello too. To do that, we need a little bit of paperwork.

Each Go module needs to have a special file named go.mod that identifies it by name. It’s just a text file, so we could create it by hand, but an easier way is to use the go tool itself:

go mod init github.com/bitfield/hello

go: creating new go.mod: module github.com/bitfield/hello

One folder, one package

But we still have a problem when running go test, because our old main.go file from listing hello/1 is still kicking around in this folder:

found packages hello (hello_test.go) and main (main.go)

The rule in Go is that each distinct package must occupy a separate folder. That is to say, you can’t have a file that declares package hello in the same folder as another file that declares package main, for example. And right now that’s what we have.

We still need a main package somewhere, because we’re going to build an executable binary. But this folder ain’t big enough for both main and the hello package to co-exist: one of them must move to a subfolder.

It doesn’t really matter which one we move, but it’s worth asking: which of these two packages is the more important? hello contains all the substantive code in our program, and it’s the package that other users will be importing. By contrast, main does almost nothing at all. It’s a very thin shim around the hello package which is required only to make it executable, and no one else will ever want to refer to the main package except us.

It seems logical to me that the more important package should be at the top level, that is, the root folder of the repository. Subordinate packages should go in subordinate folders. And since hello is clearly the boss, it should occupy the penthouse. Let’s move main.go to a subfolder, then:

mkdir -p cmd/hello mv main.go cmd/hello

It doesn’t matter what we call the subfolder, but there’s a loose convention among Gophers that code implementing “commands” (that is, executable binaries) is placed under a folder named cmd.

A null implementation

Let’s try go test again:

go build hello: no non-test Go files

That’s absolutely correct. In order to run tests, there must be some package to build, and there isn’t. So let’s create a hello.go file that declares package hello. What else shall we put in it?

Well, we need the PrintTo function to be at least defined, or our test won’t compile. On the other hand, we don’t want it to actually do anything yet, because we want to verify that our test can tell when it doesn’t. So let’s write a null implementation: just the same empty function we saw before.

package hello

func PrintTo(w io.Writer) {}

For it to compile, the function must take a parameter, and we know its type should be io.Writer. Nothing else is actually required, since Go is quite happy for you to write empty functions that do nothing. But we think the test should fail, so let’s see what happens:

--- FAIL: TestPrintTo_PrintsHelloMessageToGivenWriter (0.00s)
    hello_test.go:16: want "Hello, world", got ""

Nice! That’s exactly what we hoped for, even though it looks like something bad happened. Don’t think of it as a failing test: think of it instead as a succeeding bug detector. We know there is a bug, so if the test passed at this stage, we would have to conclude that our bug-detecting machinery is faulty.

The real implementation

The test is doing one half of its job: detecting when the function doesn’t work. The other half of its job is to detect when the function does work. So does it? Let’s write the real implementation of PrintTo and find out:

func PrintTo(w io.Writer) {
    fmt.Fprintln(w, "Hello, world")
}

Here’s the result:

go test

PASS

Ideal. We now have an importable, testable package that makes our “print hello to some io.Writer” behaviour available to other users, provided that we remember to publish it with an appropriate software licence.

Refactoring to use our new package

While we wait for the world to rush to our door to take advantage of our generous contribution to the Go universal library, let’s use the new package ourselves to rebuild our hello command.

In fact, we only need to make minor changes to our main package from listing hello/2. Specifically, we now need to pass in the destination for PrintTo to print to:

package main

import (
    "os"

    "github.com/bitfield/hello"
)

func main() {
    hello.PrintTo(os.Stdout)
}

(Listing hello/3)

Let’s run it and see if that works:

go run cmd/hello/main.go

Hello, world

This is great. We’re back precisely where we started. From one point of view, we’ve added no value at all, since the program’s behaviour is exactly the same. But from another point of view, we’ve added all the value in the world, since we created an importable package.

Someone else who wants this behaviour can now add it to their own program as a pluggable component. And because we’ve made it flexible about where it prints to, that program needn’t even be a command-line tool. It could be, for example, a web server:

func main() {
    fmt.Println("Listening on http://localhost:9001")
    http.ListenAndServe(":9001", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        hello.PrintTo(w)
    }))
}

(Listing hello/3)

Going further

Here’s a mini-project for you to practice building an importable package, guided by tests, in the way that we’ve talked about.

• Instead of a program that simply prints “Hello, world”, try writing a program that asks the user for their name, then prints “Hello, [NAME]”. Put all the substantive behaviour of the program in an importable package that’s used by main.

  You’ll find one possible solution in listing greet/1.