Don't fear the pointer

From For the Love of Go

You will understand pointers in Go after reading this, or your money back

Go pointers aren't as scary as they might sound, especially if you're new to programming or don't have a computer sciencey background. In fact, they're so extremely straightforward that everyone I explain them to says "But that isn't complicated at all!"

Exactly. And I hope you'll be saying the same by the end of this piece, which aims to explain pointers in Go in simple terms: what they are, why we need them, and what to watch out for when using them.

Why do we need pointers?

Let's create a new package so that we can play around with some ideas and see how they work. Create a new folder, wherever you keep your Go code, and name it pointerplay.

Inside this folder, run go mod init pointerplay, to tell Go you're creating a new module called pointerplay, and create a new empty file called pointerplay_test.go.

We're going to write a test to experiment with function calls and values. Suppose we write a test for a function in the pointerplay package called Double. It should take one parameter, an integer, and multiply it by 2.

Here's something that should do the job:

package pointerplay_test

import (
    "pointerplay"
    "testing"
)

func TestDouble(t *testing.T) {
    t.Parallel()
    var x int = 12
    want := 24
    pointerplay.Double(x)
    if want != x {
        t.Errorf("want %d, got %d", want, x)
    }
}

Nothing fancy here: all Double needs to do is multiply its input by 2. Here's my first attempt at an implementation, in the pointerplay.go file:

package pointerplay

func Double(input int) {
    input *= 2
}

You may know that input *= 2 is a handy short form for input = input * 2. Its effect is to double the value of input.

So does the test pass now? Well, let's run go test and see:

--- FAIL: TestDouble (0.00s)
        pointerplay_test.go:14: want 24, got 12

Oh no!

The mystery of the failing test

What's going on? Is the Double function broken?

Actually, Double is doing exactly what we asked it to do. It receives a parameter we call input, and it multiplies that value by 2:

input *= 2

So why is it that, in the test, when we set x to 12, and call Double(x), the value of x remains 12? Shouldn't it now be 24?

var x int = 12
want := 24
pointerplay.Double(x)

Parameters are passed by value

The answer to this puzzle lies in what happens when we pass a value as a parameter to a Go function. It's tempting to think, if we have some variable x, and we call Double(x), that the input parameter inside Double is simply the variable x. But that's not the case.

In fact, the value of input will be the same as the value of x, but they're two independent variables. The Double function can modify its local input variable as much as it likes, but that will have no effect on the x variable back in the test function—as we've just proved.

The technical name for this way of passing function parameters is pass by value, because Double receives only the value of x, not the original x variable itself. This explains why x wasn't modified in the test.

Creating a pointer

Is there any way, then, to write a function that can modify a variable we pass to it? For example, we'd like to write a version of Double that will actually have an effect on x when we call Double(x). Instead of just taking a copy of the value of x at the moment of the function call, we want to pass Double some kind of reference to x. That way, Double could modify the original x directly.

Go lets us do exactly that. We can create what's called a pointer to x, using this syntax:

pointerplay.Double(&x)

You can think of the & (pronounced 'ampersand') here as the sharing operator; it lets you share a variable with the function you're passing it to, so that the function can modify it.

Declaring pointer parameters

There's still something missing, though, because our modified test doesn't compile:

cannot use &x (type *int) as type int in argument to pointerplay.Double

The compiler is saying that Double takes a parameter of type int, but what we tried to pass it was a value of type *int (pronounced "pointer to int").

These are two distinct types, and we can't mix them. If we want the function to be able to take a *int, we'll need to update its signature accordingly:

func Double(input *int) {

It's worth adding that the type here is not just "pointer", but specifically "pointer to int". For example, if we tried to pass a *float64 here, that wouldn't work. A *float64 and a *int are both pointers, but since they're pointers to different types, they are also different from each other.

What can we do with pointers?

We're still not quite done, because even with our updated function signature, the compiler isn't happy with this line:

input *= 2

It complains:

invalid operation: input *= 2 (mismatched types *int and int)

Another type mismatch. It's saying "you tried to multiply two different kinds of thing". The numeric constant literal 2 is interpreted as an int, while input is a pointer.

The * operator

Instead of trying to do math with the pointer itself, we need the value that the pointer points to. Because a pointer is a reference to some variable, the fancy name for this is dereferencing the pointer.

To get the value pointed to by input, we write *input ("star-input"):

*input *= 2

Nil pointers and panics

You know that every data type in Go has some default value. If you declare a variable of type int, for example, it automatically has the value 0 unless you assign some other value to it.

So what's the default value of a pointer type? If you declare a variable of type *int, what value does it have?

The answer is the special value nil, which you've encountered many times already in connection with error values. Just as a nil error value means "no error", a nil pointer value means "doesn't point to anything".

It makes no sense to dereference a nil pointer, then, and if this situation arises while your program is running, Go will stop execution and give you a message like:

panic: runtime error: invalid memory address or nil pointer dereference

This shouldn't happen under normal circumstances, so "panic" in this context means something like "unrecoverable internal program error".

Pointer methods

If functions can take pointers as parameters, then can the receiver of a method also be a pointer? Yes, it can. Such a method is called a pointer method, and it's useful because you can write methods that modify the variable they're called on.

Could we modify Double, using our new knowledge about pointers, to turn it into a method? Let's find out. Here's what the relevant part of our test looks like right now:

pointerplay.Double(&x)

How do we turn this into a method call on &x? We might try something like this:

&x.Double()

But that doesn't quite work:

x.Double undefined (type int has no field or method Double)

The compiler misunderstood what we wanted. What we wanted was to create a pointer to x, using the sharing operator, and then to call the Double method on that pointer. But what our syntax actually said was to create a pointer to the value returned by x.Double()!

There's some ambiguity here about which of the two operators should be applied first: the method call, or the sharing operator? We can clarify this using parentheses:

(&x).Double()

While this satisfies the compiler, it's rather cumbersome, and we'd prefer not to smash together the two operations of creating a pointer and calling a method into the same statement. Instead, let's create the pointer first, then call the method:

p := &x
p.Double()

Much clearer. The compiler will now prompt us to complete the refactoring by changing Double from a function to a method:

p.Double undefined (type *int has no field or method Double)

Creating custom types

How can we update the definition of Double to make the test pass?

A completely reasonable guess at this might be:

func (input *int) Double() {

But straight away we run into a problem:

cannot define new methods on non-local type int

We're not allowed to add methods on a type we didn't define. That's easy to deal with, though. We can define a new type MyInt:

type MyInt int

func (input *MyInt) Double() {
    *input *= 2
}

This solves the method definition problem, but we also need to update the test to use values of MyInt rather than plain old int:

func TestDouble(t *testing.T) {
    t.Parallel()
    x := pointerplay.MyInt(12)
    want := pointerplay.MyInt(24)
    p := &x
    p.Double()
    if want != x {
        t.Errorf("want %d, got %d", want, x)
    }
}

And we're there!

PASS

Pointers vs values

Now that you understand when and why we use pointers in Go, you might still be wondering when to write a value method (one that takes a value) and when to write a pointer method (one that takes a pointer to a value).

There's a very simple way to decide whether to use a value or a pointer receiver. Ask yourself:

Does this method need to modify the receiver?

If the answer is yes, then it should take a pointer. You can see why, can't you? If it took a value, then it could modify that value as much as it liked, but the change wouldn't affect the original variable (like our first version of the Double function).

On the other hand, if the method doesn't need to modify the receiver, it doesn't need to take a pointer (and by taking a value, it signals that fact to anyone reading the code, which is useful).

If we're writing some method that modifies its receiver, but we don't take a pointer, then any changes to it we might make in the method don't persist. This can lead to subtle bugs which are hard to detect without careful testing. (I write this bug about once a day, so if the same thing happens to you, don't feel too bad.)

In fact, the staticcheck linter (which Visual Studio Code and some other Go editors run for you automatically) will warn you about this problem:

ineffective assignment to variable input (SA4005)

Takeaways

• When you pass a variable to a function, the function actually receives a copy of that variable's value.

• So if the function modifies the value, that change won't be reflected in the original variable.

• When you want the function to modify the original variable, you can create a pointer to the variable, and pass that instead.

• A pointer gives permission to modify the thing it points to.

• The & operator creates a pointer, so &x gives a pointer to x.

• A function that takes a pointer must declare the appropriate parameter type: *int for "pointer to int", for example.

• Just as ordinary types are distinct and non-interchangeable, so are pointers to those types: for example, *int and *float64 are different types.

• The only thing we're allowed to do with a pointer value is dereference it using the * operator to get the value it points to, so *p gives the value of whatever variable p points to.

• The default (zero) value of any pointer type is nil, meaning "doesn't point to anything".

• Trying to dereference a pointer that happens to be nil will cause the program to panic and terminate with an error message.

• Just as functions can take pointer parameters, so methods can take pointer receivers, and indeed they must do so if they want to modify the receiver.

Gopher image courtesy of the wonderful MariaLetta .