An API client in Go

An API client in Go

In general, it is best to assume that the network is filled with malevolent entities that will send in packets designed to have the worst possible effect.

RFC 1122

Suppose we want to write a Go program that produces a short summary of the current weather conditions at our location:

Clouds

We could use our tool to show the current weather conditions in our terminal prompt, or menu bar, for example. Where can we get that data from?

The OpenWeatherMap API isn’t a bad choice. We’ll need an API key, or token, which we can get by signing up to the OpenWeatherMap site and entering an email address. It doesn’t cost anything to use the API; the key just helps prevent the service being abused.

The first test

What can we write a test for? The API returns JSON data, so we’ll need to parse that JSON response into a Go value, which we can then print as a string.

So suppose there were some function ParseResponse that turned the raw JSON data from OpenWeatherMap into a Go struct. Could we write a test for that?

Let’s use curl to quickly download an example of the API’s JSON response (substitute your API key for XXX in the URL):

curl "https://api.openweathermap.org/data/2.5/weather?q=London,UK&appid=XXX"

We’ll put this in a file in testdata; that’s our test input. Here’s a test that tries to parse it:

func TestParseResponse_CorrectlyParsesJSONData(t *testing.T) {
    t.Parallel()
    data, err := os.ReadFile("testdata/weather.json")
    if err != nil {
        t.Fatal(err)
    }
    want := weather.Conditions{
        Summary: "Clouds",
    }
    got, err := weather.ParseResponse(data)
    if err != nil {
        t.Fatal(err)
    }
    if !cmp.Equal(want, got) {
        t.Error(cmp.Diff(want, got))
    }
}

(Listing weather/3)

If we create a weather module, add the sample JSON data in testdata, define the Conditions struct, and provide a null implementation of ParseResponse, we should have a failing test:

-   Summary: "Clouds",
+   Summary: "",

That makes sense: we’re not actually parsing anything yet. How should we do that?

A temporary struct type

We’ll need some kind of struct to unmarshal the JSON data into. Let’s take a closer look at the test data. Here’s the part we’re interested in:

"weather": [
  {
    "id": 801,
    "main": "Clouds",
    "description": "few clouds",
    "icon": "02d"
  }
],

We’re going to need a Go struct definition matching the parts of this schema that we want to extract:

type OWMResponse struct {
    Weather []struct {
        Main string
    }
}

(Listing weather/3)

Now that we have the test, the JSON data, and the struct type required to decode it, we’re ready to go ahead and write ParseResponse.

Implementing ParseResponse

Here’s a first attempt:

func ParseResponse(data []byte) (Conditions, error) {
    var resp OWMResponse
    err := json.Unmarshal(data, &resp)
    if err != nil {
        return Conditions{}, fmt.Errorf("invalid API response %s: %w", data, err)
    }
    conditions := Conditions{
        Summary: resp.Weather[0].Main,
    }
    return conditions, nil
}

What could go wrong?

Not bad, but we can see from the fact that there are two return statements that we need another test for “invalid input” behaviour. The simplest JSON document that won’t parse successfully into an OWMResponse is an empty []byte, so let’s write a test that tries to parse that:

func TestParseResponse_ReturnsErrorGivenEmptyData(t *testing.T) {
    t.Parallel()
    _, err := weather.ParseResponse([]byte{})
    if err == nil {
        t.Fatal("want error parsing empty response, got nil")
    }
}

(Listing weather/3)

While we’re playing the “what could go wrong?” game, a closer look at this version of ParseResponse reveals another potential issue. Can you spot it?

We know that referring to a slice element without a protective len check can panic. So we need to write a test that will panic unless we check the slice:

func TestParseResponse_ReturnsErrorGivenInvalidJSON(t *testing.T) {
    t.Parallel()
    data, err := os.ReadFile("testdata/weather_invalid.json")
    if err != nil {
        t.Fatal(err)
    }
    _, err = weather.ParseResponse(data)
    if err == nil {
        t.Fatal("want error parsing invalid response, got nil")
    }
}

(Listing weather/3)

We’ll need to write some new test data for this: we want something that’s valid JSON, but whose Weather array contains no elements. We can arrange that by copying the valid data and setting its weather field to an empty array:

"weather": [],

(Listing weather/3)

To pass this test, we’ll need to add the requisite len check, with a suitably helpful error message. Here’s the result:

func ParseResponse(data []byte) (Conditions, error) {
    var resp OWMResponse
    err := json.Unmarshal(data, &resp)
    if err != nil {
        return Conditions{}, fmt.Errorf("invalid API response %q: %w", data, err)
    }
    if len(resp.Weather) < 1 {
        return Conditions{}, fmt.Errorf("invalid API response %q: want at least one Weather element", data)
    }
    conditions := Conditions{
        Summary: resp.Weather[0].Main,
    }
    return conditions, nil
}

(Listing weather/3)

Formatting the request URL

Another chunk of behaviour that we could pull out into a function and unit-test is constructing the request URL:

URL := fmt.Sprintf("%s/data/2.5/weather?q=London,UK&appid=%s", BaseURL, key)

We’ll need to be able to supply the location, too. Let’s say there’s some function FormatURL that takes the base URL, location, and key, and returns a string containing the complete request URL.

This sounds like something we could write a test for. Let’s try:

func TestFormatURL_ReturnsCorrectURLForGivenInputs(t *testing.T) {
    t.Parallel()
    baseURL := weather.BaseURL
    location := "Paris,FR"
    key := "dummyAPIKey"
    want := "https://api.openweathermap.org/data/2.5/weather?q=Paris,FR&appid=dummyAPIKey"
    got := weather.FormatURL(baseURL, location, key)
    if !cmp.Equal(want, got) {
        t.Error(cmp.Diff(want, got))
    }
}

(Listing weather/3)

Having checked this test against the null implementation of FormatURL, we can now fill in the real code:

func FormatURL(baseURL, location, key string) string {
    return fmt.Sprintf("%s/data/2.5/weather?q=%s&appid=%s", baseURL, location, key)
}

(Listing weather/3)

A paperwork-reducing GetWeather function

Now that we have these two components, we can start to see how we might connect them together to solve the problem. Something like this, for example:

URL := weather.FormatURL(weather.BaseURL, location, key)
data, err := weather.MakeAPIRequest(URL)
if err != nil {
    log.Fatal(err)
}
conditions, err := weather.ParseResponse(data)

This is okay, but not great. It feels a bit fussy. Why get a URL from some function only to pass it back to some other function to actually make the request?

What we’d like is to call some function that only needs the location and the key:

conditions, err := weather.GetWeather(location, key)

Testing against a local HTTP server

Testing GetWeather is a challenge, because we don’t want it to call the real OpenWeatherMap API. We need it to call some local HTTP server instead, that will respond to a GET request with our test JSON data.

But we don’t currently have a way to pass the test server’s URL to GetWeather. Instead, GetWeather will presumably use FormatURL to construct the URL itself, incorporating the OpenWeatherMap base URL.

How can we inject the test URL instead? We could make BaseURL a global variable instead of a constant, and set it from the test, but that feels wrong, and we wouldn’t be able to parallelise our tests.

We’re saying, in fact, that up to now we’ve effectively been using some default OpenWeatherMap client, and now we’d like one that we can customise.

This is a familiar pattern:

c := weather.NewClient(key)

Now that we have a client object, it makes sense that we should get the weather by calling some method on it. GetWeather, perhaps:

conditions, err := c.GetWeather(location)

Writing the client constructor

What information does the client struct need to store? First, the API key, which will be passed in to the constructor. Next, for test purposes, we’ll want to be able to set both the base URL and an arbitrary HTTP client:

type Client struct {
    APIKey  string
    BaseURL string
    HTTPClient *http.Client
}

The constructor should set usable defaults for the URL and client fields, then:

func NewClient(apiKey string) *Client {
    return &Client{
        APIKey:  apiKey,
        BaseURL: "https://api.openweathermap.org",
        HTTPClient: &http.Client{
            Timeout: 10 * time.Second,
        },
    }
}

(Listing weather/4)

A “canned JSON” test handler

There’s one more thing we need for the GetWeather test: an HTTP handler that returns the JSON data from our test file. Something like this will do the job:

func TestGetWeather_ReturnsExpectedConditions(t *testing.T) {
    t.Parallel()
    ts := httptest.NewTLSServer(http.HandlerFunc(
        func(w http.ResponseWriter, r *http.Request) {
            http.ServeFile(w, r, "testdata/weather.json")
        }))
    defer ts.Close()
    c := weather.NewClient("dummyAPIKey")
    c.BaseURL = ts.URL
    c.HTTPClient = ts.Client()
    want := weather.Conditions{
        Summary: "Clouds",
    }
    got, err := c.GetWeather("Paris,FR")
    if err != nil {
        t.Fatal(err)
    }
    if !cmp.Equal(want, got) {
        t.Error(cmp.Diff(want, got))
    }
}

(Listing weather/4)

Implementing GetWeather

We don’t need to do any clever problem-solving here, just careful refactoring:

func (c *Client) GetWeather(location string) (Conditions, error) {
    URL := c.FormatURL(location)
    resp, err := c.HTTPClient.Get(URL)
    if err != nil {
        return Conditions{}, err
    }
    defer resp.Body.Close()
    if resp.StatusCode != http.StatusOK {
        return Conditions{}, fmt.Errorf("unexpected response status %q", resp.Status)
    }
    data, err := io.ReadAll(resp.Body)
    if err != nil {
        return Conditions{}, err
    }
    conditions, err := ParseResponse(data)
    if err != nil {
        return Conditions{}, fmt.Errorf("invalid API response %s: %w", data, err)
    }
    return conditions, nil
}

(Listing weather/4)

A convenience wrapper

Having a weather client object makes it easy for us to customise its behaviour in tests, but that’s not usually what users want. They would appreciate being able to call a convenience wrapper around an implicit default client, just like with http.Get.

They’d like to be able to call a single function, such as weather.Get, that constructs and uses the client for them, transparently. Let’s write that:

func Get(location, key string) (Conditions, error) {
    c := NewClient(key)
    conditions, err := c.GetWeather(location)
    if err != nil {
        return Conditions{}, err
    }
    return conditions, nil
}

(Listing weather/4)

Adding a Main function

Finally, let’s write a Main function that uses these components to implement the command-line tool:

func Main() {
    if len(os.Args) < 2 {
        fmt.Println(Usage)
        return 0
    }
    key := os.Getenv("OPENWEATHERMAP_API_KEY")
    if key == "" {
        fmt.Fprintln(os.Stderr, "Please set the environment \
            variable OPENWEATHERMAP_API_KEY.")
        return 1
    }
    location := os.Args[1]
    conditions, err := Get(location, key)
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        return 1
    }
    fmt.Println(conditions)
    return 0
}

(Listing weather/4)

We can now use this weather.Main function in our real main package, like this:

package main

import (
    "os"

    "github.com/bitfield/weather"
)

func main() {
    os.Exit(weather.Main())
}

(Listing weather/4)

The patterns and components that we’ve seen here are useful for creating clients for a wide range of other APIs, too. Pick your favourite API and write a Go client for it!

Tips for Time Lords

Tips for Time Lords

Scripting with Go

Scripting with Go

0