An API client in Go
In general, it is best to assume that the network is filled with malevolent entities that will send in packets designed to have the worst possible effect.
Suppose we want to write a Go program that produces a short summary of the current weather conditions at our location:
Clouds
We could use our tool to show the current weather conditions in our terminal prompt, or menu bar, for example. Where can we get that data from?
The OpenWeatherMap API isn’t a bad choice. We’ll need an API key, or token, which we can get by signing up to the OpenWeatherMap site and entering an email address. It doesn’t cost anything to use the API; the key just helps prevent the service being abused.
The first test
What can we write a test for? The API returns JSON data, so we’ll need to parse that JSON response into a Go value, which we can then print as a string.
So suppose there were some function ParseResponse
that turned the raw JSON data from OpenWeatherMap into a Go struct. Could we write a test for that?
Let’s use curl
to quickly download an example of the API’s JSON response (substitute your API key for XXX
in the URL):
curl "https://api.openweathermap.org/data/2.5/weather?q=London,UK&appid=XXX"
We’ll put this in a file in testdata
; that’s our test input. Here’s a test that tries to parse it:
func TestParseResponse_CorrectlyParsesJSONData(t *testing.T) {
.Parallel()
t, err := os.ReadFile("testdata/weather.json")
dataif err != nil {
.Fatal(err)
t}
:= weather.Conditions{
want : "Clouds",
Summary}
, err := weather.ParseResponse(data)
gotif err != nil {
.Fatal(err)
t}
if !cmp.Equal(want, got) {
.Error(cmp.Diff(want, got))
t}
}
If we create a weather
module, add the sample JSON data in testdata
, define the Conditions
struct, and provide a null implementation of ParseResponse
, we should have a failing test:
- Summary: "Clouds",
+ Summary: "",
That makes sense: we’re not actually parsing anything yet. How should we do that?
A temporary struct type
We’ll need some kind of struct to unmarshal the JSON data into. Let’s take a closer look at the test data. Here’s the part we’re interested in:
"weather": [
{
"id": 801,
"main": "Clouds",
"description": "few clouds",
"icon": "02d"
}
],
We’re going to need a Go struct definition matching the parts of this schema that we want to extract:
type OWMResponse struct {
[]struct {
Weather string
Main }
}
Now that we have the test, the JSON data, and the struct type required to decode it, we’re ready to go ahead and write ParseResponse
.
Implementing ParseResponse
Here’s a first attempt:
func ParseResponse(data []byte) (Conditions, error) {
var resp OWMResponse
:= json.Unmarshal(data, &resp)
err if err != nil {
return Conditions{}, fmt.Errorf("invalid API response %s: %w", data, err)
}
:= Conditions{
conditions : resp.Weather[0].Main,
Summary}
return conditions, nil
}
What could go wrong?
Not bad, but we can see from the fact that there are two return
statements that we need another test for “invalid input” behaviour. The simplest JSON document that won’t parse successfully into an OWMResponse
is an empty []byte
, so let’s write a test that tries to parse that:
func TestParseResponse_ReturnsErrorGivenEmptyData(t *testing.T) {
.Parallel()
t, err := weather.ParseResponse([]byte{})
_if err == nil {
.Fatal("want error parsing empty response, got nil")
t}
}
While we’re playing the “what could go wrong?” game, a closer look at this version of ParseResponse
reveals another potential issue. Can you spot it?
We know that referring to a slice element without a protective len
check can panic. So we need to write a test that will panic unless we check the slice:
func TestParseResponse_ReturnsErrorGivenInvalidJSON(t *testing.T) {
.Parallel()
t, err := os.ReadFile("testdata/weather_invalid.json")
dataif err != nil {
.Fatal(err)
t}
, err = weather.ParseResponse(data)
_if err == nil {
.Fatal("want error parsing invalid response, got nil")
t}
}
We’ll need to write some new test data for this: we want something that’s valid JSON, but whose Weather
array contains no elements. We can arrange that by copying the valid data and setting its weather
field to an empty array:
"weather": [],
To pass this test, we’ll need to add the requisite len
check, with a suitably helpful error message. Here’s the result:
func ParseResponse(data []byte) (Conditions, error) {
var resp OWMResponse
:= json.Unmarshal(data, &resp)
err if err != nil {
return Conditions{}, fmt.Errorf("invalid API response %q: %w", data, err)
}
if len(resp.Weather) < 1 {
return Conditions{}, fmt.Errorf("invalid API response %q: want at least one Weather element", data)
}
:= Conditions{
conditions : resp.Weather[0].Main,
Summary}
return conditions, nil
}
Formatting the request URL
Another chunk of behaviour that we could pull out into a function and unit-test is constructing the request URL:
:= fmt.Sprintf("%s/data/2.5/weather?q=London,UK&appid=%s", BaseURL, key) URL
We’ll need to be able to supply the location, too. Let’s say there’s some function FormatURL
that takes the base URL, location, and key, and returns a string containing the complete request URL.
This sounds like something we could write a test for. Let’s try:
func TestFormatURL_ReturnsCorrectURLForGivenInputs(t *testing.T) {
.Parallel()
t:= weather.BaseURL
baseURL := "Paris,FR"
location := "dummyAPIKey"
key := "https://api.openweathermap.org/data/2.5/weather?q=Paris,FR&appid=dummyAPIKey"
want := weather.FormatURL(baseURL, location, key)
got if !cmp.Equal(want, got) {
.Error(cmp.Diff(want, got))
t}
}
Having checked this test against the null implementation of FormatURL
, we can now fill in the real code:
func FormatURL(baseURL, location, key string) string {
return fmt.Sprintf("%s/data/2.5/weather?q=%s&appid=%s", baseURL, location, key)
}
A paperwork-reducing GetWeather
function
Now that we have these two components, we can start to see how we might connect them together to solve the problem. Something like this, for example:
:= weather.FormatURL(weather.BaseURL, location, key)
URL , err := weather.MakeAPIRequest(URL)
dataif err != nil {
.Fatal(err)
log}
, err := weather.ParseResponse(data) conditions
This is okay, but not great. It feels a bit fussy. Why get a URL from some function only to pass it back to some other function to actually make the request?
What we’d like is to call some function that only needs the location and the key:
, err := weather.GetWeather(location, key) conditions
Testing against a local HTTP server
Testing GetWeather
is a challenge, because we don’t want it to call the real OpenWeatherMap API. We need it to call some local HTTP server instead, that will respond to a GET
request with our test JSON data.
But we don’t currently have a way to pass the test server’s URL to GetWeather
. Instead, GetWeather
will presumably use FormatURL
to construct the URL itself, incorporating the OpenWeatherMap base URL.
How can we inject the test URL instead? We could make BaseURL
a global variable instead of a constant, and set it from the test, but that feels wrong, and we wouldn’t be able to parallelise our tests.
We’re saying, in fact, that up to now we’ve effectively been using some default OpenWeatherMap client, and now we’d like one that we can customise.
This is a familiar pattern:
:= weather.NewClient(key) c
Now that we have a client object, it makes sense that we should get the weather by calling some method on it. GetWeather
, perhaps:
, err := c.GetWeather(location) conditions
Writing the client constructor
What information does the client struct need to store? First, the API key, which will be passed in to the constructor. Next, for test purposes, we’ll want to be able to set both the base URL and an arbitrary HTTP client:
type Client struct {
string
APIKey string
BaseURL *http.Client
HTTPClient }
The constructor should set usable defaults for the URL and client fields, then:
func NewClient(apiKey string) *Client {
return &Client{
: apiKey,
APIKey: "https://api.openweathermap.org",
BaseURL: &http.Client{
HTTPClient: 10 * time.Second,
Timeout},
}
}
A “canned JSON” test handler
There’s one more thing we need for the GetWeather
test: an HTTP handler that returns the JSON data from our test file. Something like this will do the job:
func TestGetWeather_ReturnsExpectedConditions(t *testing.T) {
.Parallel()
t:= httptest.NewTLSServer(http.HandlerFunc(
ts func(w http.ResponseWriter, r *http.Request) {
.ServeFile(w, r, "testdata/weather.json")
http}))
defer ts.Close()
:= weather.NewClient("dummyAPIKey")
c .BaseURL = ts.URL
c.HTTPClient = ts.Client()
c:= weather.Conditions{
want : "Clouds",
Summary}
, err := c.GetWeather("Paris,FR")
gotif err != nil {
.Fatal(err)
t}
if !cmp.Equal(want, got) {
.Error(cmp.Diff(want, got))
t}
}
Implementing GetWeather
We don’t need to do any clever problem-solving here, just careful refactoring:
func (c *Client) GetWeather(location string) (Conditions, error) {
:= c.FormatURL(location)
URL , err := c.HTTPClient.Get(URL)
respif err != nil {
return Conditions{}, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return Conditions{}, fmt.Errorf("unexpected response status %q", resp.Status)
}
, err := io.ReadAll(resp.Body)
dataif err != nil {
return Conditions{}, err
}
, err := ParseResponse(data)
conditionsif err != nil {
return Conditions{}, fmt.Errorf("invalid API response %s: %w", data, err)
}
return conditions, nil
}
A convenience wrapper
Having a weather client object makes it easy for us to customise its behaviour in tests, but that’s not usually what users want. They would appreciate being able to call a convenience wrapper around an implicit default client, just like with http.Get
.
They’d like to be able to call a single function, such as weather.Get
, that constructs and uses the client for them, transparently. Let’s write that:
func Get(location, key string) (Conditions, error) {
:= NewClient(key)
c , err := c.GetWeather(location)
conditionsif err != nil {
return Conditions{}, err
}
return conditions, nil
}
Adding a Main
function
Finally, let’s write a Main
function that uses these components to implement the command-line tool:
func Main() {
if len(os.Args) < 2 {
.Println(Usage)
fmtreturn 0
}
:= os.Getenv("OPENWEATHERMAP_API_KEY")
key if key == "" {
.Fprintln(os.Stderr, "Please set the environment \
fmt variable OPENWEATHERMAP_API_KEY.")
return 1
}
:= os.Args[1]
location , err := Get(location, key)
conditionsif err != nil {
.Fprintln(os.Stderr, err)
fmtreturn 1
}
.Println(conditions)
fmtreturn 0
}
We can now use this weather.Main
function in our real main
package, like this:
package main
import (
"os"
"github.com/bitfield/weather"
)
func main() {
.Exit(weather.Main())
os}
The patterns and components that we’ve seen here are useful for creating clients for a wide range of other APIs, too. Pick your favourite API and write a Go client for it!