An API client in Go
In general, it is best to assume that the network is filled with malevolent entities that will send in packets designed to have the worst possible effect.
Suppose we want to write a Go program that produces a short summary of the current weather conditions at our location:
CloudsWe could use our tool to show the current weather conditions in our terminal prompt, or menu bar, for example. Where can we get that data from?
The OpenWeatherMap API isn’t a bad choice. We’ll need an API key, or token, which we can get by signing up to the OpenWeatherMap site and entering an email address. It doesn’t cost anything to use the API; the key just helps prevent the service being abused.
The first test
What can we write a test for? The API returns JSON data, so we’ll need to parse that JSON response into a Go value, which we can then print as a string.
So suppose there were some function ParseResponse that turned the raw JSON data from OpenWeatherMap into a Go struct. Could we write a test for that?
Let’s use curl to quickly download an example of the API’s JSON response (substitute your API key for XXX in the URL):
curl "https://api.openweathermap.org/data/2.5/weather?q=London,UK&appid=XXX"We’ll put this in a file in testdata; that’s our test input. Here’s a test that tries to parse it:
func TestParseResponse_CorrectlyParsesJSONData(t *testing.T) {
t.Parallel()
data, err := os.ReadFile("testdata/weather.json")
if err != nil {
t.Fatal(err)
}
want := weather.Conditions{
Summary: "Clouds",
}
got, err := weather.ParseResponse(data)
if err != nil {
t.Fatal(err)
}
if !cmp.Equal(want, got) {
t.Error(cmp.Diff(want, got))
}
}If we create a weather module, add the sample JSON data in testdata, define the Conditions struct, and provide a null implementation of ParseResponse, we should have a failing test:
- Summary: "Clouds",
+ Summary: "",That makes sense: we’re not actually parsing anything yet. How should we do that?
A temporary struct type
We’ll need some kind of struct to unmarshal the JSON data into. Let’s take a closer look at the test data. Here’s the part we’re interested in:
"weather": [
{
"id": 801,
"main": "Clouds",
"description": "few clouds",
"icon": "02d"
}
],We’re going to need a Go struct definition matching the parts of this schema that we want to extract:
type OWMResponse struct {
Weather []struct {
Main string
}
}Now that we have the test, the JSON data, and the struct type required to decode it, we’re ready to go ahead and write ParseResponse.
Implementing ParseResponse
Here’s a first attempt:
func ParseResponse(data []byte) (Conditions, error) {
var resp OWMResponse
err := json.Unmarshal(data, &resp)
if err != nil {
return Conditions{}, fmt.Errorf("invalid API response %s: %w", data, err)
}
conditions := Conditions{
Summary: resp.Weather[0].Main,
}
return conditions, nil
}What could go wrong?
Not bad, but we can see from the fact that there are two return statements that we need another test for “invalid input” behaviour. The simplest JSON document that won’t parse successfully into an OWMResponse is an empty []byte, so let’s write a test that tries to parse that:
func TestParseResponse_ReturnsErrorGivenEmptyData(t *testing.T) {
t.Parallel()
_, err := weather.ParseResponse([]byte{})
if err == nil {
t.Fatal("want error parsing empty response, got nil")
}
}While we’re playing the “what could go wrong?” game, a closer look at this version of ParseResponse reveals another potential issue. Can you spot it?
We know that referring to a slice element without a protective len check can panic. So we need to write a test that will panic unless we check the slice:
func TestParseResponse_ReturnsErrorGivenInvalidJSON(t *testing.T) {
t.Parallel()
data, err := os.ReadFile("testdata/weather_invalid.json")
if err != nil {
t.Fatal(err)
}
_, err = weather.ParseResponse(data)
if err == nil {
t.Fatal("want error parsing invalid response, got nil")
}
}We’ll need to write some new test data for this: we want something that’s valid JSON, but whose Weather array contains no elements. We can arrange that by copying the valid data and setting its weather field to an empty array:
"weather": [],To pass this test, we’ll need to add the requisite len check, with a suitably helpful error message. Here’s the result:
func ParseResponse(data []byte) (Conditions, error) {
var resp OWMResponse
err := json.Unmarshal(data, &resp)
if err != nil {
return Conditions{}, fmt.Errorf("invalid API response %q: %w", data, err)
}
if len(resp.Weather) < 1 {
return Conditions{}, fmt.Errorf("invalid API response %q: want at least one Weather element", data)
}
conditions := Conditions{
Summary: resp.Weather[0].Main,
}
return conditions, nil
}Formatting the request URL
Another chunk of behaviour that we could pull out into a function and unit-test is constructing the request URL:
URL := fmt.Sprintf("%s/data/2.5/weather?q=London,UK&appid=%s", BaseURL, key)We’ll need to be able to supply the location, too. Let’s say there’s some function FormatURL that takes the base URL, location, and key, and returns a string containing the complete request URL.
This sounds like something we could write a test for. Let’s try:
func TestFormatURL_ReturnsCorrectURLForGivenInputs(t *testing.T) {
t.Parallel()
baseURL := weather.BaseURL
location := "Paris,FR"
key := "dummyAPIKey"
want := "https://api.openweathermap.org/data/2.5/weather?q=Paris,FR&appid=dummyAPIKey"
got := weather.FormatURL(baseURL, location, key)
if !cmp.Equal(want, got) {
t.Error(cmp.Diff(want, got))
}
}Having checked this test against the null implementation of FormatURL, we can now fill in the real code:
func FormatURL(baseURL, location, key string) string {
return fmt.Sprintf("%s/data/2.5/weather?q=%s&appid=%s", baseURL, location, key)
}A paperwork-reducing GetWeather function
Now that we have these two components, we can start to see how we might connect them together to solve the problem. Something like this, for example:
URL := weather.FormatURL(weather.BaseURL, location, key)
data, err := weather.MakeAPIRequest(URL)
if err != nil {
log.Fatal(err)
}
conditions, err := weather.ParseResponse(data)This is okay, but not great. It feels a bit fussy. Why get a URL from some function only to pass it back to some other function to actually make the request?
What we’d like is to call some function that only needs the location and the key:
conditions, err := weather.GetWeather(location, key)Testing against a local HTTP server
Testing GetWeather is a challenge, because we don’t want it to call the real OpenWeatherMap API. We need it to call some local HTTP server instead, that will respond to a GET request with our test JSON data.
But we don’t currently have a way to pass the test server’s URL to GetWeather. Instead, GetWeather will presumably use FormatURL to construct the URL itself, incorporating the OpenWeatherMap base URL.
How can we inject the test URL instead? We could make BaseURL a global variable instead of a constant, and set it from the test, but that feels wrong, and we wouldn’t be able to parallelise our tests.
We’re saying, in fact, that up to now we’ve effectively been using some default OpenWeatherMap client, and now we’d like one that we can customise.
This is a familiar pattern:
c := weather.NewClient(key)Now that we have a client object, it makes sense that we should get the weather by calling some method on it. GetWeather, perhaps:
conditions, err := c.GetWeather(location)Writing the client constructor
What information does the client struct need to store? First, the API key, which will be passed in to the constructor. Next, for test purposes, we’ll want to be able to set both the base URL and an arbitrary HTTP client:
type Client struct {
APIKey string
BaseURL string
HTTPClient *http.Client
}The constructor should set usable defaults for the URL and client fields, then:
func NewClient(apiKey string) *Client {
return &Client{
APIKey: apiKey,
BaseURL: "https://api.openweathermap.org",
HTTPClient: &http.Client{
Timeout: 10 * time.Second,
},
}
}A “canned JSON” test handler
There’s one more thing we need for the GetWeather test: an HTTP handler that returns the JSON data from our test file. Something like this will do the job:
func TestGetWeather_ReturnsExpectedConditions(t *testing.T) {
t.Parallel()
ts := httptest.NewTLSServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
http.ServeFile(w, r, "testdata/weather.json")
}))
defer ts.Close()
c := weather.NewClient("dummyAPIKey")
c.BaseURL = ts.URL
c.HTTPClient = ts.Client()
want := weather.Conditions{
Summary: "Clouds",
}
got, err := c.GetWeather("Paris,FR")
if err != nil {
t.Fatal(err)
}
if !cmp.Equal(want, got) {
t.Error(cmp.Diff(want, got))
}
}Implementing GetWeather
We don’t need to do any clever problem-solving here, just careful refactoring:
func (c *Client) GetWeather(location string) (Conditions, error) {
URL := c.FormatURL(location)
resp, err := c.HTTPClient.Get(URL)
if err != nil {
return Conditions{}, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return Conditions{}, fmt.Errorf("unexpected response status %q", resp.Status)
}
data, err := io.ReadAll(resp.Body)
if err != nil {
return Conditions{}, err
}
conditions, err := ParseResponse(data)
if err != nil {
return Conditions{}, fmt.Errorf("invalid API response %s: %w", data, err)
}
return conditions, nil
}A convenience wrapper
Having a weather client object makes it easy for us to customise its behaviour in tests, but that’s not usually what users want. They would appreciate being able to call a convenience wrapper around an implicit default client, just like with http.Get.
They’d like to be able to call a single function, such as weather.Get, that constructs and uses the client for them, transparently. Let’s write that:
func Get(location, key string) (Conditions, error) {
c := NewClient(key)
conditions, err := c.GetWeather(location)
if err != nil {
return Conditions{}, err
}
return conditions, nil
}Adding a Main function
Finally, let’s write a Main function that uses these components to implement the command-line tool:
func Main() {
if len(os.Args) < 2 {
fmt.Println(Usage)
return 0
}
key := os.Getenv("OPENWEATHERMAP_API_KEY")
if key == "" {
fmt.Fprintln(os.Stderr, "Please set the environment \
variable OPENWEATHERMAP_API_KEY.")
return 1
}
location := os.Args[1]
conditions, err := Get(location, key)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
fmt.Println(conditions)
return 0
}We can now use this weather.Main function in our real main package, like this:
package main
import (
"os"
"github.com/bitfield/weather"
)
func main() {
os.Exit(weather.Main())
}The patterns and components that we’ve seen here are useful for creating clients for a wide range of other APIs, too. Pick your favourite API and write a Go client for it!
