Elephants for breakfast
Photo by Rohit Varma on Unsplash
If you have a tough question that you can’t answer, first tackle a simpler question that you can’t answer.
—Max Tegmark, “Our Mathematical Universe”
Everyone knows how to eat an elephant (but please don’t: they’re quite endangered). The point is that apparently intimidating tasks can always be dealt with by breaking them up into more tractable sub-tasks, and the same applies to testing.
For example, in Here comes the
sun, we did the bare minimum to create a simple weather client
program in Rust, and now we’re interested in testing the
get_weather function that we wrote. Here’s our first
attempt at a test:
#[cfg(test)]
mod tests {
use std::env;
use super::*;
#[test]
fn get_weather_fn_returns_correct_weather_for_location() {
let api_key = env::var("WEATHERSTACK_API_KEY").unwrap();
let weather = get_weather("London,UK", &api_key).unwrap();
assert_eq!(
weather,
Weather {
temperature: 11.2,
summary: "Sunny".into(),
},
"wrong weather"
);
}
}The get_weather function calls the Weatherstack API to
get the current weather conditions for London, and checks the result.
The problem, as I dare say you spotted, is that we don’t necessarily
know in advance what that will be!
A problem of determinism
I mean, as long as it happens to be 11.2ºC and sunny in London, this test will pass, but that’s not really good enough. If it gets colder, or starts raining, the test will start failing, and that’s wrong, because the function is correct (we suppose).
What we’re really saying is that, if the function is working, then the weather it returns should be whatever the weather actually is in London, but how can we know that? It would be silly to try to get the weather by calling Weatherstack in the test, for example, because if we made a mistake doing that in the function, we could also make the same mistake in the test.
In any case, that’s not really testing anything useful: we’d end up just asserting that if we make the same call to Weatherstack twice, we get the same answer both times. Worse, we might not! We don’t know how often the data is updated, but it must be updated sometimes, and if we were unlucky, that might happen between the first and second calls.
What can we test about
get_weather?
This might all seem very obvious, and I’m sorry if I appear to be
labouring the point. It’s just that there’s something different about
get_weather from any other function we’ve tested (or
written) before, and that’s that it’s non-deterministic: we
can’t know in advance what the result will be.
So how do we test a function like that? I mean, if the answer can be more or less anything, how can we test that the answer we’re getting is correct?
Well, we can’t, but that’s not the same as saying that we can’t test the function. We just need to test something else about it.
After all, despite the name, the get_weather function
isn’t really about weather. Its job is to make a correctly-formatted
request to Weatherstack, and to correctly interpret the response as a
Weather struct.
One bite at a time
And when you put the problem that way, it becomes clear that there’s a way to bypass this problem altogether: just break it up into two separate problems that are easier to solve.
In other words, suppose we write an imaginary version of
get_weather that uses two more magic functions to
implement these “request” and “response” behaviours:
pub fn get_weather(
location: &str,
api_key: &str,
) -> Result<Weather> {
let resp = request(location, api_key).send()?;
let weather = deserialize(&resp.text()?)?;
Ok(weather)
}In other words, we’re saying that if we had a request
function that generated the necessary request object, all we’d have to
do is call send() on it in order to make the
request to the Weatherstack API.
That would give us a response object, containing the JSON data with
the actual weather for the location, and if we had a
deserialize function that could turn that data into a
Weather struct, then we’d be done!
The key insight is that we can test both of those functions,
request and deserialize, in a completely
deterministic way, without calling the API at all. Yes, we’d still have
to make sure that get_weather correctly glues together
those two magic functions, but at least we’d have offloaded the bulk of
the testing problem.
In my book The Secrets of Rust: Tools, you’ll learn all about this and dozens of other useful techniques for crafting practical programs in Rust, including a line counter, a logbook, a memo manager, and a Cargo plugin. Despite Rust’s somewhat-deserved reputation for complexity, it’s not at all difficult to build clear, simple, and powerful tools in Rust, if you know how to go about it the right way.
Similarly, in this case, we can turn a complicated function
(get_weather) into two simple functions
(request and deserialize). Let’s do just
that.
Testing request
First, what about request? Its job is pretty simple:
take the location and key strings provided by the user, encode them as a
query to the Weatherstack API, create a request object that’s ready to
send, and return it.
The most direct way to test this would be to create the request, and then inspect it to make sure it contains all the information it’s supposed to.
use url::Host::Domain;
#[test]
fn request_builds_correct_request() {
let req = request("London,UK", "dummy API key");
let req = req.build().unwrap();
assert_eq!(req.method(), "GET", "wrong method");
let url = req.url();
assert_eq!(
url.host(),
Some(Domain("api.weatherstack.com")),
"wrong host"
);
assert_eq!(url.path(), "/current", "wrong path");
let params: Vec<(_, _)> = url.query_pairs().collect();
assert_eq!(
params,
vec![
("query".into(), "London,UK".into()),
("access_key".into(), "dummy API key".into())
],
"wrong params"
);
}There might be more things that we could or should check about this request, but I think we’ve covered the main ones: the HTTP method, the host (are we calling the right API?), the path (are we calling the right endpoint?), and the all-important parameters. If those are correct, the request should be fine.
Maybe this all seems a bit paperworky, and I agree—but the point of this function is to do paperwork! What we’re really saying is that, if we fill out the request properly according to the Weatherstack API docs, then making the request should give the right answer.
In other words, we’re testing that we’ve upheld our part of the contract. The rest is up to Weatherstack, and there’s no point testing their code—if it doesn’t work, it’s not our problem. (Let’s hope they have their own tests, but you never know.)
Implementing request
GOAL: Write the request function so
that it passes this test.
HINT: We already have code that works in
main, so see if you can lift and shift it across to
lib.rs and put it in the request function. In
fact, request does less than the code we wrote
earlier, because it doesn’t actually send() the request,
just return it.
There’s one little wrinkle, which is that it turns out to be more
convenient to have request return a
RequestBuilder rather than a finished
Request—this builder pattern is very common in
Rust APIs.
If we do that, then get_weather can simply call
send() on the value returned by request.
SOLUTION: Here’s one possibility:
fn request(location: &str, api_key: &str) -> RequestBuilder {
reqwest::blocking::Client::new()
.get("https://api.weatherstack.com/current")
.query(&[("query", location), ("access_key", api_key)])
}Blocking or non-blocking?
By the way, we keep using that word “blocking”; we had to opt in to
this feature of reqwest, and we’re explicitly creating a
blocking Client, as opposed to some other kind. So, what
other kind could there be?
Well, a non-blocking one, obviously! All the Rust code we’ve written so far has been what’s called synchronous, which means the same as “blocking”. In other words, when we call a function, we have to wait for it to finish and return before we can do anything else.
Asynchronous code, or just “async” for short, would be able to do more than one thing at once. For example, we can imagine making more than one HTTP request at once. In that case, it would be sensible to fire them all off together and then wait for the responses concurrently, instead of waiting for one to complete before starting the next.
This multi-tasking approach makes sense any time we’re dealing with things we might have to wait for: instead of just waiting, we can do useful work in the meantime. On the other hand, async Rust programs are a bit more complicated to write, because there are many things happening at once. It’s a topic that really deserves a whole book of its own (and I’m looking forward to writing one).
Since we’re only making one request in this program, though, it
doesn’t need to be async, which helps to keep things simple. That’s why
we’re using the blocking mode of reqwest:
mystery solved!
So far, so good, then, and we can have some confidence that this code
is correct even without making actual requests to Weatherstack, and
burning through our quota. But if we did make a request, we’d
get some JSON data back which needs to be deserialized and turned into a
Weather struct, so let’s tackle that job in the next
post.
