Conditions and concurrency
In Part 1 of this series, we got
started with the testscript
package. In Part 2 we learned how to test Go CLI tools
using a test script, and in Part
3 we explored the txtar
format and ways to manipulate
files and folders.
Let’s look at some advanced testscript
techniques now,
including conditions, environment variables, and
concurrency.
Conditions
Because we’re running real programs on a real computer that we have limited information about, we may need to specify some conditions governing whether or not to perform an action.
For example, we can’t always guarantee in advance that the program we want to run will actually be available on the test system. We can check for its existence by prefixing the script line with a condition:
[exec:sh] exec echo yay, we have a shell
The square brackets contain a condition, which can be true or false. If the condition is true, the rest of the line is executed. Otherwise, it’s ignored.
In this example, the condition [exec:sh]
is true if
there is a program named sh
in our $PATH
that
we have permission to execute. If there isn’t, this line of the script
will be ignored.
On the other hand, if we need to check for a program at a specific path, we can give that path as part of the condition:
[exec:/bin/sh] exec echo yay, we have /bin/sh
There are a few other built-in conditions available, but perhaps the most useful to us are the current Go version, the current operating system, and the CPU architecture of this machine:
# 'go1.x' is true if this is Go 1.x or higher
[go1.16] exec echo 'We have at least Go 1.16'
# Any known value of GOOS is also a valid condition
[darwin] exec echo 'We''re on macOS'
# As is any known value of GOARCH
[!arm64] exec echo 'This is a non-arm64 machine'
As you can see from that example, we can also negate a
condition by prefixing it with !
. For example, the
condition [!arm64]
is true only if the value of
GOARCH
is not arm64
.
We can use a condition to control other script statements, too, not
just exec
. For example, we can use the skip
statement to skip the test in some circumstances:
# Skip this test unless we have Go 1.18 or later
[!go1.18] skip
# Skip this test on Linux
[linux] skip
The unix
condition is true when the target OS is one of
those that Go considers “Unix-like”, including Linux, macOS, FreeBSD,
and others.
[unix] exec echo 'It''s a UNIX system! I know this!'
In Go 1.19, the only supported operating systems for which
unix
is not true are js
,
nacl
, plan9
, windows
, and
zos
.
Setting environment
variables with env
Since command-line tools often use environment variables, it’s
important to be able to set and use these in scripts. To set a
particular variable for the duration of the script, use an
env
statement:
env MYVAR=hello
For example, to test that some program myprog
fails and
prints an error message when the environment variable it needs has no
value, we could write:
env AUTH_TOKEN=
! exec myprog
stderr 'AUTH_TOKEN must be set'
In scripts, we can refer to the value of an environment variable by
using the $
token followed by the name of the variable,
just like in a shell script. This will be replaced by the value of the
variable when the script runs. For example:
exec echo $PATH
This will cause echo
to print the contents of
$PATH
, whatever it happens to be when the script is
run.
Each script starts with an empty environment, apart from
$PATH
, and a few other predefined variables. For example,
HOME
is set to /no-home
, because many programs
expect to be able to find the user’s home directory in
$HOME
.
If scripts need to know the absolute path of their work directory, it
will be available as $WORK
. For example, we could use
$WORK
as part of the value for another environment
variable:
env CACHE_DIR=$WORK/.cache
To make scripts deterministic, though, the actual value of
this variable will be replaced in the testscript
output by
the literal string $WORK
. In fact, the value of
$WORK
will be different for every run and every test, but
it will always print as simply $WORK
.
Because some programs also expect to read the variable
$TMPDIR
to find out where to write their temporary files,
testscript
sets this too, and its value will be
$WORK/.tmp
.
A useful variable for cross-platform testing is $exe
.
Its value on Windows is .exe
, but on all other platforms,
it’s the empty string. So you can specify the name of some program
prog
like this:
exec prog$exe
On Windows, this evaluates to running prog.exe
, but on
non-Windows systems, it will be simply prog
.
Passing values to scripts via environment variables
Sometimes we need to supply dynamic information to a script, for
example some value that’s not known until the test is actually running.
We can pass values like this to scripts using the environment, by using
the Setup
function supplied to testscript.Run
as part of Params
.
For example, suppose the program run by the script needs to connect to some server, but the required port number won’t be known until we start the server in the test. Let’s say it’s generated randomly.
Once we know the generated address, we can store it in a suitable
environment variable, using the Setup
function supplied to
testscript.Run
:
func TestScriptWithExtraEnvVars(t *testing.T) {
.Parallel()
t:= randomLocalAddr(t)
addr .Run(t, testscript.Params{
testscript: "testdata/script",
Dir: func(env *testscript.Env) error {
Setup.Setenv("SERVER_ADDR", addr)
envreturn nil
},
})
}
The Setup
function is run after any files in the test
script have been extracted, but before the script itself starts. It
receives an Env
object containing the predefined
environment variables to be supplied to the script.
At this point, our Setup
function can call
env.Setenv
to add any extra environment variables it wants
to make available in the script. For example, we can set the
SERVER_ADDR
variable to whatever the test server’s address
is.
A script can then get access to that value by referencing
$SERVER_ADDR
, just as it would with any other environment
variable:
exec echo $SERVER_ADDR
stdout '127.0.0.1:[\d]+'
The stdout
assertion here is just to make sure that we
did successfully set the SERVER_ADDR
variable to something
plausible. In a real test we could use this value as the address for the
program to connect to.
Running programs in
background with &
One nice feature of typical shells is the ability to run programs in the background: that is, concurrently. For example, in most shells we can write something like:
sleep 10 &
This starts the sleep
program running as a separate
process, but continues to the next line of the script so that we can do
something else concurrently. We can do the same in a test script:
exec sleep 10 &
The trailing &
(“ampersand”) character tells
testscript
to execute the sleep
program in the
background, but carry on. Its output will be buffered so that we can
look at it later, if we need to. We can start as many programs in the
background as necessary, and they will all run concurrently with the
rest of the script.
Because backgrounding a program with &
doesn’t wait
for it to complete before continuing, we need a way to do that
explicitly. Otherwise, the script might end before the background
program is done. To do that, we use the wait
statement:
exec sleep 10 &
wait
This pauses the script until all backgrounded programs have exited,
and then continues. At this point the output from those programs, in the
order they were started, will be available for testing using the
stdout
, stderr
, and cmp
assertions, just as if they’d been run in the foreground.
Why might we want to do this? Well, some programs might need some background service to be available in order for us to test them, such as a database or a web server.
At the end of the script, all programs still running in background
will be stopped using the os.Interrupt
signal (that is, the
equivalent of typing Ctrl-C at the terminal), or, if necessary,
os.Kill
.
Let’s see an example. Suppose we have some Go program that implements
a web server, and we want to start it on some arbitrary port, then
connect to it with curl
and check that we get the expected
welcome message. How could we test that?
First, let’s set up a delegate
Main
function to run the server program, taking its
listen address from os.Args
. We’ll associate this with a
custom program named listen
, using
testscript.RunMain
in the same way that we’ve done for
previous examples.
Again, we’ll generate a random listen address, and put this value in
the environment variable SERVER_ADDR
, using
Setup
. The details are in Listing
env/2
, but they’re essentially the same as in the
previous examples.
Now we can write the script itself:
exec listen $SERVER_ADDR &
exec curl -s --retry-connrefused --retry 1 $SERVER_ADDR
stdout 'Hello from the Go web server'
wait
The first thing we do is use exec
to start our
listen
program on the generated address. Notice that we use
the &
token to execute the program concurrently, in
background.
The next line of the script uses the well-known curl
tool to make an HTTP connection to the given address, and print the
response. Because it takes some non-zero time for our server program to
actually start, if we try to connect instantly it probably
won’t work. Instead, we need to tell curl
to retry if the
first connection attempt is refused.
Once the connection has succeeded, we use stdout
to
assert that the server responds with the message “Hello from the Go web
server”.
If the script simply ended here, the background listen
process would be killed with an interrupt signal, which would cause the
test to fail. We don’t want that, so we add a wait
statement.
But if the server listened forever, then the wait
statement would also wait for it forever (or, at least, until the test
timed out). We don’t want that either, so we’ve arranged
for the server to politely shut itself down after serving exactly one
request. Normal servers wouldn’t do this, but it just makes this
demonstration clearer.
Therefore, the wait
statement pauses the script just
long enough to allow this tidy shutdown to happen, and then exits.
Thanks for reading Part 4. In Part
5 we’ll talk about standalone test scripts and the
testscript
CLI tool.
Previous: Files in test scripts
Next: Standalone test scripts