Advanced Testing Tips in Go
If you haven’t looked yet please take a look at the 5 Testing Tips in Go post. I will try to list more testing functions and concepts here.
Test Caches
Go creates and uses caching for build ing and test ing. If nothing has changed in your program Go does not wait to change the unit test result as well so just gets the results from cache. OK but what does a “cacheable” test mean? How does Go identify and detect them anyway. How to get away with this kind of caching. Let’s take a look at the official documentation.
The rule for a match in the cache is that the run involves the same test binary and the flags on the command line come entirely from a restricted set of ‘cacheable’ test flags, defined as -cpu, -list, -parallel, -run, -short, and -v. If a run of go test has any test or non-test flags outside this set, the result is not cached. To disable test caching, use any test flag or argument other than the cacheable flags. The idiomatic way to disable test caching explicitly is to use -count=1. Tests that open files within the package’s source root (usually $GOPATH) or that consult environment variables only match future runs in which the files and environment variables are unchanged.
As clearly mentioned caching does not include environmental variables or outside files. Changes on those may affect the tests’ behavior of course. So you just need to keep in mind the caching effect on tests. For cleaning all the caches that are related to the tests only you can also use the command:
go clean -testcache
Fuzz Testing
First of all, let’s explain Fuzz Testing for those hearing the term for the first time. Let’s look at what Wikipedia says:
Fuzzing or fuzz testing is an automated software testing technique that involves providing invalid, unexpected, or random data as inputs to a computer program.
So it’s a great way to test your units for possible crashes, maybe handling them or possible security breaches and, so on. Since fuzzing can easily increase the RAM consumption or affect the test running performance badly, they’re also being separated in Go’s built-in support.
Fuzz tests are identified with their function signatures and the files they’re held in as the other tests. A fuzz target must be in a *_test.go file as a function in the form FuzzXxx. This function must be passed a *testing.F argument, much like a *testing.T argument is passed to a TestXxx function.
Please check the https://blog.golang.org/fuzz-beta post for more information.
Fault Injection Testing & Chaos Engineering
Fault injection is one of the cornerstones of Chaos Engineering. The term refers to a methodology developed to test how the system will respond to unexpected errors. As you can understand as a concept, it stands close to Fuzz Testing. I want to introduce a helpful library on this. Go Fault. With the middleware, you will be able to mimic some erroneous cases on a bigger picture.
Shuffle Your Tests
Unit tests are mean to test their units in an isolated way. So unit tests that are depending each other clearly or secretly, is a not-wanted implementation for sure. Go is going to allow shuffling unit tests’ orders with this new flag.
As of Go 1.17 (not released yet), you will be able to run your tests with -shuffle option.
I am leaving the Release Notes here if you want to take a look.
Count Flag and Stress Testing
As mentioned in the first topic Go’s test tool has a parameter named count . Using this parameter you will be able to benchmark your tests for n times. The default value is of course 1 so giving the parameter count=1 just means skip the caches and nothing else.
When running stress tests, race flag can be very useful to detect the possible race conditions on your design. Of course, this flag needs tests to run with data close to real data. Although this flag can also be used when building, you should take into account more RAM and CPU consumption in production compared to normal.
To investigate benchmark results deeper there are lots of flags that can be useful. E.g. benchmem , cpuprofile , mutexprofile and so on…
Thanks for reading.
