Golang testing — gotest.tools assertions

Equal uses the == operator to assert two values are equal and fails the test if they are not equal.

If the comparison fails Equal will use the variable names for x and y as part of the failure message to identify the actual and expected values.

If either x or y are a multi-line string the failure message will include a unified diff of the two values. If the values only differ by whitespace the unified diff will be augmented by replacing whitespace characters with visible characters to identify the whitespace difference.

On the other hand…

DeepEqual uses google/go-cmp (http://bit.do/go-cmp) to assert two values are equal and fails the test if they are not equal.

Package https://godoc.org/gotest.tools/assert/opt provides some additional commonly used Options.

Using one or the other is as simple as : if you wrote your if with == then use Equal, otherwise use DeepEqual. DeepEqual (and usually reflect.DeepEqual) is used when you want to compare anything more complex than primitive types. One advantage of using cmp.DeepEqual over reflect.DeepEqual (in an if), is that you get a well crafted message that shows the diff between the expected and the actual structs compared – and you can pass options to it.

assert.Assert(t, cmp.DeepEqual([]string{"a", "b"}, []string{"b", "a"}))
// Will print something like
// --- result
// +++ exp
// {[]string}[0]:
//         -: "a"
//         +: "b"
// {[]string}[1]:
//         -: "b"
//         +: "a"
foo := &someType(a: "with", b: "value")
bar := &someType(a: "with", b: "value")
// the following will succeed as foo and bar are _DeepEqual_
assert.Assert(t, cmp.DeepEqual(foo, bar))

When using DeepEqual, you may end up with really weird behavior(s). You may want to ignore some fields, or consider nil slice or map the same as empty ones ; or more common, your struct contains some unexported fields that you cannot use when comparing (as they are not exported 😓). In those case, you can use go-cmp options.

Some existing one are :

• EquateEmpty returns a Comparer option that determines all maps and slices with a length of zero to be equal, regardless of whether they are nil.
• IgnoreFields returns an Option that ignores exported fields of the given names on a single struct type. The struct type is specified by passing in a value of that type.
• IgnoreUnexported returns an Option that only ignores the immediate unexported fields of a struct, including anonymous fields of unexported types.
• SortSlices returns a Transformer option that sorts all []V
• … and more 👼

gotest.tools also defines some and you can define yours ! As an example, gotest.tools defines TimeWithThreshold and DurationWithThreshold that allows to not fails if the time (or duration) is not exactly the same but in the specified threshold we specified. Here is the code for DurationWithThreshold for inspiration.

// DurationWithThreshold returns a gocmp.Comparer for comparing time.Duration. The
// Comparer returns true if the difference between the two Duration values is
// within the threshold and neither value is zero.
func DurationWithThreshold(threshold time.Duration) gocmp.Option {
        return gocmp.Comparer(cmpDuration(threshold))
}

func cmpDuration(threshold time.Duration) func(x, y time.Duration) bool {
        return func(x, y time.Duration) bool {
                if x == 0 || y == 0 {
                        return false
                }
                delta := x - y
                return delta <= threshold && delta >= -threshold
        }
}

Another good example for those options is when you want to skip some field. In docker/docker we want to be able to easily check for equality between two service specs, but those might have different CreatedAt and UpdatedAt values that we usually don’t care about – what we want is to make sure it happens in the past 20 seconds. You can easily define an option for that.

func cmpServiceOpts() cmp.Option {
        const threshold = 20 * time.Second

        // Apply withinThreshold only for the following fields
        metaTimeFields := func(path cmp.Path)bool {
                switch path.String() {
                case "Meta.CreatedAt", "Meta.UpdatedAt":
                        return true
                }
                return false
        }
        // have a 20s threshold for the time value that will be passed
        withinThreshold := cmp.Comparer(func(x, y time.Time) bool {
                delta := x.Sub(y)
                return delta < threshold && delta > -threshold
        })

        return cmp.FilterPath(metaTimeFields, withinThreshold)
}

I recommend you look at the gotest.tools/assert/opt documentation to see which one are defined and how to use them.

Checking for errors is very common in Go, having Comparison function for it was a requirement.

• Error fails if the error is nil or the error message is not the expected one.
• ErrorContains fails if the error is nil or the error message does not contain the expected substring.
• ErrorType fails if the error is nil or the error type is not the expected type.

Let’s first look at the most used : Error and ErrorContains.

var err error
// will fail with : expected an error, got nil
assert.Check(t, cmp.Error(err, "message in a bottle"))
err = errors.Wrap(errors.New("other"), "wrapped")
// will fail with : expected error "other", got "wrapped: other"
assert.Check(t, cmp.Error(err, "other"))
// will succeed
assert.Check(t, cmp.ErrorContains(err, "other"))

As you can see ErrorContains is especially useful when working with wrapped errors. Now let’s look at ErrorType.

var err error
// will fail with : error is nil, not StubError
assert.Check(t, cmp.ErrorType(err, StubError{}))

err := StubError{"foo"}
// will succeed
assert.Check(t, cmp.ErrorType(err, StubError{}))

// Note that it also work with a function returning an error
func foo() error {}
assert.Check(t, cmp.ErrorType(foo, StubError{}))

Sometimes, a code is supposed to panic, see Effective Go (#Panic) for more information. And thus, you may want to make sure you’re code panics in such cases. It’s always a bit tricky to test a code that panic as you have to use a deferred function to recover the panic — but then if the panic doesn’t happen how do you fail the test ?

This is where Panics comes handy.

func foo(shouldPanic bool) {
        if shouldPanic {
                panic("booooooooooh")
        }
        // don't worry, be happy
}
// will fail with : did not panic
assert.Check(t, cmp.Panics(foo(false)))
// will succeed
assert.Check(t, cmp.Panics(foo(true)))

Those last three built-in Comparison are pretty straightforward.

• Contains succeeds if item is in collection. Collection may be a string, map, slice, or array.

  If collection is a string, item must also be a string, and is compared using strings.Contains(). If collection is a Map, contains will succeed if item is a key in the map. If collection is a slice or array, item is compared to each item in the sequence using =reflect.DeepEqual()=.

• Len succeeds if the sequence has the expected length.
• Nil succeeds if obj is a nil interface, pointer, or function.

// Contains works on string, map, slice or arrays
assert.Check(t, cmp.Contains("foobar", "foo"))
assert.Check(t, cmp.Contains([]string{"a", "b", "c"}, "b"))
assert.Check(t, cmp.Contains(map[string]int{"a": 1, "b": 2, "c": 4}, "b"))

// Len also works on string, map, slice or arrays
assert.Check(t, cmp.Len("foobar", 6))
assert.Check(t, cmp.Len([]string{"a", "b", "c"}, 3))
assert.Check(t, cmp.Len(map[string]int{"a": 1, "b": 2, "c": 4}, 3))

// Nil
var foo *MyStruc
assert.Check(t, cmp.Nil(foo))
assert.Check(t, cmp.Nil(bar()))

But let’s not waste more time and let’s see how to write our own Comparison !

One of the main aspect of gotest.tools/assert is to make it easy for developer to write as less boilerplate code as possible while writing tests. Writing your own Comparison allows you to write a well named function that will be easy to read and that can be re-used across your tests.

Let’s look back at the cmp.Comparison and cmp.Result types.

type Comparison func() Result

type Result interface {
        Success() bool
}

A Comparison for assert.Check or assert.Check is a function that return a Result, it’s pretty straightforward to implement, especially with cmp.ResultSuccess and cmp.ResultFailure(…) (as seen previously).

func regexPattern(value string, pattern string) cmp.Comparison {
        return func() cmp.Result {
                re := regexp.MustCompile(pattern)
                if re.MatchString(value) {
                        return cmp.ResultSuccess
                }
                return cmp.ResultFailure(
                        fmt.Sprintf("%q did not match pattern %q", value, pattern))
        }
}

// To use it
assert.Check(t, regexPattern("12345.34", `\d+.\d\d`))

As you can see, it’s pretty easy to implement, and you can do quite a lot in there easily. If a function call returns an error inside of your Comparison function, you can use cmp.ResultFromError for example. Having something like assert.Check(t, isMyServerUp(":8080")) is way more readable than a 30-line of code to check it.