gotestingtestify

Table-driven test always shows parent as failing when using testify


When using testify with table-driven tests as in:

func TestFoo(t *testing.T) {
    a := assert.New(t)
    tc := []struct {
        desc string
        foo  string
    }{
        {
            desc: "fail abc",
            foo:  "abc",
        },
        {
            desc: "fail def",
            foo:  "def",
        },
    }
    for _, tC := range tc {
        t.Run(tC.desc, func(t *testing.T) {
            tC := tC
            a.Equal(tC.foo, "ghi")
        })
    }
}

Your failures will come back ambiguous:

--- FAIL: TestFoo (0.00s)
    /test.go:27: 
            Error Trace:    /test.go:27
            Error:          Not equal: 
                            expected: "abc"
                            actual  : "ghi"
                            
                            Diff:
                            --- Expected
                            +++ Actual
                            @@ -1 +1 @@
                            -abc
                            +ghi
            Test:           TestFoo
    /test.go:27: 
            Error Trace:    /test.go:27
            Error:          Not equal: 
                            expected: "def"
                            actual  : "ghi"
                            
                            Diff:
                            --- Expected
                            +++ Actual
                            @@ -1 +1 @@
                            -def
                            +ghi
            Test:           TestFoo
FAIL

Rather than telling you which subtest failed, they all show the parent test having failed (e.g. TestFoo instead of TestFoo/fail_abc). If there are many tests in the table, this can make the test completely useless.


Solution

  • When initializing the testify shorthand, it must use the *testing.T from the sub-test. If you have the same habit as me of putting a := assert.New(t) at the top of every test, you'll end up passing *testing.T from the parent, so the failures will always show as coming from the parent.

    Rewriting as

    func TestFoo(t *testing.T) {
        tc := []struct {
            desc string
            foo  string
        }{
            {
                desc: "fail abc",
                foo:  "abc",
            },
            {
                desc: "fail def",
                foo:  "def",
            },
        }
        for _, tC := range tc {
            t.Run(tC.desc, func(t *testing.T) {
                tC := tC
                a := assert.New(t) // Now using *testing.T from within the sub-test
                a.Equal(tC.foo, "ghi")
            })
        }
    }
    

    Gets the expected legible output (i.e. failures broken up by sub-test and test names including the description):

    --- FAIL: TestFoo (0.00s)
        --- FAIL: TestFoo/fail_abc (0.00s)
            boop_test.go:27: 
                    Error Trace:    /test.go:27
                    Error:          Not equal: 
                                    expected: "abc"
                                    actual  : "ghi"
                                
                                    Diff:
                                    --- Expected
                                    +++ Actual
                                    @@ -1 +1 @@
                                    -abc
                                    +ghi
                    Test:           TestFoo/fail_abc
        --- FAIL: TestFoo/fail_def (0.00s)
            boop_test.go:27: 
                    Error Trace:    /test.go:27
                    Error:          Not equal: 
                                    expected: "def"
                                    actual  : "ghi"
                                
                                    Diff:
                                    --- Expected
                                    +++ Actual
                                    @@ -1 +1 @@
                                    -def
                                    +ghi
                    Test:           TestFoo/fail_def
    FAIL