Saving/accessing DefaultTestSet created from Test.@testset?

I am looking to parse the results from the @testset macro from Test.jl, but am having trouble accessing the DefaultTestSet that is created implicitly. I am looking to run a testset, and save the results to a variable for later parsing and analysis.

I have tried from a couple different angles without success. For example, here is a "solution" that I found elsewhere online, which doesn't get me anywhere.

julia> mytestresults = @testset "My test set" begin
           @test true == true
           @test true == false
       end
My test set: Test Failed at REPL[26]:3
  Expression: true == false
   Evaluated: true == false

Stacktrace:
 [1] macro expansion
...

 [4] top-level scope
   @ REPL[26]:2
Test Summary: | Pass  Fail  Total  Time
My test set   |    1     1      2  0.0s
ERROR: Some tests did not pass: 1 passed, 1 failed, 0 errored, 0 broken.

julia> mytestresults
ERROR: UndefVarError: `mytestresults` not defined in `Main`
Suggestion: check for spelling errors or missing imports.

I have also attempted more complex solutions: nesting test sets in functions which return testsets, creating a modified copies of @testset.

This seems like a functionality that would be generally useful, and so I am sure I am overlooking a straightforward solution. I suspect the functionality exists in Test.jl, but I am yet to find it.

Edit: I am in 1.11

Solution

Another way to capture a testset object with failure(s), is to wrap the capture in another @testset. In contrast to calling Test.get_testset(), this does not depend on calling an internal function.

test1.jl:

using Test

mytestresults = nothing

@testset "" begin
  global mytestresults = @testset "My test set" begin
    @test true == true
    @test true == false
  end
end

Sample run:

julia> include("test1.jl")

My test set: Test Failed at ... test1.jl:7
  Expression: true == false
   Evaluated: true == false

Stacktrace:
...
Test Summary: | Pass  Fail  Total  Time
              |    1     1      2  0.0s
  My test set |    1     1      2  0.0s
ERROR: LoadError: Some tests did not pass: 1 passed, 1 failed, 0 errored, 0 broken.
in expression starting at ... test1.jl:4

julia> mytestresults

Test.DefaultTestSet("My test set", Any[Test Failed at ... test1.jl:7
  Expression: true == false
   Evaluated: true == false
], 1, true, false, true, 1.755207325961e9, 1.755207325977e9, false, "... test1.jl")

Why is the wrapper @testset needed? Because the outermost @testset sometimes throws instead of returning the testset object, while a nested @testset always returns the testset.

The confusing documentation gives clues to this solution.

? @testset

... DefaultTestSet records all the results and, if there are any Fails or Errors, throws an exception at the end of the top-level (non-nested) test set, along with a summary of the test results.

...

By default the @testset macro will return the testset object itself, ...

If that's not clear, the (internal) function Test.finish(::DefaultTestSet) code comments may help.

# Called at the end of a @testset, behaviour depends on whether
# this is a child of another testset, or the "root" testset
function finish(ts::DefaultTestSet; print_results::Bool=TESTSET_PRINT_ENABLE[])
    ...
    # If we are a nested test set, do not print a full summary
    # now - let the parent test set do the printing
    if get_testset_depth() != 0
        ...
        return ts
    end
    ...
    # Finally throw an error as we are the outermost test set
    if total != total_pass + total_broken
        throw(TestSetException(...))
    end

    # return the testset so it is returned from the @testset macro
    ts
end