I wrote some test framework using Perl's Test::more, specifically using the ok($test_result, $test_description).
As some tests "cluster", I wrote a subroutine that performs multiple ok() tests, and the main program calls such subroutines.
Now the problem for a failed test is this:
Test::More outputs the line number within the subroutine (direct caller), but I'd like to have the "caller's caller" (indirect caller) line to be output instead.
Is that possible?
Imagine the code to be similar as this:
#!/usr/bin/perl
use 5.18.2;
use warnings;
use strict;
use Test::More;
sub foo(@)
{
# do something magical
return undef;
}
sub foo_OK(@)
{
ok(foo(@_), 'foo_OK: ' . join(' ', @_)); # actually less trivial
}
sub complex_test(@)
{
foo_OK(qw(something special), @_);
foo_OK(qw(something else), @_);
#...
}
sub main()
{
complex_test(qw(abra kadabra));
complex_test(qw(more magic));
#...
}
main();
done_testing();
So I'd like to see the line of main, not the line of test_OK if ok() fails.
As shown the output would be:
not ok 1 - foo_OK: something special abra kadabra
# Failed test 'foo_OK: something special abra kadabra'
# at /tmp/test.pl line 15.
not ok 2 - foo_OK: something else abra kadabra
# Failed test 'foo_OK: something else abra kadabra'
# at /tmp/test.pl line 15.
not ok 3 - foo_OK: something special more magic
# Failed test 'foo_OK: something special more magic'
# at /tmp/test.pl line 15.
not ok 4 - foo_OK: something else more magic
# Failed test 'foo_OK: something else more magic'
# at /tmp/test.pl line 15.
1..4
# Looks like you failed 4 tests of 4.
Based on https://stackoverflow.com/a/78083924/6607497 I used Test::Builder to create sub-tests via the child() method, so a failing test would show the line number, but also causing the parent test to fail (that would also print the line number).
That way I get the information I wanted, even with more output that ideally wanted.
A modified example would look like this (line numbers added to understand the output better):
1:#!/usr/bin/perl
2:use 5.18.2;
3:use warnings;
4:use strict;
5:
6:use Test::Builder;
7:
8:sub foo($$$)
9:{
10: my ($TB, $test, $desc) = @_;
11:
12: $TB->plan('tests' => 1);
13: $test = int(rand() * 2); # simulate random failure
14: $TB->ok($test, $desc);
15: $TB->finalize();
16:}
17:
18:sub bar($$$)
19:{
20: my ($TB, $test, $desc) = @_;
21:
22: $TB->plan('tests' => 1);
23: $test = int(rand() * 2); # simulate random failure
24: $TB->ok($test, $desc);
25: $TB->finalize();
26:}
27:
28:sub foo_bar($$$)
29:{
30: my ($TB, $test, $desc) = @_;
31: my $subtest = sub($) { $TB->diag($_[0]); return $TB->child($_[0]); };
32:
33: $TB->plan('tests' => 2);
34: foo($subtest->('foo'), $test, $desc . ' foo_bar foo-param');
35: bar($subtest->('bar'), $test, $desc . ' foo_bar bar-param');
36: $TB->finalize();
37:}
38:
39:sub complex_test($$$)
40:{
41: my ($TB, $test, $desc) = @_;
42: my $subtest = sub($) { $TB->diag($_[0]); return $TB->child($_[0]); };
43:
44: $TB->plan('tests' => 3);
45: foo($subtest->('foo'), $test, $desc . ' complex_test foo-param');
46: bar($subtest->('bar'), $test, $desc . ' complex_test bar-param');
47: foo_bar($subtest->('foobar'), $test, $desc . ' complex_test foo_bar-param');
48: $TB->finalize();
49:}
50:
51:sub main()
52:{
53: my $TB = Test::Builder->create();
54: my $subtest = sub($) { $TB->diag($_[0]); return $TB->child($_[0]); };
55:
56: $TB->plan('tests' => 2);
57: complex_test($subtest->('sub1'), 0, q(abra kadabra));
58: complex_test($subtest->('sub2'), 0, q(more magic));
59: #...
60: $TB->done_testing();
61:}
62:
63:main();
And an example output would be:
1..2
# sub1
1..3
# foo
1..1
ok 1 - abra kadabra complex_test foo-param
ok 1 - foo
# bar
1..1
ok 1 - abra kadabra complex_test bar-param
ok 2 - bar
# foobar
1..2
# foo
1..1
ok 1 - abra kadabra complex_test foo_bar-param foo_bar foo-param
ok 1 - foo
# bar
1..1
ok 1 - abra kadabra complex_test foo_bar-param foo_bar bar-param
ok 2 - bar
ok 3 - foobar
ok 1 - sub1
# sub2
1..3
# foo
1..1
ok 1 - more magic complex_test foo-param
ok 1 - foo
# bar
1..1
not ok 1 - more magic complex_test bar-param
# Failed test 'more magic complex_test bar-param'
# at /tmp/test.pl line 46.
# Looks like you failed 1 test of 1.
not ok 2 - bar
# Failed test 'bar'
# at /tmp/test.pl line 46.
# foobar
1..2
# foo
1..1
not ok 1 - more magic complex_test foo_bar-param foo_bar foo-param
# Failed test 'more magic complex_test foo_bar-param foo_bar foo-param'
# at /tmp/test.pl line 34.
# Looks like you failed 1 test of 1.
not ok 1 - foo
# Failed test 'foo'
# at /tmp/test.pl line 34.
# bar
1..1
not ok 1 - more magic complex_test foo_bar-param foo_bar bar-param
# Failed test 'more magic complex_test foo_bar-param foo_bar bar-param'
# at /tmp/test.pl line 35.
# Looks like you failed 1 test of 1.
not ok 2 - bar
# Failed test 'bar'
# at /tmp/test.pl line 35.
# Looks like you failed 2 tests of 2.
not ok 3 - foobar
# Failed test 'foobar'
# at /tmp/test.pl line 47.
# Looks like you failed 2 tests of 3.
not ok 2 - sub2
# Failed test 'sub2'
# at /tmp/test.pl line 58.