bashunit-testingerror-handlingbats-core

BATS assert_failure test not recognising exit 1?


While trying to test a method that checks for 5 seconds whether a GitLab server is running, I am experiencing some difficulties in detecting the error message.

Function checking GitLab server status

check_for_n_seconds_if_gitlab_server_is_running() {
    duration=$1
    echo "duration=$duration"
    running="false"
    if [ "$running" == "false" ]; then
        echo "ERROR, did not find the GitLab server running within $duration seconds!"
        exit 1
    fi
}

Test Code

#!./test/libs/bats/bin/bats

load 'libs/bats-support/load'
load 'libs/bats-assert/load'
load 'libs/bats-file/load'

source src/helper.sh


@test "If error is thrown if the GitLab server is not running within 5 seconds after uninstallation." {
        
    # run the tested method
    check_for_n_seconds_if_gitlab_server_is_running 4
    
    assert_failure
    assert_output --partial "ERROR, did not find the GitLab server running within 4 seconds!"
}

Expected behaviour

I would expect the test to pass, because the exit 1 is reached and I think it throws a failure.

Observed behaviour

When the exit 1 is included, the test fails and the output of the test is:

✗ If error is thrown if the GitLab server is not running within 5 seconds after uninstallation.

When the exit 1 is commented out, the test fails and the output of the test is:

✗ If error is thrown if the GitLab server is not running within 5 seconds after uninstallation.
   (from function `assert_failure' in file test/libs/bats-assert/src/assert.bash, line 140,
    in test file test/long_test_helper.bats, line 17)
     `assert_failure' failed
   duration=4
   ERROR, did not find the GitLab server running within 4 seconds!
   
   -- command succeeded, but it was expected to fail --
   output : 
   --

Question

How can I ensure the test detects error that is thrown?/how should I throw the error/exit 1 command to ensure the assert_failure test passes?


Solution

  • The issue was that I was trying to run a function from the test function, instead of from a separate bash shell. I found out by reproducing another working example that used the run bash -c command, which did behave as expected on the same function. So in practice the following works:

    Test code

    #!./test/libs/bats/bin/bats
    
    load 'libs/bats-support/load'
    load 'libs/bats-assert/load'
    load 'libs/bats-file/load'
    
    @test "If error is thrown if the GitLab server is not running within 5 seconds after uninstallation." {
        
        run bash -c "source src/helper.sh && check_for_n_seconds_if_gitlab_server_is_running"
        assert_failure 
        assert_output --partial "ERROR, did not find the GitLab server running within 4 seconds!"
    }
    

    Tested function code

    check_for_n_seconds_if_gitlab_server_is_running() {
        duration=$1
        echo "duration=$duration"
        running="false"
        if [ "$running" == "false" ]; then
            echo "ERROR, did not find the GitLab server running within $duration seconds!"
            exit 1
        fi
    }
    

    Expected behaviour

    By calling the function via a the run bash -c command, the test passes:

    ✓ If error is thrown if the GitLab server is not running within 5 seconds after uninstallation.

    1 test, 0 failures

    Note

    If someone is able to include a MWE that works without calling the function from a separate run bash -c command, but directly as a function from within the test function, please post it as a separate answer.