@Order annotations appear ignored

I'm testing our own library, that uses JDBC. For the specified database-name, the code needs to find the value javax.sql.DataSource, connect using the DataSource.getConnection(), and then run some SQL-statements.

I'd like for all tests to be skipped, if there is mp $TESTDB in the environment. I'd also want to test those initial steps separately -- skipping the rest if, for example, finding the dataSource failed.

The test code so far is:

    public static final String DBNAME = System.getenv("TESTDB");

    private static DataSource DS = null;
    private static Connection CONN = null;

    @BeforeAll
    public static void checkEnvironment() {
        assumeTrue(DBNAME != null, "Environment variable DBNAME");
        logger.info("DBNAME is {}", DBNAME);
    }

    @Test
    @Order(20)
    public void getDataSource() {
        DS = DatasourceFactory.getDataSourceByKey(DBNAME);
        assertNotNull(DS, "Failed to find details for datasource " +
            DBNAME);

        logger.info("Datasource for {}: {}", DBNAME, DS);
    }

    @Test
    @Order(30)
    public void getConnection() {
        try {
            CONN = DS.getConnection();
        } catch (SQLException e) {
            fail("Connetion to " + DBNAME + " failed", e);
        }
    }

The @BeforeAll annotation seems to work fine -- the method is invoked first, and logs the DBNAME obtained from the environment variable. If the environment variable is not set, all tests are skipped just as I want.

My problem is, despite the explicit @Order annotations, the getConnection() test is attempted before the getDataSource() test has a chance to set the static DS. The getDataSource() itself succeeds -- later, logging something like:

Datasource for SQL_DEV4: org.apache.commons.dbcp.BasicDataSource@4a3668

but the getConnection() fails due to a null-pointer exception, because DS is still null, when it is attempted.

Why is my ordering not working?

You are probably missing:

@TestMethodOrder(OrderAnnotation.class)

defined at the class level.