Writing safe, non-ugly Swing tests

I keep hearing this recommendation to always access Swing components on the EDT, including in tests. In plain Java, it usually means calling invokeLater() / invokeAndWait().

Indeed, some tests do encounter what seem to be race issues. However, I'm still not totally comfortable with how it looks. FEST's GuiActionRunner makes it even a little bit uglier since GuiTask, GuiQuery are abstract classes and cannot be expressed as lambdas.

// example test
    @Test
    @SuppressWarnings({"OptionalGetWithoutIsPresent", "unchecked"})
    void givenMatchPredicate_ifNoSearchTextSet_andNextButtonClicked_nothingHappens() {
        /*
        A
        |__B
        |  |__C
        |__D
         */
        DefaultMutableTreeNode root = new DefaultMutableTreeNode(TreeObject.from("A"));
        DefaultMutableTreeNode nodeB = new DefaultMutableTreeNode(TreeObject.from("B"));
        root.add(nodeB);
        TreeObject objectC = TreeObject.from("C");
        DefaultMutableTreeNode nodeC = new DefaultMutableTreeNode(objectC);
        nodeB.add(nodeC);
        DefaultMutableTreeNode nodeD = new DefaultMutableTreeNode(TreeObject.from("D"));
        root.add(nodeD);

        SwingUtilities.invokeLater(() -> {
            TreeSelectDialog<TreeObject> dialog = TreeSelectDialog.of(null, () -> root);
            BiPredicate<TreeObject, String> predicateMock = mock();
            dialog.setMatchPredicate(predicateMock);
            JTextComponent searchField = finder.find(dialog, JTextComponent.class).get();
            JButton nextButton = finder.find(dialog, JButton.class, "nextButton").get();
            tree = finder.find(dialog, JTree.class).get();
            SwingUtilities.invokeLater(dialog::doModal);
            robot.waitForIdle();
            assumeTrue(searchField instanceof ValueField);
            assumeFalse(((ValueField<?>) searchField).getPlaceholderText() == null);
            Object initialSelectedObject = getSelectedObject();

            nextButton.doClick();

            then(predicateMock).shouldHaveNoInteractions();
            assertEquals(initialSelectedObject, getSelectedObject());
            assertEquals(Colors.placeholder(), searchField.getForeground());
            dialog.dispose();
        });
    }

I tried to refactor it using InvocationInterceptor. However, my waitForIdle() calls fail since you "cannot call method from the event dispatcher thread". invokeLater() also executes the code on EDT, including waitForIdle(), but the exception is swallowed and no assertions are actually executed.

Wrapping doModal() in invokeLater() and then waitForIdle() was my attempt to defer assumptions/assertions before the dialog gets actually displayed (doModal() is essentially a decorated setVisible(true)).

@ExtendWith(EdtExtension.class)
class TreeSelectDialogTest {
    // outer SwingUtilities.invokeLater(...) removed

import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.InvocationInterceptor;
import org.junit.jupiter.api.extension.ReflectiveInvocationContext;
import org.junit.platform.commons.util.ExceptionUtils;

import javax.swing.SwingUtilities;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;

public class EdtExtension implements InvocationInterceptor {

    @Override
    public void interceptTestMethod(Invocation<Void> invocation, ReflectiveInvocationContext<Method> invocationContext, ExtensionContext extensionContext) {
        invokeInEdt(invocation);
    }

    private void invokeInEdt(Invocation<Void> invocation) {
        try {
            SwingUtilities.invokeAndWait(() -> {
                try {
                    invocation.proceed();
                } catch (Throwable e) {
                    ExceptionUtils.throwAsUncheckedException(e);
                }
            });
        } catch (InterruptedException | InvocationTargetException e) {
            ExceptionUtils.throwAsUncheckedException(e);
        }
    }
}

Caused by: java.lang.IllegalThreadStateException: Cannot call method from the event dispatcher thread
    at org.fest.swing.core.BasicRobot.waitForIdle(BasicRobot.java:669)
    at org.fest.swing.core.BasicRobot.waitForIdle(BasicRobot.java:654)

What it the most concise and readable way to refactor my test? If FEST can help, you can use it.

Java 8.

Solution

Invoking Tests on EDT

If you're using JUnit Jupiter then you can write an InvocationInterceptor extension that invokes tests on the EDT. You just have to make sure you wait for the test to complete and that any errors thrown by the test are propagated to JUnit. This will avoid needing invokeLater and/or invokeAndWait calls inside your tests.

Here's a proof-of-concept. It uses java.awt.Robot instead of FEST's or AssertJ Swing's BasicRobot. It also uses SwingUtilities::invokeLater instead of GuiActionRunner. Both these things can easily be changed for your real code.

import static org.junit.platform.commons.util.ExceptionUtils.throwAsUncheckedException;

import java.awt.AWTException;
import java.awt.Robot;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import javax.swing.SwingUtilities;
import org.junit.jupiter.api.extension.DynamicTestInvocationContext;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.ExtensionContext.Namespace;
import org.junit.jupiter.api.extension.InvocationInterceptor;
import org.junit.jupiter.api.extension.ReflectiveInvocationContext;

public class EdtExtension implements InvocationInterceptor {

  private static final Namespace NAMESPACE = Namespace.create(EdtExtension.class);
  private static final Object ROBOT_KEY = Robot.class;

  @Override
  public void interceptTestMethod(
      Invocation<Void> invocation,
      ReflectiveInvocationContext<Method> invocationContext,
      ExtensionContext extensionContext)
      throws Throwable {
    // Invoke @Test methods on EDT
    invokeOnEdt(invocation, extensionContext);
  }

  @Override
  public void interceptDynamicTest(
      Invocation<Void> invocation,
      DynamicTestInvocationContext invocationContext,
      ExtensionContext extensionContext)
      throws Throwable {
    // Invoke DynamicNode tests on EDT
    invokeOnEdt(invocation, extensionContext);
  }

  @Override
  public void interceptTestTemplateMethod(
      Invocation<Void> invocation,
      ReflectiveInvocationContext<Method> invocationContext,
      ExtensionContext extensionContext)
      throws Throwable {
    // Invoke @TestTemplate (e.g., @ParameterizedTest) methods on EDT
    invokeOnEdt(invocation, extensionContext);
  }

  private void invokeOnEdt(Invocation<Void> invocation, ExtensionContext context) throws Throwable {
    Robot robot = getAwtRobot(context);
    try {
      SwingUtilities.invokeAndWait(() -> {
        try {
          invocation.proceed();
        } catch (Throwable t) {
          // Note: ExceptionUtils is considered internal JUnit API
          throw throwAsUncheckedException(t);
        }
      });
    } catch (InvocationTargetException ex) {
      Throwable cause = ex.getCause();
      throw cause != null ? cause : ex;
    } finally {
      robot.waitForIdle();
    }
  }

  private Robot getAwtRobot(ExtensionContext context) {
    ExtensionContext.Store store = context.getRoot().getStore(NAMESPACE);
    return store.getOrComputeIfAbsent(ROBOT_KEY, this::createAwtRobot, Robot.class);
  }

  private Robot createAwtRobot(Object unused) {
    try {
      return new Robot();
    } catch (AWTException ex) {
      // Note: ExceptionUtils is considered internal JUnit API
      throw throwAsUncheckedException(ex);
    }
  }
}

Trying it Out

Using the above extension with the following test class:

import static org.junit.jupiter.api.Assertions.assertTrue;
import static org.junit.jupiter.api.Assertions.fail;

import javax.swing.SwingUtilities;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.DynamicNode;
import org.junit.jupiter.api.DynamicTest;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.TestFactory;
import org.junit.jupiter.api.extension.ExtendWith;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.ValueSource;

@ExtendWith(EdtExtension.class)
@DisplayName("Test EdtExtension")
public class EdtExtensionTests {

  @Test
  @DisplayName("invokes regular tests on EDT")
  void test() {
    assertTrue(SwingUtilities.isEventDispatchThread(), "EDT");
  }

  @TestFactory
  DynamicNode dynamicTest() {
    return DynamicTest.dynamicTest(
        "invokes dynamic tests on EDT",
        () -> assertTrue(SwingUtilities.isEventDispatchThread(), "EDT"));
  }

  @ParameterizedTest(name = "invokes templated tests on EDT")
  @ValueSource(strings = "foo")
  void templateTest(String param) {
    assertTrue(SwingUtilities.isEventDispatchThread(), "EDT");
  }

  @Test
  @DisplayName("does not swallow exceptions thrown out of test (FAILURE EXPECTED)")
  void failingTest() {
    fail("This should cause the test to fail");
  }
}

Gives me the following output when the tests are executed by Gradle:

Test EdtExtension > templateTest(String) > invokes templated tests on EDT PASSED

Test EdtExtension > dynamicTest() > invokes dynamic tests on EDT PASSED

Test EdtExtension > does not swallow exceptions thrown out of test (FAILURE EXPECTED) FAILED
    org.opentest4j.AssertionFailedError at GuiTests.java:42

Test EdtExtension > invokes regular tests on EDT PASSED

Now, that's not a particularly robust or automatic way to test the extension, but it does give the expected results.

Waiting for Idle

I tried to refactor it using InvocationInterceptor. However, my waitForIdle() calls fail since you "cannot call method from the event dispatcher thread". invokeLater() also executes the code on EDT, including waitForIdle(), but the exception is swallowed and no assertions are actually executed.

Just like FEST's and AssertJ Swing's BasicRobot, the java.awt.Robot::waitForIdle method cannot be invoked on the EDT. The above calls that method just after invokeAndWait returns, thus off the EDT.

If you need to waitForIdle before executing any assertions, then the extension needs to be modified. The cleanest solution, I believe, will be to implement a way to invoke "setup methods" that setup the GUI before the test. Then the extension will wait-for-idle some time before invoking the test. How you want to specify "setup methods" is up to you, but here's an example that simply searches for all methods with a certain annotation.

SwingSetup.java (annotation)

import java.lang.annotation.Documented;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.ANNOTATION_TYPE, ElementType.METHOD})
public @interface SwingSetup {}

EdtExtension.java (modified from above)

Now implements BeforeEachCallback to invoke instance methods annotated with @SwingSetup before each test.

import static org.junit.platform.commons.support.AnnotationSupport.findAnnotatedMethods;
import static org.junit.platform.commons.support.HierarchyTraversalMode.TOP_DOWN;
import static org.junit.platform.commons.util.ExceptionUtils.throwAsUncheckedException;

import java.awt.AWTException;
import java.awt.Robot;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.List;
import javax.swing.SwingUtilities;
import org.junit.jupiter.api.extension.BeforeEachCallback;
import org.junit.jupiter.api.extension.DynamicTestInvocationContext;
import org.junit.jupiter.api.extension.ExecutableInvoker;
import org.junit.jupiter.api.extension.ExtensionContext;
import org.junit.jupiter.api.extension.ExtensionContext.Namespace;
import org.junit.jupiter.api.extension.InvocationInterceptor;
import org.junit.jupiter.api.extension.ReflectiveInvocationContext;
import org.junit.platform.commons.support.ModifierSupport;

public class EdtExtension implements BeforeEachCallback, InvocationInterceptor {

  private static final Namespace NAMESPACE = Namespace.create(EdtExtension.class);
  private static final Object ROBOT_KEY = Robot.class;

  @Override
  public void beforeEach(ExtensionContext context) throws Exception {
    // Invoke all instance @SwingSetup methods
    Object instance = context.getRequiredTestInstance();
    List<Method> methods = getSetupMethods(context);
    ExecutableInvoker invoker = context.getExecutableInvoker();
    for (Method method : methods) {
      // Using ExecutableInvoker allows parameter injection
      SwingUtilities.invokeAndWait(() -> invoker.invoke(method, instance));
    }
    getAwtRobot(context).waitForIdle();
  }

  @SuppressWarnings("unchecked")
  private List<Method> getSetupMethods(ExtensionContext context) {
    Class<?> testClass = context.getRequiredTestClass();
    ExtensionContext.Store store = context.getRoot().getStore(NAMESPACE);
    return store.getOrComputeIfAbsent(
        testClass,
        key -> {
          List<Method> methods = findAnnotatedMethods(key, SwingSetup.class, TOP_DOWN);
          return methods.stream().filter(ModifierSupport::isNotStatic).toList();
        },
        List.class);
  }

  @Override
  public void interceptTestMethod(
      Invocation<Void> invocation,
      ReflectiveInvocationContext<Method> invocationContext,
      ExtensionContext extensionContext)
      throws Throwable {
    // Invoke @Test methods on EDT
    invokeOnEdt(invocation, extensionContext);
  }

  @Override
  public void interceptDynamicTest(
      Invocation<Void> invocation,
      DynamicTestInvocationContext invocationContext,
      ExtensionContext extensionContext)
      throws Throwable {
    // Invoke DynamicNode tests on EDT
    invokeOnEdt(invocation, extensionContext);
  }

  @Override
  public void interceptTestTemplateMethod(
      Invocation<Void> invocation,
      ReflectiveInvocationContext<Method> invocationContext,
      ExtensionContext extensionContext)
      throws Throwable {
    // Invoke @TestTemplate (e.g., @ParameterizedTest) methods on EDT
    invokeOnEdt(invocation, extensionContext);
  }

  private void invokeOnEdt(Invocation<Void> invocation, ExtensionContext context) throws Throwable {
    Robot robot = getAwtRobot(context);
    try {
      SwingUtilities.invokeAndWait(() -> {
        try {
          invocation.proceed();
        } catch (Throwable t) {
          // Note: ExceptionUtils is considered internal JUnit API
          throw throwAsUncheckedException(t);
        }
      });
    } catch (InvocationTargetException ex) {
      Throwable cause = ex.getCause();
      throw cause != null ? cause : ex;
    } finally {
      robot.waitForIdle();
    }
  }

  private Robot getAwtRobot(ExtensionContext context) {
    ExtensionContext.Store store = context.getRoot().getStore(NAMESPACE);
    return store.getOrComputeIfAbsent(ROBOT_KEY, this::createAwtRobot, Robot.class);
  }

  private Robot createAwtRobot(Object unused) {
    try {
      return new Robot();
    } catch (AWTException ex) {
      // Note: ExceptionUtils is considered internal JUnit API
      throw throwAsUncheckedException(ex);
    }
  }
}

Trying it Out

Using the modified extension with the following test class:

import static org.junit.jupiter.api.Assertions.assertFalse;
import static org.junit.jupiter.api.Assertions.assertTrue;

import javax.swing.SwingUtilities;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;

@ExtendWith(EdtExtension.class)
@DisplayName("Test EdtExtension")
public class EdtExtensionTests {

  private boolean setupGuiCalled;

  @SwingSetup
  void setupGui() {
    assertTrue(SwingUtilities.isEventDispatchThread(), "isEventDispatchThread");
    assertFalse(setupGuiCalled, "setupGui called previously");
    setupGuiCalled = true;
  }

  @Test
  @DisplayName("invokes @SwingSetup methods before tests")
  void testGui() {
    assertTrue(SwingUtilities.isEventDispatchThread(), "isEventDispatchThread");
    assertTrue(setupGuiCalled, "setupGui was not called before test");
  }
}

Gives the following results (Gradle):

Test EdtExtension > invokes @SwingSetup methods before tests PASSED

Again, not the best way to test extensions but it gives the expected results.

Notes

I'm not sure how well either of the above EdtExtension classes behave in every situation, such as with @Nested tests or test classes with a lifecycle of PER_CLASS. You may want or need to make the extension more robust. I recently made a similar answer for a question related to JavaFX and TestFX (a testing framework for JavaFX apps). That may help provide some inspiration.