I'm looking to distribute tests to multiple instances of the junit5 standalone console whereby each instance reads off of a queue. Each instance of the runner would use the same test.jar on the classpath, so I'm not trying to distribute the byte code of the actual tests here, just the names of the tests / filter pattern strings.

From the junit 5 advanced topics doc, I think the appropriate place to extend junit 5 to do this is using the platform launcher api. I cobbled this snippet together largely with the sample code in the guide. I think this is what I need to write but I'm concerned I'm oversimplifying the effort involved here:

// keep pulling test classes off the queue until its empty
while(myTestQueue.isNotEmpty()) {
    String classFromQueue = myTestQueue.next(); //returns "org.myorg.foo.fooTests"
    LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
        .selectors(selectClass(classFromQueue)).build();
        
    SummaryGeneratingListener listener = new SummaryGeneratingListener();
    try (LauncherSession session = LauncherFactory.openSession()) {
        Launcher launcher = session.getLauncher();
        launcher.registerTestExecutionListeners(listener);
        TestPlan testPlan = launcher.discover(request);
        launcher.execute(testPlan);
    }   
    TestExecutionSummary summary = listener.getSummary();
    addSummary(summary);
}

Questions:

1. Will repeatedly discovering and executing in a while loop violate the normal test lifecycle? I'm a little fuzzy on whether discovery is a one time thing that's supposed to happen before all executions.
2. If I assume that it's ok to repeatedly discover then execute, I see the HierarchicalTestEngine may be an even better place to read from a queue since that seems to be used for implementing parallel execution. Is this more suitable for my use case? Would the implementation be essentially the same as what I have above except maybe I wouldn't need to handle accumulating test summaries?

Approaches I do not want to take: I am not looking to use the new features of junit 5 aimed at parallelizing test execution within the same jvm. I'm also not looking to divide the tests or classes up ahead of time; starting each console runnner instance with a pre-determined subset of tests.

Short Answer

The code posted in the question (loosely) illustrates a valid approach. There is no need to create a custom engine. Leveraging the platform launcher api to repeatedly discover and execute tests does work. I think it's worth highlighting that you do not have to extend junit5 This isn't executed through an extension that you need to register as I'd originally assumed. You're just simply leveraging the platform launcher api to discover and execute tests.

Long Answer

Here is some sample code with a simple queue of tests class names that exist on the class path. While the queue is not empty, an instance of the testNode class will discover and execute each of the three test classes and write a LegacyXmlReport.

TestNode Code:

package org.sample.node;
import org.junit.platform.launcher.Launcher;
import org.junit.platform.launcher.LauncherDiscoveryRequest;
import org.junit.platform.launcher.LauncherSession;
import org.junit.platform.launcher.TestPlan;
import org.junit.platform.launcher.core.LauncherConfig;
import org.junit.platform.launcher.core.LauncherDiscoveryRequestBuilder;
import org.junit.platform.launcher.core.LauncherFactory;
import org.junit.platform.launcher.listeners.SummaryGeneratingListener;
import org.junit.platform.reporting.legacy.xml.LegacyXmlReportGeneratingListener;

import java.io.FileNotFoundException;
import java.io.FileOutputStream;
import java.io.PrintWriter;
import java.nio.file.Paths;
import java.util.LinkedList;
import java.util.Queue;

import static org.junit.platform.engine.discovery.DiscoverySelectors.selectClass;
public class TestNode {

    public void run() throws FileNotFoundException {

        // keep pulling test classes off the queue until its empty
        Queue<String> queue = getQueue();
        while(!queue.isEmpty()) {
            String testClass = queue.poll();
            LauncherDiscoveryRequest request = LauncherDiscoveryRequestBuilder.request()
                    .selectors(selectClass(testClass)).build();


            LauncherConfig launcherConfig = LauncherConfig.builder()
                    .addTestExecutionListeners(new LegacyXmlReportGeneratingListener(Paths.get("target"), new PrintWriter(new FileOutputStream("log.txt"))))
                    .build();

            SummaryGeneratingListener listener = new SummaryGeneratingListener();
            try (LauncherSession session = LauncherFactory.openSession(launcherConfig)) {
                Launcher launcher = session.getLauncher();
                launcher.registerTestExecutionListeners(listener);
                TestPlan testPlan = launcher.discover(request);
                launcher.execute(testPlan);
            }
        }
    }

    private Queue<String> getQueue(){
        Queue<String> queue = new LinkedList<>();
        queue.add("org.sample.tests.Class1");
        queue.add("org.sample.tests.Class2");
        queue.add("org.sample.tests.Class3");
        return queue;
    }

    public static void main(String[] args) throws FileNotFoundException {
        TestNode node = new TestNode();
        node.run();
    }
}

Tests executed by TestNode

I'm just showing 1 of the three test classes since they're all the same thing with different class names. They reside in src/main/java and NOT src/test/java. This is an admittedly weird yet common pattern in maven for packaging tests into a fat jar.

package org.sample.tests;

import org.junit.jupiter.api.Test;

public class Class1 {
    @Test
    void test1() {
        System.out.println("Class1 Test 1");
    }

    @Test
    void test2() {
        System.out.println("Class1 Test 2");
    }

    @Test
    void test3() {
        System.out.println("Class1 Test 3");
    }
}