Custom state-full rule not working with Volcano Planner after calcite upgrade from 1.21.0 to 1.32.0/1.34.0

public class StateFullRule extends RelOptRule {

    // *** some state
    //private boolean isProccessed = false;
    //private Map<RelNodeKey, Object> ancestorsMap;

    public StateFullRule() {
        super(operand(LogicalUnion.class, operand(LogicalProject.class, any()), new RelOptRuleOperand[]{operand(LogicalProject.class, any())}), "StateFullRule");
    }

    @Override
    public boolean matches(RelOptRuleCall call) {
            // Build intermediate state:subsetAncestorsMap using input call
            Boolean matches = determineIfRuleMatches();  // can return true or false
            if(matches) {
                //*** updateState
                //ancestorsMap = subsetAncestorsMap;
            }
            return matches;
    }

    private Boolean determineIfRuleMatches() {
        // using intermediate and actual state determine if the rule matches.
        return false;
    }
    

    public void onMatch(RelOptRuleCall call)   {
        // Transformation logic using state.
    }
}

The above rule fails with AssertionError: null whenever the matches function returns false. here's the stacktrace

java.lang.AssertionError: null
    at org.apache.calcite.plan.volcano.IterativeRuleDriver.drive(IterativeRuleDriver.java:57) ~[calcite-core-1.34.0.jar:1.34.0]
    at org.apache.calcite.plan.volcano.VolcanoPlanner.findBestExp(VolcanoPlanner.java:523) ~[calcite-core-1.34.0.jar:1.34.0]

Is it like , it is no longer allowed to return false from Calcite Rule now? Since I could see the assert over matches call in IterativeRuleDriver.java as well as TopDownRuleDriver.java.

I have also tried implementing the TransformationRule interface but no luck. Note the same rule was working calcite version 1.21.0 as there were no such assertions back then.

Calcite doesn't support stateful rules. If you are going to make rules stateful, Calcite can't stop you, but must make those rules comply with Calcite's assumptions. One of which is that whether a rule matches depends only on its RelNode arguments.

I hesitate to recommend workarounds, but maybe you can change your rule so that matches() returns true until the onMatch(RelOptRuleCall) method has been called.