Question
What is the proper way to create a Java annotation processor, which makes use of annotations that it itself generates?
Context
I'm looking at annotation processing as a means of generating repetitive/boilerplate code and currently in my crosshair are annotations that use an enum. From what I understand only enums which are explicitly referenced can be used, however I'd like to be able to use any client Enum (thus not something that is known to the annotation processor at its compile time).
public @interface GenericEnumAnnotation() {
Enum<?> value();
}
doesn't work, rather this has to be done as
public @interface MyEnumAnnotation() {
MyEnum value();
}
So code generation to the rescue! Rather than having the client create a custom annotation for each Enum, I have it setup to generate this annotation based on a @GenerateAnnotation annotation. Thus
@GenerateAnnotation
public enum MyEnum {...}
will generate the valid MyEnumAnnotation
@EnumAnnotation
public @interface MyEnumAnnotation() {
MyEnum value();
}
Client code can then make use of the generated @MyEnumAnnotation. Now that the enum is generated, I want to now use this @MyEnumAnnotation to generate some additional code for client code that is annotated with it. The newly generated annotation becomes available in the second pass of the annotation processor, and thanks to the @EnumAnnotation I can tell that this is the annotation that I want to use for code generation, however when I make the attempt no usages are found.
@SupportedAnnotationTypes("com.company.generator.EnumAnnotation")
@AutoService(Processor.class)
public class EnumAnnotationProcessor extends AbstractProcessor {
@Override
public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment env) {
annotations.forEach(enumAnnotation -> { //@EnumAnnotation
env.getElementsAnnotatedWith(enumAnnotation).forEach(customAnnontation -> { //@MyEnumAnnotation
env.getElementsAnnotatedWith(customAnnotation -> { // Elements using the @MyEnumAnnotation
// Never entered - nothing annotated is found
});
});
});
}
}
From experimentation I've determined that this is due to the second pass only looking at the "new files" rather than the full scope/scale of the classes. The client code (which uses the annotation) is only processed during the initial pass and as such it is no longer searchable/accessible in the second pass when the annotation processor actually knows of this generated annotation.
The only method that I have found that allows me to go back and "reprocess" the original file set is by means of a separate processor which just purely holds on to the environment from the first pass, and using it rather than the environment from subsequent passes.
@SupportedAnnotationTypes("*")
@SupportedSourceVersion(SourceVersion.RELEASE_21)
@AutoService(Processor.class)
public class FirstPassCollector extends AbstractProcessor {
public static RoundEnvironment firstPassEnvironment = null;
@Override
public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment roundEnv) {
if (firstPassEnvironment == null)
FirstPassCollector.firstPassEnvironment = roundEnv;
return false;
}
}
@SupportedAnnotationTypes("com.company.generator.EnumAnnotation")
@AutoService(Processor.class)
public class EnumAnnotationProcessor extends AbstractProcessor {
@Override
public boolean process(Set<? extends TypeElement> annotations, RoundEnvironment env) {
annotations.forEach(enumAnnotation -> {
env.getElementsAnnotatedWith(enumAnnotation).forEach(customAnnontation -> {
FirstPassCollector.firstPassEnvironment.getElementsAnnotatedWith(customAnnotation -> {
// Now searching the files from the first pass, and annotated classes are now found!
});
});
});
}
}
I know there are deficiencies in the code as written (i.e.: no null check on the firstPassEnvironment when using it), however as a concept this is something that works, but feels like a rather brittle/hacked solution. Is there a better way of accomplishing this end goal?
I would be excited to tackle the question:
What is the proper way to create a Java annotation processor, which makes use of annotations that it itself generates?
This particular answer however addresses the following (slightly changed) question:
How to create a Java annotation processor, which makes use of annotations that itself generates?
The difference is that I am not 100% certain (or able to guarantee) the "proper" part.
Note though that the original problem you are presenting seems to be another one:
I'd like to be able to use any client Enum
...which I will give a shot at another answer here later.
By the problem statement itself one can devise the following sequence of steps needed to fulfil (in order):
enums.That's only two steps (hence the name of the processor in the implementation below). One could extend this concept to more steps, for example if in step 2 the generated code needs further processing, such as when step 2 would generate more annotations (and so on until as many steps as needed are complete). This answer addresses this only-two step concept.
Long story short, one more detailed sequence of steps for solving the problem is the following:
enums which we want to generate annotations for.enums found at step 1....which is intended to be repeated per round and I am going to use it as reference in the answer parts which follow.
For step 1, we can just create an annotation which will annotate each enum the user wants to fit into the category. We will then find these enums through the getElementsAnnotatedWith method of RoundEnvironment (on each round). This idea is taken from your GenerateAnnotation (and is named as GenerateEnumAnnotation in this answer).
For step 2, we should use the Filer to generate the annotations. We want to create them through the Filer so that they are taken into account for the compilation.
Then, according to the documentation of Processor:
On each round, a processor may be asked to process a subset of the annotations found on the source and class files produced by a prior round.
as well as also according to the documentation of Filer:
This interface supports the creation of new files by an annotation processor. ... Source and class files so created will be considered for processing by the tool in a subsequent round of processing after the close method has been called on the Writer or OutputStream used to write the contents of the file.
So when we generate source files (which includes generated annotations) through the Filer interface, we get their corresponding Elements as the root ones in subsequent rounds. We can then use these to find which other Elements seen so far are annotated with them (fulfilling step 3).
Step 4 now is ready to generate the code you are requiring, since we now have each enum annotation at our disposal from the 3rd step.
Annotation to generate the enum annotations:
package annotations.soq78648395;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.SOURCE)
public @interface GenerateEnumAnnotation {
}
Processor:
package annotations.soq78648395;
import java.io.IOException;
import java.io.PrintStream;
import java.util.HashMap;
import java.util.HashSet;
import java.util.Map;
import java.util.Objects;
import java.util.Set;
import javax.annotation.processing.AbstractProcessor;
import javax.annotation.processing.Messager;
import javax.annotation.processing.ProcessingEnvironment;
import javax.annotation.processing.RoundEnvironment;
import javax.annotation.processing.SupportedAnnotationTypes;
import javax.annotation.processing.SupportedSourceVersion;
import javax.lang.model.SourceVersion;
import javax.lang.model.element.AnnotationMirror;
import javax.lang.model.element.Element;
import javax.lang.model.element.ElementKind;
import javax.lang.model.element.TypeElement;
import javax.lang.model.util.Elements;
import javax.tools.Diagnostic;
import javax.tools.JavaFileObject;
@SupportedSourceVersion(SourceVersion.RELEASE_8)
@SupportedAnnotationTypes("annotations.soq78648395.GenerateEnumAnnotation")
public class TwoStepEnumAnnotationProcessor extends AbstractProcessor {
/**
* We need an {@code Element} representation which can be independent of the round. That's
* because {@code Element} information is populated as new rounds are coming (for example for
* the generated annotations), so we need to reload {@code Element}s on every round.
*/
private static final class InterRoundElement {
//All public final to avoid getters and setters in order to save space for the answer post itself.
public final String simpleName, packageName, qualifiedName;
public InterRoundElement(final Elements elementUtils,
final Element element) {
this(element.getSimpleName().toString(), elementUtils.getPackageOf(element).getQualifiedName().toString());
}
public InterRoundElement(final String simpleName,
final String packageName) {
this.simpleName = simpleName;
this.packageName = packageName;
qualifiedName = packageName.isEmpty()? simpleName: (packageName + '.' + simpleName);
}
@Override
public String toString() {
return qualifiedName;
}
@Override
public int hashCode() {
return Objects.hashCode(qualifiedName);
}
@Override
public boolean equals(final Object obj) {
if (this == obj)
return true;
if (obj == null || getClass() != obj.getClass())
return false;
return Objects.equals(qualifiedName, ((InterRoundElement) obj).qualifiedName);
}
}
private boolean isAnnotatedWith(final AnnotationMirror annotationMirror,
final TypeElement annotation) {
final TypeElement other = (TypeElement) annotationMirror.getAnnotationType().asElement();
//Note here: 'other.getKind()' may actually be 'CLASS' rather than 'ANNOTATION_TYPE' (it happens for annotations generated by annotation processing).
return Objects.equals(annotation.getQualifiedName().toString(), other.getQualifiedName().toString());
}
/**
* As <i>early elements</i> are named the {@code Element}s which are potentially annotated with
* an enum annotation which is going to be generated. For example root elements of the first
* round will not appear again in the following rounds, but they may be already annotated with
* an enum annotation which is not yet generated, so we need to maintain them until we find out
* what happens.
*/
private final Set<InterRoundElement> earlyElements = new HashSet<>();
/**
* A {@code Map} from generated enum annotations to the {@code Element}s being annotated with
* them. If an enum annotation is registered as a key of this map, then its code is already
* generated even if no {@code Elements} are found to be annotated with it (ie for empty map
* value).
*/
private final Map<InterRoundElement, Set<InterRoundElement>> processedElements = new HashMap<>();
/**
* Just a zero based index of the processing round.
*/
private int roundSerial = -2;
/**
* For debugging messages.
* @param tokens
*/
private void debug(final Object... tokens) {
System.out.print(String.format(">>>> [Round %2d]", roundSerial));
for (final Object token: tokens) {
System.out.print(' ');
System.out.print(token);
}
System.out.println();
}
/**
* Opens a {@code PrintStream} for writing/generating code.
* @param interRoundElement
* @param originatingElements
* @return
* @throws IOException
*/
private PrintStream create(final InterRoundElement interRoundElement,
final Element... originatingElements) throws IOException {
debug("Will generate output for", interRoundElement);
final JavaFileObject outputFileObject = processingEnv.getFiler().createSourceFile(interRoundElement.qualifiedName, originatingElements);
return new PrintStream(outputFileObject.openOutputStream());
}
/**
* Generates an enum annotation.
* @param origin
* @param output
* @param originatingElements
* @return {@code true} for success, otherwise {@code false}.
*/
private boolean generateEnumAnnotation(final InterRoundElement origin,
final InterRoundElement output,
final Element... originatingElements) {
try (final PrintStream outputFileOutput = create(output, originatingElements)) {
if (!output.packageName.isEmpty()) { //The default package is represented as an empty 'packageName'.
outputFileOutput.println("package " + output.packageName + ";");
outputFileOutput.println();
}
for (final Object line: new Object[]{ //We obviously here need to utilize text blocks of the newer Java versions...
"@java.lang.annotation.Target(java.lang.annotation.ElementType.TYPE)",
"@java.lang.annotation.Retention(java.lang.annotation.RetentionPolicy.SOURCE)",
"public @interface " + output.simpleName + " {",
" " + origin.qualifiedName + " value();",
"}"
})
outputFileOutput.println(line);
return true;
}
catch (final IOException ioe) {
ioe.printStackTrace(System.out);
return false;
}
}
private void reset() {
processedElements.clear();
earlyElements.clear();
roundSerial = -1;
}
@Override
public synchronized void init(final ProcessingEnvironment processingEnv) {
super.init(processingEnv);
if (super.isInitialized())
reset(); //Initialize, prepare for new processes.
}
/**
* This method handles elements annotated with {@code GenerateEnumAnnotation}.
* @param annotatedElement
*/
private void handleGenerationAnnotation(final Element annotatedElement) {
final Messager messager = processingEnv.getMessager();
if (annotatedElement.getKind() != ElementKind.ENUM)
messager.printMessage(Diagnostic.Kind.ERROR, "Only enums are supported.", annotatedElement);
else {
final InterRoundElement origin = new InterRoundElement(processingEnv.getElementUtils(), (TypeElement) annotatedElement);
final String pack = (origin.packageName.isEmpty()? "": (origin.packageName + '.')) + "enum_annotations";
final InterRoundElement output = new InterRoundElement(origin.simpleName + "Annotation", pack); //The enum annotation element to generate...
if (generateEnumAnnotation(origin, output, annotatedElement))
processedElements.computeIfAbsent(output, dejaVu -> new HashSet<>()); //Store the generated enum annotation information.
else
messager.printMessage(Diagnostic.Kind.ERROR, "Failed to create annotation " + output + " (for " + origin + ").", annotatedElement);
}
}
/**
* Handles {@code Element}s annotated with a generated enum annotation. Modify this according to
* your requirements. It assumes that generated enum annotations are not repeatable (otherwise
* it should take a {@code Collection} of {@code AnnotationMirror}s instead of a single one).
* @param enumAnnotation The generated enum annotation.
* @param annotatedElement The {@code Element} annotated with {@code enumAnnotation}.
* @param annotationMirror The mirror of annotating {@code annotatedElement} with {@code enumAnnotation}.
*/
private void handleEnumAnnotation(final TypeElement enumAnnotation,
final Element annotatedElement,
final AnnotationMirror annotationMirror) {
//The current implementation just prints some messages of the annotation we've found...
debug("Processing", annotatedElement.getKind(), "element", annotatedElement);
debug(" Annotated by enum annotation", enumAnnotation);
debug(" With the following applicable mirror:", annotationMirror);
}
/**
* Find out if any "early elements" are annotated with a generated enum annotation, and handle them.
*/
private void processEarlyElements() {
final Elements elementUtils = processingEnv.getElementUtils();
//We need a defensive shallow copy of 'earlyElements' because it is going to be modified inside the loop:
final Set<InterRoundElement> defensiveCopiedEarlyElements = new HashSet<>(earlyElements);
processedElements.forEach((annotationInterRoundElement, annotatedElements) -> {
final TypeElement generatedEnumAnnotation = elementUtils.getTypeElement(annotationInterRoundElement.qualifiedName);
//'roundEnv.getElementsAnnotatedWith(generatedEnumAnnotation)' will actually return an empty List here, so we have to find elements annotated with 'generatedEnumAnnotation' via its annotation mirrors...
defensiveCopiedEarlyElements.forEach(interRoundElement -> {
final TypeElement annotatedElement = elementUtils.getTypeElement(interRoundElement.qualifiedName); //Reload annotated element for the current round (ie don't rely on its previous Element occurences), because its annotations may not yet be ready.
//The following code assumes generated enum annotations are not repeatable...
annotatedElement.getAnnotationMirrors().stream()
.filter(annotationMirror -> isAnnotatedWith(annotationMirror, generatedEnumAnnotation)) //Continue only for mirrors of type generatedEnumAnnotation.
.filter(annotationMirror -> annotatedElements.add(interRoundElement)) //If we've seen the early element before then skip it, otherwise add it to annotatedElements and process it...
.findAny()
.ifPresent(annotationMirror -> {
earlyElements.remove(interRoundElement); //No need to store it any more.
handleEnumAnnotation(generatedEnumAnnotation, annotatedElement, annotationMirror);
});
});
});
}
@Override
public boolean process(final Set<? extends TypeElement> annotations,
final RoundEnvironment roundEnv) {
++roundSerial;
final Elements elementUtils = processingEnv.getElementUtils();
final Set<? extends Element> rootElements = roundEnv.getRootElements();
//Store only root elements of type class in 'earlyElements':
rootElements.stream()
.filter(rootElement -> rootElement.getKind() == ElementKind.CLASS)
.map(rootElement -> new InterRoundElement(elementUtils, rootElement))
.forEachOrdered(earlyElements::add);
debug("Annotations:", annotations);
debug("Root elements:", rootElements);
/*First process early elements and then generate enum annotations. The sequence of these two
calls is ought to how their methods' body is implemented, and we know we won't loose any
enum annotations because any generated enum annotations in the current round will be
supplied as root elements in the next round and we already add "early elements" in the
beginning of the processor's 'process' method.*/
processEarlyElements();
roundEnv.getElementsAnnotatedWith(GenerateEnumAnnotation.class).forEach(this::handleGenerationAnnotation);
debug("Current early elements:", earlyElements);
processedElements.forEach((annotation, elements) -> debug("Created annotation", annotation, "with the following elements processed for it so far:", elements));
//Cleanup, prepare for later processes (if reused):
if (roundEnv.processingOver())
reset();
/*GenerateEnumAnnotations are always consumed. Even if there is a failure in generating the
resulting code from a GenerateEnumAnnotation annotated element, this doesn't mean that
repeating the handleGenerationAnnotation has the potential in succeeding in later rounds,
so we are not repeating the attempt (ie handling is finished, errors are handled, we move on).*/
return true;
}
}
On the user's code, annotate the enums you need with GenerateEnumAnnotation:
package soq78648395;
import annotations.soq78648395.GenerateEnumAnnotation;
@GenerateEnumAnnotation
public enum MyEnum {
MY_A, MY_B;
}
...and then use the generated enum annotation like so:
package soq78648395;
import soq78648395.enum_annotations.MyEnumAnnotation;
@MyEnumAnnotation(MyEnum.MY_A)
public class MyClass {
}
enums are supported to be annotated with GenerateEnumAnnotation.TypeElement kinds through a simple condition check, but for all other Element kinds more modifications are required (depends on the kinds you would want).handleEnumAnnotation method) to find out if they are annotated with more than one generated enum annotation, but only if you generate all the enum annotations at the same round. For more rounds, you would need to extend the implementation. I think this should potentially also solve the case of the generated enum annotations being repeatable."*") and then claiming only the ones we generate (along with GenerateEnumAnnotation of course). The only thing I think that enables us to verify the correctness of the implementation of this answer (as far as the implicit claiming of generated annotations goes) is whether my first assumption holds true. If you go down the supporting any annotation path then try to be careful not to claim annotations other than the ones you really generate and know (eg through maintaining a data structure between rounds with these annotations).According to my experimental observations, the following steps are how processing happens (in the case of the code in this answer at least):
RoundEnvironment and nothing more. This includes the fact that MyClass is annotated with MyEnumAnnotation, but its corresponding AnnotationMirror currently indicates the annotation type is of kind CLASS, rather than ANNOTATION_TYPE and it seems like there is more information missing (eg you can't obtain its parameters), which should be normal judging from the fact that the code for MyEnumAnnotation is not yet generated and processed. In this same round we are fully generating enum annotations (ie MyEnumAnnotation) (that is, we create the source file and close it's OutputStream, thus making the code available to the compiler tool).Elements (suggesting that the compiler has now processed the representation of the generated annotations from the previous round, so that they are now reachable as Elements). We are not getting MyClass in root elements in the second and subsequent rounds, but they are reachable through the RoundEnvironment (we need to find them, eg with elementUtils.getTypeElement(...) method). At this point we know enum annotation Elements so one could say we should just use roundEnv.getElementsAnnotatedWith(myEnumAnnotationElement);, but according to my testing this just returns an empty Set here (maybe going with @SupportedAnnotationTypes("*") could fix the issue, but I haven't tested it). Another option to find AnnotationMirrors would be to go the other way around, which I mean is to obtain the AnnotationMirrors of the annotated Element and find the ones of interest. This option is what I used, so, if we hadn't maintained a set of seen client Elements then we wouldn't know what to look for.Element information may become outdated as new rounds are coming (for example in the case of @MyEnumAnnotation's type). I don't know the whole annotation processing documentation by heart, but I can't find information if the above experimental results can be backed by the documentation (nor contradicted by it), so I assumed that Elements kept by reference between rounds may not be up to date with their latest internal state in upcoming rounds. Hence the need for an Element representation which can endure changing rounds. Obviously, in this context, it is not practical to maintain all the information about an Element (because it's too much information to maintain, we don't need it all and finally it is expected to change between rounds), and the only information needed is an identifier of the Element to find it on each upcoming round. Since we are specifically looking into TypeElements then their qualified names seem enough for such identification and there is also the handy method elementUtils.getTypeElement to spare us from more involved searches. This identifying information is kept in InterRoundElements in the context of this answer.RoundEnvironment you are referring to as being reimplemented here. So, if possible, let me know if there is anything more you could clarify in your 4th comment.