Aggregating Annotation Processors Across Repos

Mar 16, 2021 · 5 minute read · Comments
code
android

Note - A plethora of excellent articles and talks describing how to write an annotation processor exist - consequently, this blog post will not talk about the details on how to build an annotation processor. Instead, it addresses a specific case that I had a much more difficult time finding answers for online and a strategy for solving it.

Scenario

Suppose we are shipping an app with multiple libraries pulled from a company Maven repository. Let’s say we want to write an annotation processor that outputs a list of all classes annotated with a certain annotation. In other words, given:

// in repo1, artifact com.company.app:module1
@CompanyModule
class Foo : CommonModule

// in repo2, artifact com.company.app:module2
@CompanyModule
class Baz : CommonModule

// in the app repo
@CompanyModule
class Bin : CommonModule

// we want an implementation of this generated
@CompanyModuleRepository
interface CompanyRepository {
  val modules: List<CommonModule>
}

we want a class in our app module generated that looks like:

class CompanyRepositoryImpl : CompanyRepository {
   override val modules: List<CommonModule> = listOf(Foo(), Baz(), Bin())
}

Note - Uber has open sourced an annotation processor, Crumb, that handles this case painlessly.

Initial Attempts at a Picasso

Given this problem, let’s write an annotation processor to do this. Let’s take the obvious approach first, one in which we get all classes annotated with @CompanyModuleRepository and all the classes annotated with @CompanyModule. Using this information, we’ll write code to generate the CompanyRepositoryImpl class.

If we do this, we’ll find that the only modules we’ll be able to pick up are the ones in app - so in the above case, the Bin module is the only one that will be added.

Roadblocks and Imperfect Frescos

If we debug our annotation processor code, we’ll find that a line that looks for all the CompanyModules -

val modules = roundEnv.getElementsAnnotatedWith(CompanyModule::class.java)

Only finds the single module, Bin, within our app module. If we add another module in app, it will also be found, but all our modules from our artifacts won’t be found.

Why? When the annotation processor runs, it will run against a particular module. In this case, we’re running the annotation processor against our app module. It will find any annotated classes in app with no problem.

On the other hand, our dependencies, module1 and module2, are coming from a Maven repository as binary dependencies - i.e. they’re precompiled. Consequently, the annotation processor will not run on them at this point, since it’s too late for that. This is true even if the RETENTION on the annotations is properly set to AnnotationRetention.BINARY as it should be in this case. To solve this, we’ll need something to run while module1 and module2 are compiling, and take this result into consideration while compiling app at the end.

Gliding to Completion

After a lot of digging, I found people pointing to Glide as one of the canonical reference annotation processors that people look at while writing their own annotation processors. Reading their code and stepping through it with a sample project, here’s a summary of what Glide does:

For each LibraryGlideModule or GlideExtension (CompanyModule in our example), an Indexer class is generated in a consistent package (irrespective of the aforementioned annotated class’s package) with an @Index annotation and the fully qualified path of the original annotated class. In Glide’s case, all of these generated indices are written to com.bumptech.glide.annotation.compiler. This output generated Indexer class looks something like this:

@Index(modules = "com.bumptech.glide.LibraryGlideModule")
public class Indexer_GlideModule_com_bumptech_glide_LibraryGlideModule

For each AppGlideModule (Glide restricts these to only 1 - this is our CompanyModuleRepository), the processor looks for all files in the aforementioned directory (com.bumptech.glide.annotation.compiler), filters out only the ones with an @Index annotation, and uses those to generate the list of modules.

val glideGenPackage =     processingEnv.elementUtils.getPackageElement(COMPILER_PACKAGE_NAME);

The full code for this can be seen here.

Putting the Coils in Place

Knowing how Glide does this, we can apply the same strategy to our problem. In our case, we’ll generate an @Index annotated class for each @CompanyModule and we’ll write that to a common directory - com.company.generated.module for example.

Using the beginning example, our modules that will be pushed to Maven will look something like this:

module1.jar
- com/company/app/module1/foo/Foo.class
- com/company/generated/module/Indexer_Foo.class

module2.aar
- classes.jar
    com/company/app/module2/baz/Baz.class
    com/company/generated/module/Indexer_Baz.class

When the annotation processor is run against App, it can then find all files present in com.company.generated.module, check which ones properly have the @Index annotation (and read the full path from them), and use that information to build the list.

One Last Potential Pitfall

Glide’s annotation processor is outputting Java code using JavaPoet. If, instead, we decide to generate Kotlin code using KotlinPoet, there’s one more gotcha we need to look out for.

Consider the case where we have a single module with 1 class - running our code, we might find nothing generated, and re-running it, something might be generated. Huh? Why? The point to remember here is that kapt does not process newly generated Kotlin sources across multiple rounds (see bug here). The workaround for this is to generate our Indexer classes using JavaPoet instead of KotlinPoet (see this KotlinPoet issue for some useful extension functions for making this easier).

Special thanks to Efeturi for reviewing this post.

helw.net