javajava-9java-platform-module-systemjmod

Why did Java 9 introduce the JMOD file format?

Java 9 has three ways to package compiled code in files:

JIMAGE is optimized for speed and space and used by the JVM at runtime so it makes sense why JIMAGE was introduced. JIMAGE files are not supposed to be published to maven repos or used at compile or link time.

The docs claim that JMOD can store native code and other things that can't be stored by JAR files and that developers can make and distribute their own JMOD files. The JDK ships with jmods/ directory containing all the modules of the JDK for users to depend on.

Questions:

Solution

  • The purpose of JMODs are not well documented and existing documentation is rather sparse. Here is an in-depth explanation of system, from my understanding.

    Parts of this answer are rather long, verbose, partially redundant, and a tough read. Constructive, structural, or grammatical edits are more than welcome to improve readability for future readers.

    Short(er) Answer

    The Java Platform Module System (Project Jigsaw) in Java 9+ introduces the notion of a new optional link time phase. This phase occurs when using the CLI tool jlink (JEP 282) to build a custom space-optimized JRE.

    The jlink utility bundles all explicit/transitive JAR modules/JMOD dependencies into a minified JRE. All other unreachable dependencies in the dependency graph (starting from specified root modules) are omitted from the built JRE. As of JDK 9+, all of Java's standard library has been broken up into JMODs. These JMODs are located at <jdk>/jmods.

    Whereas JARs can only contain .class and resource files, JMODs (i.e. .jmod files) contain additional files that are consumed specifically in the new optional link time phase to customize the JRE (e.g. executables, native libraries, configurations, legal licenses, etc). These additional files are not available as resources at run time in the classpath, but are instead installed under various locations in the built JRE (e.g. executables and native libraries are placed under <jre>/bin). From the relevant bundled JARs and JMODs dependencies, classes and file resources will be written into a single optimized JIMAGE file, located at <jre>/lib/modules (replacing <jre>/lib/rt.jar in Java 8 and prior versions). The role of JMODs is at compile time and link time, and are not designed to be used at run time.

    For the average library/application, only JARs should be built and pushed, instead of JMODs; only under certain conditions will JMODs offer critical functionality that is needed during the link time phase. At the time of writing, Maven does not appear to offer strong support for JMODs beyond the alpha release plugin org.apache.maven.plugins:maven-jmod-plugin.

    Long Answer

    This long-winded answer is more complexly motivated and sheds some light into how the new module system fundamentally operates. There is a strong emphasis throughout this post on the CLI tool jlink, since JMODs are designed specifically for this new optional link time phase that the tool introduces.

    The Introduction of Project Jigsaw

    Java 9 introduced Project Jigsaw in 'JEP 261: Module System', a novel module system that can be used to minimize startup times and the size of JREs. As part of this release, the CLI utilities jmod, jimage, and jlink were introduced along with new file formats for JMODs/.jmods (ZIP-based) and JIMAGEs/.jimages.

    A significant takeaway of this new module system is that the CLI tool jlink enables developers to build a custom JRE that contains only relevant standard library and external dependencies for their applications. This introduces a new notion of an optional link time phase between the traditional phases in the compile time -> run time pipeline.

    For an example of the advantages of using jlink, a minimalist JRE built from JDK 15 with only the java.base module comes out to roughly ~40MB in size, in stark juxtaposition to JDK 15's ~310MB size. This is especially useful for shipping a minimal custom JRE, such as for lean Docker images. The new module system brings significant benefits to the Java ecosystem that have been discussed at length elsewhere, and are thus not further elaborated in detail here.

    The 3 J's: JARs, JMODs, and JIMAGEs

    The high level description of a JARs, JMODs, and JIMAGEs do not quickly lend themselves to an explanation that strongly differentiates between the roles of the three file formats. Here is a non-exhaustive overview of the purposes of each:

    Note: There may be a way to add easily JMODs to the classpath at run time; however, research did not explicitly state any functionality relating to this. Merely adding a JMOD to the classpath will not be sufficient for using the classes and resources. A custom ClassLoader could be used to resolve class and resource files correctly in the JMOD archive at run time, however; this is generally not recommended and is not the purpose of JMODs.

    The Substance: Detailed Purpose of JMOD

    A New, Optional Link Time Phase

    As stated a few times previously, the CLI tool jlink introduces a new optional stage in the normal Java pipeline - the link time phase. This link time phase is used to generate a custom built JRE from a set of Java 9 modules (either a JAR with a module-info.java descriptor or a JMOD).

    The high level stages are briefly described as follows:

    Introduction of JMODs

    During the link time phase, all classes and resources from modules (valid JAR modules or form JMODs' classes) are compiled into a single optimized JIMAGE runtime image located at <jre>/lib/modules. Modules not explicitly or transitively included will not be included into this final JIMAGE, saving a significant amount of space. However, when building a custom JRE, some additional files might be necessary inside of the JRE; e.g. executable commands or native libraries. For JAR modules, the story ends here - this is no way for a JAR to add files (beyond the classes included in the JIMAGE) into the built JRE without ambiguities.

    Introducing JMODs: JMODs have the ability to add additional files into the custom built JRE; some examples (but not necessarily exhaustive): executable commands, configuration files, header files, legal notices and licenses, native libraries, and manual pages. This allows a module dependency to shape the built JRE in its own way. The behavior of how these additional files are inserted into the built JRE by the CLI tool jlink are documented within the next section.

    JMODs are destined for solely for the compile time and link time phases, as described in 'JEP 261: Module System':

    JMOD files can be used at compile time and link time, but not at run time. To support them at run time would require, in general, that we be prepared to extract and link native-code libraries on-the-fly. This is feasible on most platforms, though it can be very tricky, and we have not seen many use cases that require this capability, so for simplicity we have chosen to limit the utility of JMOD files in this release.

    The New Format - No Backwards Compatibility with JARs

    A good question might be "why not enable JARs to add link-time behavior?". A sneaking suspicion here is that this does not enable sufficient backwards-compatibility support with existing JARs and tooling. There is no specification for reserved filenames in the JAR archive file format. If an existing library stores any resources under the directories intended for link time, jlink could not accurately guess whether it is meant to be consumed during link time or needed at run time. A new file format specification with reserved directory names would resolve this clashing issue - such as the new JMOD format. With JMODs, there is no ambiguity about what resources are designated for link time and run time. Furthermore, the JMOD format can be also extended to add new functionalities in later JDK versions, without backwards-compatibility issues.

    The JMOD file format is similar to a JAR in that it is based on the ZIP file format. A JMOD file has the following reserved directory names with the following behavior (this is not necessarily an exhaustive list!):

    For the curiously inclined, standard library JMODs (located under $JAVA_HOME/jmods in a JDK 9+) can be inspected with any application that reads ZIP archives.

    Mainstream Support...?

    A significant part of the reason that JMODs have not been rapidly adopted and have poor documentation availability is that, quite simply put, they are not necessary for the vast majority of libraries and module dependencies. While they still are useful for specific use cases, modules should use the JAR format that already has mainstream support since it was defined with JDK 1.1 in 1997 (with module-info.java module support added with JDK 9 in 2017).

    From the documentation of the CLI tool jmod:

    For most development tasks, including deploying modules on the module path or publishing them to a Maven repository, continue to package modules in modular JAR files. The jmod tool is intended for modules that have native libraries or other configuration files or for modules that you intend to link, with the jlink tool, to a runtime image.

    An opinion: JMODs will likely not gain any significant adoption by developers for at least a very long time. Most developers will never hear or know the purpose of a JMOD - nor will they need to. JMODs serve a critical purpose behind the scenes for building JREs (all of the Java standard library modules are JMODs), but do not affect the vast majority of applications and projects due to their niche use case at link time. Java 9 was released in 2017 and dependencies in the Java ecosystem still struggle to reliably have a module-info.class descriptor to make a JAR a valid fully-fledged module...

    Takeaways

    Disclaimer

    At the time of writing this answer, there was sparse documentation on the purpose of JMODs for Java 9 and onward. In fact, the Google search phrases "java jmods" and "jmod format" bring this very same StackOverflow question as the second search hit result. Therefore, some aspects may not be accurately explained, but are generally "directionally correct"; furthermore, it may not paint the full picture. If you find any issues or caveats, leave a comment and I will try to reconcile it with this answer.