Jars and JDK modules

The classpath

init! takes a :classpath of jars and directories, searched in order like java -cp. With no :classpath it uses CHAMPACA_CLASSPATH's entries, which an app bundle's launcher sets, so a bundled program needn't know where its jars went.

A class in a package one of the JDK's modules owns is never taken from the classpath, as on the JVM, so an old jar carrying its own copy of javax.xml classes gets the JDK's (test/java/ShadowCheck.java).

(ldk/init! {:classpath ["target/lib.jar"]})
(ldk/invoke-static "champaca.lib.Shelf" "sum" [1 2 3 4])   ;=> 10

(target/lib.jar is the small test library jolt test builds from test/jar. examples/jars.clj uses it.)

A directory holds .class files in package folders, the way javac -d writes them, and a jar is read in place without unpacking. Classes load lazily. Each one is read, parsed and registered the first time something names it, and each method compiles the first time it's called.

Pass the whole classpath on the first init!, because a second call does nothing and can't add jars later.

Classes compiled with javac --enable-preview that use one of JDK 25's preview features need :preview true on that first call, as java needs --enable-preview: (ldk/init! {:classpath ["classes"] :preview true}). Without it, HotSpot's UnsupportedClassVersionError refuses such a class the first time something names it. The option also turns on the JDK's own preview features, such as structured concurrency and stable values. Running Java programs has the version rules, and examples/preview.clj loads one.

A module outside the JDK's default module graph, such as the Vector API's jdk.incubator.vector, needs :add-modules on that first call, as java needs --add-modules. (ldk/init! {:add-modules ["jdk.incubator.vector"]}) puts it in the boot layer (examples/vector.clj); without it, a call naming one of the module's classes throws NoClassDefFoundError.

With the module in, lane-wise arithmetic, compares, masks, shuffles and reductions give HotSpot's results. So do the operations that change a vector's shape or lane type (castShape, reinterpretAs*, test(IS_NAN)) and loads and stores through a MemorySegment. None of it is accelerated, though (limitations).

A real library

Twenty-three libraries have been run this way and checked against the JVM, with identical output:

LibraryWhat was exercised
commons-lang3 3.12.0StringUtils
commons-lang 2.4SerializationUtils round trips and a failed one, StringUtils, WordUtils, DurationFormatUtils; built for Java 1.2, so its finally blocks are jsr/ret subroutines
commons-compress 1.27.1a tar archive over gzip written and read back, bzip2 both ways, a zip entry with a UTF-8 name, entry sizes, times and CRCs
commons-io 2.17.0FileUtils reading, writing, copying (which keeps the file's times, through setattrlist on macOS), listing with filters and deleting a tree, FilenameUtils, IOUtils, BOM detection, counting and tee streams
gson 2.13.2parsing, toJson of a tree, a plain object and a Map, fromJson into a class with no default constructor, pretty printing
jsoup 1.18.1parsing HTML, CSS selectors, absolute URLs, Jsoup.clean with a safelist
Jackson 2.20readTree, writeValueAsString of maps and annotated classes, @JsonProperty, @JsonIgnore, readValue into a bean with a List<Point> property, indented output
Guice 7.0.0constructor, field and provider injection, a @Named constant, a singleton scope, a Multibinder set, a @Provides method, a missing binding's error, the injector's bindings
Joda-Time 2.13.0a daylight-saving gap in America/New_York from the jar's own zone data, the next transition, formatting in another zone and locale, ISO parsing, periods and their wording, day counts, intervals
Kryo 5.6.2registered classes, a record, an enum and a map with an int[] written and read back through Unsafe, Objenesis and ReflectASM, the exact bytes, a deep copy
Jackson XML 2.18.1 with Woodstox 7.0.0an annotated bean written as XML with an attribute, a wrapped list and escaped text, read back, a tree, pretty printing, the StAX factory ServiceLoader finds in Woodstox's jar
Log4j 2.24.3a configuration built in code, plugins found from its jar, a pattern layout with level, logger, thread, marker and thread-context fields, an exception's message, a logger kept quiet by level, a second thread
Netty 4.1.115an NIO event loop running a line-based echo server and client over the loopback, UTF-8 frames, pooled direct and composite buffers
Bouncy Castle 1.79a signed jar as a JCE provider: SHA-3 and BLAKE2 digests, AES-GCM with associated data, ECDSA keys and a signature the JDK's own SunEC verifies, HMAC, PBKDF2
Commons Math 3.6.1LU, eigenvalue and singular value decompositions, descriptive statistics, a regression, normal, t and gamma distributions, a root solver, an FFT, Mersenne Twister, big fractions
Lucene 10.0.0an index written to disk and searched through MMapDirectory (memory segments), term, phrase, Boolean, wildcard, fuzzy and point range queries, sorting by doc values
Caffeine 3.2.2size, weight, write- and access-based eviction against a fake clock, removal listeners, refresh, getAll, async and loading caches, stats
Guava 33.4.8CRC32C, CRC32, Adler-32, Murmur3, FarmHash, SHA-256, SHA-512 and MD5 hashing, a Bloom filter, immutable multimaps, Joiner, Splitter
SLF4J 2.0.16 with slf4j-simplelogging through the provider ServiceLoader finds in slf4j-simple's jar
Byte Buddy 1.17.5subclasses generated at run time and loaded by each strategy (a wrapper and a child-first loader, Unsafe injection into the application loader, Lookup.defineClass), fixed values, delegation to a static method, interception with @SuperCall, a field accessor implementing Supplier, a generated toString
H2 2.3.232an in-memory database through JDBC: tables with keys, a batch of prepared inserts with decimals, dates, arrays and JSON, joins, grouping, window functions, a recursive query, a rolled-back transaction, constraint and syntax errors, metadata, four threads querying at once
picocli 4.7.6annotated options and positional parameters with type conversion, split values, a map option, subcommands with a mixin and @ParentCommand, usage and version help, error messages, a command defined by an interface (a Proxy), parseArgs results
SnakeYAML 2.3loading and dumping maps, beans with generic List and Map properties, public fields and enums, anchors and merge keys, multi-document streams, block style, timestamps, error messages with their line and column marks

Five other JVM languages run in the same gate, each through its own launcher on a probe in test/libs. They are Clojure 1.12.4 (clojure.main on Clj.clj: protocols, records, multimethods, spec, refs, agents, futures, pprint, eval), Groovy 5.0.8 (GroovyMain on Grv.groovy, every call one of Groovy's own invokedynamic sites: closures, metaclasses, traits, @CompileStatic, JSON, XML), Scala 2.13.18 (MainGenericRunner, which compiles Scl.scala in memory with scalac and runs it), Kawa 3.1.1 (kawa.repl on Kwa.scm, Scheme compiled to bytecode form by form: records, syntax-rules, tail calls, call/cc, dynamic-wind, guard, Java interop), and ABCL 1.9.2 (org.armedbear.lisp.Main on Abcl.lisp: CLOS, conditions, restarts, compile, Java interop; its boot defines classes of the same name from many loaders). Each is a :run entry in libs.edn, a main class and its arguments, so javac has nothing to compile. From a fresh cache the five take about five minutes between them, mostly spent translating Clojure's and scalac's compilers for the first time.

jolt libs reruns all twenty-three and the five languages from their probes. Each runs on the JVM and through champaca, from a fresh cache, and the outputs and exit codes have to match (stderr too, for SLF4J, which logs there). jolt libs Gua Yml runs just those two, and an unknown name is an error.

The jars come from $HOME/.m2/repository, each checked against its SHA-1 pin in test/libs/libs.edn before use. jolt libs:fetch downloads missing jars from Maven Central and keeps one only when its SHA-1 matches. A library whose jars are still missing is skipped and named, and a skip fails the run unless you pass --allow-skip. A run that compares nothing fails either way. None of this is part of jolt test, which needs no network.

To call one of them from jolt, with the jars in ~/.m2:

(ldk/init! {:classpath ["/path/to/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.20.1/jackson-databind-2.20.1.jar"
                        "/path/to/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.20.1/jackson-core-2.20.1.jar"
                        "/path/to/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.20/jackson-annotations-2.20.jar"]})

(def mapper (ldk/construct "com.fasterxml.jackson.databind.ObjectMapper"))
(def json (ldk/invoke mapper "writeValueAsString" {:name "champaca" :tags ["a" "b"]}))
;=> "{\"name\":\"champaca\",\"tags\":[\"a\",\"b\"]}"
(ldk/->clj (ldk/invoke mapper "readValue" json (ldk/invoke-static "java.lang.Class" "forName" "java.util.Map")))
;=> {"name" "champaca", "tags" ["a" "b"]}

Champaca doesn't resolve Maven coordinates, so you list a library's dependencies too, as with Jackson's three jars here. The exception is a jar that names them in its manifest's Class-Path, which is followed as on the JVM.

Resources

A jar's other files are reachable the way Java code expects. getResourceAsStream reads them, and getResource gives a jar: URL that opens like any other. For a directory on the classpath it's a file: URL instead; there a directory is a resource too, and "" is the entry's root, as on the JVM.

ServiceLoader finds the providers listed under META-INF/services, which is how SLF4J 2 finds its logging backend. Without that, SLF4J prints "No SLF4J providers were found" and logs nothing. test/java/JarCheck.java loads a service, a properties file and a URL from test/jar.

A class loader of your program's own, such as a URLClassLoader over a jar a plugin system found, answers getResource, getResourceAsStream and getResources as the JDK does: its parent first, then its own findResource. A class it defines finds its resources through it, and has the getProtectionDomain() the loader gave defineClass. A classpath class's code source is its entry's URL, a jar's or a directory's. Quarkus's launcher and Spring Boot find their own jar that way (test/java/ResourceCheck.java).

Resources inside the JDK's own modules, such as the ICU data the JDK uses for Unicode normalization, can be read as streams but have no URL.

JDK modules

The JDK's own classes come from $JAVA_HOME/lib/modules, the runtime image that jlink writes and the JVM loads its classes from. Boot starts in java.base, and you can reach any other module too:

(ldk/invoke (ldk/construct "java.sql.Date" 0) "getTime")                              ;=> 0       java.sql
(ldk/invoke (ldk/get-static "java.util.logging.Level" "WARNING") "getName")           ;=> "WARNING"  java.logging

The image's format is jimage, read the way the JDK's own jdk.internal.jimage.BasicImageReader reads it. A header and an index come first, then every entry's bytes. The index is a perfect hash over full names like /java.base/java/lang/Object.class, and each package has an entry of its own, /packages/java.lang, naming its module. A class lookup finds its package's module, then the class, and reads just those bytes in place. Only the index stays in memory, about 1.5 MB for JDK 25, and nothing is copied or indexed beforehand.

Every JDK has an image, so any build of JDK 25 works, Temurin's included, though they ship no jmods (JDK 24's JEP 493 made them optional). The image is also what the JVM runs. It differs a little from the jmods, because jlink generates a few classes as it links, such as jdk.internal.module.SystemModules$all and the java.lang.invoke holder classes: 21 of the roughly 27,850 in JDK 25.0.2.

A runtime built with jlink works as JAVA_HOME too, if it has the modules your code uses. One made with --compress zip-6 (or any zip-N) is inflated as it's read. jlink's older string-sharing compression, --compress 1, isn't supported; champaca says so the first time it reads a class from such an image.

The module system boots on top of that, as on the JVM. System.initPhase2() builds the boot layer from the module graph jlink resolved into the image, so ModuleLayer.boot().modules() lists what the JVM's does for the same JDK. ServiceLoader finds the providers those modules declare (jdk.localedata's locale data, jdk.charsets' charsets, jdk.jartool's ToolProvider). A JDK class's getModule() is its named module, and reflection into a package the module doesn't export or open fails as it does on the JVM.

Your own classes sit in the application loader's unnamed module and report that loader, as on the JVM. A module-info on the class path is never read, as on the JVM, so there are no checks between the packages there. Named modules of your own come from a module path (-p, -m, running-java-programs), with the differences limitations lists.

sun.misc.Unsafe

Some libraries still reach for sun.misc.Unsafe, which lives in jdk.unsupported. It works, and Shelf.bumpThroughUnsafe in the test jar uses it. The first time a program touches memory through it, JDK 25 prints a warning on stderr (the default, warn), and champaca prints the same one word for word. As on the JVM, -Dsun.misc.unsafe.memory.access=allow turns it off and deny makes the call throw.