Building dynamic Java Module System layers to integrate JavaFX 11 in Equinox

One of the most fundamental features of the e(fx)clipse runtime is to integrate JavaFX into the Equinox OSGi-Container and even a running Eclipse IDE.

We currently support the following setups:

JavaFX 8
JavaFX 9/10
JavaFX 11

and the integration for all those versions is a bit different. I don’t want to go into details but starting with JavaFX-11 we need to spin up a new Java-Module-System-Layer at runtime because we can not assume JavaFX being part of the JRE running your OSGi-Container (Eclipse IDE).

Since JavaFX-9 we spin up a dynamic layer to implement JavaFX-SWT-Integration and we adapted that logic for JavaFX-11 to load all JavaFX-11 modules.

The code we have works like this

	package javamodules;

	import java.lang.module.Configuration;
	import java.lang.module.ModuleFinder;
	import java.net.URL;
	import java.net.URLClassLoader;
	import java.nio.file.Files;
	import java.nio.file.Path;
	import java.nio.file.Paths;
	import java.util.Set;
	import java.util.stream.Collectors;
	import java.util.stream.Stream;

	public class SimpleLayerCreation {
	private static Path JAVAFX_MODULES = Paths.get("/Users/tomschindl/Downloads/javafx-sdk-11/lib");

	@SuppressWarnings("deprecation")
	public static void main(String[] args) throws Throwable {
	Path[] array = Files.list(JAVAFX_MODULES) //
	.filter(p -> {
	String name = p.getFileName().toString();
	return name.startsWith("javafx.") && name.endsWith(".jar");
	}) //
	.toArray(i -> new Path[i]);
	Set<String> modules = Stream.of(array) //
	.map(Path::getFileName) //
	.map(Path::toString) //
	.map(n -> n.substring(0, n.length() - ".jar".length())) //
	.collect(Collectors.toSet());

	URL[] urls = Stream.of(array).map( Path::toFile).map( f -> {
	try {
	return f.toURL();
	} catch(Throwable t) {
	return null;
	}
	} ).toArray( i -> new URL[i]);


	URLClassLoader c = new URLClassLoader(urls, FeatureRichLayerCreation.class.getClassLoader());

	ModuleFinder fxModuleFinder = ModuleFinder.of(array);
	ModuleFinder empty = ModuleFinder.of(new Path[0]);
	ModuleLayer bootLayer = ModuleLayer.boot();
	Configuration configuration = bootLayer.configuration();
	Configuration newConfiguration = configuration.resolve(fxModuleFinder, empty, modules);

	ModuleLayer moduleLayer = bootLayer.defineModulesWithManyLoaders(newConfiguration, FeatureRichLayerCreation.class.getClassLoader());

	ClassLoader loader = moduleLayer.findLoader("javafx.base");

	System.err.println(loader.loadClass("javafx.beans.property.SimpleStringProperty").newInstance());
	System.err.println(loader.loadClass("com.sun.javafx.runtime.VersionInfo").newInstance());

	c.close();
	}
	}

view raw SimpleLayerCreation.java hosted with ❤ by GitHub

and it works prefectly fine until someone like ControlsFX comes along and does not play by the rules trying to load classes from unexported packages like com.sun.javafx.runtime.VersionInfo.

The standard answer from ControlsFX to fix that problem temporarily is to force the module-system to export them using –add-exports=javafx.base/com.sun.javafx.runtime=ALL-UNNAMED.

Unfortunately this workaround does not work in our case because the command-line flag only allows to modify modules of the Boot-Layer but not those created in dynamic ones like those we construct inside our JavaFX-OSGi integration.

I was investigating yesterday how one could fix this problem but could not come up with a good solution (one that does not call into internals of the module system) until I tweeted

so @java experts following - one can modify modules of the boot-layer (eg --add-exports,..) but if one constructs a dynamic layer this is not possible - what API did I miss in https://t.co/VdVF9kUIqT /cc @nipafx @Java_Champions
— Tom Schindl (@tomsontom) October 23, 2018

about it and Tom Watson (one of the maintainers of Equinox) pointed me into the right direction.

So the solution is

	/*
	* For modules on the boot-layer one can add exports using --add-exports but it looks like there's no
	* public API to do the same when constructing a custom layer
	*/
	package javamodules;

	import java.lang.ModuleLayer.Controller;
	import java.lang.module.Configuration;
	import java.lang.module.ModuleFinder;
	import java.net.URL;
	import java.net.URLClassLoader;
	import java.nio.file.Files;
	import java.nio.file.Path;
	import java.nio.file.Paths;
	import java.util.Arrays;
	import java.util.Set;
	import java.util.stream.Collectors;
	import java.util.stream.Stream;


	public class FeatureRichLayerCreation {

	private static Path JAVAFX_MODULES = Paths.get("/Users/tomschindl/Downloads/javafx-sdk-11/lib");

	@SuppressWarnings("deprecation")
	public static void main(String[] args) throws Throwable {
	System.err.println("VERSION: " + System.getProperty("java.version"));
	Path[] array = Files.list(JAVAFX_MODULES) //
	.filter(p -> {
	String name = p.getFileName().toString();
	return name.startsWith("javafx.") && name.endsWith(".jar");
	}) //
	.toArray(i -> new Path[i]);
	Set<String> modules = Stream.of(array) //
	.map(Path::getFileName) //
	.map(Path::toString) //
	.map(n -> n.substring(0, n.length() - ".jar".length())) //
	.collect(Collectors.toSet());

	URL[] urls = Stream.of(array).map( Path::toFile).map( f -> {
	try {
	return f.toURL();
	} catch(Throwable t) {
	return null;
	}
	} ).toArray( i -> new URL[i]);


	URLClassLoader c = new URLClassLoader(urls, FeatureRichLayerCreation.class.getClassLoader()) {
	protected java.lang.Class<?> findClass(String moduleName, String name) {
	try {
	return findClass(name);
	} catch (ClassNotFoundException e) {}
	return null;
	}

	protected URL findResource(String moduleName, String name) throws java.io.IOException {
	return findResource(name);
	}
	};

	ModuleFinder fxModuleFinder = ModuleFinder.of(array);
	ModuleFinder empty = ModuleFinder.of(new Path[0]);
	ModuleLayer bootLayer = ModuleLayer.boot();
	Configuration configuration = bootLayer.configuration();
	Configuration newConfiguration = configuration.resolve(fxModuleFinder, empty, modules);

	Controller moduleLayerController = ModuleLayer.defineModules(newConfiguration, Arrays.asList(bootLayer), s -> c);
	ModuleLayer moduleLayer = moduleLayerController.layer();

	moduleLayerController.addExports(moduleLayer.findModule("javafx.base").get(), "com.sun.javafx.runtime", FeatureRichLayerCreation.class.getModule());

	ClassLoader loader = moduleLayer.findLoader("javafx.base");

	System.err.println(loader.loadClass("javafx.beans.property.SimpleStringProperty").newInstance());
	System.err.println(loader.loadClass("com.sun.javafx.runtime.VersionInfo").newInstance());
	Class<?> loadClass = loader.loadClass("com.sun.glass.utils.NativeLibLoader");
	System.err.println(loadClass.getModule());

	// we need to overwrite findResource(String,String)
	System.err.println(loadClass.getResource("NativeLibLoader.class"));
	// we need to overwrite findClass(String,String)
	System.err.println(Class.forName(moduleLayer.findModule("javafx.base").get(), "javafx.beans.property.SimpleIntegerProperty"));


	c.close();
	}
	}

view raw FeatureRichLayerCreation.java hosted with ❤ by GitHub

and now I have to think how we expose that to in our OSGi-Integration.

6 Responses to Building dynamic Java Module System layers to integrate JavaFX 11 in Equinox

Mordechai Meisels says:

February 13, 2020 at 7:16 pm

This doesn’t solve what you can’t say `–add-exports=module.in.my.layer`. Looks like https://github.com/AdoptOpenJDK/openjdk-jdk11/blob/master/src/java.base/share/classes/java/lang/ModuleLayer.java#L214 has a hardcoded block to this behaviour, otherwise, I’d be glad if you can point me to how to do this.

- Tom Schindl says:
  
  February 13, 2020 at 7:44 pm
  
  I’m not sure what you are aiming at for me the code from this blog works perfectly fine – do you get an exception when trying to build your own Module-Layer?
  
  - mordechaimeisels says:
    
    February 13, 2020 at 9:32 pm
    
    My bad I didn’t explain myself enough. So say you have a module in the new layer that needs access to a package but that package is in the boot layer, in such a case I think we’re out of luck.
    
    Using command-line flags won’t work since that target module doesn’t exist, and using a controller will not work either as it disallows exporting from the parent module.
    
    one solution which I haven’t tried could be to add a dummy module with the same name of the target module and put that in the boot layer just to satisfy the JVM flag, then shadow that boot dummy module in the child layer.
    
    SO again, your code works but still doesn’t solve one issue.
  - Tom Schindl says:
    
    February 14, 2020 at 8:11 am
    
    Ok now i got it – you want to expose a package explicitly to one module in another layer – this sounds like a fair use case to me. Did you ever asked at the openjdk mailing list?
  - mordechaimeisels says:
    
    February 13, 2020 at 9:34 pm
    
    We discussed this blog post here: https://github.com/update4j/update4j/issues/81
  - Tom Schindl says:
    
    February 14, 2020 at 8:33 am
    
    I’ve subscribed myself to that issue and comment there