How I generate the JavaFlight Recorder Docu

From a tweet von Gunnar Morling

I learned today that my work on JFR-Doc is used for JfrUnit. In a follow up Gunnar asked me how those JSON-Files are generated and I promised to write up a blog what I hacked together in an evening to get this going.

Get the input

The first and most important thing is to get all JFR-Events (those commands are executed eg in your Java-17-Install-Dir)

./java -XX:StartFlightRecording:filename=/tmp/out-bin.jfr \
-version # Dump JFR Data
./jfr metadata /tmp/out-bin.jfr > /tmp/openjdk-17.jfr

The final file holds content like this

class boolean {
}
class byte {
}
class char {
}
class double {
}
class float {
}
class int {
}
class long {
}
class short {
}

@Name("java.lang.Class")
@Label("Java Class")
class Class {
  @Label("Class Loader")
  ClassLoader classLoader;

  @Label("Name")
  String name;

  @Label("Package")
  Package package;

  @Label("Access Modifiers")
  int modifiers;

  @Label("Hidden")
  boolean hidden;
}
...

Looks like these are Java-Classes so one strategy could be to just compile those and use Reflection to extract meta informations but I went another route

Parsing the .jfr-File

Handcrafting a parser is certainly not the way to go. I needed something that could provide me a fairly simple Logical-AST. There are BNF-Definitions for Java but I wanted something much simpler so I fired up my Eclipse IDE and created an Xtext-Project using the wizards and replaced the content in the .xtext-File with

grammar at.bestsolution.jfr.JFRMeta with org.eclipse.xtext.common.Terminals

generate jFRMeta "http://www.bestsolution.at/jfr/JFRMeta"

Model:
  classes+=Clazz*;

Clazz:
  annotations+=Annotation*
  'class' name=ID ( 'extends' super=QualifiedName )? '{'
    attributes += Attribute*
  '}';

Attribute:
  annotations+=Annotation*
  type=[Clazz|ID] array?='[]'? name=ID ';'
;

Annotation:
  '@' type=[Clazz|ID] ('(' (values+=AnnotationValue |
   ('{' values+=AnnotationValue
   (',' values += AnnotationValue)* '}')) ')')?
;

AnnotationValue:
  valueString=STRING | valueBoolean=Boolean | valueNum=INT
;

enum Boolean:
  TRUE="true" | FALSE="false"
;

QualifiedName:
  ID ('.' ID)*;

That’s all required because the .jfr-File is extremly simple so we don’t need a more complex definition.

How to convert

Well although Xtext is primarily used to develop DSL-Editors for the Eclipse IDE one can run the generated parser in plain old Java. So all now needed is to write a generator who parses the .jfr-File(s) and generate different output from it (HTML, JSON, …) and because although Java now has multiline strings Xtend is the much better choice to write a “code”-generator.

package at.bestsolution.jfr

import org.eclipse.xtext.resource.XtextResourceSet
import org.eclipse.xtext.resource.XtextResource
import java.util.ArrayList
import org.eclipse.emf.common.util.URI
import java.nio.file.Files
import java.nio.file.Paths
import at.bestsolution.jfr.jFRMeta.Model
import java.nio.file.StandardOpenOption
import at.bestsolution.jfr.jFRMeta.Clazz

import static extension at.bestsolution.jfr.GenUtil.*
import at.bestsolution.jfr.jFRMeta.Attribute

class JSONGen {
     def static void main(String[] args) {
         val versions = createVersionList(Integer.parseInt(args.get(0)))
         val injector = new JFRMetaStandaloneSetup().createInjectorAndDoEMFRegistration();
         val resourceSet = injector.getInstance(XtextResourceSet);
         resourceSet.addLoadOption(XtextResource.OPTION_RESOLVE_ALL, Boolean.TRUE);

         val models = new ArrayList
         for( v : versions ) {
             val resource = resourceSet.getResource(
                 URI.createURI("file:/Users/tomschindl/git/jfr-doc/openjdk-"+v+".jfr"), true);
             val model = resource.getContents().head as Model;
             models.add(model)
         }

         for( pair : models.indexed ) {
             val model = pair.value
             var version = versions.get(pair.key)
             val preModel = pair.key == 0 ? null : models.get(pair.key - 1)
             Files.writeString(Paths.get("/Users/tomschindl/git/jfr-doc/openjdk-"+version+".json"),model.generate(preModel,version), StandardOpenOption.TRUNCATE_EXISTING, StandardOpenOption.CREATE)
         }
     }

     def static generate(Model model, Model prevModel, String ver) '''
         {
             "version": "«ver»",
             "distribution": "openjdk",
             "events": [
                 «val evts = model.classes.filter[c|c.super == "jdk.jfr.Event"]»
                 «FOR e : evts»
                     «e.generateEvent»«IF e !== evts.last»,«ENDIF»
                 «ENDFOR»
             ],
             "types": [
                 «val types = model.classes.filter[c|c.super === null]»
                 «FOR t : types»
                     «t.generateType»«IF t !== types.last»,«ENDIF»
                 «ENDFOR»
             ]
         }
     '''

     def static generateEvent(Clazz clazz) '''
         {
             "name": "«clazz.name»",
             "description": "«clazz.description»",
             "label": "«clazz.label»",
             "categories": [
                 «val cats = clazz.categories»
                 «FOR cat : cats»
                     "«cat»"«IF cat !== cats.last»,«ENDIF»
                 «ENDFOR»
             ],
             "attributes": [
                 «FOR a : clazz.attributes»
                     «a.generateAttribute»«IF a !== clazz.attributes.last»,«ENDIF»
                 «ENDFOR»
             ]
         }
     '''

     def static generateType(Clazz clazz) '''
         {
             "name": "«clazz.name»",
             "attributes": [
                 «FOR a : clazz.attributes»
                     «a.generateAttribute»«IF a !== clazz.attributes.last»,«ENDIF»
                 «ENDFOR»
             ]
         }
     '''

     def static generateAttribute(Attribute a) '''
         {
             "name": "«a.name»",
             "type": "«a.type.name»",
             "contentType": "«a.contentType»",
             "description": "«a.description»"
         }
     '''
}

All sources are available at https://github.com/BestSolution-at/jfr-doc if you look at this code keep in mind that it was hacked together in an evening

Posted in Uncategorized | Leave a comment

Building a cmd-line application to notarize Apple applications using Quarkus and GraalVM Native Image support

So I’ve been searching for a long time for a small side project where I could give Quarkus’ Native-Image support a spin. While we are using Quarkus in JDK mode in almost all of our server applications there was no need yet to compile a native binary.

This week although I found the perfect usecase: I’ve been banging my head against codesigning and notarizing an e(fx)clipse application (shipped with a JRE) the whole week.

Doing that requires to execute a bunch of cmd-utilities one by one. So I came up with the plan to write an Quarkus command line application, compiled to a native executable to automate this process a bit more. Yeah there are go and python solutions and I could have simply written a shell-script but why not try something cooler.

The result of this work is a native OS-X executable allowing me to codesign, create a dmg/pkg, notarize and finally staple the result as you can see from the screenshot below

115746928-0f796280-a395-11eb-9ec8-abe9fd94d591

As of now this does not include anything special for Java application so it can be used for any application (I currently have an artifical restriction that you can only use .app)

All sources are available at https://github.com/BestSolution-at/mac-codesigner and I added a pre-release of the native executable. Miss a feature, found a bug? Feel free to file a ticket and provide a PR.

Posted in Uncategorized | Leave a comment

e(fx)clipse 3.7.0 is released

We are happy to announce that e(fx)clipse 3.7.0 has been released. This release contains the following repositories/subprojects:

There are almost no new features (eg the new boxshadow) but only bugfixes who are very important if you use OpenJFX in an OSGi-Environment.

For those of you who already use our pom-First approache the new bits have been pushed to https://maven.bestsolution.at/efxclipse-releases/ and the Sample application at https://github.com/BestSolution-at/e4-efxclipse-maven-sample/tree/efxclipse-3.7.0 has been updated to use the latest release.

Posted in Uncategorized | Leave a comment

Browser like BoxShadow for JavaFX coming with e(fx)clipse 3.7.0

Using BoxShadow is a very common thing in modern UIs, so it might not be suprising that designers defining UIs often also use them heavily.

Unfortunately JavaFX has NO 100% compatible effect and even worse one who is closest (DropShadow) leads to a massive performance hit as shown in this video

On the left hand side is a Node who has a DropShadow-Effect applied to it and you notice that once the effect is applied that the animation isn’t smooth any more. On the right hand side you see a new Node we’ll release with e(fx)clipse 3.7.0 who provides a new BoxShadow-Node (named BoxShadow2).

Beside getting a huge performance win, the BoxShadow-Node uses the same semantics the browser counterpart does so you can port CSS definition to your JavaFX-Application.

For completness here’s the code for this demo video.

package org.eclipse.fx.ui.controls.demo;

import org.eclipse.fx.ui.controls.effects.BoxShadow2;

import javafx.animation.Animation;
import javafx.animation.Animation.Status;
import javafx.animation.TranslateTransition;
import javafx.application.Application;
import javafx.geometry.Insets;
import javafx.geometry.Pos;
import javafx.scene.Node;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.effect.DropShadow;
import javafx.scene.layout.BorderPane;
import javafx.scene.layout.Region;
import javafx.scene.layout.StackPane;
import javafx.scene.paint.Color;
import javafx.stage.Stage;
import javafx.util.Duration;

public class InTheShadow extends Application {

	public static void main(String[] args) {
		launch(args);
	}

	@Override
	public void start(Stage primaryStage) throws Exception {
		BorderPane p = new BorderPane();
		p.setPadding(new Insets(20));

		Button shadowMe = new Button("Toggle Shadow");
		p.setTop(shadowMe);

		Region pane;

		if (Boolean.getBoolean("efxclipse-shadow")) {
			BoxShadow2 shadow = new BoxShadow2(createComplexUI());
			shadow.setShadowVisible(false);
			pane = shadow;
		} else {
			pane = new StackPane(createComplexUI());
		}

		p.setCenter(pane);

		shadowMe.setOnAction(evt -> toggleShadow(pane));

		Scene s = new Scene(p, 1200, 800);
		primaryStage.setTitle("efxclipse-shadow: " + Boolean.getBoolean("efxclipse-shadow"));
		primaryStage.setScene(s);
		primaryStage.show();
	}

	private void toggleShadow(Region pane) {
		if (pane instanceof BoxShadow2) {
			BoxShadow2 s = (BoxShadow2) pane;
			s.setShadowVisible(!s.isShadowVisible());
		} else {
			if (pane.getEffect() != null) {
				pane.setEffect(null);
			} else {
				DropShadow dropShadow = new DropShadow();
				dropShadow.setRadius(5.0);
				dropShadow.setOffsetX(3.0);
				dropShadow.setOffsetY(3.0);
				dropShadow.setColor(Color.color(0.4, 0.5, 0.5));
				pane.setEffect(dropShadow);
			}
		}
	}

	private Node createComplexUI() {
		StackPane pane = new StackPane();
		pane.setStyle("-fx-background-color: white");

		for (int i = 0; i < 100; i++) {
			Button b = new Button("Button " + i);
			b.setTranslateX(i % 100);
			pane.getChildren().add(b);
		}

		Button animated = new Button("Animated");
		StackPane.setAlignment(animated, Pos.BOTTOM_CENTER);

		TranslateTransition t = new TranslateTransition(Duration.millis(1000), animated);
		t.setAutoReverse(true);
		t.setFromX(-300);
		t.setToX(300);
		t.setCycleCount(Animation.INDEFINITE);
		animated.setOnAction(evt -> {
			if (t.getStatus() == Status.RUNNING) {
				t.pause();
			} else {
				t.play();
			}
		});

		pane.getChildren().add(animated);

		return pane;
	}

}
Posted in e(fx)clipse, Uncategorized | 1 Comment

JavaFlightRecorder-Event Documentation

So while exploring the JFR-Event-Stream-System I came across the problem that there’s no (human readable) documentation what JFR-Events are available in a given JDK-Version.

Erik Gahlin who works on JavaFlightRecorder pointed me to a command one can use to dump the information

The output you get from such a command is something like:

So this is some structured text (well it’s fairly simple Java-Code). So I crafted a simple parser using Xtext:

wrote an HTML-Generator using Xtend, threw some CSS-Framework (https://getuikit.com/) on it to get

Created a github-repo so that you can browse that yourself – https://bestsolution-at.github.io/jfr-doc/openjdk-14.html

Posted in Uncategorized | Leave a comment

Observing your Quarkus Server with “JFR Event Streaming” via Websockets

Having reviewed thousands lines if code last week I had the urgent need to code something for fun and inspired by Gunnar Morlings tweets on “JFR Event Streaming – JEP 349”

I hacked this afternoon in the sun and I’m now able to observe an Quarkus-Server via Websockets retrieving JFR Events rendering them via https://www.chartjs.org/

This is a very early draft implementation who is only able to observe the “jdk.CPULoad”-Event and just a PoC to see if this is possible at all but I’m quite happy with my afternoon work (well now it is already 10pm 🙂

Posted in Uncategorized | Leave a comment

News from DriftFX: Approaching v1.0

It’s been a long time since we gave an update on DriftFX (we announced our PoC roughly 1.5 years ago) but I’m pleased to share with you that we’ve worked hard on it for the past half year to turn the PoC-Code into something we think will get v1.0 of DriftFX.

While our first implementation was focused on the usecase of the main sponsor (and is used in product already) we took the stuff we learned from it and hopefully came up without repeating mistakes we made in our PoC.

Before going into detail let me repeat what DriftFX is (and what it is not):

  • DriftFX is an OpenJFX Node and an API to draw textures within this Node. (the textures could come from anywhere but the main focus is currently OpenGL)
  • DriftFX provides transfer modes to bypass main memory for Windows and MacOS (the texture never leaves the GPU memory)
  • DriftFX is NOT an OpenGL context creation/management library (use one of your own choice e.g. GLFW, ..)
  • DriftFX is NOT an OpenGL API library (use one of your own choice e.g. LWJGL, JOGL, GLEW, ..)
  • DriftFX is NOT a rendering or game engine, its sole purpose is the transport of rendered textures to OpenJFX

So let’s take a closer look what we worked on for the last couple of months:

  • Rewrote the complete internal structure and moved as much code as possible to Java (formerly a lot stuff was in C++)
  • Rewrote the complete API implementing a Swap-Chain approach allowing us to deal with the fact that the old API had to allocate textures on a per frame basis
  • Provided a Java-API so that people don’t have to write Native-Code
  • Provided an CPP-API which wraps the Java-API. It is distributed in source form, so no more linking against DriftFX binaries is needed.
  • Because of the new Java-API direct support for LWJGL is available (there’s been an experimental LWJGL-Support provided by the LWJGL-maintainers)
  • Moved the demos to its own repo, so that we may include samples which are not EPL.
  • Demo Application running out of the box
  • Building with GitHub-Actions and publishing p2 and maven artifacts

The video below shows a LWJGL-Demo we adapted to use DriftFX

You can find the source code of DriftFX at https://github.com/eclipse-efx/efxclipse-drift/ and our Demo-Project at https://github.com/BestSolution-at/efxclipse-drift-samples.

While we are still fixing some details, write some documentation we’d like to get feedback from the wider OpenSource-Community.

Posted in e(fx)clipse, Uncategorized | 1 Comment

Setting up e(fx)clipse RCP development for Java11+ and PDE

As I’m currently converting a Java-8 project to AdoptJDK-11 and JavaFX-11+ I thought it would be a good idea document the steps involved.

Prequisits

I assume you have installed:

Configure your Eclipse

Java Settings

Make AdoptJDK-11 the default JRE unless it is already the default.

Make sure AdoptJDK-11 is used for the Java-SE-11 EE

e(fx)clipse Settings

Open the JavaFX-Preference Page and point it to your JavaFX-11-SDK

This step is required because JavaFX is not part of AdoptJDK-11 and hence Eclipse won’t find the libraries and your code won’t compile inside the IDE (we’ll revisit this topic below once more)

Setup a target platform

Create your project

Bootstrap your project

Check your project setup

Check if Eclipse correctly recognized the javafx-library and magically added them to your plug-in dependendencies

Implement the UI

Add javax.annotation to your MANIFEST.MF

Before you can write the Java-Code for your UI you need to add javax.annotation-package to your bundle (this used to ship with Java-8 has been removed since then)

Create a Java-Class

package my.app.app;

import javax.annotation.PostConstruct;

import javafx.scene.control.Label;
import javafx.scene.layout.BorderPane;

public class SamplePart {
  @PostConstruct
  void init(BorderPane root) {
    root.setCenter(
      new Label(System.getProperty("javafx.version"))
    );
  }
}

Adapt your e4xmi

Running your application

While everything happily compiles running the application would fail because in the initial steps we only satisfied the Eclipse compiler by magically injecting the JavaFX-Libraries in your Plug-in-Dependency (see above).

To run the application we need to decide how we’d like to ship JavaFX:

  • next to your application in a folder
  • as part of your eclipse application inside the the plugins-directory
  • you jlink yourself a JDK

We’ll not take a look at the 3rd solution as part of this blog post!

Running with an external folder

Open the generated launch configuration and append -Defxclipse.java-modules.dir=PATH_TO_YOUR_JAVAFX_LIBS in the VM arguments-field

Running with bundled javafx-modules

We provide OSGi-Bundles who contain the original and unmodified JavaFX-Modules (note you can NOT use them are OSGi-Dependencies!) you can use them by adding http://downloads.efxclipse.bestsolution.at/p2-repos/openjfx-11/repository/

Add them to your launch configuration

Exporting your application

The project wizard already generated the basic infrastructure for you but we need to make some small changes. We assume you’ve chosen to option to ship the JavaFX-modules as part of the plugins-directory to keep it simple.

The wizard already added the JavaFX-Standard-Feature into your product-File

It also added the parts to satisfy the compiler in your releng/pom.xml

While most of the stuff is already in place we need to make 2 small modifications:

  • Update the tycho-version property to 1.5.0
  • Change the export environment to match the operation-system(s) you want to target
    • Windows: os=win32, ws=win32, arch=x86_64
    • Linux: os=linux, ws=gtk, arch=x86_64
    • OS-X: os=macosx, ws=cocoa, arch=x86_64

Producing a native launcher

As we anyway have to produce a platform-dependent we can also add the creation of a native launcher. For that open your .product-File:

  • Tick the “The product includes native launcher artifacts”
  • Change the application to main-thread-application

Posted in Uncategorized | 7 Comments

Slides from JavaFX-Days ZĂŒrich on e(fx)clipse APIs

If you could not attend my talk at the JavaFX-Days ZĂŒrich yesterday or you did and wanted to recap what you’ve presented. Here are the slides.

I enjoyed the conference and I hope you did as well. See you next year!

Posted in Uncategorized | 1 Comment

Announcing e(fx)clipse DriftFX – Integrating Native Rendering Pipelines into JavaFX

I’ve had the honor to introduce DriftFX at todays JavaFX-Days conference in ZĂŒrich.

What is DriftFX

DriftFX is a JavaFX extension allowing you to embed native rendering engines (eg OpenGL) into the JavaFX-Scenegraph.

To embed external rendering engines DriftFX exposes a new Node-Type DriftFXSurface one can put at any place in the SceneGraph and treat it like any other JavaFX-Node – you can apply transformations, effects, … on it like you do with any other JavaFX-Node. In general you can think of it as ImageView on steroids.

A more real world example what one can make happen with DriftFX can be seen in this video provide by Netallied a partner company.

What does DriftFX do?

DriftFX allows you to directly embed content rendered by native pipelines into JavaFX WITHOUT going through the Main-Memory. The rendered artifacts stay at the GPU all time!

What DriftFX does not do?

DriftFX is not a rendering engine itself and it does not provide any abstraction layer to write native rendering engines. Its only purpose is to bring your rendered content directly in JavaFX.

What platforms does DriftFX support?

Our Beta implementation currently supports all 3 major Desktop-Systems supported by OpenJFX – Windows, OS-X and Linux.

We currently targeted JavaFX-8 because this is what our customers are running their applications on. We plan to provide support for OpenJFX-11 and future releases of OpenJFX in the week/months to come.

Is DriftFX opensource?

Yes. We’ve been lucky that the sponsors of the work agreed to make DriftFX opensource. Currently there’s an open Pull-Request at Github for the e(fx)clipse project.

Note that it will take some time until the PR is merged. The reason is that we are going to run this through the Eclipse IP-Process to make sure you can safely embed it into your application.

Does DriftFX use internal “APIs”

Yes. It integrates into the JavaFX-Rendering pipeline so it needs to access internals. We are aware that those can change at any time. We are open to contribute DriftFX in future to OpenJFX but it heavily depends on the OpenJFX community and the stewards of OpenJFX.

Is DriftFX production ready?

Propably not 100% yet, we are just about to integrate it into our customers application and fix problems as they arise. So we are somewhere in between Beta and production readiness.

The reason to opensource it now is that we expect the OpenJFX community to take a look and help us improving it. We know that there are many very clever people in the OpenJFX community who can hopefully help us pushing DriftFX forward.

How can you/your company help?

First of all take a look at it and in case it looks interesting get in touch with us to help fund the ongoing work on this matter.

Acknowledgement

First of all I’d like to thank EclipseSource Munich and Netallied for their on going support on DriftFX. Special thank goes out to my co-worker (and partner in crime) Christoph and all other people at BestSolution.at who made DriftFX happen!

Posted in e(fx)clipse | 4 Comments