8349091: Charts: exception initializing in a background thread #1697

andy-goryachev-oracle · 2025-02-06T19:26:25Z

Root Cause:
(Multiple) properties are getting bound to the global Platform.accessibilityActive property. Binding (and I say, accessing) of properties is not thread-safe.

I also changed the design a bit. Originally, every symbol in a chart had its focusTraversableProperty bound to Platform.accessibilityActive in order to enable the accessibility navigation across the chart data points. This is rather inefficient, as the property has to manage (thousands?) of listeners.

Instead, a single boolean property is added to each chart, with a listener added to it which iterates over data symbols to toggle the focusTraversableProperty directly.

The exact moment when the new property gets bound is also important, and has to happen in the FX application thread.

With this change, it is safe to create and populate charts with data in a background thread.

NOTES

It looks like the Platform.accessibilityActive property never transitions back to false after it transitioned to true. Some say it is because there is no mechanism in the platform to get notified which cannot possibly be true.
The javadoc for Platform.accessibilityActiveProperty() method stipulates that "This method may be called from any thread" which is patently not true as the current implementation is simply not thread-safe.

Note to the Reviewers

To avoid merge conflicts, the preferred order of integrations:

#1697
#1713
#1717

/reviewers 2

Progress

Change must not contain extraneous whitespace
Commit message must refer to an issue
Change must be properly reviewed (2 reviews required, with at least 1 Reviewer, 1 Author)

Issue

JDK-8349091: Charts: exception initializing in a background thread (Bug - P4)

Reviewers

Alexander Zuev (@azuev-java - Author)
Kevin Rushforth (@kevinrushforth - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jfx.git pull/1697/head:pull/1697
$ git checkout pull/1697

Update a local copy of the PR:
$ git checkout pull/1697
$ git pull https://git.openjdk.org/jfx.git pull/1697/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 1697

View PR using the GUI difftool:
$ git pr show -t 1697

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jfx/pull/1697.diff

Using Webrev

Link to Webrev Comment

….safety

…ad.safety

bridgekeeper · 2025-02-06T19:26:41Z

👋 Welcome back angorya! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2025-02-06T19:27:37Z

@andy-goryachev-oracle This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8349091: Charts: exception initializing in a background thread

Reviewed-by: kizune, kcr

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been no new commits pushed to the master branch. If another commit should be pushed before you perform the /integrate command, your PR will be automatically rebased. If you prefer to avoid any potential automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

openjdk · 2025-02-06T19:50:26Z

@andy-goryachev-oracle
The total number of required reviews for this PR (including the jcheck configuration and the last /reviewers command) is now set to 2 (with at least 1 Reviewer, 1 Author).

mlbridge · 2025-02-06T19:53:45Z

Webrevs

08: Full - Incremental (509c0519)
07: Full (fc40e9d6)
06: Full (d212ffd6)
05: Full (4288d1d0)
04: Full (75d4b836)
03: Full (e60c027b)
02: Full (93fc7b93)
01: Full (4b6089e4)
00: Full (4070d8cc)

kevinrushforth · 2025-02-06T20:14:43Z

Reviewers: @kevinrushforth @azuev-java

tests/system/src/test/java/test/robot/javafx/scene/NodeInitializationStressTest.java

azuev-java · 2025-02-12T09:59:30Z

Ok, changes look good and they seems to work on Mac OS. I am curious why do we make labels in charts focus traversable when a11y is switched on? May be something to do with accessibility on Windows? I haven't tested this patch on Windows yet. On Mac OS user can move accessibility cursor to non-focusable elements such as text labels and images so we should not have a problem navigating to the chart labels. Needs to be investigated more.

…ad.safety

kevinrushforth

What you have now works in all cases I've tried. I left a couple suggestions and will reapprove if you decide to make changes.

kevinrushforth · 2025-02-26T14:34:50Z

modules/javafx.controls/src/main/java/javafx/scene/chart/Chart.java

@@ -104,6 +102,9 @@ public abstract class Chart extends Region {
    /** Animator for animating stuff on the chart */
    private final ChartLayoutAnimator animator = new ChartLayoutAnimator(chartContent);

+    // SimpleBooleanProperty or ObjectBinding
+    private volatile Object accessibilityActive;


You can use ObservableValue<?> instead of Object as the type. Alternatively, use two fields, a SimpleBooleanProperty for use by the FX app thread and an ObjectBinding for use by a background thread. They wouldn't need to be volatile in that case. What you have is OK, but using two properties might simplify the logic a bit.

It's a design decision - I won't want to waste an extra pointer.
The cpu cycles are much cheaper nowadays than bytes.
Extra bytes cost much more in cpu cycles (gc) and electricity.

While I still think the code would be cleaner with two properties, and the extra memory is not significant, what you have will work, so I won't object.

kevinrushforth · 2025-02-26T14:40:16Z

modules/javafx.controls/src/main/java/javafx/scene/chart/Chart.java

+                ObservableValue<Window> winProp = sceneProperty().flatMap(Scene::windowProperty);
+                accessibilityActive = winProp; // keep the reference so it won't get gc
+
+                // lambda cannot be used in place of a ChangeListener in removeListener()


Why not use a Subscription then? It seems tailor-made for what you are trying to do.

I don't know how to use Subscription in this case.
This does not work:

ObservableValue<Window> winProp = sceneProperty().flatMap(Scene::windowProperty); accessibilityActive = winProp; // keep the reference so it won't get gc Subscription sub = winProp.subscribe((win) -> { if (win != null) { if (accessibilityActive == winProp) { accessibilityActive = null; } if (isAccessibilityActive()) { handleAccessibilityActive(true); } //winProp.removeListener(this); sub.unsubscribe(); <-- COMPILE ERROR } });

@hjohn could you help here please? How could we use Subscription in a situation when it has to be unsubscribed from within the lambda?

I haven't been tracking these fixes for allowing initialization in background threads, but it seems to me that basically anything should be allowed as long as you're not part of a visible scene graph -- and I think there's also no expectation that all functionality of a control "works" as long as it is not yet part of such a graph (ie. the listeners are only needed once it is part of a scene graph).

If you make listeners conditional on being part of a scene graph, then I think you can handle these with a single code path. Such a condition would be:

ObservableValue<Boolean> partOfSceneGraph = node.sceneProperty() .flatMap(Scene::windowProperty) .flatMap(Window::showingProperty) .orElse(false); // The node here is "this", the chart, because as soon as it is part of the scene // graph, you want the listeners to function.

You can then safely install any listeners directly, regardless if they're on a background thread or not:

someProperty().when(partOfSceneGraph).subscribe( ... );

Or:

someProperty().when(partOfSceneGraph).addListener( ... );

Or with any mappings added, you can even pick where you put the when:

someProperty().flatMap(xyz).when(partOfSceneGraph).addListener( ... ); someProperty().when(partOfSceneGraph).flatMap(xyz).addListener( ... );

On a background thread, partOfSceneGraph will be false, and no listener gets installed (yet). As soon as it becomes true all listeners with that same condition become active. It becomes true automatically when the node has a scene belonging to a window that is visible. Vice versa, if it ever becomes false again, (which it may when the Node is removed or replaced) all listeners using this condition are removed again.

Thank you!

This is not what I asked though: my question was about using Subscription in one-off case, and more specifically, about whether it is even possible to unsubscribe from within the lambda.

What you are proposing is a slightly more involved change: for example, registering and unregistering listeners each time might introduce other issues (especially since we have no way to prioritize listeners/handlers). It also complicates the management of skin (as skins can be added/removed), something I would rather avoid.

As for this current code:

ObservableValue<Window> winProp = sceneProperty().flatMap(Scene::windowProperty); accessibilityActive = winProp; // keep the reference so it won't get gc Subscription sub = winProp.subscribe((win) -> { if (win != null) { if (accessibilityActive == winProp) { accessibilityActive = null; } if (isAccessibilityActive()) { handleAccessibilityActive(true); } //winProp.removeListener(this); sub.unsubscribe(); <-- COMPILE ERROR } });

What you want there is not possible in this way as you can't use this in a lambda, nor refer to it via a local. You can achieve it only if you store the subscription in a field, not in a local. Assigning the Lambda to a ChangeListener local, and using a ChangeListener may be simpler (unless you go for the when solution that may eliminate the need for 2 different code paths).

Note that there is no need to store winProp; flatMap does not use weak listeners (when in active use), and so as long as sceneProperty() exists, the derived property via flatMap will also exist, and also its listener. The way the fluent mapping system works is that listeners are only present when something is actually listening:

ObservableValue<Window> winProp = sceneProperty().flatMap(Scene::windowProperty); // ^^ nobody is listening, so no listeners installed and no hard references; it could GC but // it is currently in a local, so it won't ObservableValue<Window> winProp = sceneProperty().flatMap(Scene::windowProperty); winProp.subscribe(v -> { .. }); // ^^ somebody is listening; all listeners get installed automatically, and now there are // hard references. No risk of GC.

You can then write it as a single statement which won't be GC'd as long as the subscription is active:

subscriptionFIeld = sceneProperty() .flatMap(Scene::windowProperty); .subscribe(v -> { // use subscription field here if you want to unregister the lambda // Note: when you do, there will again be no listeners and thus no hard references and // thus the intermediate property created by flatMap can now be GC'd :) });

…ad.safety

hjohn · 2025-03-03T21:01:58Z

modules/javafx.controls/src/main/java/javafx/scene/chart/AreaChart.java

-            symbol.focusTraversableProperty().bind(Platform.accessibilityActiveProperty());
+            symbol.setFocusTraversable(isAccessibilityActive());


So, if I understand correctly, we can't directly do this bind:

symbol.focusTraversableProperty().bind(Platform.accessibilityActiveProperty())

Because on a background thread there may not yet be an initialized Platform? Or perhaps, Platform would need to initialize and we don't want to do that yet on a background thread?

The reason we should not be binding in this PR is because in part it's bad design(tm): we are creating a zillion of listeners (one per plot data point); a better solution would be to create one listener and use (already existing series) to toggle the data points.

The other reason is https://bugs.openjdk.org/browse/JDK-8351067 - the Platform::accessibilityActive property getter is not thread safe, but even if was, registering a listener with it may cause the change to come in before the node becomes a part of the scene graph, leading to unsynchronized multi-threaded access.

Was this solution considered:

ObservableValue<Boolean> partOfSceneGraph = this.sceneProperty() .flatMap(Scene::windowProperty) .flatMap(Window::showingProperty) .orElse(false); symbol.focusTraversableProperty() .bind(Platform.accessibilityActiveProperty().when(partOfSceneGraph));

What the above code does is create a binding on the when statement (a listener is added on the when result). However, the when property will not add a listener on Platform.accessibilityActiveProperty() until partOfSceneGraph is true. As soon as it does though, a listener is installed on Platform.accessibilityActiveProperty() and its latest value is set to focusTraversableProperty via the bind.

The above does require though that Platform.accessibilityActiveProperty() is properly synchronized. I think it may be a good idea to fix that first. Perhaps all platform provided properties (if there are more) should ensure they're only initialized once.

unrelated, but I would rather disallow background access to any platform properties.

per the earlier email, this might (read: will) create concurrent access when the node is not yet attached to the scene graph:

background thread fx app thread 1. start creating the node 2. add listener to platform prop 3. keep initializing the node <-- platform property change notification 4. some more code

step 3 is when concurrent access occurs.

Yes, but that's because there is no synchronization. Here is a version that does do synchronization; I see no more exceptions (and it runs just as fast really):

public class PropTest { public static void main(String[] args) { Thread mainThread = Thread.currentThread(); final var prop = new SimpleBooleanProperty(false) { @Override public synchronized void addListener(ChangeListener<? super Boolean> listener) { super.addListener(listener); } @Override public synchronized void set(boolean newValue) { super.set(newValue); } }; // add listeners in a background thread var thr = new Thread(() -> { for (int i = 0; i < 1000000; i++) { ChangeListener<Boolean> listener = (obs, o, n) -> { if (!mainThread.equals(Thread.currentThread())) { System.err.println("***** ERROR: listener called on wrong thread: " + Thread.currentThread()); } }; prop.addListener(listener); } }); thr.start(); // Fire events in main thread for (int jj = 0; jj < 1000000; jj++) { prop.set(!prop.get()); } } }

Of course for a generic solution, we could provide a wrapper for properties, or custom synchronized properties.

Yes, using a custom property, or a wrapper, would solve this particular synchronization problem, although that wouldn't solve all of the problems.

We would then be left with the problem that if the accessibility change listener did fire -- which necessarily happens on the FX app thread -- while the object was being set up on a background thread, the listener might try to modify properties that are being touched on the background thread. This is just another case of a more general problem we already have with animation, and which Andy fixed by not starting animation in a few places unless and until we are on the FX app thread.

So I think the solution Andy has chosen of deferring adding the listener is better than trying to fix the accessibility property, followed by fixing the listeners that use it, to be thread-safe.

Thank you, @kevinrushforth .

Getting back to the rule of allowing Node construction from a background thread but not "use", should we consider enforcing the thread access rules for global and static objects (like Platform properties)?

In other words, constructing objects that might modify internal properties is ok, but trying to access shared objects is not?

(maybe it's time to move the discussion out of this PR into the mailing list perhaps?)

Enforcing the threading restriction when using properties is a larger conversation. If we want to have it, I agree it should be on the mailing list.

hjohn · 2025-03-03T21:23:47Z

modules/javafx.controls/src/main/java/javafx/scene/chart/Chart.java

+                boolean aa = Platform.accessibilityActiveProperty().get();
+                SimpleBooleanProperty active = new SimpleBooleanProperty(aa);
+                accessibilityActive = active;
+                active.bind(Platform.accessibilityActiveProperty());
+                active.addListener((src, prev, on) -> {
+                    handleAccessibilityActive(on);
+                });
+                return active.get();


I'm confused as to what is happening here. A boolean property active is created, which is bound to Platform.accessibilityActiveProperty (this creates a listener on accessibilityActiveProperty). We add then another listener on active.

What confuses me is what this is supposed to achieve, and also why we're initialising the property, but then bind it anyway... active = new SimpleBooleanProperty(aa) and later active.bind means that the variable aa is completely unnecessary as the bind will do this get for you.

aa is unnecessary, you are right, bind() will set the value.

the property is created as a way to signal any subsequent calls that no more work is needed.

…ad.safety

andy-goryachev-oracle · 2025-03-04T18:27:02Z

Modified to use Subscription (thanks, @hjohn !)

Check out the code sample

https://github.com/andy-goryachev-oracle/Test/blob/main/src/goryachev/tests/Chart_ThreadInitTest.java

which exercises the scenario.

kevinrushforth

I'll do some more testing, but this looks OK to me.

kevinrushforth · 2025-03-04T18:44:41Z

modules/javafx.controls/src/main/java/javafx/scene/chart/Chart.java

+                    .flatMap(Scene::windowProperty)
+                    .subscribe((w) -> {
+                        if (w != null) {
+                            // also unsubscribes when appears in a window


Suggestion: Add a comment that this can only happen on the FX app thread, since you are relying on that.

added comment

kevinrushforth · 2025-03-04T18:47:15Z

modules/javafx.controls/src/main/java/javafx/scene/chart/Chart.java

@@ -104,6 +102,9 @@ public abstract class Chart extends Region {
    /** Animator for animating stuff on the chart */
    private final ChartLayoutAnimator animator = new ChartLayoutAnimator(chartContent);

+    // SimpleBooleanProperty or ObjectBinding
+    private volatile Object accessibilityActive;


While I still think the code would be cleaner with two properties, and the extra memory is not significant, what you have will work, so I won't object.

kevinrushforth

All my testing looks good.

azuev-java · 2025-03-05T16:29:03Z

I finished testing of the last iteration and i do not see any regressions in the accessibility on Mac or Windows - seems to be working fine on both platforms.

andy-goryachev-oracle · 2025-03-05T17:33:01Z

@hjohn thank you for help with Subscription. Do you have any other comments?

hjohn · 2025-03-05T18:05:57Z

@hjohn thank you for help with Subscription. Do you have any other comments?

No, but admittedly I did not look beyond the subscription issue. I think Kevin and Alexander have got you covered on this one :)

andy-goryachev-oracle · 2025-03-05T18:13:26Z

/integrate

openjdk · 2025-03-05T18:14:05Z

Going to push as commit fc770fb.

openjdk · 2025-03-05T18:14:17Z

@andy-goryachev-oracle Pushed as commit fc770fb.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

andy-goryachev-oracle added 17 commits January 29, 2025 15:52

initial test

40857f5

more tests

4539e56

all controls

aba8476

thread count

e9259a7

pagination

02f3272

cleanup

8786bab

review comments

d1356dc

fast loop

e0b3033

better name

3fde513

jiggler

a05d391

skip tests

e25f6f6

Merge remote-tracking branch 'origin/master' into 8348423.node.thread…

597e39d

….safety

more jitter

30700a6

chart tests only

756bda5

tests pass

e9a6519

cleanup

c74744a

Merge remote-tracking branch 'origin/master' into 8349091.charts.thre…

125ee60

…ad.safety

whitespace

4070d8c

andy-goryachev-oracle marked this pull request as ready for review February 6, 2025 19:49

openjdk bot added the rfr label Feb 6, 2025

kevinrushforth self-requested a review February 6, 2025 20:13

Merge branch 'master' into 8349091.charts.thread.safety

4b6089e

azuev-java reviewed Feb 10, 2025

View reviewed changes

tests/system/src/test/java/test/robot/javafx/scene/NodeInitializationStressTest.java Show resolved Hide resolved

azuev-java approved these changes Feb 12, 2025

View reviewed changes

andy-goryachev-oracle added 3 commits February 25, 2025 07:29

Merge remote-tracking branch 'origin/master' into 8349091.charts.thre…

757c640

…ad.safety

review comments

75d4b83

Merge remote-tracking branch 'origin/master' into 8349091.charts.thre…

4288d1d

…ad.safety

kevinrushforth self-requested a review February 26, 2025 14:23

kevinrushforth approved these changes Feb 28, 2025

View reviewed changes

Merge remote-tracking branch 'origin/master' into 8349091.charts.thre…

4976c58

…ad.safety

kevinrushforth self-requested a review March 3, 2025 14:30

hjohn reviewed Mar 3, 2025

View reviewed changes

andy-goryachev-oracle added 3 commits March 3, 2025 13:48

unnecessary

d212ffd

Merge remote-tracking branch 'origin/master' into 8349091.charts.thre…

643d4cf

…ad.safety

use subscription

fc40e9d

kevinrushforth reviewed Mar 4, 2025

View reviewed changes

kevinrushforth approved these changes Mar 4, 2025

View reviewed changes

review comments

509c051

kevinrushforth approved these changes Mar 4, 2025

View reviewed changes

azuev-java approved these changes Mar 5, 2025

View reviewed changes

openjdk bot added the ready label Mar 5, 2025

openjdk bot added the integrated label Mar 5, 2025

openjdk bot closed this Mar 5, 2025

openjdk bot removed ready rfr labels Mar 5, 2025

andy-goryachev-oracle deleted the 8349091.charts.thread.safety branch March 5, 2025 18:14

		symbol.focusTraversableProperty().bind(Platform.accessibilityActiveProperty());
		symbol.setFocusTraversable(isAccessibilityActive());

8349091: Charts: exception initializing in a background thread #1697

8349091: Charts: exception initializing in a background thread #1697

Conversation

andy-goryachev-oracle commented Feb 6, 2025 • edited by openjdk bot Loading

NOTES

Note to the Reviewers

Progress

Issue

Reviewers

Reviewing

bridgekeeper bot commented Feb 6, 2025

openjdk bot commented Feb 6, 2025 • edited Loading

openjdk bot commented Feb 6, 2025

mlbridge bot commented Feb 6, 2025 • edited Loading

Webrevs

kevinrushforth commented Feb 6, 2025

azuev-java commented Feb 12, 2025

kevinrushforth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hjohn Mar 3, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andy-goryachev-oracle Mar 3, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

andy-goryachev-oracle commented Mar 4, 2025

kevinrushforth left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kevinrushforth left a comment

Choose a reason for hiding this comment

azuev-java commented Mar 5, 2025

andy-goryachev-oracle commented Mar 5, 2025

hjohn commented Mar 5, 2025

andy-goryachev-oracle commented Mar 5, 2025

openjdk bot commented Mar 5, 2025

openjdk bot commented Mar 5, 2025

andy-goryachev-oracle commented Feb 6, 2025 •

edited by openjdk bot

Loading

openjdk bot commented Feb 6, 2025 •

edited

Loading

mlbridge bot commented Feb 6, 2025 •

edited

Loading

hjohn Mar 3, 2025 •

edited

Loading

andy-goryachev-oracle Mar 3, 2025 •

edited

Loading