8335409: Can't allocate and retain memory from resource area in frame::oops_interpreted_do oop closure after 8329665 #20012

pchilano · 2024-07-03T16:24:20Z

The ResourceMark added in 8329665 to address the case of having to allocate extra memory for the _bit_mask, prevents code in the closure from allocating and retaining memory from the resource area across the closure, relying on some ResourceMark in scope further up the stack from frame::oops_interpreted_do(). There is in fact one case today in JFR code where this kind of allocation happens.

The amount of locals and expression stack entries a method can have before having to allocate extra memory for the _bit_mask is 4*64/2 = 128. This is already big enough that we almost never have to allocate. A test run through mach5 tiers1-6 shows only a handful of methods that fall into this case, and most are artificial ones created to trigger this condition. So moving the allocation to the C heap shouldn't have any performance penalty as the comment otherwise says. This comment dates back from 2002 where instead of 128 entries we could have only 32, considering 32 bits cpus as still in main use (see bug for more history details).

The current code in InterpreterOopMap::resource_copy() has a comment expecting the InterpreterOopMap object to be recently created and empty, but it also has an assert in the allocation case path where it considers the entry might be in use already. This assert actually looks wrong since a used InterpreterOopMap object will not necessarily contain a pointer to resource area memory in _bit_mask[0]. I added an example case in the bug details. In any case, since we don't have any such cases in the codebase I added an explicit assert to verify each InterpreterOopMap is only used one.

I tested the patch by running it through mach5 tiers 1-6.

Thanks,
Patricio

Progress

Change must be properly reviewed (1 review required, with at least 1 Reviewer)
Change must not contain extraneous whitespace
Commit message must refer to an issue

Issue

JDK-8335409: Can't allocate and retain memory from resource area in frame::oops_interpreted_do oop closure after 8329665 (Bug - P2)

Reviewers

David Holmes (@dholmes-ora - Reviewer)
Coleen Phillimore (@coleenp - Reviewer) 🔄 Re-review required (review applies to 7ce559cb)
Aleksey Shipilev (@shipilev - Reviewer) 🔄 Re-review required (review applies to 7ce559cb)
Thomas Stuefe (@tstuefe - Reviewer)

Reviewing

Using git

Checkout this PR locally:
$ git fetch https://git.openjdk.org/jdk.git pull/20012/head:pull/20012
$ git checkout pull/20012

Update a local copy of the PR:
$ git checkout pull/20012
$ git pull https://git.openjdk.org/jdk.git pull/20012/head

Using Skara CLI tools

Checkout this PR locally:
$ git pr checkout 20012

View PR using the GUI difftool:
$ git pr show -t 20012

Using diff file

Download this PR as a diff file:
https://git.openjdk.org/jdk/pull/20012.diff

Webrev

Link to Webrev Comment

bridgekeeper · 2024-07-03T16:25:05Z

👋 Welcome back pchilanomate! A progress list of the required criteria for merging this PR into master will be added to the body of your pull request. There are additional pull request commands available for use with this pull request.

openjdk · 2024-07-03T16:26:36Z

@pchilano This change now passes all automated pre-integration checks.

ℹ️ This project also has non-automated pre-integration requirements. Please see the file CONTRIBUTING.md for details.

After integration, the commit message for the final commit will be:

8335409: Can't allocate and retain memory from resource area in frame::oops_interpreted_do oop closure after 8329665

Reviewed-by: dholmes, stuefe, coleenp, shade

You can use pull request commands such as /summary, /contributor and /issue to adjust it as needed.

At the time when this comment was updated there had been 75 new commits pushed to the master branch:

dcf4e0d: 8335966: Remove incorrect problem listing of java/lang/instrument/NativeMethodPrefixAgent.java in ProblemList-Virtual.txt
1472124: 8333364: Minor cleanup could be done in com.sun.crypto.provider
7e11fb7: 8335688: Fix -Wzero-as-null-pointer-constant warnings from fflush calls in jvmti tests
531a6d8: 8335911: Document ccls indexer in doc/ide.md
0e0dfca: 8330806: test/hotspot/jtreg/compiler/c1/TestLargeMonitorOffset.java fails on ARM32
f3ff4f7: 8335882: platform/cgroup/TestSystemSettings.java fails on Alpine Linux
8f62f31: 8335906: [s390x] Test Failure: GTestWrapper.java
2a29647: 8334777: Test javax/management/remote/mandatory/notif/NotifReconnectDeadlockTest.java failed with NullPointerException
564a72e: 8335955: JDK-8335742 wrongly used a "JDK-" prefix in the problemlist bug number
9c7a6ea: 8312125: Refactor CDS enum class handling
... and 65 more: https://git.openjdk.org/jdk/compare/fac74b118f5fda4ec297e46238d34ce5b9be1e21...master

As there are no conflicts, your changes will automatically be rebased on top of these commits when integrating. If you prefer to avoid this automatic rebasing, please check the documentation for the /integrate command for further details.

➡️ To integrate this PR with the above commit message to the master branch, type /integrate in a new comment.

openjdk · 2024-07-03T16:27:09Z

@pchilano The following label will be automatically applied to this pull request:

hotspot

When this pull request is ready to be reviewed, an "RFR" email will be sent to the corresponding mailing list. If you would like to change these labels, use the /label pull request command.

mlbridge · 2024-07-03T16:43:26Z

Webrevs

dholmes-ora

Thanks for the detailed explanations. A couple of minor (pre-existing) nits but changes are good.

Thanks

dholmes-ora · 2024-07-04T04:47:31Z

src/hotspot/share/interpreter/oopMapCache.cpp

-    "Should not resource allocate the _bit_mask");
-  assert(from->has_valid_mask(),
-    "Cannot copy entry with an invalid mask");
+  // The expectation is that this InterpreterOopMap is a recently created


s/is a recently/is recently/

dholmes-ora · 2024-07-04T04:49:53Z

src/hotspot/share/interpreter/oopMapCache.cpp

 #ifdef ASSERT
-  _resource_allocate_bit_mask = true;
+  _used = false;
 #endif


Nit pre-existing: use of DEBUG_ONLY would be more consistent with later setting of _used.

dholmes-ora · 2024-07-04T04:52:49Z

src/hotspot/share/interpreter/oopMapCache.hpp

@@ -128,11 +129,12 @@ class InterpreterOopMap: ResourceObj {

 public:
  InterpreterOopMap();
+  ~InterpreterOopMap();

  // Copy the OopMapCacheEntry in parameter "from" into this
  // InterpreterOopMap.  If the _bit_mask[0] in "from" points to
  // allocated space (i.e., the bit mask was to large to hold


Nit pre-existing: s/to/too/

dholmes-ora · 2024-07-04T04:53:59Z

src/hotspot/share/interpreter/oopMapCache.hpp

-  // in-line), allocate the space from a Resource area.
+  // in-line), allocate the space from the C heap.
  void resource_copy(OopMapCacheEntry* from);


The name resource_copy seems somewhat of a misnomer given it may be C heap. Is it worth changing?

I agree, this should probably be copy_from, and rename the parameter src. Or something like that.

I also thought about renaming it but ended up leaving it as is in v1. I changed it to Coleen's suggestion.

coleenp

Also a couple of nits, but this looks good. Thanks for tracking down the history and verifying that its an unusual situation that we were optimizing for.

coleenp · 2024-07-05T12:32:53Z

src/hotspot/share/interpreter/oopMapCache.hpp

-// a resource area for better performance.  InterpreterOopMap
+// For InterpreterOopMap the bit_mask is allocated in the C heap
+// to avoid issues with allocations from the resource area that have
+// to live accross the oop closure (see 8335409). InterpreterOopMap


We don't usually put bug numbers in the code and after this change nobody will want to move this back to resource area, so putting the bug number as a caution shouldn't be needed. If one wants to know the details, they can git blame this file.

coleenp · 2024-07-05T12:33:42Z

src/hotspot/share/interpreter/oopMapCache.hpp

-// a resource area for better performance.  InterpreterOopMap
+// For InterpreterOopMap the bit_mask is allocated in the C heap
+// to avoid issues with allocations from the resource area that have
+// to live accross the oop closure (see 8335409). InterpreterOopMap
 // should only be created and deleted during same garbage collection.


Can you add 'the' to "during the same garbage collection."

coleenp · 2024-07-05T12:34:55Z

src/hotspot/share/interpreter/oopMapCache.hpp

-  // in-line), allocate the space from a Resource area.
+  // in-line), allocate the space from the C heap.
  void resource_copy(OopMapCacheEntry* from);


I agree, this should probably be copy_from, and rename the parameter src. Or something like that.

coleenp · 2024-07-05T14:34:57Z

src/hotspot/share/interpreter/oopMapCache.hpp

@@ -89,7 +90,7 @@ class InterpreterOopMap: ResourceObj {

 protected:
 #ifdef ASSERT
-  bool _resource_allocate_bit_mask;
+  bool           _used;


Can you make this a DEBUG_ONLY() too?

coleenp

One tiny nit.

coleenp

Perfect, thanks!

dholmes-ora

Still good.

shipilev

Looks fine. I am tracking this for backport to 21.0.5, which already got the ResourceMark in frame::oops_interpreted_do due to JDK-8329665 backport.

tstuefe

Small nits, otherwise good. Thanks a lot for fixing.

tstuefe · 2024-07-08T12:19:48Z

src/hotspot/share/interpreter/oopMapCache.cpp

+InterpreterOopMap::~InterpreterOopMap() {
+  if (mask_size() > small_mask_limit) {
+    assert(!Thread::current()->resource_area()->contains((void*)_bit_mask[0]),
+           "The bit mask should be allocated from the C heap");


Arguably, this assert is not needed. In debug builds, we have NMT enabled, and that does a check on os::free.

However, an assert that _bit_mask[0] != 0 would make sense, since the free quielty swallows null pointers.

Fixed. We could had such case if the mask was never filled due to invalid bci, so I also improved the conditional.

tstuefe · 2024-07-08T12:26:48Z

src/hotspot/share/interpreter/oopMapCache.cpp

-    "Should not resource allocate the _bit_mask");
-  assert(from->has_valid_mask(),
-    "Cannot copy entry with an invalid mask");
+void InterpreterOopMap::copy_from(OopMapCacheEntry* src) {


Possibly for another RFE: src pointer should be const

Fixed, should be fine to do it in this PR.

tstuefe · 2024-07-08T12:24:44Z

src/hotspot/share/interpreter/oopMapCache.hpp

-#endif
-  int            _num_oops;
-  intptr_t       _bit_mask[N];    // the bit mask if
+  DEBUG_ONLY(bool _used;)


Minor nit. This changes memory layout between debug and release builds, and this is used as part of OopMapCache. Not a big concern, but I usually prefer having the same layout between debug and release to test what we ship.

Can't we not just assert that mask size == USHRT_MAX?

Yes, fixed.

tstuefe · 2024-07-08T12:26:16Z

src/hotspot/share/interpreter/oopMapCache.cpp

    assert(_bit_mask[0] != 0, "bit mask was not allocated");
-    memcpy((void*) _bit_mask[0], (void*) from->_bit_mask[0],
-      mask_word_size() * BytesPerWord);
+    memcpy((void*) _bit_mask[0], (void*) src->_bit_mask[0], mask_word_size() * BytesPerWord);


Are the (void*) cast really needed?

We need them here otherwise we get a compilation error on the conversion from intptr_t to void*. But we don't need them above so I removed those.

tstuefe · 2024-07-08T12:29:01Z

src/hotspot/share/interpreter/oopMapCache.cpp

-    // from the C heap as is done for OopMapCache has a significant
-    // performance impact.
-    _bit_mask[0] = (uintptr_t) NEW_RESOURCE_ARRAY(uintptr_t, mask_word_size());
+    _bit_mask[0] = (uintptr_t) NEW_C_HEAP_ARRAY(uintptr_t, mask_word_size(), mtClass);
    assert(_bit_mask[0] != 0, "bit mask was not allocated");


The assert can be removed, no? NEW_C_HEAP_ARRAY does a null check by default.

Right, removed.

tstuefe

good. thanks!

dholmes-ora

Still good.

pchilano · 2024-07-10T16:24:47Z

Thanks for the reviews @dholmes-ora, @coleenp, @shipilev and @tstuefe!

pchilano · 2024-07-10T16:25:26Z

/integrate

openjdk · 2024-07-10T16:26:16Z

Going to push as commit 7ab96c7.
Since your change was applied there have been 82 commits pushed to the master branch:

fb66716: 8331725: ubsan: pc may not always be the entry point for a VtableStub
fb9a227: 8313909: [JVMCI] assert(cp->tag_at(index).is_unresolved_klass()) in lookupKlassInPool
e6c5aa7: 8336012: Fix usages of jtreg-reserved properties
e0fb949: 8335779: JFR: Hide sleep events
537d20a: 8335766: Switch case with pattern matching and guard clause compiles inconsistently
a44b60c: 8335778: runtime/ClassInitErrors/TestStackOverflowDuringInit.java fails on ppc64 platforms after JDK-8334545
b5909ca: 8323242: Remove vestigial DONT_USE_REGISTER_DEFINES
dcf4e0d: 8335966: Remove incorrect problem listing of java/lang/instrument/NativeMethodPrefixAgent.java in ProblemList-Virtual.txt
1472124: 8333364: Minor cleanup could be done in com.sun.crypto.provider
7e11fb7: 8335688: Fix -Wzero-as-null-pointer-constant warnings from fflush calls in jvmti tests
... and 72 more: https://git.openjdk.org/jdk/compare/fac74b118f5fda4ec297e46238d34ce5b9be1e21...master

Your commit was automatically rebased without conflicts.

openjdk · 2024-07-10T16:26:22Z

@pchilano Pushed as commit 7ab96c7.

💡 You may see a message that your pull request was closed with unmerged commits. This can be safely ignored.

pchilano · 2024-07-15T15:28:23Z

/backport :jdk23

openjdk · 2024-07-15T15:30:14Z

@pchilano Could not automatically backport 7ab96c74 to openjdk/jdk due to conflicts in the following files:

src/hotspot/share/interpreter/oopMapCache.hpp

Please fetch the appropriate branch/commit and manually resolve these conflicts by using the following commands in your personal fork of openjdk/jdk. Note: these commands are just some suggestions and you can use other equivalent commands you know.

# Fetch the up-to-date version of the target branch
$ git fetch --no-tags https://git.openjdk.org/jdk.git jdk23:jdk23

# Check out the target branch and create your own branch to backport
$ git checkout jdk23
$ git checkout -b backport-pchilano-7ab96c74-jdk23

# Fetch the commit you want to backport
$ git fetch --no-tags https://git.openjdk.org/jdk.git 7ab96c74e2c39f430a5c2f65a981da7314a2385b

# Backport the commit
$ git cherry-pick --no-commit 7ab96c74e2c39f430a5c2f65a981da7314a2385b
# Resolve conflicts now

# Commit the files you have modified
$ git add files/with/resolved/conflicts
$ git commit -m 'Backport 7ab96c74e2c39f430a5c2f65a981da7314a2385b'

Once you have resolved the conflicts as explained above continue with creating a pull request towards the openjdk/jdk with the title Backport 7ab96c74e2c39f430a5c2f65a981da7314a2385b.

Below you can find a suggestion for the pull request body:

Hi all,

This pull request contains a backport of commit 7ab96c74 from the openjdk/jdk repository.

The commit being backported was authored by Patricio Chilano Mateo on 10 Jul 2024 and was reviewed by David Holmes, Thomas Stuefe, Coleen Phillimore and Aleksey Shipilev.

Thanks!

v1

ca0db02

openjdk bot added the hotspot [email protected] label Jul 3, 2024

pchilano marked this pull request as ready for review July 3, 2024 16:39

openjdk bot added the rfr Pull request is ready for review label Jul 3, 2024

dholmes-ora approved these changes Jul 4, 2024

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Jul 4, 2024

coleenp reviewed Jul 5, 2024

View reviewed changes

pchilano added 2 commits July 5, 2024 09:57

David's comments

127ae6e

Coleen's comments

805358e

openjdk bot removed the ready Pull request is ready to be integrated label Jul 5, 2024

coleenp reviewed Jul 5, 2024

View reviewed changes

use DEBUG_ONLY on _used declaration

7ce559c

coleenp approved these changes Jul 5, 2024

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Jul 5, 2024

dholmes-ora approved these changes Jul 8, 2024

View reviewed changes

shipilev approved these changes Jul 8, 2024

View reviewed changes

tstuefe reviewed Jul 8, 2024

View reviewed changes

address Thomas' comments

88d866b

openjdk bot removed the ready Pull request is ready to be integrated label Jul 8, 2024

tstuefe approved these changes Jul 8, 2024

View reviewed changes

openjdk bot added the ready Pull request is ready to be integrated label Jul 8, 2024

dholmes-ora approved these changes Jul 10, 2024

View reviewed changes

openjdk bot added the integrated Pull request has been integrated label Jul 10, 2024

openjdk bot closed this Jul 10, 2024

openjdk bot removed ready Pull request is ready to be integrated rfr Pull request is ready for review labels Jul 10, 2024

8335409: Can't allocate and retain memory from resource area in frame::oops_interpreted_do oop closure after 8329665 #20012

8335409: Can't allocate and retain memory from resource area in frame::oops_interpreted_do oop closure after 8329665 #20012

Uh oh!

Conversation

pchilano commented Jul 3, 2024 • edited by openjdk bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Progress

Issue

Reviewers

Reviewing

Webrev

Uh oh!

bridgekeeper bot commented Jul 3, 2024

Uh oh!

openjdk bot commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openjdk bot commented Jul 3, 2024

Uh oh!

mlbridge bot commented Jul 3, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Webrevs

Uh oh!

dholmes-ora left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

coleenp left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

coleenp left a comment

Choose a reason for hiding this comment

Uh oh!

coleenp left a comment

Choose a reason for hiding this comment

Uh oh!

dholmes-ora left a comment

Choose a reason for hiding this comment

Uh oh!

shipilev left a comment

Choose a reason for hiding this comment

Uh oh!

tstuefe left a comment

Choose a reason for hiding this comment

Uh oh!

tstuefe Jul 8, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pchilano commented Jul 3, 2024 •

edited by openjdk bot

Loading

openjdk bot commented Jul 3, 2024 •

edited

Loading

mlbridge bot commented Jul 3, 2024 •

edited

Loading

tstuefe Jul 8, 2024 •

edited

Loading