JPA not able to render entities with relationships in JHipster 7.6

[mraible]

Overview of the issue

If you try to view an entity's list screen that has relationships to other entities, you'll get an error.

Could not write JSON: Unable to perform requested lazy initialization [com.mycompany.myapp.domain.Post.tags] - 
no session and settings disallow loading outside the Session; nested exception is com.fasterxml.jackson.databind.JsonMappingException: 
Unable to perform requested lazy initialization [com.mycompany.myapp.domain.Post.tags] - 
no session and settings disallow loading outside the Session (through reference chain: 
java.util.Collections$UnmodifiableRandomAccessList[0]->com.mycompany.myapp.domain.Post[\"tags\"])

Motivation for or Use Case

Relationships between entities should work.

Reproduce the error

Create a blog app with PostgreSQL, run it, and try to view posts. If there is no data, try to create a new entity and you'll see the failure. You can also run npm run e2e to see the failures.

You can also do the following if you'd like to clone an existing example:

git clone https://github.com/mraible/spring-native-examples.git
cd spring-native-examples/postgres-mvc
./mvnw

Related issues

I thought this was a Spring Native issue, but it happens in JVM mode too.

#16498

Edit: this only happens with Spring Native, not in JVM mode.

JHipster Version(s)

7.6.0. No, it is not a regression.

JHipster configuration, a .yo-rc.json file generated in the root folder

.yo-rc.json file

{
  "generator-jhipster": {
    "applicationType": "monolith",
    "authenticationType": "oauth2",
    "baseName": "PostgresMVC",
    "blueprints": [
      {
        "name": "generator-jhipster-native",
        "version": "0.0.0"
      }
    ],
    "buildTool": "maven",
    "cacheProvider": "no",
    "clientFramework": "angularX",
    "clientPackageManager": "npm",
    "clientTheme": "none",
    "clientThemeVariant": "",
    "creationTimestamp": 1632872179205,
    "cypressCoverage": false,
    "databaseType": "sql",
    "devDatabaseType": "h2Disk",
    "devServerPort": 4200,
    "dtoSuffix": "DTO",
    "enableGradleEnterprise": false,
    "enableHibernateCache": false,
    "enableSwaggerCodegen": false,
    "enableTranslation": false,
    "entities": ["Blog", "Post", "Tag"],
    "entitySuffix": "",
    "jhiPrefix": "jhi",
    "jhipsterVersion": "7.6.0",
    "languages": ["en", "fr"],
    "lastLiquibaseTimestamp": 1632939293000,
    "messageBroker": false,
    "monorepository": true,
    "nativeLanguage": "en",
    "otherModules": [
      {
        "name": "generator-jhipster-native",
        "version": "0.0.0"
      }
    ],
    "packageName": "com.mycompany.myapp",
    "pages": [],
    "prodDatabaseType": "postgresql",
    "reactive": false,
    "searchEngine": false,
    "serverPort": "8080",
    "serverSideOptions": [],
    "serviceDiscoveryType": "no",
    "skipCheckLengthOfIdentifier": false,
    "skipClient": false,
    "skipCommitHook": true,
    "skipFakeData": false,
    "skipJhipsterDependencies": true,
    "skipServer": false,
    "skipUserManagement": true,
    "testFrameworks": ["cypress"],
    "websocket": false,
    "withAdminUi": true
  }
}

JDL for the Entity configuration(s) entityName.json files generated in the .jhipster directory

JDL entity definitions

entity Blog {
  name String required minlength(3)
  handle String required minlength(2)
}
entity Post {
  title String required
  content TextBlob required
  date Instant required
}
entity Tag {
  name String required minlength(2)
}
relationship ManyToOne {
  Blog{user(login)} to User
  Post{blog(name)} to Blog
}
relationship ManyToMany {
  Post{tag(name)} to Tag{entry}
}

paginate Post, Tag with infinite-scroll

Environment and Tools

openjdk version "17.0.2" 2022-01-18
OpenJDK Runtime Environment GraalVM CE 22.0.0.2 (build 17.0.2+8-jvmci-22.0-b05)
OpenJDK 64-Bit Server VM GraalVM CE 22.0.0.2 (build 17.0.2+8-jvmci-22.0-b05, mixed mode, sharing)

git version 2.32.0 (Apple Git-132)

node: v16.13.2

npm: 8.1.2

Docker version 20.10.12, build e91ed57

Docker Compose version v2.2.3

[mraible]

This is probably caused by the Hibernate Enhance Maven Plugin that I added for Spring Native.

[INFO] --- hibernate-enhance-maven-plugin:5.6.4.Final:enhance (default) @ postgres-mvc ---
[INFO] Starting Hibernate enhancement for classes on /Users/mraible/dev/spring-native-examples/postgres-mvc/target/classes
Feb 06, 2022 6:17:05 PM org.hibernate.Version logVersion
INFO: HHH000412: Hibernate ORM core version 5.6.4.Final
Feb 06, 2022 6:17:06 PM org.hibernate.bytecode.enhance.internal.bytebuddy.BiDirectionalAssociationHandler wrap
INFO: Bi-directional association not managed for field [com.mycompany.myapp.domain.Blog#user]: Could not find target field in [com.mycompany.myapp.domain.User]
Feb 06, 2022 6:17:06 PM org.hibernate.bytecode.enhance.internal.bytebuddy.BiDirectionalAssociationHandler wrap
INFO: Bi-directional association not managed for field [com.mycompany.myapp.domain.User#authorities]: Could not find target field in [com.mycompany.myapp.domain.Authority]
Feb 06, 2022 6:17:06 PM org.hibernate.bytecode.enhance.internal.bytebuddy.BiDirectionalAssociationHandler wrap
INFO: Bi-directional association not managed for field [com.mycompany.myapp.domain.Post#blog]: Could not find target field in [com.mycompany.myapp.domain.Blog]

I'm using the recommended default configuration as recommended by the here.

<plugins>
    [...]
    <plugin>
        <groupId>org.hibernate.orm.tooling</groupId>
        <artifactId>hibernate-enhance-maven-plugin</artifactId>
        <version>$currentHibernateVersion</version>
        <executions>
            <execution>
                <configuration>
                    <failOnError>true</failOnError>
                    <enableLazyInitialization>true</enableLazyInitialization>
                    <enableDirtyTracking>true</enableDirtyTracking>
                    <enableAssociationManagement>true</enableAssociationManagement>
                    <enableExtendedEnhancement>false</enableExtendedEnhancement>
                </configuration>
                <goals>
                    <goal>enhance</goal>
                </goals>
            </execution>
        </executions>
    </plugin>
    [...]
</plugins>

[mshima]

ManyToMany relationships are not loaded due to this problem (lazy loading) and fetching multiples bags problem (multiples *ToMany relationships).
Both approaches above have performance problems.

Related to #13038
The fix was proposed at #13448, but never got reviewed.

[mraible]

@mshima How do you think we should go about fixing this?

[mshima]

@mraible I still think the PR is the best approach.

[mraible]

@mshima AFAICT this only happens when running your JHipster app as a native executable. It doesn't happen when running in JVM mode. I just tested with https://github.com/oktadev/auth0-full-stack-java-example (main branch).

git clone https://github.com/oktadev/auth0-full-stack-java-example example
cd example
# start keycloak
mvn
# editing posts and tags works fine
git checkout spring-native
mvn clean package -Pnative,prod -DskipTests
# start postgresql
./target/native-executable
# editing posts and tags doesn't work

[mshima]

AFAICT this only happens when running your JHipster app as a native executable

The initial bug report states that it’s both JVM and native.

I think the settings enableLazyInitialization and enableAssociationManagement should be set to false.
We use eager loading and the serialization is done outside the transaction context. So nothing is lazy loaded.
Probably we don’t need hibernate-enhance-maven-plugin at all.

[mshima]

editing posts and tags doesn't work

Try adding left join photo.album at:
https://github.com/oktadev/auth0-full-stack-java-example/blob/3402a85645e9a1f0f76fdcd6b39b23d799420040/src/main/java/com/auth0/flickr2/repository/PhotoRepository.java#L18
and https://github.com/oktadev/auth0-full-stack-java-example/blob/3402a85645e9a1f0f76fdcd6b39b23d799420040/src/main/java/com/auth0/flickr2/repository/PhotoRepository.java#L24

[mraible]

The initial bug report states that it’s both JVM and native.

I updated the description to note that it only happens in native mode.

If I change the Hibernate plugin's configuration to have the following:

<configuration>
   <failOnError>true</failOnError>
   <enableLazyInitialization>false</enableLazyInitialization>
   <enableDirtyTracking>true</enableDirtyTracking>
   <enableAssociationManagement>false</enableAssociationManagement>
   <enableExtendedEnhancement>false</enableExtendedEnhancement>
</configuration>

It fixes save for tags, but not for photos. Adding a photo still fails with the following error.

org.springframework.http.converter.HttpMessageConversionException: Type definition error: 
[collection type; class java.util.HashSet, contains [simple type, class java.lang.Object]]; nested exception is 
com.fasterxml.jackson.databind.exc.InvalidDefinitionException: Cannot construct instance of `java.util.HashSet` 
(no Creators, like default constructor, exist): no default no-arguments constructor found
 at [Source: (PushbackInputStream); line: 1, column: 131] (through reference chain: com.auth0.flickr2.domain.Photo["tags"])

If I remove the Hibernate plugin, I get the same error as above (saving albums and tags still works).

If I remove the Hibernate plugin and add your left join suggestions to PhotoRepository as follows:

public interface PhotoRepository extends JpaRepository<Photo, Long> {
    @Query(
        value = "select distinct photo from Photo photo left join fetch photo.tags left join photo.album",
        countQuery = "select count(distinct photo) from Photo photo"
    )
    Page<Photo> findAllWithEagerRelationships(Pageable pageable);

    @Query("select distinct photo from Photo photo left join fetch photo.tags left join photo.album")
    List<Photo> findAllWithEagerRelationships();

    @Query("select photo from Photo photo left join fetch photo.tags where photo.id =:id")
    Optional<Photo> findOneWithEagerRelationships(@Param("id") Long id);
}

... the error still happens too.

[mraible]

One last thing I thought of is to add java.util.HashSet.class to the type hints. This got me a bit further:

Caused by: java.lang.InstantiationException: Type `com.drew.metadata.exif.ExifIFD0Directory` 
can not be instantiated reflectively as it does not have a no-parameter constructor or the 
no-parameter constructor has not been added explicitly to the native image.

I added a type hint for this class (and others that resulted in a similar error) too:

@org.springframework.nativex.hint.TypeHint(
    types = {
        ...
        java.util.HashSet.class,
        com.drew.metadata.exif.ExifIFD0Directory.class,
        com.drew.metadata.exif.ExifSubIFDDirectory.class
    }
)

Then, I get an error about Charsets:

Caused by: java.nio.charset.UnsupportedCharsetException: Cp1252
        at java.nio.charset.Charset.forName(Charset.java:528) ~[native-executable:na]
        at com.drew.lang.Charsets.<clinit>(Charsets.java:40) ~[na:an]

So I added java.nio.charset.Charset.class to the type hints. It worked when I added the first photo, but I got the same error on the 2nd, so I added the following.

@TypeHint(types = com.drew.lang.Charsets.class, access = {TypeAccess.PUBLIC_FIELDS, TypeAccess.DECLARED_CONSTRUCTORS})

This solves the problem!

None of these classes are related to a freshly-generated JHipster app, so I tried commenting out the code that uses them in PhotoResource:

public ResponseEntity<Photo> createPhoto(@Valid @RequestBody Photo photo) throws Exception {
    ...

    /*try {
        photo = setMetadata(photo);
    } catch (ImageProcessingException ipe) {
        log.error(ipe.getMessage());
    }*/

    ...
}

Saving a photo works in this scenario with java.util.HashSet.class added as a type hint. So it seems this is all we need to do to fix JPA relationships in native mode. I also didn't have the Hibernate plugin in my pom.xml.

See this commit for what I changed.

One other thing I'm wondering: is it possible to change the native-executable name to match the artifactId? I think this might help folks who have numerous native apps.

[mraible]

@sdeleuze I discovered today that adding a type hint for java.util.HashSet.class solves our JPA issue with relationships. It seems we don't need the Hibernate plugin either. I've confirmed it in two different JHipster applications.

mraible/spring-native-examples@0c19e4e