[Feature Request]: Apache Nifi 2 support ( Java 21 version )

[ismaHenzel]

What would you like to happen?

Plc4x runs with Java 11, and Apache NiFi versions 1.x also use Java 11. However, after I updated Apache NiFi to the latest version (Version 2), Plc4x stopped working.

The error message is as follows:

GhostProcessor[id=018f1016-372a-13ca-dfb0-0a1b9d02d9d0] Encountering difficulty starting. (Validation State is INVALID: ['Missing Processor' validated against 'Any Property' is invalid because Processor is of type org.apache.plc4x.nifi.Plc4xSourceRecordProcessor, but this is not a valid Processor type]). Will continue trying to start.

Programming Languages

• plc4j
• plc4go
• plc4c
• plc4net

Protocols

• AB-Ethernet
• ADS /AMS
• BACnet/IP
• CANopen
• DeltaV
• DF1
• EtherNet/IP
• Firmata
• KNXnet/IP
• Modbus
• OPC-UA
• S7

[chrisdutz]

Could it be that the Nifi API changed? The major version change usually indicates that. I would assume you need to update the code of the adapter to the new version and API.

[ismaHenzel]

Could it be that the Nifi API changed? The major version change usually indicates that. I would assume you need to update the code of the adapter to the new version and API.

I'm sorry if I gave the impression that I was using PLC4X with custom development. I'm actually using the .nar from https://mvnrepository.com/artifact/org.apache.plc4x/plc4j-nifi-plc4x-nar/0.12.0. I also tested building the .nar 0.13 from the plc4x-extras repository, but the same error occurred.

I ran a clean version of Apache NiFi in Docker with versions 1.28.0, 2.0.0, and 2.1.0 using the following Dockerfile:
`version: "3.8"

services:
nifi:
image: apache/nifi:1.28.0
deploy:
replicas: 1
container_name: nifi
restart: always
ports:
- 8443:8443
environment:
- NIFI_WEB_HTTPS_PORT=8443
- SINGLE_USER_CREDENTIALS_USERNAME=admin
- SINGLE_USER_CREDENTIALS_PASSWORD=adminadminadmin
volumes:
- ./lib:/opt/nifi/nifi-current/extensions
- ./lib_2:/opt/nifi/nifi-current/nar_extensions
`
When running version 1.x, everything works fine, but in version 2.x, the error mentioned below appears.

I checked the logs, and when I click on the processors in the NiFi UI, this error occurs. Additionally, PLC4X is not found among the available processors.

2025-01-22 23:57:43,138 INFO [main] org.apache.nifi.runtime.Application Started Application in 20.611 seconds (20611807389 ns)
nifi  | 2025-01-22 23:57:55,506 WARN [NiFi Web Server-133] org.apache.nifi.web.api.dto.DtoFactory Unable to get extension class for [org.apache.plc4x.
nifi.Plc4xSourceRecordProcessor]
nifi  | java.lang.NoSuchFieldError: Class org.apache.nifi.expression.ExpressionLanguageScope does not have member field 'org.apache.nifi.expression.Ex
pressionLanguageScope VARIABLE_REGISTRY'
nifi  |         at org.apache.plc4x.nifi.BasePlc4xProcessor.(BasePlc4xProcessor.java:105)
nifi  |         at java.base/java.lang.Class.forName0(Native Method)
nifi  |         at java.base/java.lang.Class.forName(Class.java:534)
nifi  |         at java.base/java.lang.Class.forName(Class.java:513)
nifi  |         at org.apache.nifi.nar.StandardExtensionDiscoveringManager.getClass(StandardExtensionDiscoveringManager.java:497)
nifi  |         at org.apache.nifi.web.api.dto.DtoFactory.fromDocumentedTypes(DtoFactory.java:3296)
nifi  |         at org.apache.nifi.web.controller.ControllerFacade.getFlowFileProcessorTypes(ControllerFacade.java:486)
nifi  |         at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
nifi  |         at java.base/java.lang.reflect.Method.invoke(Method.java:580)

Running these tests with a minimal setup, I realized that my first error, "validated against 'Any Property' is invalid because Processor is of type," occurred because the processor was not installed because the error above, and I have many processgroups using PLC4X processors.

[sruehl]

there is a related PR to that, might be related apache/plc4x-extras#52

[ismaHenzel]

there is a related PR to that, might be related apache/plc4x-extras#52

Yes, this is likely the problem. I will keep the production Apache NiFi at version 1.28.1 and hope that version 0.13, when officially released, will support nifi 2.x versions.

I tried to build this PR locally for testing, but I'm encountering compilation errors. As my knowledge of Java is minimal, I can't provide much assistance, but I'm available to help test the new release before its official launch.

Thanks for the responses, everyone !

[chrisdutz]

To me it looks as if one module can only support NiFi 1 or NiFi 2, but not both ... I hope the PR is adding a second module for 2.0 but even if not, it should not be hard to update it to do so.

However I am not sure there's currently anyone working on the PR ... also do I not know if it's finished for testing ...

[ismaHenzel]

Certainly, I believe it's not that difficult to compile for version 2. I ran a mvn clean package inside the NiFi folder from the branch with the possible fix, but errors occurred during the build. Maybe later I'll try to build it again and take a closer look at what was going wrong. For now, NiFi 1.28.1 is working very well, and the update would be more about exploring the new features and keeping it on the latest version. I'll see if I can generate a .nar for Java 21 that runs on NiFi 2 in the coming days.

[chrisdutz]

I did notice a day ago that I needed to build the plc4x-maven-plugin in order to build PLC4X and that I needed to downgrade to Java 21 from 23 to have the build running ... perhals this is also what you need to do?

[ismaHenzel]

Hello!

I used Java 21 to build the project, but I’m unsure whether I correctly built the plc4x-maven-plugin. I will attempt the build again tomorrow and update this issue with the results.

[ismaHenzel]

I recreated the changes from PR #52 locally within the current develop branch of plc4x-extras, and I was able to build the NiFi NAR successfully.

Afterward, I tried building it by cloning the branch from the PR itself, but I encountered some build issues that seem to have been resolved in later commits on the develop branch.

When testing the import in NiFi, everything worked fine. However, I don’t have any CLP devices at home to test with, so I plan to test this NAR at my company tomorrow.

I have a few questions:

• Do you think it would be better to rebuild these changes based on the main branch instead of the develop branch?
• At home, I built the package on my Arch Linux system. Should I build it on another OS instead? I noticed that PLC4X uses Ubuntu images. Would you recommend building it from a clean Docker image?
• If everything works, is there a place to store the built NAR for other users who might encounter the same problem and want to use NiFi 2.0, at least until a new official integration release is available?

[sruehl]

maybe @ottobackwards can help with those questions

[chrisdutz]

You should definitely base your PR on develop ... but I just noticed that we seem to have a "develop" and a "main" branch. I would say we should get rid of the main branch as soon as possible and stick with the develop branch. However I am unsure if it contains anything that's not in develop.

If you want to update your PR, just merge the changes from develop (of the original plc4x-extras repo) into your branch.

[ismaHenzel]

I created this fork: https://github.com/ismaHenzel/plc4x-extras-nifi-2.x/tree/feature/fix-plc4x-nar-incompatibility, and the NAR works fine in Apache NiFi, but only after I build the Nifi NAR on the machine. If I simply copy the NAR into Apache NiFi, it doesn't run. I initially thought that the .nar file contains all the dependencies, but it seems that's not the case. Am I missing something, or is it always necessary to build the package? I’m trying to understand if it’s possible to create a .nar file that can just be copied and pasted into NiFi, or if I need to create a Dockerfile that runs NiFi, then compiles the NAR using the pom.xml and installs all the dependencies.

My tests on CLP devices are working as expected. I will continue testing until the end of the week and aim to try a pre-production version, migrating from NiFi 2.

[ottobackwards]

Distribution as a .nar file is the canonical way to distribute nifi components and should work.
We need more information from the nifi logs as to what isn't working.

Nifi publishes it's NAR files to the maven central repository, but I don't think you'll want to do that.