Skip to content

Multiple ES-Hadoop versions detected - false positive #2371

@frensjan

Description

@frensjan

org.elasticsearch.hadoop.util.Version throws a RuntimeException if it finds the class itself multiple times on the class-path:

Multiple ES-Hadoop versions detected in the classpath; please use only one
[list of jars containing the Version class]

This probably is a great help in cases where issues arise from actually having different versions on the class-path. However, it actually doesn't check for whether there actually are different versions.

When deploying with Flink we run into the issue that ES Hadoop is added to the class-path twice. Once because it is packaged in a user-lib, but it is also added to the class-path for YARN deployments. This is unfortunate, not that easy to work around, especially because it also influences a lot of other class-path issues.

I'd like to provide a PR which actually checks if there are multiple different versions on the class-path.

I'm thinking about something like:

Set versions = get versions
if(versions.size() > 1) {
   throw ...
}

instead of the current implementation of

List jars = get normalized jars containing `org.elasticsearch.hadoop.util.Version`
if(jars.size() > 1) {
   throw ...
}

Any (early) feedback is more than welcome!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions