Kibana upgrade failed while upgrading from 7.17.28-SNAPSHOT>8.18.0-SNAPSHOT>9.9.0-SNAPSHOT

[amolnater-qasource]

Description:

• 7.17.28-SNAPSHOT>8.18.0-SNAPSHOT -Upgraded Successfully.
  • UPDATE: Failed in latest tests details here: Kibana upgrade failed while upgrading from 7.17.28-SNAPSHOT>8.18.0-SNAPSHOT>9.9.0-SNAPSHOT #207048 (comment)
• 8.18.0-SNAPSHOT>9.9.0-SNAPSHOT issue: Enterprise search failed while upgrading from 8.18.0-SNAPSHOT-9.0.0-SNAPSHOT #206541
• 7.17.28-SNAPSHOT>8.18.0-SNAPSHOT[Success till here]>9.9.0-SNAPSHOT [Failed]

Screenshot:

Deployment ID:
5914e76b70d54178a58d91748beca436

[amolnater-qasource]

@muskangulati-qasource Please review.

[muskangulati-qasource]

Secondary review is Done for this ticket!

[jlind23]

According to the error message shown it seems that the issue is coming from the ES version not being validated. @kpollich was this suppose to work already?

[amolnater-qasource]

Hi @jlind23

While retesting this today, we have observed that Kibana upgrade also failed while upgrading from 7.17.28-SNAPSHOT>8.18.0-SNAPSHOT.

Screenshot:

Deployment ID:
77baafca81724255898364f8c1f768e3

Hence, we have updated the bug description accordingly.

Please let us know if anything else is required from our end.

Thanks!!

[kpollich]

According to the error message shown it seems that the issue is coming from the ES version not being validated. @kpollich was this suppose to work already?

AFAIK the snapshot version upgrades should be working so this seems unexpected to me. Will investigate.

[elasticmachine]

Pinging @elastic/kibana-core (Team:Core)

[kpollich]

Hey @jen-huang - did you run into anything like what we're seeing here with your tests?

[jen-huang]

Hey @jen-huang - did you run into anything like what we're seeing here with your tests?

I didn't, I tested using a local cluster and not Cloud. It seems all these errors are stemming from ES failing to upgrade first. @amolnater-qasource Is there any additional logging around the ES failures we can look at?

[amolnater-qasource]

Hi @jen-huang
Thank you for looking into this.

Currently 7.17.28-SNAPSHOT fails to upgrade to 8.18.0-SNAPSHOT.

Please find below deployment id for detailed logs, as we don't have the admin access:

• d16e4835ff14448cb3c7975c6c2320cd

UPDATE:
Today, some issue is going on in cloud environment where 8.18.0-SNAPSHOT is not deploying but earlier this wasn't the case.
We will revalidate once 8.18.0-SNAPSHOT will be deployed.

Screenshot:

Please let us know if anything else is required from our end.

Thanks!!

[kpollich]

From ES logs

instance-0000000000] fatal exception while booting Elasticsearch java.lang.IllegalArgumentException: unknown setting [xpack.inference.eis.gateway.url] did you mean [xpack.inference.elastic.url]? at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:564) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:510) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:480) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.common.settings.AbstractScopedSettings.validate(AbstractScopedSettings.java:450) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:133) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.common.settings.SettingsModule.(SettingsModule.java:51) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.node.NodeConstruction.validateSettings(NodeConstruction.java:535) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.node.NodeConstruction.prepareConstruction(NodeConstruction.java:285) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.node.Node.(Node.java:201) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.bootstrap.Elasticsearch$2.(Elasticsearch.java:290) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.bootstrap.Elasticsearch.initPhase3(Elasticsearch.java:290) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?] at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:88) ~[elasticsearch-8.18.0-SNAPSHOT.jar:?]

I see this a few times.