Problem with migrations location when switched to 0.4.3

[alexeychalov]

After switching to 0.4.3 from 0.4.2 starter does not see migration script directory in packaged boot jar. I.e. when one clean up elastic and then start migration via boot jar app startup. Everything worked fine in 0.4.2 version

[alexeychalov]

Attach logs for both versions

0.4.3
2023-11-14 16:10:06,834 INFO [main] c.s.e.e.c.ElasticsearchEvolution: Created ElasticsearchEvolution with config='ElasticsearchEvolutionConfig{enabled=true, locations=[classpath:es/migration], encoding=UTF-8, lineSeparator='\n', de
faultContentType='application/json; charset=UTF-8', esMigrationPrefix='V', esMigrationSuffixes=[.http], placeholders={}, placeholderPrefix='${', placeholderSuffix='}', placeholderReplacement=true, historyIndex='search_support_evolut
ion', historyMaxQuerySize=1000, validateOnMigrate=true, baselineVersion='1.0', outOfOrder='false'}' and client='[[host=http://localhost:9200]]'
2023-11-14 16:10:06,838 INFO [main] c.s.e.e.c.ElasticsearchEvolution: start elasticsearch migration...
2023-11-14 16:10:06,839 INFO [main] c.s.e.e.c.ElasticsearchEvolution: reading migration scripts...
2023-11-14 16:10:07,567 INFO [main] c.s.e.e.c.ElasticsearchEvolution: parsing migration scripts...
2023-11-14 16:10:07,568 INFO [main] c.s.e.e.c.ElasticsearchEvolution: execute migration scripts...
2023-11-14 16:10:07,570 INFO [main] c.s.e.e.s.b.s.a.ElasticsearchEvolutionInitializer: ElasticsearchEvolution executed successfully 0 migration scripts in 732 ms

0.4.2
2023-11-14 16:12:10,294 INFO [main] c.s.e.e.c.ElasticsearchEvolution: Created ElasticsearchEvolution with config='ElasticsearchEvolutionConfig{enabled=true, locations=[classpath:es/migration], encoding=UTF-8, lineSeparator='\n', de
faultContentType='application/json; charset=UTF-8', esMigrationPrefix='V', esMigrationSuffixes=[.http], placeholders={}, placeholderPrefix='${', placeholderSuffix='}', placeholderReplacement=true, historyIndex='search_support_evolut
ion', historyMaxQuerySize=1000, validateOnMigrate=true, baselineVersion='1.0'}' and client='[[host=http://localhost:9200]]'
2023-11-14 16:12:10,298 INFO [main] c.s.e.e.c.ElasticsearchEvolution: start elasticsearch migration...
2023-11-14 16:12:10,298 INFO [main] c.s.e.e.c.ElasticsearchEvolution: reading migration scripts...
2023-11-14 16:12:10,341 INFO [main] c.s.e.e.c.ElasticsearchEvolution: parsing migration scripts...
2023-11-14 16:12:10,347 INFO [main] c.s.e.e.c.ElasticsearchEvolution: execute migration scripts...
2023-11-14 16:12:10,466 WARN [main] o.e.c.RequestLogger: request [POST http://localhost:9200/search_support_evolution/_search?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true] returned 1 wa
rnings: [299 Elasticsearch-8.8.1-f8edfccba429b6477927a7c1ce1bc6729521305e "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2023-11-14 16:12:10,773 WARN [main] o.e.c.RequestLogger: request [GET http://localhost:9200/search_support_evolution/_refresh?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true] returned 1 wa
rnings: [299 Elasticsearch-8.8.1-f8edfccba429b6477927a7c1ce1bc6729521305e "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2023-11-14 16:12:10,780 WARN [main] o.e.c.RequestLogger: request [GET http://localhost:9200/search_support_evolution/_count?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true] returned 1 warn
ings: [299 Elasticsearch-8.8.1-f8edfccba429b6477927a7c1ce1bc6729521305e "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2023-11-14 16:12:10,794 WARN [main] o.e.c.RequestLogger: request [GET http://localhost:9200/search_support_evolution/_count?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true] returned 1 warn
ings: [299 Elasticsearch-8.8.1-f8edfccba429b6477927a7c1ce1bc6729521305e "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2023-11-14 16:12:10,879 WARN [main] o.e.c.RequestLogger: request [POST http://localhost:9200/search_support_evolution/_search?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true] returned 1 wa
rnings: [299 Elasticsearch-8.8.1-f8edfccba429b6477927a7c1ce1bc6729521305e "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2023-11-14 16:12:10,880 INFO [main] c.s.e.e.c.i.m.e.MigrationServiceImpl: executing script V1.0.0.001__initial.http
2023-11-14 16:12:11,039 INFO [main] c.s.e.e.c.i.m.e.MigrationServiceImpl: executed migration script MigrationScriptProtocol{version=1.0.0.1, indexName='null', description='initial', scriptName='V1.0.0.001__initial.http', checksum=-
548431403, executionTimestamp=2023-11-14T16:12:11.039053700+03:00, executionRuntimeInMillis=159, success=true, locked=true}
2023-11-14 16:12:11,068 WARN [main] o.e.c.RequestLogger: request [GET http://localhost:9200/search_support_evolution/_refresh?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true] returned 1 wa
rnings: [299 Elasticsearch-8.8.1-f8edfccba429b6477927a7c1ce1bc6729521305e "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]
2023-11-14 16:12:11,095 WARN [main] o.e.c.RequestLogger: request [POST http://localhost:9200/search_support_evolution/_update_by_query?requests_per_second=-1&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true&ignor
e_throttled=false&refresh=true] returned 1 warnings: [299 Elasticsearch-8.8.1-f8edfccba429b6477927a7c1ce1bc6729521305e "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen
tiers in place of frozen indices."]
2023-11-14 16:12:11,128 WARN [main] o.e.c.RequestLogger: request [POST http://localhost:9200/search_support_evolution/_update_by_query?requests_per_second=-1&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true&ignor
e_throttled=false&refresh=true] returned 1 warnings: [299 Elasticsearch-8.8.1-f8edfccba429b6477927a7c1ce1bc6729521305e "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen
tiers in place of frozen indices."]
2023-11-14 16:12:11,130 INFO [main] c.s.e.e.s.b.s.a.ElasticsearchEvolutionInitializer: ElasticsearchEvolution executed successfully 1 migration scripts in 832 ms

[alexeychalov]

BTW for elastic version 8.8.1 three are some warnings

[xtermi2]

thanks @alexeychalov
This must be the update of org.reflections:reflections from 0.9.12 to 0.10.2: #233
Toughed we had an integration test for this, because we had this issue some years ago
@RiVogel do you have the same issue with 0.4.3?

[RiVogel]

Hi @xtermi2, @alexeychalov: I just tried it out in my project. All my migration scripts are getting collected with the new version 4.3.0, see debugger screenshot and a log screenshot. My configuration in this case is the default one.

After migrating there is a validation mechanism of Hibernate Search which does not fail - so index seems to be completely migrated.

[RiVogel]

@xtermi2: I see that @alexeychalov got a baseline version set and I see some minor changes between 4.2 and 4.3 regarding the baseline validation. Could that be the case?

[alexeychalov]

@RiVogel: actually, baseline version is default one, I didn't set it purposefully. Moreover, my migrations run on application packaged in boot jar startup, if there no problems in your case I guess it may classloader/class path lookup issue. But everything was fine in 0.4.2 version

I'll later will make a test with 0.4.3 version and downgraded reflections version and report the result

[alexeychalov]

Well, actually I cannot use evolution version 0.4.3 with reflections 0.9.12:
Caused by: java.lang.NoClassDefFoundError: org/reflections/scanners/Scanners
at com.senacor.elasticsearch.evolution.core.internal.migration.input.MigrationScriptReaderImpl.readScriptsFromClassPath(MigrationScriptReaderImpl.java:150)

Running just main class is ok, running via gradle bootRun is ok, the problem appears only if I run bootJar via java -jar
So, I think there is a problem scanning bootJar classpath and receiving resources (migrations) from BOOT_INF/classes directory.
I.e. some mechanism changed in reflections library, that leads to error.

[RiVogel]

@alexeychalov: Yes, downgrading did not work since org.reflections also changed the usage of the ResourceScanner somehow.

In the meantime I could reproduce your problem within a minimal reproducer and starting the Spring Boot application with "java -jar". Let's see...

[RiVogel]

I can't believe it. It looks like you ran into this issue: ronmamo/reflections#373

My mimimal reproducer faced the same problem and after downgrading to reflections version 0.10.1 it worked. @xtermi2: Shall we change the dependency to reflections 0.10.1 or just mention it in the documentation?

[xtermi2]

Maybe both. But we also need an unit- or integration test.
I also think about migrating to the spring classpath Scanner. Initially I wanted to avoid the spring dependency in the core lib, but org.reflectiins is also obviously not the best choice.
Maybe https://github.com/classgraph/classgraph is also worth a try, it's actively maintained and has zero dependencies.

[xtermi2]

@RiVogel and/or @alexeychalov can you checkout the branch bugfix/replace_org.reflections_with_classgraph from PR #241 and locally build the 0.5.0-SNAPSHOT version to verify this issue is fixed in your application / reproducer-app?

[alexeychalov]

@xtermi2 works fine, indices are created. Thanks, guys.
There is log warning that warries me:

2023-11-16 09:58:22,887 WARN [main] o.e.c.RequestLogger: request [POST http://localhost:9200/search_support_evolution/_search?ignore_throttled=false&ignore_unavailable=true&expand_wildcards=open&allow_no_indices=true] returned 1 wa
rnings: [299 Elasticsearch-8.8.1-f8edfccba429b6477927a7c1ce1bc6729521305e "[ignore_throttled] parameter is deprecated because frozen indices have been deprecated. Consider cold or frozen tiers in place of frozen indices."]

ES8. Should I create another issue?

[xtermi2]

OK, thanks for testing @alexeychalov. Then I will create a release shortly in the next days.

• ignore_throttled is in my opinion not a problem. The drop of this deprecated parameter is normally done by elastic with the next major update (=Elasticsearch 9) and even then I expect from elasticsearch to just ignore this parameter. This parameter just says in which kind indices you want to search in and we search here in an exact index, search_support_evolution in your case - the index where elasticsearch evolution stores its state.
  • and yes, this parameter should be removed, because it's not required at this part - you can create an issue if you want.

[RiVogel]

@xtermi2: Yes, for me the branch also works. Maybe add in the documentation of 0.4.3 that one should downgrade reflections to 0.10.1 if they run spring as jar.