Error "IndexDeletionPolicy changed, normally only one commit should be stored"

[mishka81]

I have these errors when indexing projects :

opengrok  | 07-Jul-2023 12:12:31.189 SEVERE [ForkJoinPool-497-worker-3] org.opengrok.suggest.Suggester.lambda$getRebuildRunnable$2 Could not rebuild suggester
opengrok  |     java.lang.IllegalStateException: IndexDeletionPolicy changed, normally only one commit should be stored
opengrok  |             at org.opengrok.suggest.SuggesterProjectData.getCommitVersion(SuggesterProjectData.java:173)
opengrok  |             at org.opengrok.suggest.SuggesterProjectData.initSearchCountMap(SuggesterProjectData.java:331)
opengrok  |             at org.opengrok.suggest.SuggesterProjectData.rebuild(SuggesterProjectData.java:241)
opengrok  |             at org.opengrok.suggest.Suggester.lambda$getRebuildRunnable$2(Suggester.java:367)
opengrok  |             at java.base/java.util.concurrent.ForkJoinTask$AdaptedRunnableAction.exec(ForkJoinTask.java:1407)
opengrok  |             at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
opengrok  |             at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
opengrok  |             at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
opengrok  |             at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
opengrok  |             at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)

I recently migrated from 1.11 to 1.12.11
I this a bug ? how can I avoid it ?

Regards,

[vladak]

How often does this happen ? Is reindex running while this happens ?

I wonder if this is somehow related to #3983 that allowed the number of segments in an index to be higher than 1. Looking into the indexer, it does not seem that the IndexDeletionPolicy is changed from the default KeepOnlyLastCommitDeletionPolicy.

[vladak]

If this was indeed introduced after switching from 1.11 to 1.12.11 then the only related change is the upgrade to Lucene 9.6 in 1.12.6 (#4345), however that does not seem too likely given it was upgrade from Lucene 9.5.

[vladak]

Also, what is the file-system type used for the index data ?

[vladak]

Checking the logs on a production system, I can see this exception being thrown as early as March 2021; there are no older logs on the system. At that point the system was running OpenGrok 1.6.3, so this is definitely not introduced in the recent versions. Also, the removal of the optimization step was done in 1.7.35 (#3983 mentioned above) so that hypothesis is not viable either.

Looking into the logs, this is happening every now and then - several times per month. This system has been running the indexer (in per project workflow) every 4 hours for bunch of years.

[vladak]

OpenGrok 1.12.12 contains improved logging for the above exception, it might help to shed a bit more light on the problem.