Regex search

[JJK96]

Describe the pull request content
I added a regular expression search option.

To do this, I had to update Apache Lucene 5 major versions (from 3.6 to 8.11). I tested several languages and searching still works, however testing by native speakers might be necessary. Depends on AndBible/jsword#15. After merging, it might be necessary to invalidate the indexes for all documents.

Screenshots

[tuomas2]

What requires this?

[JJK96]

Not sure, I'll check what breaks if I put it back

[JJK96]

MethodHandle.invoke and MethodHandle.invokeExact are only supported starting with Android O (--min-api 26): Lorg/tartarus/snowball/SnowballProgram;find_among([Lorg/tartarus/snowball/Among;I)I

[JJK96]

Lucene requires this:

[tuomas2]

This is rather conserning in my opinion. We will drop support from very big amount of mobile phones if we drop android 6 and 7. Maybe we need to release two versions, one for old phones and one for new phones if we will merge this...

• I will check and analyze play store statistics.

[tuomas2]

About 2% of our users are using Android versions 6 / 7 / 7.1.

[tuomas2]

• todo: check japanese documents

[JJK96]

I performed a quick check by copy-pasting part of the document and searching for it, which worked. But I don't speak/read Japanese, so I don't know if the results made any sense.

[tuomas2]

I will try to consult my friend chatgpt. He speaks japanese for sure 😊

[tuomas2]

• implement invalidating old indices

[JJK96]

• Bug: Regex search always searches whole bible, not the current book.

[JJK96]

@tuomas2 I have fixed all my open to-do's for this one. Let me know when you have finished the things you still wanted to check. Also, if you have feedback on my changes, let me know.

[tuomas2]

@tuomas2 I have fixed all my open to-do's for this one. Let me know when you have finished the things you still wanted to check. Also, if you have feedback on my changes, let me know.

Thank you! It's now on top of my TODO list, probably next month having better time...

[tuomas2]

this file does not belong to this PR, right?

[JJK96]

Oh, I did not mean to push that, indeed

[tuomas2]

looks like identical to y1ot1nt1_chronological.properties

[tuomas2]

Removing this.

[tuomas2]

• To be investigated: why production built apk is now 24.4 megabytes! Can it be reduced? There must be something unnecessary coming with new lucene libraries, I think. Before this change it was only 13.2 MB.

[timbze]

• To be investigated: why production built apk is now 24.4 megabytes! Can it be reduced? There must be something unnecessary coming with new lucene libraries, I think. Before this change it was only 13.2 MB.

Android Studio has apk analyzer with which it's quite easy to see what is taking up the space

[tuomas2]

Analyzers bundle a lot of new stuff, especially chinese and japanese analyzers.

[tuomas2]

• Crashes when indexing bible modules (like KJV or NASB)

[tuomas2]

Leaving out these

//implementation("org.apache.lucene:lucene-analyzers-smartcn:8.11.2")
//implementation("org.apache.lucene:lucene-analyzers-kuromoji:8.11.2")

will save some space. But still it's big.

[tuomas2]

I wish those japanese / chinese analyzers could be bundled as an extension package. Could investigate that option.

-->

seem to be possible. Something like this:

File jarFile = new File(getFilesDir(), "lucene-analyzers-smartcn-8.11.4.jar");
String optimizedDir = getDir("dex", Context.MODE_PRIVATE).getAbsolutePath();

DexClassLoader classLoader = new DexClassLoader(
    jarFile.getAbsolutePath(),
    optimizedDir,
    null,
    getClassLoader()
);

try {
    Class<?> analyzerClass = classLoader.loadClass("org.apache.lucene.analysis.cn.smart.SmartChineseAnalyzer");
    Object analyzerInstance = analyzerClass.newInstance();

    Method analyzeMethod = analyzerClass.getMethod("analyze", String.class);
    analyzeMethod.invoke(analyzerInstance, "测试文本");

} catch (Exception e) {
    e.printStackTrace();
}

So we could bundle these as add-on module. Need to build support for this in jsword/andbible.

BUT. That is probably disallowed by Google. Instead should use Play Feature Delivery. Which probably won't work outside of Google Play-enabled devices.

https://developer.android.com/guide/playcore/feature-delivery

[tuomas2]

Can you @JJK96 try to figure out why it is crashing and fix it?