Temporarily Disable Java Integration Tests

[tustvold]

Which issue does this PR close?

Closes #.

Rationale for this change

Seeing failures on master, this might fix it

https://github.com/apache/arrow-rs/actions/runs/6574785367/job/17860535457

What changes are included in this PR?

Are there any user-facing changes?

[tustvold]

@pitrou Any ideas, this at least appears to coincide with apache/arrow#38248

[pitrou]

Right, you will also need to add this line to the build steps:
https://github.com/apache/arrow/blob/23dfd0e8643799b803b286e88ca6253303ecb703/docker-compose.yml#L1746

and make sure you pass these env vars:
https://github.com/apache/arrow/blob/23dfd0e8643799b803b286e88ca6253303ecb703/docker-compose.yml#L1737-L1740

[tustvold]

Is there perhaps some way that we can script these in a way that can be reused by arrow-rs as opposed to hard-coded in a docker-compose file, that we keep having to incorporate changes from as things change?

[pitrou]

That's a good question. I'm not sure how to do that as these environment variables are influencing build steps for various implementations. Perhaps we need a single overarching environment variable such as ARROW_INTEGRATION_ONLY=1. @kou Any idea?

[kou]

How about adding ci/scripts/integration_build.sh to apache/arrow and use it in apache/arrow's docker-compose.yml and apache/arrow-rs's .github/workflows/integration.yml?

diff --git a/docker-compose.yml b/docker-compose.yml
index e54c609e5..6b5b777c9 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -1734,18 +1734,8 @@ services:
       # Tell Archery where the arrow C++ binaries are located
       ARROW_CPP_EXE_PATH: /build/cpp/debug
       ARROW_GO_INTEGRATION: 1
-      ARROW_JAVA_CDATA: "ON"
-      JAVA_JNI_CMAKE_ARGS: >-
-        -DARROW_JAVA_JNI_ENABLE_DEFAULT=OFF
-        -DARROW_JAVA_JNI_ENABLE_C=ON
     command:
-      ["/arrow/ci/scripts/rust_build.sh /arrow /build &&
-        /arrow/ci/scripts/cpp_build.sh /arrow /build &&
-        /arrow/ci/scripts/csharp_build.sh /arrow /build &&
-        /arrow/ci/scripts/go_build.sh /arrow &&
-        /arrow/ci/scripts/java_jni_build.sh /arrow $${ARROW_HOME} /build /tmp/dist/java/$$(arch) &&
-        /arrow/ci/scripts/java_build.sh /arrow /build /tmp/dist/java &&
-        /arrow/ci/scripts/js_build.sh /arrow /build &&
+      ["/arrow/ci/scripts/integration_build.sh /arrow /build &&
         /arrow/ci/scripts/integration_arrow.sh /arrow /build"]
 
   ################################ Docs #######################################

diff --git a/.github/workflows/integration.yml b/.github/workflows/integration.yml
index 62d2d2cb1a..0c3e4213fd 100644
--- a/.github/workflows/integration.yml
+++ b/.github/workflows/integration.yml
@@ -88,42 +88,10 @@ jobs:
         with:
           path: rust
           fetch-depth: 0
-      - name: Install pythonnet
-        run: conda run --no-capture-output pip install pythonnet
-      - name: Install archery
-        run: conda run --no-capture-output pip install -e dev/archery[integration]
-      - name: Make build directory
-        run: mkdir /build
-      - name: Build Rust
-        run: conda run --no-capture-output ci/scripts/rust_build.sh $PWD /build
-      - name: Build C++
-        run: conda run --no-capture-output ci/scripts/cpp_build.sh $PWD /build
-      - name: Build C#
-        run: conda run --no-capture-output ci/scripts/csharp_build.sh $PWD /build
-      - name: Build Go
-        run: conda run --no-capture-output ci/scripts/go_build.sh $PWD
-      - name: Build Java
-        run: conda run --no-capture-output ci/scripts/java_build.sh $PWD /build
-      - name: Build JS
-        run: conda run --no-capture-output ci/scripts/js_build.sh $PWD /build
+      - name: Build
+        run: conda run --no-capture-output ci/scripts/integration_build.sh $PWD /build
       - name: Run integration tests
-        run: |
-          conda run --no-capture-output archery integration \
-            --run-flight \
-            --run-c-data \
-            --run-ipc \
-            --with-cpp=1 \
-            --with-csharp=1 \
-            --with-java=1 \
-            --with-js=1 \
-            --with-go=1 \
-            --with-rust=1 \
-            --gold-dirs=testing/data/arrow-ipc-stream/integration/0.14.1 \
-            --gold-dirs=testing/data/arrow-ipc-stream/integration/0.17.1 \
-            --gold-dirs=testing/data/arrow-ipc-stream/integration/1.0.0-bigendian \
-            --gold-dirs=testing/data/arrow-ipc-stream/integration/1.0.0-littleendian \
-            --gold-dirs=testing/data/arrow-ipc-stream/integration/2.0.0-compression \
-            --gold-dirs=testing/data/arrow-ipc-stream/integration/4.0.0-shareddict
+        run: conda run --no-capture-output ci/scripts/integration_arrow.sh $PWD /build
 
   # test FFI against the C-Data interface exposed by pyarrow
   pyarrow-integration-test:

[tustvold]

Something like that would be perfect. One small comment is it it would be good if we preserved the ability to override the tested languages, as in the past this has been very useful for temporarily disabling certain languages when they are having issues

[kou]

It makes sense. We can implement it by environment variables such as ARROW_INTEGRATION_${LANG}=1. We use the environment variable approach in our verification script: https://github.com/apache/arrow/blob/a0e58f18dc643f9bfd4efa32331ddf477643fed2/dev/release/verify-release-candidate.sh#L1160-L1191

[tustvold]

I've opted to temporarily disable the Java tests in order to get master green, and filed #4963 to track a better solution