[BUG] polaris-service:test, two tests fails

[sreev]

Is this a possible security vulnerability?

• This is NOT a possible security vulnerability

Describe the bug

Update Documentation:
start iceberg spark docker (for rest api 8181, 8182 ports) before running the polaris full build with tests.

WHY:

1. What version of Apache Polaris are you using?
   main branch

2. What operating system and processor architecture are you using?
   macOS 14.6.1 (23G93)

3. What did you do?
   very first build on git clone
   % ./gradlew build

4. What did you expect to see?
   successful build

5. What did you see instead?
   two tests failed in polaris-service module

port 8181 and/or 8182 connection timeout

% git clone https://github.com/sreev/polaris.git
% cd polaris
% ./gradlew build
...
...
...
Successfully generated code to ~/polaris/polaris-service/build/generated
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended

Task :polaris-service:javadoc
~/polaris/polaris-service/build/generated/src/main/java/org/apache/polaris/service/catalog/api/IcebergRestCatalogApi.java:489: warning: invalid input: '<'

• Load a table from the catalog. The response contains both configuration and table metadata. The configuration, if non-empty is used as additional configuration for the table that overrides catalog configuration. For example, this configuration may change the FileIO implementation to be used for the table. The response also contains the table's full metadata, matching the table metadata JSON file. The catalog configuration may contain credentials that should be used for subsequent requests for the table. The configuration key "token" is used to pass an access token to be used as a bearer token for table requests. Otherwise, a token may be passed using a RFC 8693 token type as a configuration key. For example, "urn:ietf:params:oauth:token-type:jwt=".
  ^
  ~/polaris/polaris-service/build/generated/src/main/java/org/apache/polaris/service/catalog/api/IcebergRestCatalogApi.java:527: warning: invalid input: '<'
• Load a view from the catalog. The response contains both configuration and view metadata. The configuration, if non-empty is used as additional configuration for the view that overrides catalog configuration. The response also contains the view's full metadata, matching the view metadata JSON file. The catalog configuration may contain credentials that should be used for subsequent requests for the view. The configuration key "token" is used to pass an access token to be used as a bearer token for view requests. Otherwise, a token may be passed using a RFC 8693 token type as a configuration key. For example, "urn:ietf:params:oauth:token-type:jwt=".
  ^
  2 warnings
  OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
  OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended

Task :polaris-service:test

PolarisApplicationIntegrationTest > testIcebergCreateNamespaceInExternalCatalog(TestInfo) FAILED
jakarta.ws.rs.ProcessingException at PolarisApplicationIntegrationTest.java:236
Caused by: java.net.SocketTimeoutException at PolarisApplicationIntegrationTest.java:236

PolarisOverlappingTableTest > Test restrictions on table locations > [1] extension=strict, catalog=default, status=Forbidden FAILED
jakarta.ws.rs.ProcessingException at PolarisOverlappingTableTest.java:206
Caused by: java.net.SocketTimeoutException at PolarisOverlappingTableTest.java:206

OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
INFO [2024-09-14 01:18:57,518] org.apache.spark.util.ShutdownHookManager: Shutdown hook called
INFO [2024-09-14 01:18:57,519] org.apache.spark.util.ShutdownHookManager: Deleting directory /private/var/folders/qf/1xhrknbj7f33hz2wry75k3_m0000gn/T/spark-75985dac-f980-4164-83ef-01217f28b076

Task :polaris-service:test

467 tests completed, 2 failed, 6 skipped

Task :polaris-service:test FAILED

FAILURE: Build failed with an exception.

• What went wrong:
  Execution failed for task ':polaris-service:test'.

There were failing tests. See the report at: file:///~/polaris/polaris-service/build/reports/tests/test/index.html

• Try:

Run with --scan to get full insights.

BUILD FAILED in 11m 27s
75 actionable tasks: 73 executed, 1 from cache, 1 up-to-date

testIcebergCreateNamespaceInExternalCatalog(TestInfo)

jakarta.ws.rs.ProcessingException: java.net.SocketTimeoutException: Read timed out
at app//org.glassfish.jersey.apache5.connector.Apache5Connector.apply(Apache5Connector.java:544)
at app//org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:300)
at app//org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$0(JerseyInvocation.java:674)
at app//org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:709)
at app//org.glassfish.jersey.client.JerseyInvocation.lambda$runInScope$3(JerseyInvocation.java:703)
at app//org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at app//org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at app//org.glassfish.jersey.internal.Errors.process(Errors.java:205)
at app//org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
at app//org.glassfish.jersey.client.JerseyInvocation.runInScope(JerseyInvocation.java:703)
at app//org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:673)
at app//org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:439)
at app//org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:345)
at app//org.apache.polaris.service.PolarisApplicationIntegrationTest.createCatalog(PolarisApplicationIntegrationTest.java:236)
at app//org.apache.polaris.service.PolarisApplicationIntegrationTest.createCatalog(PolarisApplicationIntegrationTest.java:190)
at app//org.apache.polaris.service.PolarisApplicationIntegrationTest.lambda$before$0(PolarisApplicationIntegrationTest.java:184)
at [email protected]/java.util.Optional.ifPresent(Optional.java:178)
at app//org.apache.polaris.service.PolarisApplicationIntegrationTest.before(PolarisApplicationIntegrationTest.java:180)
at [email protected]/java.lang.reflect.Method.invoke(Method.java:580)
at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1597)
at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1597)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:278)
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.implRead(Socket.java:1108)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1095)
at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
at org.apache.hc.core5.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:247)
at org.apache.hc.core5.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:54)
at org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:299)
at org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:175)
at org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:218)
at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint.execute(PoolingHttpClientConnectionManager.java:717)
at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.execute(InternalExecRuntime.java:216)
at org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:116)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
at org.glassfish.jersey.apache5.connector.Apache5Connector.apply(Apache5Connector.java:496)

[1] extension=strict, catalog=default, status=Forbidden

jakarta.ws.rs.ProcessingException: java.net.SocketTimeoutException: Read timed out
at app//org.glassfish.jersey.apache5.connector.Apache5Connector.apply(Apache5Connector.java:544)
at app//org.glassfish.jersey.client.ClientRuntime.invoke(ClientRuntime.java:300)
at app//org.glassfish.jersey.client.JerseyInvocation.lambda$invoke$0(JerseyInvocation.java:674)
at app//org.glassfish.jersey.client.JerseyInvocation.call(JerseyInvocation.java:709)
at app//org.glassfish.jersey.client.JerseyInvocation.lambda$runInScope$3(JerseyInvocation.java:703)
at app//org.glassfish.jersey.internal.Errors.process(Errors.java:292)
at app//org.glassfish.jersey.internal.Errors.process(Errors.java:274)
at app//org.glassfish.jersey.internal.Errors.process(Errors.java:205)
at app//org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:390)
at app//org.glassfish.jersey.client.JerseyInvocation.runInScope(JerseyInvocation.java:703)
at app//org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:673)
at app//org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:439)
at app//org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:345)
at app//org.apache.polaris.service.admin.PolarisOverlappingTableTest.createTable(PolarisOverlappingTableTest.java:206)
at app//org.apache.polaris.service.admin.PolarisOverlappingTableTest.testTableLocationRestrictions(PolarisOverlappingTableTest.java:237)
at [email protected]/java.lang.reflect.Method.invoke(Method.java:580)
at [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:194)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
at [email protected]/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:782)
at [email protected]/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:291)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:1024)
at [email protected]/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
at [email protected]/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
at [email protected]/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at [email protected]/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
at [email protected]/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:611)
at [email protected]/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:291)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:212)
at [email protected]/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
at [email protected]/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
at [email protected]/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
at [email protected]/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at [email protected]/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
at [email protected]/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:611)
at [email protected]/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:291)
at [email protected]/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1709)
at [email protected]/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:556)
at [email protected]/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:546)
at [email protected]/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at [email protected]/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at [email protected]/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
at [email protected]/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:611)
at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1597)
at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1597)
Caused by: java.net.SocketTimeoutException: Read timed out
at java.base/sun.nio.ch.NioSocketImpl.timedRead(NioSocketImpl.java:278)
at java.base/sun.nio.ch.NioSocketImpl.implRead(NioSocketImpl.java:304)
at java.base/sun.nio.ch.NioSocketImpl.read(NioSocketImpl.java:346)
at java.base/sun.nio.ch.NioSocketImpl$1.read(NioSocketImpl.java:796)
at java.base/java.net.Socket$SocketInputStream.implRead(Socket.java:1108)
at java.base/java.net.Socket$SocketInputStream.read(Socket.java:1095)
at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:149)
at org.apache.hc.core5.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:280)
at org.apache.hc.core5.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:247)
at org.apache.hc.core5.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:54)
at org.apache.hc.core5.http.impl.io.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:299)
at org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:175)
at org.apache.hc.core5.http.impl.io.HttpRequestExecutor.execute(HttpRequestExecutor.java:218)
at org.apache.hc.client5.http.impl.io.PoolingHttpClientConnectionManager$InternalConnectionEndpoint.execute(PoolingHttpClientConnectionManager.java:717)
at org.apache.hc.client5.http.impl.classic.InternalExecRuntime.execute(InternalExecRuntime.java:216)
at org.apache.hc.client5.http.impl.classic.MainClientExec.execute(MainClientExec.java:116)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.ConnectExec.execute(ConnectExec.java:188)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.ProtocolExec.execute(ProtocolExec.java:192)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.HttpRequestRetryExec.execute(HttpRequestRetryExec.java:113)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.RedirectExec.execute(RedirectExec.java:116)
at org.apache.hc.client5.http.impl.classic.ExecChainElement.execute(ExecChainElement.java:51)
at org.apache.hc.client5.http.impl.classic.InternalHttpClient.doExecute(InternalHttpClient.java:170)
at org.apache.hc.client5.http.impl.classic.CloseableHttpClient.execute(CloseableHttpClient.java:87)
at org.glassfish.jersey.apache5.connector.Apache5Connector.apply(Apache5Connector.java:496)
... 59 more

To Reproduce

1. What version of Apache Polaris are you using?
   main branch

% git clone https://github.com/sreev/polaris.git
% cd polaris
% ./gradlew build

Actual Behavior

5. What did you see instead?
   two tests failed in polaris-service module

port 8181 and/or 8182 connection timeout

Expected Behavior

4. What did you expect to see?
   successful build

Additional context

% git clone https://github.com/sreev/docker-spark-iceberg.git
% code docker-spark-iceberg
docker-spark-iceberg % docker-compose up

% ./gradlew build
Configuration on demand is an incubating feature.

Task :polaris-build-logic:compileKotlin UP-TO-DATE
Kotlin does not yet support 22 JDK target, falling back to Kotlin JVM_21 JVM target

Configure project :
Kotlin does not yet support 22 JDK target, falling back to Kotlin JVM_21 JVM target
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
OpenJDK 64-Bit Server VM warning: Sharing is only supported for boot loader classes because bootstrap classpath has been appended
INFO [2024-09-14 03:18:37,554] org.apache.spark.util.ShutdownHookManager: Shutdown hook called
INFO [2024-09-14 03:18:37,555] org.apache.spark.util.ShutdownHookManager: Deleting directory /private/var/folders/qf/1xhrknbj7f33hz2wry75k3_m0000gn/T/spark-72409bf0-7ebd-425e-b656-956504834fa4

BUILD SUCCESSFUL in 3m 56s
76 actionable tasks: 13 executed, 63 up-to-date

System information

2. What operating system and processor architecture are you using?
   macOS 14.6.1 (23G93)

[flyrain]

The above failed tests don't fail in github pipelines and my local. I suspect it is related to network setting/status in your MacOS. Can you run two test cases separately to check it works?