Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Function test Cassandra Unit connection errors with Cassandra Server v3.x and JDK11. #1621

Closed
whitingjr opened this issue May 12, 2020 · 10 comments

Comments

@whitingjr
Copy link
Contributor

When running the Function Tests locally I am seeing Cassandra client connection errors. When Cassandra server is using JDK11.

In this example I am running the RemoteRepoGetErrorTest test.

13:26:19.663 [RemoteRepoGetErrorTest.run] DEBUG c.d.driver.core.SystemProperties - com.datastax.driver.USE_NATIVE_CLOCK is undefined, using default value true
13:26:19.663 [RemoteRepoGetErrorTest.run] INFO c.datastax.driver.core.ClockFactory - Using native clock to generate timestamps.
13:26:19.663 [RemoteRepoGetErrorTest.run] DEBUG o.c.s.p.p.datastax.CassandraPathDB - Connecting to Cassandra, host:localhost, port:9042
13:26:19.663 [RemoteRepoGetErrorTest.run] DEBUG com.datastax.driver.core.Cluster - Starting new cluster with contact points [localhost/127.0.0.1:9042]
13:26:19.722 [RemoteRepoGetErrorTest.run] DEBUG com.datastax.driver.core.Host.STATES - [localhost/127.0.0.1:9042] preparing to open 1 new connections, total = 1
13:26:19.724 [cluster2-nio-worker-0] DEBUG com.datastax.driver.core.Connection - Connection[localhost/127.0.0.1:9042-1, inFlight=0, closed=false] Connection established, initializing transport
13:26:19.740 [cluster2-nio-worker-0] DEBUG com.datastax.driver.core.Host.STATES - Defuncting Connection[localhost/127.0.0.1:9042-1, inFlight=0, closed=false] because: [localhost/127.0.0.1:9042] Error initializing connection: java.lang.RuntimeException: Unable to access address of buffer
13:26:19.740 [cluster2-nio-worker-0] DEBUG com.datastax.driver.core.Host.STATES - [localhost/127.0.0.1:9042] preventing new connections for the next 1000 ms
13:26:19.740 [cluster2-nio-worker-0] DEBUG com.datastax.driver.core.Host.STATES - [localhost/127.0.0.1:9042] Connection[localhost/127.0.0.1:9042-1, inFlight=0, closed=false] failed, remaining = 0
13:26:19.740 [cluster2-nio-worker-0] DEBUG com.datastax.driver.core.Connection - Connection[localhost/127.0.0.1:9042-1, inFlight=0, closed=true] closing connection
13:26:19.741 [cluster2-nio-worker-0] DEBUG com.datastax.driver.core.Connection - Connection[localhost/127.0.0.1:9042-1, inFlight=0, closed=true] has already terminated
13:26:19.741 [RemoteRepoGetErrorTest.run] DEBUG c.d.driver.core.ControlConnection - [Control connection] error on localhost/127.0.0.1:9042 connection, no more host to try
com.datastax.driver.core.exceptions.TransportException: [localhost/127.0.0.1:9042] Error initializing connection: java.lang.RuntimeException: Unable to access address of buffer
at com.datastax.driver.core.Connection$5.apply(Connection.java:332)
at com.datastax.driver.core.Connection$5.apply(Connection.java:320)
at com.google.common.util.concurrent.Futures$ChainingListenableFuture.run(Futures.java:906)
at com.google.common.util.concurrent.Futures$1$1.run(Futures.java:635)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutorService.execute(MoreExecutors.java:299)
at com.google.common.util.concurrent.Futures$1.run(Futures.java:632)
at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:457)
at com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
at com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
at com.google.common.util.concurrent.AbstractFuture.set(AbstractFuture.java:185)
at com.datastax.driver.core.Connection$Future.onSet(Connection.java:1486)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1240)
at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1158)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:310)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:284)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
at com.datastax.driver.core.InboundTrafficMeter.channelRead(InboundTrafficMeter.java:38)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:808)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:417)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:317)
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:884)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:748)
13:26:19.742 [RemoteRepoGetErrorTest.run] DEBUG com.datastax.driver.core.Cluster - Shutting down
13:26:19.743 [RemoteRepoGetErrorTest.run] ERROR o.c.i.f.d.DefaultGalleyStorageProvider - [Indy] Can not create CacheProvider for some error.
org.commonjava.maven.galley.GalleyInitException: Create cache provider failed
at org.commonjava.maven.galley.cache.pathmapped.PathMappedCacheProviderFactory.create(PathMappedCacheProviderFactory.java:87)
at org.commonjava.indy.filer.def.DefaultGalleyStorageProvider.getCacheProvider(DefaultGalleyStorageProvider.java:377)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:88)
at org.jboss.weld.injection.StaticMethodInjectionPoint.invoke(StaticMethodInjectionPoint.java:78)
at org.jboss.weld.injection.producer.ProducerMethodProducer.produce(ProducerMethodProducer.java:100)
at org.jboss.weld.injection.producer.AbstractMemberProducer.produce(AbstractMemberProducer.java:161)
at org.jboss.weld.bean.AbstractProducerBean.create(AbstractProducerBean.java:180)
at org.jboss.weld.context.unbound.DependentContextImpl.get(DependentContextImpl.java:70)
at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:100)
at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
at org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:786)
at org.jboss.weld.manager.BeanManagerImpl.getInjectableReference(BeanManagerImpl.java:886)
at org.jboss.weld.injection.FieldInjectionPoint.inject(FieldInjectionPoint.java:92)
at org.jboss.weld.util.Beans.injectBoundFields(Beans.java:358)
at org.jboss.weld.util.Beans.injectFieldsAndInitializers(Beans.java:369)
at org.jboss.weld.injection.producer.ResourceInjector$1.proceed(ResourceInjector.java:70)
at org.jboss.weld.injection.InjectionContextImpl.run(InjectionContextImpl.java:48)
at org.jboss.weld.injection.producer.ResourceInjector.inject(ResourceInjector.java:72)
at org.jboss.weld.injection.producer.BasicInjectionTarget.inject(BasicInjectionTarget.java:117)
at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:159)
at org.jboss.weld.context.unbound.DependentContextImpl.get(DependentContextImpl.java:70)
at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:100)
at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
at org.jboss.weld.manager.BeanManagerImpl.getReference(BeanManagerImpl.java:786)
at org.jboss.weld.manager.BeanManagerImpl.getInjectableReference(BeanManagerImpl.java:886)
at org.jboss.weld.injection.FieldInjectionPoint.inject(FieldInjectionPoint.java:92)
at org.jboss.weld.util.Beans.injectBoundFields(Beans.java:358)
at org.jboss.weld.util.Beans.injectFieldsAndInitializers(Beans.java:369)
at org.jboss.weld.injection.producer.ResourceInjector$1.proceed(ResourceInjector.java:70)
at org.jboss.weld.injection.InjectionContextImpl.run(InjectionContextImpl.java:48)
at org.jboss.weld.injection.producer.ResourceInjector.inject(ResourceInjector.java:72)
at org.jboss.weld.injection.producer.BasicInjectionTarget.inject(BasicInjectionTarget.java:117)
at org.jboss.weld.bean.ManagedBean.create(ManagedBean.java:159)
at org.jboss.weld.context.AbstractContext.get(AbstractContext.java:96)
at org.jboss.weld.bean.ContextualInstanceStrategy$DefaultContextualInstanceStrategy.get(ContextualInstanceStrategy.java:100)
at org.jboss.weld.bean.ContextualInstanceStrategy$ApplicationScopedContextualInstanceStrategy.get(ContextualInstanceStrategy.java:140)
at org.jboss.weld.bean.ContextualInstance.get(ContextualInstance.java:50)
at org.jboss.weld.bean.proxy.ContextBeanInstance.getInstance(ContextBeanInstance.java:99)
at org.jboss.weld.bean.proxy.ProxyMethodHandler.getInstance(ProxyMethodHandler.java:125)
at org.commonjava.indy.httprox.HttpProxy$Proxy$_$$WeldClientProxy.getStartupPriority(Unknown Source)
at org.commonjava.indy.action.IndyLifecycleManager$3.compare(IndyLifecycleManager.java:331)
at org.commonjava.indy.action.IndyLifecycleManager$3.compare(IndyLifecycleManager.java:327)
at java.util.TimSort.binarySort(TimSort.java:296)
at java.util.TimSort.sort(TimSort.java:221)
at java.util.Arrays.sort(Arrays.java:1512)
at java.util.ArrayList.sort(ArrayList.java:1462)
at java.util.Collections.sort(Collections.java:177)
at org.commonjava.indy.action.IndyLifecycleManager.runStartupActions(IndyLifecycleManager.java:234)
at org.commonjava.indy.action.IndyLifecycleManager.start(IndyLifecycleManager.java:144)
at org.commonjava.indy.action.IndyLifecycleManager$Proxy$
$$_WeldClientProxy.start(Unknown Source)
at org.commonjava.indy.boot.jaxrs.JaxRsBooter.startLifecycle(JaxRsBooter.java:108)
at org.commonjava.propulsor.boot.Booter.start(Booter.java:278)
at org.commonjava.indy.test.fixture.core.CoreServerFixture.start(CoreServerFixture.java:119)
at org.commonjava.indy.ftest.core.AbstractIndyFunctionalTest.start(AbstractIndyFunctionalTest.java:121)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24)
at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.TransportException: [localhost/127.0.0.1:9042] Error initializing connection: java.lang.RuntimeException: Unable to access address of buffer))
at com.datastax.driver.core.ControlConnection.reconnectInternal(ControlConnection.java:268)
at com.datastax.driver.core.ControlConnection.connect(ControlConnection.java:107)
at com.datastax.driver.core.Cluster$Manager.negotiateProtocolVersionAndConnect(Cluster.java:1652)
at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1571)
at com.datastax.driver.core.Cluster.init(Cluster.java:208)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376)
at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:355)
at com.datastax.driver.core.Cluster.connect(Cluster.java:305)
at org.commonjava.storage.pathmapped.pathdb.datastax.CassandraPathDB.(CassandraPathDB.java:116)
at org.commonjava.maven.galley.cache.pathmapped.PathMappedCacheProviderFactory.create(PathMappedCacheProviderFactory.java:75)
... 72 common frames omitted
13:26:19.754 [RemoteRepoGetErrorTest.run] INFO o.c.indy.action.IndyLifecycleManager - Running startup actions...
13:26:19.755 [RemoteRepoGetErrorTest.run] INFO o.c.indy.action.IndyLifecycleManager - Running startup action: 'koji-client'
13:26:19.755 [RemoteRepoGetErrorTest.run] INFO o.c.indy.action.IndyLifecycleManager - Running startup action: 'folo-backup-sealed-records'
13:26:19.762 [RemoteRepoGetErrorTest.run] INFO o.c.indy.core.inject.CoreProvider - Apply nfc provider: ispn
13:26:19.774 [RemoteRepoGetErrorTest.run] DEBUG o.i.commons.util.ServiceFinder - No service impls found: ProgrammaticSearchMappingProvider

The knock on effect is the fixture server cannot instanciate the cacheProvider field. Which results in a NullPointerException.

@whitingjr
Copy link
Contributor Author

whitingjr commented May 26, 2020

I found this issue only happens when the jar artifacts have been compiled/generated using JDK8.
Not certain this is a cause for concern. The workaround is to use JDK11 to compile/generate everything.

@geored
Copy link
Member

geored commented May 26, 2020 via email

@whitingjr
Copy link
Contributor Author

It seems like Cassandra connections must be "warm" up before trying to query it?!

Could be necessary yes. I don't know. It requires further testing.

@whitingjr
Copy link
Contributor Author

whitingjr commented Jul 24, 2020

Looking at an earlier exception I notice this. Likely to be relevant.
Netty has a phase to discover the abilities of the JVM. Here we can see in the case of JDK11 direct buffer construction is not available.

16:02:49.182 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.DirectByteBuffer.<init>(long, int): unavailable
16:02:49.182 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available
16:02:49.186 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent - maxDirectMemory: 3061841920 bytes (maybe)
16:02:49.186 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
16:02:49.186 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
16:02:49.187 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
16:02:49.178 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false
16:02:49.188 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.maxDirectMemory: -1 bytes
16:02:49.188 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.uninitializedArrayAllocationThreshold: -1
16:02:49.188 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent0 - Java version: 11
16:02:49.189 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG io.netty.util.internal.CleanerJava9 - java.nio.ByteBuffer.cleaner(): available
16:02:49.189 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
16:02:49.189 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
16:02:49.190 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
16:02:49.193 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG c.d.driver.core.SystemProperties - com.datastax.driver.FORCE_NIO is undefined, using default value false
16:02:49.191 [StoreOneAndSourceStoreUrlInHtmlListingTest.null] DEBUG i.n.util.internal.PlatformDependent0 - direct buffer constructor: unavailable
java.lang.UnsupportedOperationException: Reflective setAccessible(true) disabled
        at io.netty.util.internal.ReflectionUtil.trySetAccessible(ReflectionUtil.java:31)
        at io.netty.util.internal.PlatformDependent0$4.run(PlatformDependent0.java:224)
        at java.base/java.security.AccessController.doPrivileged(Native Method)
        at io.netty.util.internal.PlatformDependent0.<clinit>(PlatformDependent0.java:218)
        at io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:208)
        at io.netty.util.internal.PlatformDependent.<clinit>(PlatformDependent.java:79)
        at io.netty.channel.group.DefaultChannelGroup.<init>(DefaultChannelGroup.java:47)
        at io.netty.channel.group.DefaultChannelGroup.<init>(DefaultChannelGroup.java:83)
        at io.netty.channel.group.DefaultChannelGroup.<init>(DefaultChannelGroup.java:64)
        at com.datastax.driver.core.Connection$Factory.<init>(Connection.java:909)
        at com.datastax.driver.core.Cluster$Manager.init(Cluster.java:1499)
        at com.datastax.driver.core.Cluster.init(Cluster.java:208)
        at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:376)
        at com.datastax.driver.core.Cluster.connectAsync(Cluster.java:355)
        at com.datastax.driver.core.Cluster.connect(Cluster.java:305)
        at org.commonjava.indy.subsys.cassandra.CassandraClient.lambda$getSession$0(CassandraClient.java:106)


Whereas with JDK8 in is

18:44:56.690 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false
18:44:56.690 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent0 - Java version: 8
18:44:56.690 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
18:44:56.691 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
18:44:56.691 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
18:44:56.692 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent0 - direct buffer constructor: available
18:44:56.692 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true
18:44:56.692 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable prior to Java9
18:44:56.692 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent0 - java.nio.DirectByteBuffer.<init>(long, int): available
18:44:56.692 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent - sun.misc.Unsafe: available
18:44:56.693 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.tmpdir: /tmp (java.io.tmpdir)
18:44:56.693 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
18:44:56.693 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
18:44:56.693 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.maxDirectMemory: 5602017280 bytes
18:44:56.693 [MetaListingRescheduleTimeoutTest.timeout] DEBUG i.n.util.internal.PlatformDependent - -Dio.netty.uninitializedArrayAllocationThreshold: -1

Considering the connector driver is having this issue "Unable to access address of buffer" we might have narrowed down the culprit a little.

@emerkle826
Copy link

Looking at an earlier exception I notice this. Likely to be relevant.
Netty has a phase to discover the abilities of the JVM. Here we can see in the case of JDK11 direct buffer construction is not available.

That stack trace from Netty is a bit misleading. It's not causing the issue you have, just reporting that some of Netty's "optimizations" aren't available when running under JDK 9+. See here for more: netty/netty#7769.

I'm looking at the Driver ticket here and with a simple project that uses the driver to connect to Cassandra and execute a query, I see the same DEBUG log from Netty (when running with JDK11), but the connection and query are successful. I'm going through the combinations of compile/runtime JVMs and I hope to find the solution or fix soon.

@whitingjr
Copy link
Contributor Author

@emerkle826 If you need help configuring the Indy project to quickly execute an individual failing test do ask.
I've been using OpenJDK. Not tested with Oracle's JVM.

@whitingjr
Copy link
Contributor Author

whitingjr commented Aug 18, 2020

Some more investigation about re-creating the issue. The default distribution has some influence here when running the ftests with Cassandra. The JAVA_HOME env variable is ignored when booting the failsafe tests.
When the default java is jdk8 the tests are a SUCCESS. When it is jdk11 then the test has FAILUREs.

@whitingjr
Copy link
Contributor Author

whitingjr commented Aug 19, 2020

Bah, Cassandra 3.x is only compatible with jdk8. Cassandra 4.0 supports JDK11. This issue tracks the update.
Using the org.cassandraunit:cassandra-unit:3.7.1.0 means being coupled to Cassandra 3.x release for the time being. This cassandra-unit issue discusses the state of the project with JDK11.
This isn't a blocker to JDK11 adoption on Indy. The ftests run to completion using JDK11.
The caveat is developers must ensure the default jvm on the platform is JDK8. To ensure the forked Cassandra 3.x is using a supported JVM. Otherwise the datastax connector will fail to establish a connection throwing a RuntimeException with the message Unable to access address of buffer.

@emerkle826
Copy link

@whitingjr Sorry I haven't replied sooner.
I didn't realize you were using cassandra-unit (and to be honest, I am not terribly familiar with it). Just to reiterate, you should be able to use the 3.x driver in an application that runs in JRE/JDK11. The 3.x driver should work with Java 6 and up, though the Maven artifacts are compiled with Java 8.

@whitingjr whitingjr changed the title Function test Cassandra client connection errors with JDK11. Function test Cassandra Unit connection errors with Cassandra Server v3.x and JDK11. Sep 28, 2020
@jdcasey
Copy link
Member

jdcasey commented Sep 28, 2020

I've filed an issue in our team to avoid Cassandra for local functional test runs, then use pre- and post- steps in our CI pipeline to manage keyspaces in a long-running Cassandra instance in our test environment. This should allow us to sidestep the issues with cassandra-unit. I don't think that issue made the cut for the current sprint, so I'm hoping to get it done in the next month or less.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants