Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NPE when Clean.apply called with null as first argument #3736

Open
wants to merge 1 commit into
base: trunk
Choose a base branch
from

Conversation

zybreak
Copy link

@zybreak zybreak commented Dec 11, 2024

Our Cassandra 5.0.2 cluster is struggling when consuming large amounts of embeddings (>200M) and at some point (a node dies with the following stack trace:

java.lang.NullPointerException: Cannot invoke "java.lang.Boolean.booleanValue()" because "res" is null
	at org.apache.cassandra.utils.memory.MemtableCleanerThread$Clean.apply(MemtableCleanerThread.java:97)
	at org.apache.cassandra.utils.concurrent.ListenerList$CallbackBiConsumerListener.run(ListenerList.java:244)
	at org.apache.cassandra.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:140)
	at org.apache.cassandra.utils.concurrent.ListenerList.safeExecute(ListenerList.java:166)
	at org.apache.cassandra.utils.concurrent.ListenerList.notifyListener(ListenerList.java:157)
	at org.apache.cassandra.utils.concurrent.ListenerList$CallbackBiConsumerListener.notifySelf(ListenerList.java:250)
	at org.apache.cassandra.utils.concurrent.ListenerList.lambda$notifyExclusive$0(ListenerList.java:124)
	at org.apache.cassandra.utils.concurrent.IntrusiveStack.forEach(IntrusiveStack.java:195)
	at org.apache.cassandra.utils.concurrent.ListenerList.notifyExclusive(ListenerList.java:124)
	at org.apache.cassandra.utils.concurrent.ListenerList.notify(ListenerList.java:96)
	at org.apache.cassandra.utils.concurrent.AsyncFuture.trySet(AsyncFuture.java:104)
	at org.apache.cassandra.utils.concurrent.AbstractFuture.tryFailure(AbstractFuture.java:148)
	at org.apache.cassandra.utils.concurrent.AsyncPromise.tryFailure(AsyncPromise.java:139)
	at org.apache.cassandra.db.memtable.AbstractAllocatorMemtable.lambda$flushLargestMemtable$0(AbstractAllocatorMemtable.java:306)
	at org.apache.cassandra.concurrent.ImmediateExecutor.execute(ImmediateExecutor.java:140)
	at org.apache.cassandra.utils.concurrent.ListenerList.safeExecute(ListenerList.java:166)
	at org.apache.cassandra.utils.concurrent.ListenerList.notifyListener(ListenerList.java:157)
	at org.apache.cassandra.utils.concurrent.ListenerList$RunnableWithExecutor.notifySelf(ListenerList.java:345)
	at org.apache.cassandra.utils.concurrent.ListenerList.lambda$notifyExclusive$0(ListenerList.java:124)
	at org.apache.cassandra.utils.concurrent.IntrusiveStack.forEach(IntrusiveStack.java:195)
	at org.apache.cassandra.utils.concurrent.ListenerList.notifyExclusive(ListenerList.java:124)
	at org.apache.cassandra.utils.concurrent.ListenerList.notify(ListenerList.java:96)
	at org.apache.cassandra.utils.concurrent.AsyncFuture.trySet(AsyncFuture.java:104)
	at org.apache.cassandra.utils.concurrent.AbstractFuture.tryFailure(AbstractFuture.java:148)
	at org.apache.cassandra.concurrent.FutureTask.tryFailure(FutureTask.java:87)
	at org.apache.cassandra.concurrent.FutureTask.run(FutureTask.java:75)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
	at java.base/java.lang.Thread.run(Thread.java:840)

Which results from the Clean.apply method being called with null for the first argument, which seems to be intentional given the code in ListenerList.java:244, and since the apply function in MemtableCleanerThread.java:97 accepts a Boolean and not a bool primitive, we should handle it being NULL, otherwise the parameter should be of a primitive type to disallow NULL.

When the res parameter is NULL, i treat it as if it is false which seems correct from the context but this needs to be reviewed.

Addresses the immediate issue in: CASSANDRA-20141

@bbotella
Copy link
Contributor

Can you please create a Jira ticket for this?

@zybreak
Copy link
Author

zybreak commented Dec 12, 2024

@bbotella Yes, just got the account activated so here is a issue connected to this pull request https://issues.apache.org/jira/browse/CASSANDRA-20141

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants