Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection allocation errors propagate to default Operators.onErrorDropped handler #216

Open
joedj opened this issue Aug 29, 2024 · 1 comment

Comments

@joedj
Copy link

joedj commented Aug 29, 2024

Bug Report

Hi,

I'm not entirely sure whether this is an r2dbc-pool issue, or a reactor-pool issue, or whether it should be considered an issue at all.

I notice that io.r2dbc.pool.ConnectionPool installs a context-specific hook for reactor.onNextDropped.local, but it does not install any such hook for reactor.onErrorDropped.local.

We see errors like this in our logs. These errors never propagate to the application layer, which makes me think they are caused by something happening in the background - perhaps when reactor-pool tries to maintain spare idle connections to satisfy the minIdle setting:

{
  "@timestamp": "2024-07-31T08:09:36.315Z",
  "message": "Operator called default onErrorDropped",
  "logger": "Operators",
  "thread": "parallel-4",
  "level": "ERROR",
  "stacktrace": "<see below>"
}

java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 5000ms in 'Connection Allocation' (and no fallback has been configured)
	at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.handleTimeout(FluxTimeout.java:296)
	at reactor.core.publisher.FluxTimeout$TimeoutMainSubscriber.doTimeout(FluxTimeout.java:281)
	at reactor.core.publisher.FluxTimeout$TimeoutTimeoutSubscriber.onNext(FluxTimeout.java:420)
	at reactor.core.publisher.FluxOnErrorReturn$ReturnSubscriber.onNext(FluxOnErrorReturn.java:162)
	at reactor.core.publisher.MonoDelay$MonoDelayRunnable.propagateDelay(MonoDelay.java:270)
	at reactor.core.publisher.MonoDelay$MonoDelayRunnable.run(MonoDelay.java:285)
	at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68)
	at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

My questions are:

  • since these errors are not being handled, is there some cleanup step (e.g. disposeConnection?) that is not occurring?
  • would it be possible/appropriate for r2dbc-pool or reactor-pool to handle these errors in a way that does not log at ERROR level?

Versiouns

  • Driver: org.postgresql:r2dbc-postgresql:jar:1.0.5.RELEASE
  • Database: Aurora Serverless PostgreSQL 15.5
  • Java: OpenJDK 64-Bit Server VM Temurin-17.0.11+9 (build 17.0.11+9, mixed mode)
  • OS: MacOS Darwin Kernel Version 23.5.0

Current Behavior

Errors propagate to the default Operators.onErrorDropped handler, and log at ERROR level.

Expected behavior/code

  • Any appropriate cleanup is performed, if this is not already happening.
  • Errors are handled in a way that does not result in an ERROR log entry.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
@joedj and others