Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable. #3794

Closed
morph166955 opened this issue Sep 2, 2023 · 3 comments
Labels
bug An unexpected problem or unintended behavior of the Core

Comments

@morph166955
Copy link

Using my neeo remote tonight to control a device through the http binding, system became unstable. Logs seem to indicate that I hit this again. Probably related to #3702

2023-09-01 22:17:10.795 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:10.865 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:10.865 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:10.876 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:10.893 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:10.893 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:10.894 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:10.958 [WARN ] [p.internal.http.HttpResponseListener] - Requesting 'http://10.255.0.52:8060/query/device-info' (method='GET', content='null') failed: java.util.concurrent.TimeoutException: Total timeout 3000 ms elapsed
2023-09-01 22:17:10.958 [WARN ] [p.internal.http.HttpResponseListener] - Requesting 'http://10.255.0.52:8060/query/active-app' (method='GET', content='null') failed: java.util.concurrent.TimeoutException: Total timeout 3000 ms elapsed
2023-09-01 22:17:11.102 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.102 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.143 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.143 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.144 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.145 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.145 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.146 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.146 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.147 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.147 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.
2023-09-01 22:17:11.148 [WARN ] [ab.core.internal.events.EventHandler] - The queue for a subscriber of type 'class org.openhab.core.internal.items.ItemUpdater' exceeds 5000 elements. System may be unstable.

@morph166955 morph166955 added the bug An unexpected problem or unintended behavior of the Core label Sep 2, 2023
@morph166955
Copy link
Author

Unsure if this is related. I'm having issues with an enormous thread count. I have over 4000 active threads of type "HttpClient-1-SelectorManager" (with different numbers obviously). All resemble:

"HttpClient-14819-SelectorManager" Id=38543 in RUNNABLE (running in native)
    at [email protected]/sun.nio.ch.EPoll.wait(Native Method)
    at [email protected]/sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:118)
    at [email protected]/sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:129)
      - locked sun.nio.ch.Util$2@11af4028
      - locked sun.nio.ch.EPollSelectorImpl@13b07a12
    at [email protected]/sun.nio.ch.SelectorImpl.select(SelectorImpl.java:141)
    at platform/[email protected]/jdk.internal.net.http.HttpClientImpl$SelectorManager.run(HttpClientImpl.java:889)

@morph166955
Copy link
Author

This looks to be tied to the neeo integration addon. I removed both neeo pieces and reloaded, the count sat at 1. I added only the integration and the count jumps to over 300 immediately.

It also is throwing this:

2023-09-07 21:04:18.043 [WARN ] [ache.cxf.phase.PhaseInterceptorChain] - Interceptor for {http://10.255.0.201:3000/v1/api/unregisterSdkDeviceAdapter}WebClient has thrown exception, unwinding now
org.apache.cxf.interceptor.Fault: Could not send Message.
        at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:67) ~[?:?]
        at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307) ~[?:?]
        at org.apache.cxf.jaxrs.client.AbstractClient.doRunInterceptorChain(AbstractClient.java:710) ~[?:?]
        at org.apache.cxf.jaxrs.client.WebClient.doChainedInvocation(WebClient.java:1086) ~[?:?]
        at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:932) ~[?:?]
        at org.apache.cxf.jaxrs.client.WebClient.doInvoke(WebClient.java:901) ~[?:?]
        at org.apache.cxf.jaxrs.client.WebClient.invoke(WebClient.java:461) ~[?:?]
        at org.apache.cxf.jaxrs.client.SyncInvokerImpl.method(SyncInvokerImpl.java:150) ~[?:?]
        at org.apache.cxf.jaxrs.client.SyncInvokerImpl.method(SyncInvokerImpl.java:145) ~[?:?]
        at org.apache.cxf.jaxrs.client.SyncInvokerImpl.post(SyncInvokerImpl.java:85) ~[?:?]
        at org.apache.cxf.jaxrs.client.spec.InvocationBuilderImpl.post(InvocationBuilderImpl.java:153) ~[?:?]
        at org.openhab.io.neeo.internal.net.HttpRequest.sendPostJsonCommand(HttpRequest.java:96) ~[?:?]
        at org.openhab.io.neeo.internal.NeeoApi.deregisterApi(NeeoApi.java:495) ~[?:?]
        at org.openhab.io.neeo.internal.NeeoApi.registerApi(NeeoApi.java:424) ~[?:?]
        at org.openhab.io.neeo.internal.NeeoApi.connect(NeeoApi.java:398) ~[?:?]
        at org.openhab.io.neeo.internal.NeeoApi.lambda$0(NeeoApi.java:351) ~[?:?]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539) ~[?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) ~[?:?]
        at java.lang.Thread.run(Thread.java:833) ~[?:?]
Caused by: java.io.IOException: IOException invoking http://10.255.0.201:3000/v1/api/unregisterSdkDeviceAdapter: java.lang.InterruptedException
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) ~[?:?]
        at jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[?:?]
        at java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) ~[?:?]
        at java.lang.reflect.Constructor.newInstance(Constructor.java:480) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.mapException(HTTPConduit.java:1430) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1411) ~[?:?]
        at org.apache.cxf.io.AbstractWrappedOutputStream.close(AbstractWrappedOutputStream.java:77) ~[?:?]
        at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:695) ~[?:?]
        at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63) ~[?:?]
        ... 21 more
Caused by: java.io.IOException: java.lang.InterruptedException
        at org.apache.cxf.transport.http.HttpClientHTTPConduit$HttpClientWrappedOutputStream.getResponse(HttpClientHTTPConduit.java:460) ~[?:?]
        at org.apache.cxf.transport.http.HttpClientHTTPConduit$HttpClientWrappedOutputStream.getResponseCode(HttpClientHTTPConduit.java:469) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.doProcessResponseCode(HTTPConduit.java:1631) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1662) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1604) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1398) ~[?:?]
        at org.apache.cxf.io.AbstractWrappedOutputStream.close(AbstractWrappedOutputStream.java:77) ~[?:?]
        at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:695) ~[?:?]
        at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63) ~[?:?]
        ... 21 more
Caused by: java.lang.InterruptedException
        at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:386) ~[?:?]
        at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:2096) ~[?:?]
        at org.apache.cxf.transport.http.HttpClientHTTPConduit$HttpClientWrappedOutputStream.getResponse(HttpClientHTTPConduit.java:440) ~[?:?]
        at org.apache.cxf.transport.http.HttpClientHTTPConduit$HttpClientWrappedOutputStream.getResponseCode(HttpClientHTTPConduit.java:469) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.doProcessResponseCode(HTTPConduit.java:1631) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponseInternal(HTTPConduit.java:1662) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.handleResponse(HTTPConduit.java:1604) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit$WrappedOutputStream.close(HTTPConduit.java:1398) ~[?:?]
        at org.apache.cxf.io.AbstractWrappedOutputStream.close(AbstractWrappedOutputStream.java:77) ~[?:?]
        at org.apache.cxf.transport.AbstractConduit.close(AbstractConduit.java:56) ~[?:?]
        at org.apache.cxf.transport.http.HTTPConduit.close(HTTPConduit.java:695) ~[?:?]
        at org.apache.cxf.interceptor.MessageSenderInterceptor$MessageSenderEndingInterceptor.handleMessage(MessageSenderInterceptor.java:63) ~[?:?]
        ... 21 more



@morph166955
Copy link
Author

Closing this and opening an issue on addons

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug An unexpected problem or unintended behavior of the Core
Projects
None yet
Development

No branches or pull requests

1 participant