-
Notifications
You must be signed in to change notification settings - Fork 922
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Provide blockhound integrations (#4493)
Motivation: Armeria users often use `BlockHound` to determine whether there are any blocking calls in their applications. Providing a basic `BlockHound` integration can help de-duplicate some code and debugging efforts. I propose the following: - Our CI checks for blocking calls in our tests to keep our `BlockHound` integration up-to-date with changes. - We provide a **lenient** list of allowed blocking calls that may be commonly used even though not related directly to Armeria - Some JDK implementations internally use locks (`ThreadPoolExecutor`). As long as these locks are verified to be short-living, we can add it to our integration. - Although somewhat subjective, if short-living locks are held by Armeria integrations (not necessarily directly related to Armeria) I think it's reasonable to add it to our list. - HdrHistogram is used internally by Micrometer. Micrometer itself doesn't expect metrics to be recorded from event loops, so it may not be reasonable to expect `Micrometer` or `HdrHistogram` to provide `Blockhound` integrations. Since Armeria uses `Micrometer` to record metrics from event loops, I think it is reasonable that we add `HdrHistogram` related calls to our `Blockhound` integration. - The integration is public so that users can opt-out of some integrations if they choose to do so. - We can possibly provide more customization points if users feel some rules are too lenient. - I don't consider the current listing as complete/finalized, and expect that users can suggest additional rules in the future. Also note that there are some blocking calls within the codebase, but this PR attempts to just introduce blockhound rather than remove such calls. Modifications: - Modify the build action so that - The blockhound flag is set for java 19 - Upload blockhound logs which contains a list of blocked threads and their stacktraces - Change the artifact upload name. Currently, because the name can be duplicated for java19 (mac, windows, self-hosted), only one of these is downloadable - Provide `BlockHound` integrations for `core`, `brave`, `grpc`, `retrofit2`, `scala`, `sangria` modules - Provide an internal testing `BlockHound` integration which allows test-specific blocking calls - This integration also adds a callback which logs blocking calls to a separate file - If the `blockhound` gradle property flag is specified, run tests with `BlockHound` enabled - Otherwise, `BlockHound.install` is not invoked and not enabled in the code-base - Introduce `ReentrantShortLock` which Armeria's `BlockHound` whitelists and change all usages of `ReentrantLock` - `BlockingFastThreadLocalThread` is now used for non-event loops since netty whitelists `FastThreadLocalThread` with `permitBlockingCalls==true` - #4493 (comment) - Note that this means that armeria requires at least Netty 4.1.85 now - `ShutdownSupport::shutdown` is now called from `startStopExecutor`. Although this method returns a CF, the call can actually be blocking - `START_STOP_EXECUTOR` is now a non-eventloop daemon thread instead of using the `GlobalEventExecutor` - This is because `GlobalEventExecutor` is an event loop. - Introduce `BlockingUtils#blockingRun` in case users want to run blocking calls from tests - Add the `kotlinx-coroutines-debug` dependency so that Coroutines `BlockHound` integration is loaded - https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-debug/kotlinx.coroutines.debug/-coroutines-block-hound-integration/ - Allow users to specify the `ThreadFactory` when creating a `EventLoopExtension` or `EventLoopGroupExtension` - This can be useful when users would like to create an event loop which allows blocking calls - `SamlService`, `SamlDecorator` now runs some calls from the blocking executor - `ZookeeperEndpointGroup` now uses a non-event-loop thread factory Result: - Users can now use `BlockHound` integrations provided by armeria
- Loading branch information
Showing
74 changed files
with
719 additions
and
132 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
34 changes: 34 additions & 0 deletions
34
brave/src/main/java/com/linecorp/armeria/common/brave/BraveBlockHoundIntegration.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| /* | ||
| * Copyright 2022 LINE Corporation | ||
| * | ||
| * LINE Corporation licenses this file to you under the Apache License, | ||
| * version 2.0 (the "License"); you may not use this file except in compliance | ||
| * with the License. You may obtain a copy of the License at: | ||
| * | ||
| * https://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT | ||
| * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the | ||
| * License for the specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| package com.linecorp.armeria.common.brave; | ||
|
|
||
| import com.linecorp.armeria.common.annotation.UnstableApi; | ||
|
|
||
| import reactor.blockhound.BlockHound.Builder; | ||
| import reactor.blockhound.integration.BlockHoundIntegration; | ||
|
|
||
| /** | ||
| * A {@link BlockHoundIntegration} for the brave module. | ||
| */ | ||
| @UnstableApi | ||
| public final class BraveBlockHoundIntegration implements BlockHoundIntegration { | ||
|
|
||
| @Override | ||
| public void applyTo(Builder builder) { | ||
| builder.allowBlockingCallsInside("zipkin2.reporter.AsyncReporter$BoundedAsyncReporter", "report"); | ||
| } | ||
| } |
1 change: 1 addition & 0 deletions
1
...src/main/resources/META-INF/services/reactor.blockhound.integration.BlockHoundIntegration
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| com.linecorp.armeria.common.brave.BraveBlockHoundIntegration |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
69 changes: 69 additions & 0 deletions
69
core/src/main/java/com/linecorp/armeria/common/CoreBlockHoundIntegration.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| /* | ||
| * Copyright 2023 LINE Corporation | ||
| * | ||
| * LINE Corporation licenses this file to you under the Apache License, | ||
| * version 2.0 (the "License"); you may not use this file except in compliance | ||
| * with the License. You may obtain a copy of the License at: | ||
| * | ||
| * https://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, WITHOUT | ||
| * WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the | ||
| * License for the specific language governing permissions and limitations | ||
| * under the License. | ||
| */ | ||
|
|
||
| package com.linecorp.armeria.common; | ||
|
|
||
| import java.util.ResourceBundle; | ||
|
|
||
| import com.linecorp.armeria.common.annotation.UnstableApi; | ||
|
|
||
| import reactor.blockhound.BlockHound.Builder; | ||
| import reactor.blockhound.integration.BlockHoundIntegration; | ||
|
|
||
| /** | ||
| * A {@link BlockHoundIntegration} for the core module. | ||
| */ | ||
| @UnstableApi | ||
| public final class CoreBlockHoundIntegration implements BlockHoundIntegration { | ||
| @Override | ||
| public void applyTo(Builder builder) { | ||
| // short locks | ||
| builder.allowBlockingCallsInside("com.linecorp.armeria.client.HttpClientFactory", | ||
| "pool"); | ||
| builder.allowBlockingCallsInside("com.linecorp.armeria.internal.common.util.ReentrantShortLock", | ||
| "lock"); | ||
|
|
||
| // Thread.yield can be eventually called when PooledObjects.copyAndClose is called | ||
| builder.allowBlockingCallsInside("io.netty.util.internal.ReferenceCountUpdater", "release"); | ||
|
|
||
| // hdr histogram holds locks | ||
| builder.allowBlockingCallsInside("org.HdrHistogram.ConcurrentHistogram", "getCountAtIndex"); | ||
| builder.allowBlockingCallsInside("org.HdrHistogram.WriterReaderPhaser", "flipPhase"); | ||
|
|
||
| // StreamMessageInputStream internally uses a blocking queue | ||
| // ThreadPoolExecutor.execute internally uses a blocking queue | ||
| builder.allowBlockingCallsInside("java.util.concurrent.LinkedBlockingQueue", "offer"); | ||
|
|
||
| // a single blocking call is incurred for the first invocation, but the result is cached. | ||
| builder.allowBlockingCallsInside("com.linecorp.armeria.internal.client.PublicSuffix", | ||
| "get"); | ||
| builder.allowBlockingCallsInside("java.util.ServiceLoader$LazyClassPathLookupIterator", | ||
| "parse"); | ||
| builder.allowBlockingCallsInside(ResourceBundle.class.getName(), "getBundle"); | ||
| builder.allowBlockingCallsInside("io.netty.handler.codec.compression.Brotli", "<clinit>"); | ||
|
|
||
| // a lock is held temporarily when adding workers | ||
| builder.allowBlockingCallsInside("java.util.concurrent.ThreadPoolExecutor", "addWorker"); | ||
|
|
||
| // prometheus exporting holds a lock temporarily | ||
| builder.allowBlockingCallsInside( | ||
| "com.linecorp.armeria.server.metric.PrometheusExpositionService", "doGet"); | ||
|
|
||
| // Thread.yield can be called | ||
| builder.allowBlockingCallsInside( | ||
| "java.util.concurrent.FutureTask", "handlePossibleCancellationInterrupt"); | ||
| } | ||
| } |
Oops, something went wrong.