administration: performance

Signed-off-by: Eduardo Silva <[email protected]>

Author: edsiper

SUMMARY.md

@@ -78,6 +78,7 @@
 * [HTTP Proxy](administration/http-proxy.md)
 * [Hot Reload](administration/hot-reload.md)
 * [Troubleshooting](administration/troubleshooting.md)
+* [Performance Tips](administration/performance.md)
 
 ## Local Testing
 

administration/multithreading.md

@@ -11,8 +11,8 @@ This event loop runs in the main Fluent Bit thread.
 To free up resources in the main thread, you can configure
 [inputs](../pipeline/inputs/README.md) and [outputs](../pipeline/outputs/README.md)
 to run in their own self-contained threads. However, inputs and outputs implement
-multithreading in distinct ways: inputs can run in threaded mode, and outputs
-can use one or more workers.
+multithreading in distinct ways: inputs can run in `threaded` mode, and outputs
+can use one or more `workers`.
 
 Threading also affects certain processes related to inputs and outputs. For example,
 [filters](../pipeline/filters/README.md) always run in the main thread, but

administration/performance.md

@@ -0,0 +1,57 @@
+# Performance Tips
+
+Fluent Bit is designed for high performance and minimal resource usage. Depending on your use case, you can optimize further using specific configuration options to achieve faster performance or reduce resource consumption.
+
+## Reading Files with Tail
+
+The `Tail` input plugin is used to read data from files on the filesystem. By default, it uses a small memory buffer of `32KB` per monitored file. While this is sufficient for most generic use cases and helps keep memory usage low when monitoring many files, there are scenarios where you may want to increase performance by using more memory.
+
+If your files are typically larger than `32KB`, consider increasing the buffer size to speed up file reading. For example, you can experiment with a buffer size of `128KB`:
+
+```yaml
+pipeline:
+  inputs:
+    - name: tail
+      path: '/var/log/containers/*.log'
+      buffer_chunk_size: 128kb
+      buffer_max_size: 128kb
+```
+
+By increasing the buffer size, Fluent Bit will make fewer system calls (read(2)) to read the data, reducing CPU usage and improving performance.
+
+## Fluent Bit and SIMD for JSON Encoding
+
+Starting in Fluent Bit v3.2, performance improvements have been introduced for JSON encoding. Plugins that convert logs from Fluent Bit’s internal binary representation to JSON can now do so up to 30% faster using SIMD (Single Instruction, Multiple Data) optimizations.
+
+### Enabling SIMD Support
+
+Ensure that your Fluent Bit binary is built with SIMD support. This feature is available for architectures such as x86_64, amd64, aarch64, and arm64. As of now, SIMD is only enabled by default in Fluent Bit container images.
+
+You can check if SIMD is enabled by looking for the following log entry when Fluent Bit starts:
+
+```
+[2024/11/10 22:25:53] [ info] [fluent bit] version=3.2.0, commit=12cb22e0e9, pid=74359
+[2024/11/10 22:25:53] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
+[2024/11/10 22:25:53] [ info] [simd    ] SSE2
+[2024/11/10 22:25:53] [ info] [cmetrics] version=0.9.8
+[2024/11/10 22:25:53] [ info] [ctraces ] version=0.5.7
+[2024/11/10 22:25:53] [ info] [sp] stream processor started
+```
+
+Look for the simd entry, which will indicate the SIMD support type, such as SSE2, NEON, or none.
+
+If your Fluent Bit binary was not built with SIMD enabled and you are using a supported platform, you can build Fluent Bit from source using the CMake option `-DFLB_SIMD=On`.
+
+## Run input plugins in threaded mode
+
+By default, most of input plugins runs in the same system thread than the main event loop, however by configuration you can instruct them to run in a separate thread which will allow you to take advantage of other CPU cores in your system.
+
+To run an input plugin in threaded mode, just add `threaded: true` as in the example below:
+
+```yaml
+pipeline:
+  inputs:
+    - name: tail
+      path: '/var/log/containers/*.log'
+      threaded: true
+```