Skip to content

Commit

Permalink
Fluent bit: docs: missed some trailing spaces
Browse files Browse the repository at this point in the history
Signed-off-by: Lynette Miles <[email protected]>
  • Loading branch information
esmerel committed Sep 13, 2024
1 parent bf54105 commit 69943d9
Showing 1 changed file with 35 additions and 20 deletions.
55 changes: 35 additions & 20 deletions pipeline/outputs/s3.md
Original file line number Diff line number Diff line change
Expand Up @@ -409,15 +409,15 @@ through the configuration file.

### Command Line

The S3 plugin can read the parameters from the command line through the **-p** argument:
The S3 plugin reads parameters from the command line through the `-p` argument:

```text
$ fluent-bit -i cpu -o s3 -p bucket=my-bucket -p region=us-west-2 -p -m '*' -f 1
fluent-bit -i cpu -o s3 -p bucket=my-bucket -p region=us-west-2 -p -m '*' -f 1
```

### Configuration File

In your main configuration file append the following _Output_ section:
In your main configuration file append the following `Output` section:

```python
[OUTPUT]
Expand All @@ -430,7 +430,7 @@ In your main configuration file append the following _Output_ section:
upload_timeout 10m
```

An example that using `PutObject` instead of multipart:
An example using `PutObject` instead of multipart:

```python
[OUTPUT]
Expand All @@ -446,65 +446,76 @@ An example that using `PutObject` instead of multipart:

## AWS for Fluent Bit

Amazon distributes a container image with Fluent Bit and this plugins.
Amazon distributes a container image with Fluent Bit and plugins.

### GitHub

[github.com/aws/aws-for-fluent-bit](https://github.com/aws/aws-for-fluent-bit)

### Amazon ECR Public Gallery

[aws-for-fluent-bit](https://gallery.ecr.aws/aws-observability/aws-for-fluent-bit)
Our images are available in the Amazon ECR Public Gallery as
[aws-for-fluent-bit](https://gallery.ecr.aws/aws-observability/aws-for-fluent-bit).

Our images are available in Amazon ECR Public Gallery. You can download images with different tags by following command:
You can download images with different tags using the following command:

```text
docker pull public.ecr.aws/aws-observability/aws-for-fluent-bit:<tag>
```

For example, you can pull the image with latest version by:
For example, you can pull the image with latest version with:

```text
docker pull public.ecr.aws/aws-observability/aws-for-fluent-bit:latest
```

If you see errors for image pull limits, try log into public ECR with your AWS credentials:
If you see errors for image pull limits, try signing in to public ECR with your
AWS credentials:

```text
aws ecr-public get-login-password --region us-east-1 | docker login --username AWS --password-stdin public.ecr.aws
```

You can check the [Amazon ECR Public official doc](https://docs.aws.amazon.com/AmazonECR/latest/public/get-set-up-for-amazon-ecr.html) for more details.
See the
[Amazon ECR Public official documentation](https://docs.aws.amazon.com/AmazonECR/latest/public/get-set-up-for-amazon-ecr.html)
for more details.

### Docker Hub

[amazon/aws-for-fluent-bit](https://hub.docker.com/r/amazon/aws-for-fluent-bit/tags)
is also available from the Docker Hub.

### Amazon ECR

You can use our SSM Public Parameters to find the Amazon ECR image URI in your region:
Use our SSM Public Parameters to find the Amazon ECR image URI in your region:

```text
aws ssm get-parameters-by-path --path /aws/service/aws-for-fluent-bit/
```

For more see [the AWS for Fluent Bit github repo](https://github.com/aws/aws-for-fluent-bit#public-images).
For more information, see the
[AWS for Fluent Bit GitHub repo](https://github.com/aws/aws-for-fluent-bit#public-images).

## Advanced usage

### Use Apache Arrow for in-memory data processing

Starting from Fluent Bit v1.8, the Amazon S3 plugin includes the support for [Apache Arrow](https://arrow.apache.org/). The support is currently not enabled by default, as it depends on a shared version of `libarrow` as the prerequisite.
With Fluent Bit v1.8 or greater, the Amazon S3 plugin includes the support for
[Apache Arrow](https://arrow.apache.org/). Support isn't enabled by
default, and has a dependency on a shared version of `libarrow`.

To use this feature, `FLB_ARROW` must be turned on at compile time:
To use this feature, `FLB_ARROW` must be turned on at compile time. Use the following
commands:

```text
$ cd build/
$ cmake -DFLB_ARROW=On ..
$ cmake --build .
cd build/
cmake -DFLB_ARROW=On ..
cmake --build .
```

Once compiled, Fluent Bit can upload incoming data to S3 in Apache Arrow format. For example:
Once compiled, Fluent Bit can upload incoming data to S3 in Apache Arrow format.

For example:

```python
[INPUT]
Expand All @@ -519,9 +530,13 @@ Once compiled, Fluent Bit can upload incoming data to S3 in Apache Arrow format.
Compression arrow
```

As shown in this example, setting `Compression` to `arrow` makes Fluent Bit to convert payload into Apache Arrow format.
Setting `Compression` to `arrow` makes Fluent Bit convert payload into Apache Arrow
format.

Load, analyze, and process stored data using popular data
processing tools such as Python pandas, Apache Spark and Tensorflow.

The stored data is very easy to load, analyze and process using popular data processing tools (such as Python pandas, Apache Spark and Tensorflow). The following code uses `pyarrow` to analyze the uploaded data:
The following example uses `pyarrow` to analyze the uploaded data:

```text
>>> import pyarrow.feather as feather
Expand Down

0 comments on commit 69943d9

Please sign in to comment.