Getting 400 status in fluentd - request body is required

[aakashjhawar]

(check apply)

• [-] read the contribution guideline
• [-] (optional) already reported 3rd party upstream repository or mailing list if you use k8s addon or helm charts.

Problem

I'm getting issues when fluentd is pushing logs to elasticsearch. The status code is 400.

failed to flush the buffer.
error_class=Fluent::Plugin::ElasticsearchOutput::RecoverableRequestFailure 
error="could not push logs to Elasticsearch cluster (cluster-${tag}): [400] 
{
    "error": {
        "root_cause": [
            {
                "type": "parse_exception",
                "reason": "request body is required"
            }
        ],
        "type": "parse_exception",
        "reason": "request body is required"
    },
    "status": 400
}

...

Steps to replicate

Either clone and modify https://gist.github.com/pitr/9a518e840db58f435911

Expected Behavior or What you need to ask

Fluentd should send the data to elasticsearch without any issues.
...

Using Fluentd and ES plugin versions

• OS version
• Bare Metal or within Docker or Kubernetes or others? - Kubernetes
• Fluentd v0.12 or v0.14/v1.0
  • paste result of fluentd --version or td-agent --version
• ES plugin 3.x.y/2.x.y or 1.x.y
  • paste boot log of fluentd or td-agent
  • paste result of fluent-gem list, td-agent-gem list or your Gemfile.lock
• ES version (optional)
• ES template(s) (optional)

[eli-gc]

What version of fluentd are you using? I'm getting 400s on version 1.15.1+ but it works fine for me on 1.15.0

[applike-ss]

We are suffering from the same.
Versions:

elastic-transport (8.3.1)
elasticsearch (8.12.2)
elasticsearch-api (8.12.2)
fluent-plugin-elasticsearch (5.4.3)

fluentd 1.16.2

[applike-ss]

Same happens on fluentd 1.17.0:

elastic-transport (8.3.2)
elasticsearch (8.14.0, 7.13.3)
elasticsearch-api (8.14.0, 7.13.3)
elasticsearch-transport (7.13.3)
elasticsearch-xpack (7.13.3)
fluent-plugin-aws-elasticsearch-service (2.4.1)
fluent-plugin-elasticsearch (5.4.3)

fluentd 1.17.0