Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issues during installation in Ubuntu #79

Open
lllilllilllilili opened this issue Aug 3, 2021 · 0 comments
Open

Issues during installation in Ubuntu #79

lllilllilllilili opened this issue Aug 3, 2021 · 0 comments

Comments

@lllilllilllilili
Copy link

lllilllilllilili commented Aug 3, 2021

Hello (VM : Ubuntu 16.04LTS)

I'd like to ask you a few questions.

About fluent-plugin-gcloud-pubsub-custom Installation Issues ( If I have any problems registering Issue, please let me know )

I was practicing the configuration of installing fluid on ubuntu and sending it to pubsub. (ruby using /.rvm/gems/ruby-2.5.8)

Write the instructions in accordance with the guidelines provided by github.

I ran

$ sudo /etc/init.d/td-agent start

Failed.

[....] Starting td-agent (via systemctl): td-agent.serviceJob for td-agent.service failed because the control process exited with error code. See "systemctl status td-agent.service" and "journalctl -xe" for details.
failed!

So I looked at the failure, and I couldn't check the plugin.

config error file="/etc/td-agent/td-agent.conf" error="Unknown output plugin 'gcloud_pubsub'. Run 'gem search -rd fluent-plugin'

/etc/td-agent/td-agent.conf

Page up
<source>
  @type tail
  path /var/log/nginx/access.log
  pos_file /var/log/td-agent/nginx-access.pos
  tag example.publish
  format nginx
</source>
#<match input.twitter>
#   @type stdout
#</match>
<match example.publish>
  type gcloud_pubsub
  project ...
  key ...
  topic projects...
  autocreate_topic false
  max_messages 1000
  max_total_size 9800000
  max_message_size 4000000
  timeout 60
  <buffer>
    @type memory
    flush_interval 1s
  </buffer>
  <format>
    @type json
  </format>
</match>

** project, topic, key는 ... I've done with the secret.

I checked if the plug-in was installed, but it wasn't either.

$ sudo td-agent-gem list | egrep "pubsub|google"

/var/log/td-agent/td-agent.log

2021-08-03 07:00:01 +0000 [info]: starting fluentd-0.12.40
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-kafka' version '0.6.1'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-mongo' version '0.8.1'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.6'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-s3' version '0.8.5'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-td' version '0.10.29'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-td-monitoring' version '0.2.3'
2021-08-03 07:00:01 +0000 [info]: gem 'fluent-plugin-webhdfs' version '0.7.1'
2021-08-03 07:00:01 +0000 [info]: gem 'fluentd' version '0.12.40'
2021-08-03 07:00:01 +0000 [info]: adding match pattern="td.*.*" type="tdlog"
2021-08-03 07:00:01 +0000 [info]: adding match pattern="debug.**" type="stdout"
2021-08-03 07:00:01 +0000 [info]: adding source type="forward"
2021-08-03 07:00:01 +0000 [info]: adding source type="http"
2021-08-03 07:00:01 +0000 [info]: adding source type="debug_agent"
2021-08-03 07:00:01 +0000 [info]: using configuration file: <ROOT>
  <match td.*.*>
    @type tdlog
    apikey xxxxxx
    auto_create_table
    buffer_type file
    buffer_path /var/log/td-agent/buffer/td
    buffer_chunk_limit 33554432
    <secondary>
      @type file
      path /var/log/td-agent/failed_records
      buffer_path /var/log/td-agent/failed_records.*
    </secondary>
  </match>
  <match debug.**>
    @type stdout
  </match>
  <source>
    @type forward
  </source>
  <source>
    @type http
    port 8888
  </source>
  <source>
    @type debug_agent
    bind 127.0.0.1
    port 24230
  </source>
</ROOT>
2021-08-03 07:00:01 +0000 [info]: listening fluent socket on 0.0.0.0:24224
2021-08-03 07:00:01 +0000 [info]: listening dRuby uri="druby://127.0.0.1:24230" object="Engine"
2021-08-03 07:23:37 +0000 [info]: shutting down fluentd
2021-08-03 07:23:37 +0000 [info]: shutting down input type="forward" plugin_id="object:3feb28faeb34"
2021-08-03 07:23:37 +0000 [info]: shutting down input type="http" plugin_id="object:3feb28fa9f08"
2021-08-03 07:23:37 +0000 [info]: shutting down input type="debug_agent" plugin_id="object:3feb28fa8f2c"
2021-08-03 07:23:37 +0000 [info]: shutting down output type="tdlog" plugin_id="object:3feb28d7b40c"
2021-08-03 07:23:37 +0000 [info]: shutting down output type="stdout" plugin_id="object:3feb28ff6e70"
2021-08-03 07:23:37 +0000 [info]: process finished code=0
2021-08-03 07:23:38 +0000 [info]: reading config file path="/etc/td-agent/td-agent.conf"
2021-08-03 07:23:38 +0000 [info]: starting fluentd-0.12.40
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-mixin-plaintextformatter' version '0.2.6'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-buffer-lightening' version '0.0.2'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-kafka' version '0.6.1'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-mongo' version '0.8.1'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-rewrite-tag-filter' version '1.5.6'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-s3' version '0.8.5'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-scribe' version '0.10.14'
2021-08-03 07:23:38 +0000 [info]: gem 'fluent-plugin-td' version '0.10.29'

I'm really curious about the reason why it can't be installed..

Thank you

@lllilllilllilili lllilllilllilili changed the title asdad Issues during installation in Ubuntu Aug 3, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant