Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

when using fluent-plugin-mongo, threads didnot exit and sockets to mongo didnot close #155

Open
seamaner opened this issue Nov 27, 2020 · 3 comments

Comments

@seamaner
Copy link

the container uses many socket fds and threads, it may be a leak.
0x01
now it has 4k+ threads:
ls /proc/10/task | wc
4384 4384 21044
0x02

ps -elf | grep ruby

0 S root 9712 6033 0 80 0 - 3875 pipe_w 17:45 pts/2 00:00:00 grep --color=auto ruby
4 S 999 11386 11358 0 80 0 - 27546 poll_s 16:13 ? 00:00:04 /usr/local/bin/ruby /usr/local/bundle/bin/fluentd -c /fluentd/etc/fluent.conf

use lsof can get that process has many tcp connections to mongo:
4 S 999 11996 11386 5 80 0 - 3527387 poll_s 16:13 ? 00:04:54 /usr/local/bin/ruby -Eascii-8bit:ascii-8bit /usr/local/bundle/bin/fluentd -c /fluentd/etc/fluent.conf --under-supervisor
ruby 11996 999 4461u IPv4 145632322 0t0 TCP localhost:35948->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4462u IPv4 145630540 0t0 TCP localhost:35976->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4463u IPv4 145629667 0t0 TCP localhost:35978->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4464u IPv4 145629694 0t0 TCP localhost:35994->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4465u IPv4 145629695 0t0 TCP localhost:35996->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4466u IPv4 145630677 0t0 TCP localhost:36030->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4467u IPv4 145630681 0t0 TCP localhost:36032->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4468u IPv4 145629743 0t0 TCP localhost:36040->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4469u IPv4 145629744 0t0 TCP localhost:36042->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4470u IPv4 145631716 0t0 TCP localhost:36094->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4471u IPv4 145631717 0t0 TCP localhost:36096->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4472u IPv4 145630872 0t0 TCP localhost:36100->localhost:27017 (ESTABLISHED)
lsof: no pwd entry for UID 999
ruby 11996 999 4473u IPv4 145630876 0t0 TCP localhost:36102->localhost:27017 (ESTABLISHED)

it seems that when a message comes to Port 24224, fluentd creates a thread and a mongo connection, but when the connection to 24224 closed, the thread did not close and the connection to mongo leaked.

any suggestions ?

@seamaner
Copy link
Author

fluent.conf:
fluent.conf.txt
`

@seamaner
Copy link
Author

Dockfile used:

Dockerfile.txt

@cosmo0920 cosmo0920 transferred this issue from fluent/fluentd-docker-image Mar 25, 2021
@cosmo0920
Copy link
Contributor

fluentd-dcoker-image does not provide mongo plugin image. This is user-custom image issue.
So, I transferred this issue from fluentd-docker-image.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants