Log spam #37

masikmos · 2019-09-29T23:01:48Z

Zabbix 4.2.6 module work fine but got spam in log file

	property: ActiveState
  6678:20190930:015629.347 [systemd] getting property:
	service: org.freedesktop.systemd1
	object path: /org/freedesktop/systemd1/unit/systemd_2dnetworkd_2dwait_2donline_2eservice
	interface: org.freedesktop.systemd1.Unit
	property: ActiveState
  6678:20190930:015629.395 [systemd] in cgroup_cpu()
  6678:20190930:015629.395 [systemd] metric source file: /sys/fs/cgroup/cpu,cpuacct/system.slice/networkd-dispatcher.service/cpuacct.stat
  6678:20190930:015629.395 [systemd] looking metric total in cpuacct.stat/cpu.stat file
  6678:20190930:015629.395 [systemd] unit: networkd-dispatcher.service; metric: total; value: 2
  6679:20190930:015630.493 [systemd] in cgroup_cpu()
  6679:20190930:015630.493 [systemd] metric source file: /sys/fs/cgroup/cpu,cpuacct/system.slice/elasticsearch.service/cpuacct.stat
  6679:20190930:015630.493 [systemd] looking metric total in cpuacct.stat/cpu.stat file
  6679:20190930:015630.493 [systemd] unit: elasticsearch.service; metric: total; value: 10613
  6679:20190930:015630.534 [systemd] in cgroup_mem(()
  6679:20190930:015630.534 [systemd] metric source file: /sys/fs/cgroup/memory/system.slice/zabbix-agent.service/memory.stat
  6679:20190930:015630.535 [systemd] looking metric total_rss in memory.stat file
  6679:20190930:015630.535 [systemd] unit: zabbix-agent.service; metric: total_rss; value: 3694592
  6680:20190930:015630.536 [systemd] getting property:
	service: org.freedesktop.systemd1
	object path: /org/freedesktop/systemd1/unit/vgauth_2eservice
	interface: org.freedesktop.systemd1.Unit
	property: ActiveState
  6678:20190930:015630.565 [systemd] getting property:
	service: org.freedesktop.systemd1
	object path: /org/freedesktop/systemd1/unit/rsyslog_2eservice
	interface: org.freedesktop.systemd1.Unit
	property: ActiveState

Help disable it please...

The text was updated successfully, but these errors were encountered:

jangaraj · 2019-09-30T05:35:49Z

Configure your Zabbix debug level, please.

masikmos · 2019-10-01T09:29:40Z

Debug level 0

masikmos · 2019-10-01T14:26:40Z

Why issue closed? problem not fixed

i set debug level 0 and steel see spam

jangaraj · 2019-10-01T17:28:37Z

Please provide more details: Zabbix agent configuration, OS, full logs, .... I'm happy to reopen issue (so no need to open new one), if you prove, that is a real issue.

masikmos · 2019-10-01T17:44:26Z

Oh.... Sry me please ))) i idiot )

Ubuntu 18.04

Agent CONF

ListenIP=0.0.0.0
ListenPort=10050
PidFile=/var/run/zabbix/zabbix_agentd.pid

# zabbix server connect configuration
Server=zabbix, 10.50.0.200
ServerActive=zabbix, 10.50.0.200
Hostname=srv01

# global configuration options
Timeout=20
EnableRemoteCommands=0

# logs configuration
LogFile=/var/log/zabbix/zabbix_agentd.log
LogFileSize=0
DebugLevel=3

# additional scripts and user parameters

Include=/opt/zabbix/etc

Loaded conf systemd

cat /opt/zabbix/etc/systemd.conf

LoadModulePath=/opt/zabbix/modules/
LoadModule=libzbxsystemd.so`

Log is too many...
Example:

sudo tail -f /var/log/zabbix/zabbix_agentd.log
 12517:20191001:204301.147 [systemd] getting property:
	service: org.freedesktop.systemd1
	object path: /org/freedesktop/systemd1/unit/atop_2eservice
	interface: org.freedesktop.systemd1.Unit
	property: ActiveState
 12518:20191001:204301.179 [systemd] getting property:
	service: org.freedesktop.systemd1
	object path: /org/freedesktop/systemd1/unit/systemd_2dresolved_2eservice
	interface: org.freedesktop.systemd1.Unit
	property: ActiveState
 12520:20191001:204302.237 [systemd] in cgroup_mem(()
 12520:20191001:204302.237 [systemd] metric source file: /sys/fs/cgroup/memory/system.slice/libvirt-guests.service/memory.stat
 12520:20191001:204302.237 [systemd] looking metric total_rss in memory.stat file
 12520:20191001:204302.237 [systemd] unit: libvirt-guests.service; metric: total_rss; value: 0
 12520:20191001:204302.239 [systemd] getting property:
	service: org.freedesktop.systemd1
	object path: /org/freedesktop/systemd1/unit/blk_2davailability_2eservice
	interface: org.freedesktop.systemd1.Unit
	property: ActiveState
 12520:20191001:204302.289 [systemd] in cgroup_cpu()
 12520:20191001:204302.289 [systemd] metric source file: /sys/fs/cgroup/cpu,cpuacct/system.slice/keyboard-setup.service/cpuacct.stat
 12520:20191001:204302.289 [systemd] looking metric total in cpuacct.stat/cpu.stat file
 12520:20191001:204302.289 [systemd] unit: keyboard-setup.service; metric: total; value: 0
 12518:20191001:204302.406 [systemd] in cgroup_mem(()
 12518:20191001:204302.406 [systemd] metric source file: /sys/fs/cgroup/memory/system.slice/systemd-networkd-wait-online.service/memory.stat
 12518:20191001:204302.406 [systemd] looking metric total_rss in memory.stat file
 12518:20191001:204302.406 [systemd] unit: systemd-networkd-wait-online.service; metric: total_rss; value: 0
 12520:20191001:204302.410 Cannot find the [Id] item in the received JSON object
 12520:20191001:204302.414 [systemd] getting property:
	service: org.freedesktop.systemd1
	object path: /org/freedesktop/systemd1/unit/ssh_2eservice
	interface: org.freedesktop.systemd1.Unit
	property: ActiveState
 12517:20191001:204303.424 [systemd] getting property:
	service: org.freedesktop.systemd1
	object path: /org/freedesktop/systemd1/unit/lvm2_2dmonitor_2eservice
	interface: org.freedesktop.systemd1.Unit
	property: ActiveState
 12517:20191001:204303.489 [systemd] in cgroup_cpu()
 12517:20191001:204303.489 [systemd] metric source file: /sys/fs/cgroup/cpu,cpuacct/system.slice/docker.service/cpuacct.stat
 12517:20191001:204303.489 [systemd] looking metric total in cpuacct.stat/cpu.stat file
 12517:20191001:204303.489 [systemd] unit: docker.service; metric: total; value: 87070
 12518:20191001:204303.558 [systemd] getting property:
	service: org.freedesktop.systemd1
	object path: /org/freedesktop/systemd1/unit/accounts_2ddaemon_2eservice
	interface: org.freedesktop.systemd1.Unit
	property: ActiveState
 12518:20191001:204303.596 Cannot find the [Id] item in the received JSON object
 12518:20191001:204303.599 [systemd] in cgroup_mem(()
 12518:20191001:204303.599 [systemd] metric source file: /sys/fs/cgroup/memory/system.slice/ebtables.service/memory.stat
 12518:20191001:204303.599 [systemd] looking metric total_rss in memory.stat file
 12518:20191001:204303.599 [systemd] unit: ebtables.service; metric: total_rss; value: 0
 12520:20191001:204303.606 [systemd] in cgroup_mem(()
 12520:20191001:204303.606 [systemd] metric source file: /sys/fs/cgroup/memory/system.slice/networkd-dispatcher.service/memory.stat
 12520:20191001:204303.606 [systemd] looking metric total_rss in memory.stat file
 12520:20191001:204303.606 [systemd] unit: networkd-dispatcher.service; metric: total_rss; value: 8458240
 12518:20191001:204303.613 Cannot find the [Id] item in the received JSON object
```


One day weight
`sudo ls -lah /var/log/zabbix/zabbix_agentd.log
-rw-r----- 1 zabbix zabbix 316M Oct  1 20:43 /var/log/zabbix/zabbix_agentd.log`

jangaraj · 2019-10-01T18:01:38Z

Your agent config:

DebugLevel=3

=> 3 - warnings => LOG_LEVEL_WARNING

But (some) mentioned logs are printed from LOG_LEVEL_DEBUG+:
https://github.com/cavaliercoder/zabbix-module-systemd/blob/f92597b900e950c58cba19a86d65fdce16c012d0/src/modules/systemd/cgroups.c#L86

So, you didn't provide logs for DebugLevel=3 or you didn't configure agent properly and DebugLevel=3 was not applied.

masikmos · 2019-10-01T18:25:27Z

I set for tests level=0 but log spaming

i-ky · 2019-10-01T18:37:29Z

@maksimkos, since you are not suffering from undefined symbol: __zbx_zbx_snprintf errors, I assume you have compiled the module yourself using latest Zabbix sources. Therefore I can guess that you are facing an issue similar to this one. Perhaps, you did not run ./configure in Zabbix source directory before compiling the module?

masikmos · 2019-10-02T14:46:13Z

no, I did everything right

./configure i do

i can just disable all logs of module???

i-ky · 2019-10-02T15:48:04Z

Could you please show the output of the following command?

$ ls /path/to/zabbix_sources/include/

masikmos · 2019-10-03T12:06:28Z

ls /usr/src/zabbix/include/
alias.h   common.h     daemon.h   dbschema.h   export.h  md5.h       mutexs.h   preproc.h  setproctitle.h  symbols.h  telnet.h   zbxalgo.h      zbxdbupgrade.h  zbxgetopt.h   zbxicmpping.h    zbxlld.h      zbxnix.h         zbxself.h       zbxtasks.h
base64.h  comms.h      dbcache.h  discovery.h  ipc.h     memalloc.h  perfmon.h  proxy.h    sighandler.h    sysinc.h   threads.h  zbxcompress.h  zbxembed.h      zbxhistory.h  zbxipcservice.h  zbxmedia.h    zbxprometheus.h  zbxserialize.h  zbxtypes.h
cfg.h     config.h.in  db.h       disk.h       log.h     module.h    pid.h      service.h  stamp-h1        sysinfo.h  version.h  zbxdb.h        zbxexec.h       zbxhttp.h     zbxjson.h        zbxmodules.h  zbxregexp.h      zbxserver.h

i-ky · 2019-10-03T12:19:08Z

There is no config.h file. It is produced by configure script of Zabbix and in latest versions of Zabbix it contains definitions crucial for zabbix_log() to work correctly. Because this module uses autotools as well, it has its own config.h file (which does not contain these definitions) and compilation does not fail without Zabbix' config.h file.

To fix your problem:

cd /usr/src/zabbix
./configure
cd /path/to/module_sources
make clean
make
restart Zabbix agent

masikmos · 2019-10-03T12:42:14Z

yap is fix spam, but now i see in log

14859:20191003:154129.247 Cannot open metric file: '/sys/fs/cgroup/memory/docker//runner-d4848be6-project-20-concurrent-0-postgres-0/memory.stat'
 14862:20191003:154131.399 Cannot find the [Id] item in the received JSON object
 14862:20191003:154131.399 Cannot open metric file: '/sys/fs/cgroup/memory/docker//runner-d4848be6-project-20-concurrent-0-postgres-0/memory.stat'
 14859:20191003:154132.495 Cannot find the [Id] item in the received JSON object
 14859:20191003:154133.611 Cannot find the [Id] item in the received JSON object
 14859:20191003:154133.614 Cannot find the [Id] item in the received JSON object
 14862:20191003:154138.432 Cannot find the [Id] item in the received JSON object
 14859:20191003:154138.514 Cannot find the [Id] item in the received JSON object
 14859:20191003:154138.638 Cannot find the [Id] item in the received JSON object
 14861:20191003:154139.702 Cannot find the [Id] item in the received JSON object
 14862:20191003:154139.896 Cannot find the [Id] item in the received JSON object
 14862:20191003:154139.989 Cannot find the [Id] item in the received JSON object
 14862:20191003:154139.989 Cannot open metric file: '/sys/fs/cgroup/cpu,cpuacct/docker//runner-d4848be6-project-20-concurrent-0-build-4/cpuacct.stat'
 14861:20191003:154140.203 Cannot find the [Id] item in the received JSON object
 14861:20191003:154140.203 Cannot open metric file: '/sys/fs/cgroup/memory/docker//runner-d4848be6-project-20-concurrent-0-build-4/memory.stat'
 14861:20191003:154141.670 Cannot find the [Id] item in the received JSON object
 14861:20191003:154141.670 Cannot open metric file: '/sys/fs/cgroup/memory/docker//runner-d4848be6-project-20-concurrent-0-build-4/memory.stat'

i-ky · 2019-10-03T12:46:18Z

I can't help you with this one.

masikmos · 2019-10-03T12:49:31Z

oh sry its docker metrics ) THX! You help me! ))) now in module systemd all fine!!! You Magicant! )))

i-ky · 2019-10-03T22:53:36Z

@masikmos, @jangaraj, @cavaliercoder,

Please check this bug report and add your vote to move it forward!

dimir · 2020-05-06T07:41:39Z

It's sad that judging by number of votes not many voted. :-(

jangaraj closed this as completed Sep 30, 2019

jangaraj mentioned this issue Oct 1, 2019

Log spam #38

Closed

jangaraj added a commit that referenced this issue Oct 4, 2019

Instruction for Zabbix config.h #37, ZBXNEXT-2871

92ecb08

jangaraj mentioned this issue Nov 6, 2019

Zabbix Log level zabbix-tools/zabbix-module-python#12

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Log spam #37

Log spam #37

masikmos commented Sep 29, 2019 •

edited by jangaraj

Loading

jangaraj commented Sep 30, 2019

masikmos commented Oct 1, 2019

masikmos commented Oct 1, 2019

jangaraj commented Oct 1, 2019

masikmos commented Oct 1, 2019 •

edited by jangaraj

Loading

jangaraj commented Oct 1, 2019

masikmos commented Oct 1, 2019

i-ky commented Oct 1, 2019

masikmos commented Oct 2, 2019 •

edited

Loading

i-ky commented Oct 2, 2019

masikmos commented Oct 3, 2019 •

edited by jangaraj

Loading

i-ky commented Oct 3, 2019 •

edited by jangaraj

Loading

masikmos commented Oct 3, 2019 •

edited by jangaraj

Loading

i-ky commented Oct 3, 2019

masikmos commented Oct 3, 2019

i-ky commented Oct 3, 2019

dimir commented May 6, 2020

Log spam #37

Log spam #37

Comments

masikmos commented Sep 29, 2019 • edited by jangaraj Loading

jangaraj commented Sep 30, 2019

masikmos commented Oct 1, 2019

masikmos commented Oct 1, 2019

jangaraj commented Oct 1, 2019

masikmos commented Oct 1, 2019 • edited by jangaraj Loading

jangaraj commented Oct 1, 2019

masikmos commented Oct 1, 2019

i-ky commented Oct 1, 2019

masikmos commented Oct 2, 2019 • edited Loading

i-ky commented Oct 2, 2019

masikmos commented Oct 3, 2019 • edited by jangaraj Loading

i-ky commented Oct 3, 2019 • edited by jangaraj Loading

masikmos commented Oct 3, 2019 • edited by jangaraj Loading

i-ky commented Oct 3, 2019

masikmos commented Oct 3, 2019

i-ky commented Oct 3, 2019

dimir commented May 6, 2020

masikmos commented Sep 29, 2019 •

edited by jangaraj

Loading

masikmos commented Oct 1, 2019 •

edited by jangaraj

Loading

masikmos commented Oct 2, 2019 •

edited

Loading

masikmos commented Oct 3, 2019 •

edited by jangaraj

Loading

i-ky commented Oct 3, 2019 •

edited by jangaraj

Loading

masikmos commented Oct 3, 2019 •

edited by jangaraj

Loading