POC: remote data collection over existing postgres connection #80
Conversation
* Can collect Host information: uptime, hostname, loadavg, etc. * Can collect System information: cpu utilization, context switches, etc. * Can collect Memory information. * Requires stored procs in plpythonu to be installed on the server. Could install automatically on startup assuming enough permissions. * Partition and Postgres collectors are really tricky, but should be doable.
There was a problem hiding this comment.
Looks like a viable approach to me. The problem is that creating a sproc in an untrusted language requires superuser privileges. One can solve it by running with --install option first, followed by the normal call.
As for the disk statistics, we can simply call the df and du less often.
| collectors.append(HostStatCollector()) | ||
| collectors.append(SystemStatCollector()) | ||
| collectors.append(MemoryStatCollector()) | ||
| use_local_data = not(options.host) or options.host.startswith('/') |
There was a problem hiding this comment.
host name can be also read from the configuration file
| part = groups[name].get('partitions') | ||
| if part: | ||
| pg = groups[name]['pg'] | ||
| part.ncurses_set_prefix(pg.ncurses_produce_prefix()) |
There was a problem hiding this comment.
This will not show useful information about the cluster (i.e. version, max connections, active and idle ones). I think we need to do pg.ncurses_set_prefix(pg.ncurses_produce_prefix()), attaching it to the header of pg collector.
There was a problem hiding this comment.
This is a temp hack to avoid it crashing because partition collector doesn't have a remote data source yet.
| import socket | ||
| from multiprocessing import cpu_count | ||
| try: | ||
| with open('/proc/uptime', 'rU') as f: |
There was a problem hiding this comment.
Just a nitpick, we should not use hard-coded names when generating sprocs text.
There was a problem hiding this comment.
This was just copy-paste from my psql prompt, so I don't lose the func definition ;-)
|
@alexeyklyukin I also think we need a separate call to install the funcs (with security definer option). Then we could grant some role like As for du/df: running less often makes a lot of sense, since these readings are not expected to change very quickly. Every 3-5 ticks should be reasonable. Can we then get rid of subprocesses/queues altogether? |
|
I'd probably supply a SQL file along the way and instruct users to install it with their tools of choice. |
Can collect Host information: uptime, hostname, loadavg, etc.
Can collect System information: cpu utilization, context switches, etc.
Can collect Memory information.
Requires stored procs in plpythonu to be installed on the server. Could
install automatically on startup assuming enough permissions.
Partition and Postgres collectors are really tricky, but should be doable.