-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ability to specify log format #553
Comments
👍 |
+1 |
+1 |
Any reason why this was closed? I'd like to remove the timestamps because syslog-ng does it already for example (and my log aggregator can understand syslog-ng ones) +1 |
@LouisKottmann - it looks like this is still open for Supervisor, and closed in the dokku-supervisord project (which just referenced this issue) |
Ah you're right, my bad |
+1 |
+1 |
We really need this. It's currently impossible to use supervisord with some logging tools because the format is completely random and inflexible. |
For us it's not impossible, just awkward. Of course because it's logging, the reminders are constant. |
Still no news regarding this? We'd need at least a timestamp in log lines |
+1 |
5 similar comments
+1 |
+1 |
+1 |
+1 |
+1 |
I was able to get this working by using a shell script in the following manner (definitely would be nice to not have to do this workaround but it worked)
|
+1 |
To all those people who needs this feature, here's a quick patch for it. Just apply the patch, or modify the code manually, or use my forked repo. Then do a manual installation by I referred to this #333 PR and modified the code where the logger formatter is defined. I'm not sure why that PR wasn't merged to the master. I only added timestamp, I didn't add in other config option or anything. If someone can tell me what's required, I can probably put in some time for a proper PR for this. I would also really like to have this feature in the master branch, so I don't have to keep deploying old supervisor version. Please advice, thanks. |
|
supervisord uses the % character for its own format strings in the config file. In case that after |
Let's give it another try: PR #1349 --- /usr/lib/python2.7/dist-packages/supervisor/dispatchers.py~ 2016-08-03 05:33:42.000000000 +0100
+++ /usr/lib/python2.7/dist-packages/supervisor/dispatchers.py 2020-05-21 09:04:14.502793781 +0100
@@ -87,7 +87,7 @@
if logfile:
maxbytes = getattr(process.config, '%s_logfile_maxbytes' % channel)
backups = getattr(process.config, '%s_logfile_backups' % channel)
- fmt = '%(message)s'
+ fmt = '%(asctime)s %(message)s'
if logfile == 'syslog':
fmt = ' '.join((process.config.name, fmt))
self.mainlog = process.config.options.getLogger(
|
I don't have any particular advice or insight, sorry. You may know as much about it as I do at this point. I did not write the code in question. It might be best to direct questions about its design to its author. |
Hi @mnaberez, thanks for your response. For example, I'm not sure if Is there anywhere I can read about supervisor's internals? I find the documentation to be very lacking in the code. |
I am not aware of anything. |
As I explained above, I'm not an expert on this code, sorry. I left a comment on the PR that it seems to break the Note: For visitors who came here for this ticket ("Add ability to specify a log format"), PR #1407 mentioned immediately above probably would not close this issue. This issue seems to be people requesting to be able to specify the log format, but PR #1407 is about optionally prepending a timestamp with a fixed format. |
@mnaberez Thanks for the review. Sorry I dragged you into this. Yes, it's because I saw you reviewing the 2 previous PRs related to this issue, and you are a member of "supervisor" organisation. I assumed all unit tests passed, since travis build all passed. I didn't know it break anything. I will test out the feature you mentioned I just realised that you are right about the Issue and the PR. I also noticed that there is a similar PR #1197 which seem to fit this issue better. I don't know what the maintainers/committers opinion on this, but personally I think it would benefit to have two options, one to customise the log format (be it down to specific timestamp formatting, levels, etc), and another to just prepend timestamp (which IMHO should have been implemented as default from the start). In the meantime while awaiting for other maintainers/committers, I will look into the feature that you mentioned, and into how I can change my PR to support the aforementioned first option. |
+1 |
1 similar comment
+1 |
+1 - 5 years? Come on guys, we need this, especially with docker when you want to put all logs from all applications into stdout - it becomes impossible to distinguish the apps when you have a number of them. |
So we still can't disable supervisord-added timestamps in the logs? |
This didn't work well for me, as only once every X lines I'd get the prefix. It seems the call to formatter is not done for each line, but for a buffer of some size. In the meantime as a complementary solution I edited the handler in But this still won't work well if message is truncated from previous line. |
Another hack I started using is creating another program which simple tails all logs via parallel (copied from https://unix.stackexchange.com/a/337779) - by using awk or else it would be also possible to add a date string to the output.
generates something like this
|
+1 |
1 similar comment
+1 |
One issue with this is that piping a program's output to Having spent several hours trying to change the buffering using |
+1 |
For ones who are looking for some workaround. Here is how I ended with prefixing output with process name: supervisor.conf
prefix-output.sh #!/bin/bash
# Get prefix from SUPERVISOR_PROCESS_NAME environment variable
printf -v PREFIX "%-10.10s" "${SUPERVISOR_PROCESS_NAME}"
# Prefix stdout and stderr
exec 1> >( perl -ne '$| = 1; print "'"${PREFIX}"' | $_"' >&1)
exec 2> >( perl -ne '$| = 1; print "'"${PREFIX}"' | $_"' >&2)
exec "$@" Example output
|
+1 |
This should be added as a core feature in supervisord, it's not a job for weird wrapper scripts. |
This was an excellent starting point for adding a static name to subprocess output. I wanted improve the consistency for readability and parsing by adding per-line time stamps that matched supervisord's format: '2023-06-09 17:25:36,778 INFO Included extra file "/etc/supervisor/conf.d/example.conf" during parsing' This perl in the following heredoc will calculate a timestamp at run time for each line that is sent to stdout and stderr
Example Output running within a docker container
|
+1 |
4 similar comments
+1 |
+1 |
+1 |
+1 |
It would be helpful to have control over the format of the log output. For inspiration, other tools that allow you to customize the log format include Celery (CELERYD_LOG_FORMAT) and Gunicorn (access_log_format).
The text was updated successfully, but these errors were encountered: