How to write access logs to /var/logs/nginx/access.log instead of stdout?

[noor-muradi]

How to make the Nginx ingress controller to write logs to log file instead of stdout?
I have deployed Nginx Ingress controller using the helm chart, I want to persist the logs using Persistent Volume so that I can use filebeat to push logs to Elastic Search for monitoring, the Nginx ingress by default output the logs to stdout, I have tried to make an entry to values.yaml but the logs are not written to /var/logs/access.log which is mounted to PVC and the hostPath which is configured in PVC has the permission RWX for UID "101"

controller:
  config:
    entries:
      access-log: /var/log/nginx/access.log

[shaun-nx]

Hi @noor-muradi

It's currently only possible to toggle on/off access logging using a ConfigMap.
All of our existing ConfigMap configurations are documented here: https://docs.nginx.com/nginx-ingress-controller/configuration/global-configuration/configmap-resource#summary-of-configmap-keys

You will have to use custom-templates in order to overwrite the log output directory that we set for access_log
There are steps in our examples folder that step through how to do this:
https://github.com/nginxinc/kubernetes-ingress/tree/v3.2.1/examples/shared-examples/custom-templates

For NGINX OSS the output directory for access_logs are set here:
https://github.com/nginxinc/kubernetes-ingress/blob/v3.2.1/internal/configs/version1/nginx.tmpl#L55

For NGINX Plus, it's set here:
https://github.com/nginxinc/kubernetes-ingress/blob/v3.2.1/internal/configs/version1/nginx-plus.tmpl#L83

If anything is unclear on how this works, please let me know!

[leonseng]

Just adding another data point to this.

I have a customer who would like the ability to configure a different destination for access_log. They are existing NGINX App Protect customers, and have configured NAP to send the logs to an external syslog server. They would like for NGINX Ingress Controller to have similar capability to standardize how logs are streamed.

They are currently modifying the Virtual Server template to set a custom access_log destination as a workaround, but that is not ideal as they have to update the template each time NGINX Ingress Controller gets updated.

[brianehlert]

NGINX Ingress Controller outputs to stdout following K8s convention. So that the NIC logs are captured and associated with the pods. This is normal K8s practice.

NGINX (the proxy used under the hood) can send event or access logs direct to syslog sink. This would be an override of the NIC templates and at this point require modifying the templates and building the image, or the use of a custom image.

That said, there are customers that direct NGINX logs to file (Access log most commonly). And the warning we give there is around volume filling which results in the pod being killed. Because log rotation cron jobs do not run within containers.

[vepatel]

hi @noor-muradi, we're looking at this proposal and figuring out what's the best way to design such solution. Will keep you updated here 👍🏼

[vepatel]

Hi @noor-muradi, because of how quickly the volumes can get filled with access-logs, we're a bit apprehensive of making this a default capability in NIC. Still investigating

[jjngx]

#5625