[Solved] Recurring Permission denied errors with files in /storage

[nicbou]

This is a solved problem. I found the solution as I typed this. I will leave the solution for other people.

The problem

When I try to log into the admin area, I sometimes get an Internal Server Error.

There are two types of errors in storage/logs/web.log:

[yii\base\ErrorException:2] yii\base\ErrorException: fopen(/var/www/storage/runtime/mutex/4a16982b0b160dd70f5949b82034c856.lock): failed to open stream: Permission denied in /var/vendor/yiisoft/yii2/mutex/FileMutex.php:105

[yii\caching\FileCache::setValue] Unable to write cache file '/var/www/storage/runtime/cache/9c/CraftCMS9c95f825294a84b791c15c480bde47aa.bin': file_put_contents(/var/www/storage/runtime/cache/9c/CraftCMS9c95f825294a84b791c15c480bde47aa.bin): failed to open stream: Permission denied

Some files in /storage do not have correct permissions. If I run chown -R www-data:www-data storage, the admin area works again.

The issue comes back every few days. I wake up in the morning and the Craft backend gives me an Internal Server Error.

The symptoms

• The admin area gives a 500 Internal Server Error. The public website works fine.
• The issue returns at random times, even when I am not at the keyboard.
• Most files in /storage have the correct owner (www-data:www-data), including newly-created files.
• Some files in /storage are owned by root. They cause the Internal Server Error because www-data can't access them.
• The issue happens in dev (MacOS) and production (Debian), even after rebuilding the Docker images.

My Craft setup

• Craft runs in an alpine:3.13 as the Docker image. php7-fpm to process requests.
• A separate nginx image acts as a reverse proxy. This part works fine.
• The /storage directory is not a Docker mounted volume, just a regular directory inside the image.
• I use cron to run the craft queue (* * * * * /var/www/craft queue/run >> /var/log/cron.log 2>&1)

The solution

• The host machine does not touch the /storage directory. The problem happens 100% inside the docker image.
• The PHP user is www-data (confirmed by the error log), but something is creating files in /storage as root.
• It's not caused by user interaction, so it's probably a recurring task.

...recurring task, running in the background... IT'S CRON!

cron runs the craft queue every minute as root. It creates files in /storage with root as the owner. When PHP tries to access them as www-data, it gets a permission error.

The solution: run the craft queue cronjob as www-data instead of the default root (add -u www-data). Make sure that www-data has permission to run /var/www/craft

# Dockerfile

COPY craft/crontab.conf /tmp/crontab.conf
RUN crontab -u www-data /tmp/crontab.conf

[nicbou]

The documentation should probably mention that the craft queue/run should be run as the correct user, and not as root.

https://craftcms.com/docs/3.x/config/config-settings.html#runqueueautomatically