Fails when /var/log is a tmpfs

[georgev93]

I'm trying to mount some log directories as tmpfs to avoid constant writes to storage.

This is a: Run Issue

Details

I've tried many styles of setting /var/log as a tmpfs within my docker compose file, but all of them seem to result in the contents of /var/log being clobbered after being set up. For instance, every second lighttpd fails because there is no /var/log/lighttpd directory. These messages get barfed to docker's log:

pihole  | lighttpd: no process found
pihole  | install: cannot create regular file '/var/log/lighttpd/access-pihole.log': No such file or directory
pihole  | install: cannot create regular file '/var/log/lighttpd/error-pihole.log': No such file or directory
pihole  | chown: cannot access '/var/log/lighttpd': No such file or directory
pihole  | chmod: cannot access '/var/log/lighttpd/access-pihole.log': No such file or directory
pihole  | chmod: cannot access '/var/log/lighttpd/error-pihole.log': No such file or directory
pihole  | 2024-11-19 14:52:21: configfile.c.1290) opening errorlog '/var/log/lighttpd/error-pihole.log' failed: No such file or directory
pihole  | 2024-11-19 14:52:21: server.c.1509) Opening errorlog failed. Going down.
pihole  | Stopping lighttpd

I can docker exec -it -u www-data pihole /bin/bash into /var/log and mkdir lighttpd successfully, so this isn't a permissions issue of the tmpfs being mounted. In fact, the whole /var/log directory looks pretty bare compared to a properly built container. I'm guessing this is because these directories are build/populated during the initial build phase, then the empty tmpfs directory gets plopped on top of the pre-build /var/log directory without copying over its contents. However, I believe typically docker does copy over contents when it mounts these tmpfs volumes. For reference, here is my docker-compose file:

services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "9010:80/tcp"
    environment:
      TZ: 'America/Chicago'
      PIHOLE_DNS_: '1.1.1.1;1.0.0.1'
    volumes:
      - './etc-pihole:/etc/pihole'
      - './etc-dnsmasq.d:/etc/dnsmasq.d'
      - 'run_vol:/run'
      - type: tmpfs
        target: '/var/log'
        tmpfs:
          mode: 0o777
    tmpfs:
      - '/tmp'
    restart: unless-stopped

volumes:
  run_vol:
    driver_opts:
      type: tmpfs
      device: tmpfs

You'll see three distinct methods of mounting tmpfs volumes there, one for /tmp, one for /run, and one for /var/log. The way /run is mounted was the only way any of this would work for some reason. The other two ways do result in mounting the tmpfs appropriately (as confirmed by df -hT and permissions confirmed by docker exec experimentation as well as ls -la).

Related Issues

• I have searched this repository/Pi-hole forums for existing issues and pull requests that look similar

How to reproduce the issue

1. Environment data

• Operating System: I use Arch BTW
• Hardware: PC
• Kernel Architecture: amd64
• Docker Install Info and version:
  • Software source: official docker-ce
  • Supplimentary Software: None
• Hardware architecture: x86

2. docker-compose.yml contents, docker run shell command, or paste a screenshot of any UI based configuration of containers here Included above... sorry, I don't follow directions well...
3. any additional info to help reproduce

These common fixes didn't work for my issue

• I have tried removing/destroying my container, and re-creating a new container
• I have tried fresh volume data by backing up and moving/removing the old volume data
• I have tried running the stock docker run example(s) in the readme (removing any customizations I added)
• I have tried a newer or older version of Docker Pi-hole (depending what version the issue started in for me)
• I have tried running without my volume data mounts to eliminate volumes as the cause

If the above debugging / fixes revealed any new information note it here.
Add any other debugging steps you've taken or theories on root cause that may help.

[rdwebdesign]

I'm not able to test it right now, but did you try to create a tmpfs mount on the host system (mount -t tmpfs ...) and mount it into the container with something like this?

    volumes:
      - './etc-pihole:/etc/pihole'
      - './etc-dnsmasq.d:/etc/dnsmasq.d'
      - '/mytmpfsdir:/var/log'      <-------

[dschaper]

  - type: tmpfs
    target: '/var/log'
    tmpfs:
      mode: 0o777
tmpfs:
  - '/tmp'
restart: unless-stopped

I don't think that mode is valid. https://docs.docker.com/reference/compose-file/services/#long-syntax-5

And you don't need the single quotes for the - /tmp line in the tmpfs array. https://docs.docker.com/reference/compose-file/services/#tmpfs

Can you run an inspection to see how the mounts are being created? You can grep just the tmpfs mount if you want.

docker inspect pihole --format '{{ json .Mounts }}'

[georgev93]

Thanks for the fast responses!
In response to @rdwebdesign

I'm not able to test it right now, but did you try to create a tmpfs mount on the host system (mount -t tmpfs ...) and mount it into the container with something like this?

volumes:
  - './etc-pihole:/etc/pihole'
  - './etc-dnsmasq.d:/etc/dnsmasq.d'
  - '/mytmpfsdir:/var/log'      <-------

That worked! Is this expected behavior? I can make this work for my solution whether this is expected behavior or not (I imagine this is more of a docker question than a "your pihole docker container implementation" question, but I could be wrong.

In response to @dschaper:

  - type: tmpfs
    target: '/var/log'
    tmpfs:
      mode: 0o777
tmpfs:
  - '/tmp'
restart: unless-stopped

I don't think that mode is valid. https://docs.docker.com/reference/compose-file/services/#long-syntax-5

And you don't need the single quotes for the - /tmp line in the tmpfs array. https://docs.docker.com/reference/compose-file/services/#tmpfs

Can you run an inspection to see how the mounts are being created? You can grep just the tmpfs mount if you want.

docker inspect pihole --format '{{ json .Mounts }}'

Here's what I got (without implementing @rdwebdesign's solution):

[{"Type":"bind","Source":"/home/george/Desktop/pihole/etc-pihole","Destination":"/etc/pihole","Mode":"rw","RW":true,"Propagation":"rprivate"},{"Type":"bind","Source":"/home/george/Desktop/pihole/etc-dnsmasq.d","Destination":"/etc/dnsmasq.d","Mode":"rw","RW":true,"Propagation":"rprivate"},{"Type":"volume","Name":"pihole_run_vol","Source":"/var/lib/docker/volumes/pihole_run_vol/_data","Destination":"/run","Driver":"local","Mode":"z","RW":true,"Propagation":""},{"Type":"tmpfs","Source":"","Destination":"/var/log","Mode":"","RW":true,"Propagation":""}]

[dschaper]

{"Type":"tmpfs","Source":"","Destination":"/var/log","Mode":"","RW":true,"Propagation":""}

Okay, that looks like it's being created. Try removing the

tmpfs:
  mode: 0o777

The default mode for a tmpfs mount is 777 already and I think that o in there may be throwing things off, 0o777 vs 0x777.

https://docs.docker.com/engine/storage/tmpfs/#specify-tmpfs-options

tmpfs-mode	File mode of the tmpfs in octal. For instance, 700 or 0770. Defaults to 1777 or world-writable.

[georgev93]

@dschaper, the permissions of these folders/files are actually in octal (0o...), not hexadecimal (0x...). Each digit represents three bits, so 0b000 through 0b111 is 0-7 (eight possibilities). If each one were four bits and could range from 0x0 (0b0000) to 0xF (0b1111), then they would be hex. You are correct that putting 777 results in rwxrwxrwx permissions, but so does 0o777. 0x777 on the other hand results in r-xrwSrwt permissions, which I'm actually not linux-y enough to understand what that actually means... but it almost makes sense with the hex translation (0x777 = 0b011 101 110 111 = 0o3567). In either case, these permissions are not actually my problem, although I grant you that they do look weird. It's not often we see the octal prefix of 0o.

In either case, we can count this issue as closed. @rdwebdesign's answer did, in fact, work! However, I'm a bit sad because it looks like docker does not copy existing contents of a folder into a tmpfs when creating a tmpfs on top of an existing folder. In case anyone wants to see a simple demonstration of this:

# Dockerfile
FROM alpine:latest

RUN mkdir -p /home/regMount
RUN mkdir -p /home/tmpMountVolumesSection
RUN mkdir -p /home/tmpfsSectionMount
RUN mkdir -p /home/tmpfsVolumeType

RUN touch /home/regMount/testFileFromDockerfile
RUN touch /home/tmpMountVolumesSection/testFileFromDockerfile
RUN touch /home/tmpfsSectionMount/testFileFromDockerfile
RUN touch /home/tmpfsVolumeType/testFileFromDockerfile

# Keep the container running
CMD ["tail", "-f", "/dev/null"]

# docker-compose.yml
services:
  idle:
    build: .
    container_name: tfs
    volumes:
      - ./regDir:/home/regMount
      - my_tempfs:/home/tmpMountVolumesSection
      - type: tmpfs
        target: '/home/tmpfsVolumeType'
        tmpfs:
          mode: 777
        volume:
          nocopy: false
    tmpfs:
      - /home/tmpfsSectionMount

volumes:
  my_tempfs:
    driver_opts:
      type: tmpfs
      device: tmpfs

Notice that any of the three methods of mounting a tmpfs volume on top of a directory that already has stuff in it will clobber the contents of that directory. Very sad.

But alas, a docker problem (likely intended behavior but I don't like it so I'm going to call it a problem), not a pi-hole problem.

Thanks to both of you for your time!