ssh+pam replaced /root/.google-authenticator with a zero-sized file

[mathieu-lacage]

Someone on the team managed to make ssh+pam+google authenticator to replace a good /root/.google-authenticator file with a zero-sized one.

I have no idea how they managed this but I have a log that you might find helpful:

auth.log

I suspect you guess what happened after .google-authenticator was zero-sized: no one could connect anymore on the that box: I had to use a console attached to the VM to put it back to a working state...

[mathieu-lacage]

Note: I found the whole idea of storing the last connection attempt timestamp in the configuration file somewhat fragile. i.e., trying to rewrite dynamically the configuration file sounds like a potential source of endless problems. Maybe it would make sense to store this somewhere else.

[ThomasHabets]

Strange. That should not happen. Looking at the code now I don't see how a zero byte file can be created, even in the presence of failures. This was a local filesystem, or something like NFS?

Could you try reproducing this with code at HEAD, where I have added some more logging for these failures?

If you can, while doing this, keep a separate terminal logged in so that you can fix the problem if you do manage to reproduce it.

[mathieu-lacage]

Strange. That should not happen. Looking at the code now I don't see how a zero byte file can be created, even in the presence of failures. This was a local filesystem, or something like NFS?

This is a local filesystem, ext4:
/dev/sda1 on / type ext4 (rw,relatime,data=ordered)

kernel:
Linux ssh1 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

This happened twice over the past 2 weeks on this server.

Could you try reproducing this with code at HEAD, where I have added some more logging for these failures?

If you can, while doing this, keep a separate terminal logged in so that you can fix the problem if you do manage to reproduce it.

Will get back to you with new logs if we manage to do this.

[mathieu-lacage]

This is reproducible multiple times per day now. I have depoyed the HEAD version and will report on /var/log/auth.log when it is triggered