Make sure we are not hitting systemd task limits

[itsdalmo]

While looking for resources on what KillMode to use for Concourse, I came across this:
concourse/concourse#480

From my preliminary searches:

• https://www.freedesktop.org/software/systemd/man/systemd.resource-control.html
• https://www.freedesktop.org/software/systemd/man/systemd-system.conf.html#

It seems like leaving TasksMax unset makes it default to DefaultTasksMax which defaults to:

Defaults to 15%, which equals 4915 with the kernel's defaults on the host, but might be smaller in OS containers.

MemoryLimit seems to be deprecated and replaced with MemoryMax.

So far we've not had any problems hitting high CPU/Memory utilisations on t3.large instances, but this warrants further investigation...

[itsdalmo]

Relevant issue for potential problems we could encounter:
systemd/systemd#3211

[itsdalmo]

For their smoke tests, Concourse uses the following systemd configuration (relevant parts highlighted):
https://github.com/concourse/concourse/blob/master/ci/deployments/smoke/systemd/concourse-worker.service#L9-L12

[rickardl]

Example of current limits in our own environment for workers

Limit	Soft Limit	Hard Limit	Units
Max cpu time	unlimited	unlimited	seconds
Max file size	unlimited	unlimited	bytes
Max data size	unlimited	unlimited	bytes
Max stack size	8388608	unlimited	bytes
Max core file size	0	unlimited	bytes
Max resident set	unlimited	unlimited	bytes
Max processes	30580	30580	processes
Max open files	1024	4096	files
Max locked memory	65536	65536	bytes
Max address space	unlimited	unlimited	bytes
Max file locks	unlimited	unlimited	locks
Max pending signals	30580	30580	signals
Max msgqueue size	819200	819200	bytes
Max nice priority	0	0
Max realtime priority	0	0
Max realtime timeout	unlimited	unlimited	us

I would suggest we increase our systemd concourse config to

LimitNOFILE=4096:8192