RFC: Services

[aoldershaw]

Rendered

Please comment on individual lines, not at the top-level.

[samgurtman-zz]

My preference would be to maintain the standard docker-compose configuration. Perhaps the service key can point directly to a docker-compose file. That way one could easily use a docker-compose file in a git repo if needed. Concourse would also render any variables in this compose file like it does with task files.

I guess the issue with this approach would be networking configuration may not match up with what concourse can supply. I would propose that concourse creates a compose container such that the docker host network is internal to the compose container.

[aoldershaw]

@samgurtman by "maintain the standard docker-compose configuration", do you mean be able to use your existing docker-compose.yml files as services? If so, there's nothing preventing the use of running docker-compose in a container as we currently do - but using services means that the task you run alongside it can be unprivileged.

In the RFC (the 2nd snippet under Service Configuration), I gave an example of running a service using a theoretical docker-compose Prototype, but you could also use one of the existing "docker-compose in docker" images as a service instead. This proposal just doesn't force you to use docker-compose for your services, but it's certainly possible to do so.

I guess the issue with this approach would be networking configuration may not match up with what concourse can supply

I think running docker-compose in a container should be fine, as long as your docker-compose.yml exposes ports, as Concourse won't be able to docker host network (as you mention)

[elgohr]

As the image in https://github.com/aoldershaw/rfcs/blob/services/084-services/proposal.md#networking states port 8080 specifically, I just wanted to point out that there are images

• exposing a different port
• exposing multiple ports

MySQL is a good example for that.

[aoldershaw]

@elgohr fair point, it's confusing - an earlier form of the diagram was tracing a single request to a specific example service endpoint, and I never ended up changing the port from 8080. Updated the diagram.

Worth noting that the use-case of different/multiple ports is supported - you can configure as many ports as you wish when configuring your service (https://github.com/aoldershaw/rfcs/blob/services/084-services/proposal.md#service-configuration). e.g.

task: ...
services:
- name: mysql
  ...
  ports:
  - name: default
    number: 3306
  - name: admin
    number: 33062

[jryan128]

I did an initial implementation of the Services RFC. concourse/concourse#8673. I'd want some feedback from Concourse maintainers before finishing it, so I don't waste my time if they won't accept it. It works great so far.

[drahnr]

the only thing I'd like to see, in addition to launching per task services, would be allowing tunneled access to services per worker, that can be assumed to exist.

I.e. a sccache-server/redis instance for caching compile artifacts across builds and tasks.

It would fit the tag on the can, but I am not convinced it's a) a good fit for the concept or b) bloating the RFC too much and should be extracted/not included.

[arsltan]

Hi there..
What's the status of this RFC as of now (11/2024)?
I can see it's ready to merge but it doesn't seem to be ready..
What's the concourse way to have a service running alongside the task (e.g. running tests that hit a DB)?

[paul-civitas]

I dug into the internals of gitlab services back when I was using gitlab.

The way it was working for gitlab was leaning heavily on deprecated docker behaviors, and was pretty inconsistent between runner types.

A nice as this type of functionality may be, I just want to make sure there's awareness that there's some significant technical challenges down this rabbit hole.