I can't specify on which subnet I would like service to run

[komatom]

Hi

in other words I am trying to run databases in 1 subnet, web serice in another subnet. I kind of can do this over "x-aws-cloudformation:"
but I am having trouble with NFS access points, because they are created for all subnets per service, but I limit the service to 1 subnet and CloudFormation template roll backs and can't update.

So is there an option to specify for each service section to tie it to a subnet, and respectively from that list to be generated the NFSMount targets.

Thanks

[xender69]

Hi Komatom,

I am looking to do the same. Were you able to resolve your issue?

[JohnPreston]

Hello both,

Hoping this will be helpful for you to keep docker-compose compatibility with more features that currently supported by this plugin.

You can do that with ECS Compose-X: using x-vpc via creating a new one or via Lookup for an existing VPC and its subnets, you can then define networks, the docker-compose way with networks, and map which network to run the services into, using services.<service_name>.networks.[].

As for DB/ELB and such, these resources will have a Subnets parameter that you can use to specify which subnets defined in x-vpc you want to use.

Here are some examples that might be helpful: https://docs.compose-x.io/how_tos.html
Also, check out the labs which contain some real-life usage of compose-x inspired by deployments done for work.

Any requests/feedback, let me know!

[xender69]

hi John,
Thank you for your insight. If it's not too much trouble, can you take a look at my compose file:

version: '3.8'

x-aws-vpc: "vpc-0f64c8ba9cb5bb10f"

services:
  osticket:
    container_name: osticket-web
    image: osticket/osticket
    environment:
      MYSQL_HOST: db
      MYSQL_PASSWORD: secret
    depends_on:
      - db
    ports:
      - 80:80
  db:
    container_name: osticket-db
    image: mariadb
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: secret
      MYSQL_DATABASE: osticket
      MYSQL_USER: osticket
      MYSQL_PASSWORD: secret

x-aws-cloudformation:
  Resources:
    OsticketService:
      Properties:
        NetworkConfiguration:
          AwsvpcConfiguration:
            Subnets:
              - subnet-044ddbc9a47c8744a #public subnet-1
              - subnet-0a16347f784acfb76 #public subnet-2
    DbService:
      Properties:
        NetworkConfiguration:
          AwsvpcConfiguration:
            Subnets:
              - subnet-0227671981aa9a6b8 #private subnet-1
              - subnet-09b0d7e6ca32afc59 #private subnet-2

In another question thread, they suggested that I look at the compose convert document (aws cloudformation for me) to see how to add x-aws-cloudformation informatin into the yml file.
Here is the question #921
and the answer:

Add the overlay to the end of the docker-compose file, add the subnets you'd like to use to the script for each of the services:

x-aws-cloudformation:
  Resources:
    YourService:
      Properties:
        NetworkConfiguration:
          AwsvpcConfiguration:
            Subnets:
            - subnet-xxxxxxxxx
            - subnet-yyyyyyyyy
    Your2ndService:
      Properties:
        NetworkConfiguration:
          AwsvpcConfiguration:
            Subnets:
            - subnet-xxxxxxxxx
            - subnet-yyyyyyyyy
    LoadBalancer:
      Properties:
        Subnets:
        - subnet-xxxxxxxxx
        - subnet-yyyyyyyyy

[JohnPreston]

Hey. Sorry for "delay" in reply.

Here is what you are trying to do would look like using ECS Compose-X instead of this ECS plugin...

version: '3.8'

# Define networks and map these to x-vpc Subnets, to use with the services

networks:
  application:
    x-vpc: AppSubnets
  storage:
    x-vpc: StorageSubnets

services:
  osticket:
    container_name: osticket-web
    image: osticket/osticket
    environment:
      MYSQL_HOST: db
      MYSQL_PASSWORD: secret
    depends_on:
      - db
    ports:
      - 80:80
    networks:
      - application
  db:
    container_name: osticket-db
    image: mariadb
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: secret
      MYSQL_DATABASE: osticket
      MYSQL_USER: osticket
      MYSQL_PASSWORD: secret
    networks:
      - storage

# Just as an example, if you wanted to use RDS instead.

x-rds:
  my-db:
    MacroParameters:
      Engine: mariadb # Did not check the name
      EngineVersion: 10.4 # Did not check the engine version!
    Services:
      osticket:
        Access:
          DBCluster: RO # Allows rds:Describe on Cluster and Instances
        SecretsMappings: # Compose-X will generate a new Random username/password for the DB
          Mappings: # Map the key in the secret to the environment variable name your app needs
            host: MYSQL_HOST
            port: MYSQL_PORT
            username: MYSQL_USER
            password: MYSQL_PASSWORD
            dbname: MYSQL_DATABASE

x-vpc:
  Lookup:
    VpcId:
      Tags:
        - Name: my-existing-vpc
    PublicSubnets:
      Tags:
        - Name: my-public-subnets
    AppSubnets:
      Tags:
        - Name: my-application-subnets
    StorageSubnets:
      Tags:
        - Name: my-db-storage-subnets

Does that help?

[xender69]

Thank you so much, I will try out your config without the RDS for the moment. Will report back.

[xender69]

hi John,
Thank you for such a quick reply. So in order to use ecs compose-X, I would need to install python3 and initialize ecs-compose-x before I can use these commands in the docker compose file?

Does the below configuration look right?

version: '3.8'

# Define networks and map these to x-vpc Subnets, to use with the services

networks:
  application:
    x-vpc: AppSubnets
  storage:
    x-vpc: StorageSubnets

services:
  osticket:
    container_name: osticket-web
    image: osticket/osticket
    environment:
      MYSQL_HOST: db
      MYSQL_PASSWORD: secret
    depends_on:
      - db
    ports:
      - 80:80
    networks:
      - application
  db:
    container_name: osticket-db
    image: mariadb
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: secret
      MYSQL_DATABASE: osticket
      MYSQL_USER: osticket
      MYSQL_PASSWORD: secret
    networks:
      - storage

x-vpc:
  Lookup:
    VpcId:
      Tags:
        - Name: vpc-0603884f9eaf1ebb3
#    PublicSubnets: ** not used **
#      Tags:
#        - Name: my-public-subnets
#            - subnet-093223fe760e52016 #public subnet-1
#            - subnet-08120f88feb55e3f1 #public subnet-2
    AppSubnets:
      Tags:
        - Name: my-application-subnets
            - subnet-093223fe760e52016 #public subnet-1
            - subnet-08120f88feb55e3f1 #public subnet-2
    StorageSubnets:
      Tags:
        - Name: my-db-storage-subnets
            - subnet-0c68a298227d9c2e8 #private subnet-1
            - subnet-042cae15125ba9b1b #private subnet-2

Thank you again!!!

[JohnPreston]

Hey. yes, you are correct, you need python3 and install it with that. you can run it as a container too, you just then need to mount your was creds folder accordingly for API calls to succeed. See https://gallery.ecr.aws/compose-x/compose-x

I used to have x-vpc.Use which allowed a user to hardcode subnet & VPC IDs but it proved to be very limiting in many circumstances so I shifted to API-based discovery "only". Sorry about that inconvenience.

You don't need a lot of tags on your resources to use the Lookup feature. Just enough to identify your subnets.
You do need however to have at least Public/App/Storage subnets set in the Lookup too, but you can add more too. that's because each of these is passed on as parameters to all nested stacks, to avoid any hardcoding in the CFN templates.
But that does not mean it will be used!

If you don't care about re-using the VPC you already have, you can just omit x-vpc altogether, and compose-x will generate a VPC template & stack with all of it sorted out for you (creates a 3 tiers network). I do that all the time for PoC & testing so that I can delete everything once I am done.

[xender69]

hi John,
I have installed python3 and ecs compose-X via

Install in a virtual environment # Recommended

python3 -m venv venv
source venv/bin/activate
python3 -m pip install ecs-composex

So, will the code below work now for what I am trying to do?

version: '3.8'

# Define networks and map these to x-vpc Subnets, to use with the services

networks:
  application:
    x-vpc: AppSubnets
  storage:
    x-vpc: StorageSubnets

services:
  osticket:
    container_name: osticket-web
    image: osticket/osticket
    environment:
      MYSQL_HOST: db
      MYSQL_PASSWORD: secret
    depends_on:
      - db
    ports:
      - 80:80
    networks:
      - application
  db:
    container_name: osticket-db
    image: mariadb
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: secret
      MYSQL_DATABASE: osticket
      MYSQL_USER: osticket
      MYSQL_PASSWORD: secret
    networks:
      - storage

x-vpc:
  Lookup:
    VpcId:
      Tags:
        - Name: vpc-0603884f9eaf1ebb3
#    PublicSubnets: ** not used **
#      Tags:
#        - Name: my-public-subnets
#            - subnet-093223fe760e52016 #public subnet-1
#            - subnet-08120f88feb55e3f1 #public subnet-2
    AppSubnets:
      Tags:
        - Name: my-application-subnets
            - subnet-093223fe760e52016 #public subnet-1
            - subnet-08120f88feb55e3f1 #public subnet-2
    StorageSubnets:
      Tags:
        - Name: my-db-storage-subnets
            - subnet-0c68a298227d9c2e8 #private subnet-1
            - subnet-042cae15125ba9b1b #private subnet-2

Thank you so much!!!!

[xender69]

hi John,

Installed ecs-compoose-x and when I typed: ecs-compose-x --help, I immediately get the following error:

 ~/Desktop/docker-projects/osTicket/ecs-osticket/ /System/Volumes/Data/Users/johnchang/venv/bin/ecs-compose-x --help
/Users/johnchang/venv/lib/python3.11/site-packages/ecs_composex/compose/compose_services/service_image/docker_opts.py:26: UserWarning: You must install ecs-composex[ecrscan] extra to use this functionality
  warnings.warn(
usage: ecs-compose-x [-h] {up,render,create,plan,config,init,version} ...

Can you help with this please and also I thought that I could just do docker compose up but looking at the install guide, it seems I have to run something close to the below command?

# Simple example using docker-compose file and an extension with your AWS Settings
ecs-compose-x render -d templates -n my-new-stack -f docker-compose.yaml -f aws-settings.yaml

Thank you

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.