Subscribers not running with Dockerised application

[dahlfrederik]

Hi,

I have an issue utilizing a subscriber when the Medusa backend is being run as a docker container.

My issue

I am working on a project with a Medusa JS backend & admin and my custom storefront created using NextJS. For my databases I am using PostgreSQL and Redis. I am using Solr for searching in my content. All services are supposed to be run as docker containers as part of deploying the project on a Linux server.

My issue is that when running all my containers in Docker and the event is being raised it cannot find any subscribers.

When starting the docker containers using docker-compose up --build -d my services/containers builds, starts and are all available through their designated ports.

I have subscribed to the event: product.updated

When being run in docker this is the result of invoking the product.updated event.

When I run all of my services using Docker, except for the Medusa backend which is started using medusa develop, the subscriber works.

This is the info output for medusa backend running locally:

The difference between the two scenarios is that when Medusa backend is being run locally my env variables are set in an .env file and when running in docker variables are defined in the docker-compose file.

If I go into the Medusa_backend container and open a terminal I can connect to the Redis instance and ping my redis server using the following command sequence:

docker exec -it medusa_backend bash

redis-cli -u $REDIS_URL

PING

Have anybody experienced anything similar with their subscribers and running the application in a Docker environment and if so how did you fix it?

Any input is very welcome.

Thanks in advance!

Below is all my files and I am happy to share more if needed.

My files

.env

NODE_ENV=development 
DATABASE_URL=postgres://postgres:<my-password>@localhost:5432
DATABASE_TYPE=postgres 
POSTGRES_PASSWORD=my-password
REDIS_URL=redis://localhost:6379
SOLR_URL=http://localhost:8983/solr
ADMIN_CORS=http://localhost:7000,http://localhost:7001,http://<my-server-ip>:7000
STORE_CORS=http://localhost:3000,http://<my-server-ip>:8000

Where as when running via docker-compose my env variables are set as container names instead of localhost.

Docker-compose.yml

version: "3.8"

networks:
  medusa_network:

services:
  backend:
    build:
      context: ./medusa_backend
      dockerfile: Dockerfile
    image: dahlfrederik/medusa_backend
    container_name: medusa_backend
    depends_on:
      postgres:
        condition: service_healthy
      redis:
        condition: service_healthy
    environment:
      DATABASE_URL: postgres://postgres:<my-password>@postgres:5432
      REDIS_URL: redis://redis:6379
      SOLR_URL: http://solr:8983/solr
      NODE_ENV: development
      JWT_SECRET: some_jwt_secret
      COOKIE_SECRET: some_cookie_secret
      PORT: 9000
      ADMIN_CORS: http://medusa_admin:7000,http://localhost:7001,http://<my-server-ip>:7000,http://localhost:7000
      STORE_CORS: http://localhost:3000,http://<my-server-ip>:8000,http://storefront:8000
    volumes:
      - ./medusa_backend/data:/var/lib/postgresql/data
    ports:
      - "9000:9000"
    restart: always
    networks:
      - medusa_network

  postgres:
    image: postgres:10.4
    container_name: postgres
    ports:
      - "5432:5432"
    volumes:
      - ./postgres/data:/var/lib/postgresql/data
    restart: always
    environment:
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=<my-password>
      - POSTGRES_DB=postgres
      - POSTGRES_INITDB_ARGS=--encoding=UTF-8
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U postgres -d postgres -h 127.0.0.1 -p 5432"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - medusa_network

  redis:
    image: redis
    container_name: redis
    expose: 
    - 6379 
    ports:
      - "6379:6379"
    restart: always
    environment:
    - ALLOW_EMPTY_PASSWORD=yes
    - REDIS_BIND_ADDRESS=0.0.0.0
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 10s
      timeout: 5s
      retries: 5
    networks:
      - medusa_network

  admin:
    build:
      context: ./medusa_admin
      dockerfile: Dockerfile
    image: dahlfrederik/medusa_admin
    container_name: medusa_admin
    depends_on:
      - backend
    environment:
      HOST: 0.0.0.0
      PORT: 7000
    ports:
      - "7000:7000"
    restart: always
    networks:
      - medusa_network

  storefront:
    build:
      context: ./nextjs_frontend
      dockerfile: Dockerfile
    image: dahlfrederik/medusa_storefront
    container_name: storefront
    depends_on:
      - backend
    environment:
      PORT: 8000
    ports:
      - "8000:8000"
    restart: always
    networks:
      - medusa_network

  solr:
    image: solr
    container_name: solr
    depends_on:
      - postgres
    ports:
      - "8983:8983"
    volumes:
      - data:/var/solr
      - ./solr:/opt/solr/server/solr/configsets/product_index
    entrypoint:
      - docker-entrypoint.sh
      - solr-precreate
      - product_index
    restart: always
    networks:
      - medusa_network

volumes:
  data:

Subscriber

This is the code for my subscriber

import { EventBusService } from "@medusajs/medusa";
import SolrService from "../services/solr";

class ProductPublishedSubscriber {
  private solrService: SolrService;

  /**
   * Initializes a new instance of the ProductPublishedSubscriber class.
   * @param solrService - An instance of the SolrService class.
   * @param eventBusService - An instance of the EventBusService class.
   * Destructured according as described in Medusa JS docs
   */
  constructor({
    solrService,
    eventBusService,
  }: {
    solrService: SolrService;
    eventBusService: EventBusService;
  }) {
    this.solrService = solrService;
    eventBusService.subscribe("product.updated", this.handleProductPublished);
  }

  /**
   * Handles the "product.updated" event by adding the updated product to the Solr index.
   * @param data - An object containing the ID of the updated product.
   */
  handleProductPublished = async (data: { id: string }): Promise<void> => {
    console.log("Found product with id: " + data.id);
    this.solrService.updateProduct(data.id);
  };
}

export default ProductPublishedSubscriber;

Medusa-config.js

const dotenv = require("dotenv");

let ENV_FILE_NAME = "";
switch (process.env.NODE_ENV) {
  case "production":
    ENV_FILE_NAME = ".env.production";
    break;
  case "staging":
    ENV_FILE_NAME = ".env.staging";
    break;
  case "test":
    ENV_FILE_NAME = ".env.test";
    break;
  case "development":
  default:
    ENV_FILE_NAME = ".env";
    break;
}

try {
  dotenv.config({ path: process.cwd() + "/" + ENV_FILE_NAME });
} catch (e) {}

// CORS when consuming Medusa from admin
const ADMIN_CORS =
  process.env.ADMIN_CORS 

// CORS to avoid issues when consuming Medusa from a client
const STORE_CORS =
  process.env.STORE_CORS 

// Database URL (here we use a local database called medusa-development)
const DATABASE_URL =
  process.env.DATABASE_URL 

// Medusa uses Redis, so this needs configuration as well
const REDIS_URL = process.env.REDIS_URL 

// Stripe keys
const STRIPE_API_KEY = process.env.STRIPE_API_KEY || "";
const STRIPE_WEBHOOK_SECRET = process.env.STRIPE_WEBHOOK_SECRET || "";

// This is the place to include plugins. See API documentation for a thorough guide on plugins.
const plugins = [
  `medusa-fulfillment-manual`,
  `medusa-payment-manual`,
  // Uncomment to add Stripe support.
  // You can create a Stripe account via: https://stripe.com
  // {
  //   resolve: `medusa-payment-stripe`,
  //   options: {
  //     api_key: STRIPE_API_KEY,
  //     webhook_secret: STRIPE_WEBHOOK_SECRET,
  //   },
  // },
];

module.exports = {
  projectConfig: {
    redis_url: REDIS_URL,
    // For more production-like environment install PostgresQL
    database_url: DATABASE_URL,
    database_type: "postgres",
    //database_database: "./medusa-db.sql",
    //database_type: "sqlite",
    store_cors: STORE_CORS,
    admin_cors: ADMIN_CORS,
  },
  plugins,
};

[srindom]

@dahlfrederik - can I get you to check if the events are stuck in the event outbox. You can do so by connecting to your Postgres db and running:

select * from "staged_job";

[dahlfrederik]

Hi, thanks for the response.

Below is the ouput of the command and with the content of staged_job.

[dahlfrederik]

UPDATE

I tried to console.log my EventBusService inside my subscribers constructor.
Running the Medusa_backend locally (via npm start) it logs a full eventbus object with connection parameters to my redis instance.
When executed in the docker environment no eventbus is logged - which makes me wonder if the eventbus get initialized at all.

Are there any known way to ensure the EventBusService is initialized when starting the backend?

[dahlfrederik]

SOLUTION

I figured out a solution to my problem.

When I realized that the Subscriber (and the eventBusService etc) was never initialized let me into the thought process that my project did not handle my TS files correctly.

It turns out that my subscriber and service while written in TS was not transpiled/compiled as expected.

Then I looked into MedusaJS and TypeScript and found a PR that added support for typescript (medusajs/medusa-starter-default#9)

I then decided to read through the changes and add the ones relevant for my project (for now).

I updated my package.json file with a build script:

"scripts": {
"build": "tsc -p tsconfig.json",
"start": "npm run build && medusa develop"
},

I added the following to the my tsconfig.json

{
  "compilerOptions": {
    "experimentalDecorators": true,
  },
  "extends": "./tsconfig.base.json",
  "include": ["src"],
  "exclude": ["node_modules", "**/*.spec.ts"]
}

I added a tsconfig.base.json file

{
    "compilerOptions": {
      "lib": ["es5", "es6"],
      "target": "esnext",
      "allowJs": true,
      "esModuleInterop": false,
      "module": "commonjs",
      "moduleResolution": "node",
      "emitDecoratorMetadata": true,
      "experimentalDecorators": true,
      "skipLibCheck": true,
      "skipDefaultLibCheck": true,
      "declaration": false,
      "sourceMap": false,
      "outDir": "./dist",
      "rootDir": "src",
      "baseUrl": "src"
    },
    "exclude": ["node_modules"]
  }

I updated my develop.sh script to run the same commandos as the npm start script

medusa migrations run

npm run build

medusa develop

Then I ran my docker commands

docker-compose up --build -d

Boom. The Medusa_backend log showcased that the Subscriber was initialized and the subscriber worked as intended.

[srindom]

Thanks for updating! Glad to hear you got things up and running 👍