Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Multiple RabbitSubscribe in one class, one looses connection #526

Open
wodka opened this issue Nov 4, 2022 · 8 comments
Open

Multiple RabbitSubscribe in one class, one looses connection #526

wodka opened this issue Nov 4, 2022 · 8 comments
Labels
rabbitmq stale waiting for more info

Comments

@wodka
Copy link
Contributor

wodka commented Nov 4, 2022

We have a class similar to this:

class Message {
  @RabbitSubscribe({
    exchange: 'message',
    queue: 'message-a-queue',
    queueOptions: {
      durable: true,
      maxPriority: 10,
    },
    routingKey: 'message-a',
  })
  async handleA() {}

  @RabbitSubscribe({
    exchange: 'message',
    queue: 'message-b-queue',
    queueOptions: {
      durable: true,
      maxPriority: 10,
    },
    routingKey: 'message-b',
  })
  async handleB() {}
}

after a few days running then message-a-queue consumer disconnects! There is nothing logged and I have no clue how to debug this. To me it looks like only the later one reconnects.
@zimiovid
Copy link

zimiovid commented Dec 1, 2022

Same problem!

@Andr-07
Copy link

Andr-07 commented Dec 1, 2022

Same for me -_-

@underfisk
Copy link
Contributor

@wodka Could you provide a small repo (the same exact golevelup modules version) so that we can see what's going on?

@wodka
Copy link
Contributor Author

wodka commented Dec 14, 2022

I can try - problem is that this is only happening after running for a few days :/

@underfisk
Copy link
Contributor

I can try - problem is that this is only happening after running for a few days :/

I believe there must be something that the module router that we have read might be throwing in which case the connection might be killed and not re-connected but I'd like to see if at some point I can play around with your repro and kinda attempt to force one handler to go down.
If at some point you can give us more context, that would be great but feel free to create the repro when you can

@EvgeniLeonti
Copy link

EvgeniLeonti commented Jan 2, 2023 •
edited
Loading

we have a similar setup and also experience similar issues, in our case:
this.amqpConnection.publish('EVENT_QUEUE', 'tx-created', {...});
throws an error

throw new Error('AMQP connection is not available');
/app/node_modules/@golevelup/nestjs-rabbitmq/lib/amqp/connection.js:192

this causes nestjs service to restart (if no catch handler exists) and seems like there has been a memory leak since then.

@wodka
Copy link
Contributor Author

wodka commented Jan 2, 2023 •
edited
Loading

:/ trying to get it to reproduce I've now found one problem that was earlier not logged:

Error: Channel closed by server: 404 (NOT-FOUND)

-> the Channel exits and is filling and with a restart of the service process the entries...

more detailed logs 
Failed to setup a RabbitMQ channel - name: AmqpConnection / error: Channel ended, no reply will be forthcoming Error: Channel ended, no reply will be forthcoming
    at rej (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:201:7)
    at C._rejectPending (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:207:42)
    at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:171:8)
    at C._closeChannels (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:394:18)
    at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:401:8)
    at C.onSocketError (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:355:10)
    at Connection.emit (node:events:513:28)
    at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:481:12)
    at Socket.emit (node:events:513:28)
    at emitReadable_ (node:internal/streams/readable:590:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)"}


Failed to setup a RabbitMQ channel - name: AmqpConnection / error: Operation failed: QueueDeclare; 404 (NOT-FOUND) with message "NOT_FOUND - home node 'rabbit@rabbitmq-0.rabbitmq.rabbitmq-35547411-production.svc.cluster.local' of durable queue 'pigeon-email-transactional-queue' in vhost '/' is down or inaccessible" Error: Operation failed: QueueDeclare; 404 (NOT-FOUND) with message "NOT_FOUND - home node 'rabbit@rabbitmq-0.rabbitmq.rabbitmq-35547411-production.svc.cluster.local' of durable queue 'pigeon-email-transactional-queue' in vhost '/' is down or inaccessible"
    at reply (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:134:29)
    at C.accept (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:417:7)
    at Connection.mainAccept [as accept] (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:64:33)
    at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:478:48)
    at Socket.emit (node:events:513:28)
    at emitReadable_ (node:internal/streams/readable:590:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)



Failed to setup a RabbitMQ channel - name: AmqpConnection / error: Channel ended, no reply will be forthcoming Error: Channel ended, no reply will be forthcoming
    at rej (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:201:7)
    at C._rejectPending (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:207:42)
    at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:171:8)
    at C._closeChannels (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:394:18)
    at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:401:8)
    at C.onSocketError (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:355:10)
    at Connection.emit (node:events:513:28)
    at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:481:12)
    at Socket.emit (node:events:513:28)
    at emitReadable_ (node:internal/streams/readable:590:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)



Successfully connected to RabbitMQ broker (default)
Error: Channel closed by server: 404 (NOT-FOUND) with message "NOT_FOUND - home node 'rabbit@rabbitmq-0.rabbitmq.rabbitmq-35547411-production.svc.cluster.local' of durable queue 'pigeon-email-transactional-queue' in vhost '/' is down or inaccessible"
    at C.accept (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:422:17)
    at Connection.mainAccept (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:64:33)
    at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:478:48)
    at Socket.emit (node:events:513:28)
    at emitReadable_ (node:internal/streams/readable:590:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)



Unhandled rejection Error: Channel ended, no reply will be forthcoming
    at rej (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:201:7)
    at C._rejectPending (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:207:42)
    at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:171:8)
    at C._closeChannels (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:394:18)
    at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:401:8)
    at C.onSocketError (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:355:10)
    at Connection.emit (node:events:513:28)
    at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:481:12)
    at Socket.emit (node:events:513:28)
    at emitReadable_ (node:internal/streams/readable:590:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)
Unhandled rejection Error: Channel ended, no reply will be forthcoming
    at rej (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:201:7)
    at C._rejectPending (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:207:42)
    at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/channel.js:171:8)
    at C._closeChannels (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:394:18)
    at C.toClosed (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:401:8)
    at C.onSocketError (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:355:10)
    at Connection.emit (node:events:513:28)
    at Socket.go (/usr/src/app/node_modules/@golevelup/nestjs-rabbitmq/node_modules/amqplib/lib/connection.js:481:12)
    at Socket.emit (node:events:513:28)
    at emitReadable_ (node:internal/streams/readable:590:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:81:21)

Edit: not sure that error is really related, I keep trying

@github-actions GitHub Actions
Copy link

github-actions bot commented Oct 2, 2024

This issue is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale label Oct 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
rabbitmq stale waiting for more info
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@wodka @underfisk @EvgeniLeonti @Andr-07 @zimiovid