Skip to content

Queue tutorials: should we use channel#get rather than channel#consume? #38

Open
@SimonWoolf

Description

@SimonWoolf

Inspired by intercom conversation just now.

Given we're putting emphasis on the queues as a way for people to use multiple independent workers to each pop messages off to process, rather than for streaming, should we consider having the tutorials use channel#get ('pop one message off') rather than channel#consume ('give me messages as fast as they come')?

(I know my queue-demo used consume(), but that was so we could show off how fast messages got into the queue (and 'processing time' was ~0 as it was just printing them to the console), so the streaming method was fine).

We could even showcase some basic limited concurrency, e.g. something like:

function processItem(item, cb) {
  console.log("Processing item: ", item && String(item.content))
  setTimeout(cb, 5000);
}

function getMessage(channel) {
  channel.get(QUEUE, {}, (err, item) => {
    if(!item) {
      setTimeout(() => getMessage(channel), 1000);
      return;
    }
    processItem(item, (err) => {
      if(err) {
        channel.nack(item);
      } else {
        channel.ack(item);
      }
      // Processing finished, so get another message
      getMessage(channel);
    });
  });
}

const MAX_CONCURRENCY = 5;

amqp.connect(URL, (err, conn) => {
  if (err) bail(err);
  console.log("Connected");

  conn.createChannel((err, ch) => {
    for(let i=0; i<MAX_CONCURRENCY; i++) {
      getMessage(ch);
    }
  });
});

WDYT?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions