RateLimiterQueue

RateLimiterQueue

RateLimiterQueue limits number of actions per some period of time and queues others to execute the next period.

Limit number of available tokens per interval with FIFO (first in, first out) queue on single server or distributed environment. It doesn't limit concurrency and try to consume as much points as possible, if there are waiting requests.

It strictly respects queue order with Memory and Cluster limiters only. Other limiters respect it too, but do not guarantee: some requests may go out of order because of distributed queue.

const http = require('http');
const express = require('express');
const { RateLimiterMemory, RateLimiterQueue } = require('rate-limiter-flexible');

const limiterFlexible = new RateLimiterMemory({
  points: 2,
  duration: 1, // Per one second
});

const limiter = new RateLimiterQueue(limiterFlexible, {
  maxQueueSize: 100,
});

const app = express();
app.get('/', async (req, res) => {
  try {
    const remainingTokens = await limiter.removeTokens(1)
    res.end(remainingTokens)
  } catch(err) {
    if (err instanceof Error) {
      res.status(400).end()
    } else {
      res.status(429).send('Too Many Requests');
    }
  }
});

const server = http.createServer(app);
server.listen(3002, () => {
  console.log('RateLimiterQueue service started');
});

In the above example, there is RateLimiterMemory instance created with 2 points available per one second duration. (points and tokens are the same thing in this case).

maxQueueSize option is set to 100. Default value is 4294967295 (2 ^ 32 - 1)

When is removeTokens method rejected?

1. If the queue is full and another request tries to remove token(s), removeTokens immediately rejected with RateLimiterQueueError.
2. removeTokens can be also rejected with RateLimiterQueueError, if application tries to remove more tokens than allowed per interval.
3. If you use one of store limiter like Redis, MongoDB or any other, it may be rejected with error from store.

RateLimiterQueueError can be got from components of rate-limiter-flexible:

const RateLimiterQueueError = require('rate-limiter-flexible/lib/component/RateLimiterQueueError')

getTokensRemaining

const Redis = require('ioredis');
const { RateLimiterRedis, RateLimiterQueue } = require('rate-limiter-flexible');
const redisClient = new Redis({ enableOfflineQueue: false });
const rlRedis = new RateLimiterRedis({
  storeClient: redisClient,
  points: 2, // Number of tokens
  duration: 5, // Per 5 second interval
});
const rlQueue = new RateLimiterQueue(rlRedis);
rlQueue.getTokensRemaining()
  .then((tokensRemaining) => { res.end(tokensRemaining) })
  .catch((errFromStore) => { res.status(500).end() })

Migration from limiter

This RateLimiterQueue provides the same features as rate limiter from limiter package. Advantages in comparison:

1. Works in multi-server scenario with any store limiter like Redis, MongoDB or any other from rate-limiter-flexible.
2. Respects queue order with Memory and Cluster limiters.
3. Works on top of native promises.

Example of migration:

var RateLimiter = require('limiter').RateLimiter;
var limiter = new RateLimiter(150, 'hour');
limiter.removeTokens(1, function(err, remainingRequests) {
  callMyRequestSendingFunction(...);
});

Should be changed to:

const {RateLimiterMemory, RateLimiterQueue} = require('rate-limiter-flexible');
const limiterFlexible = new RateLimiterMemory({
  points: 150,
  duration: 60 * 60, // hour
});
const limiter = new RateLimiterQueue(limiterFlexible);
app.get('/', async (req, res) => {
  const remainingTokens = await limiter.removeTokens(1);
  callMyRequestSendingFunction(...);
})

Scroll top to read more.