diff --git a/.gitignore b/.gitignore new file mode 100644 index 0000000..0a157ad --- /dev/null +++ b/.gitignore @@ -0,0 +1,4 @@ +/dist +/node_modules + +.DS_Store diff --git a/README.md b/README.md index 1c56b8c..e26290f 100644 --- a/README.md +++ b/README.md @@ -17,6 +17,14 @@ EXPIRE {[user-api-key]}:[current minute number] 59 EXEC ``` +## Prerequisites + +To run the project yourself, you will need: +- [Redis](https://redis.io/) with the [Triggers and Functions](https://github.com/RedisGears/RedisGears#run-using-docker) module. +- Redis-cli +- [git](https://git-scm.com/download) command line tools. +- [Node.js](https://nodejs.org/) and [npm](https://www.npmjs.com) +- [Python](https://www.python.org/) ## An enhanced Rate Limiter @@ -37,10 +45,19 @@ You can test this proof-of-concept using the latest Docker image including the " docker run -p 6379:6379 redislabs/redisgears:edge ``` -Clone this repository and import the Javascript library into the Redis Server: +Clone this repository and import the Javascript library into the Redis Server. Npm is used to automate the deployment of the Triggers and Functions library. +In the package.json use the scripts section to add: +```Json +"scripts": { + "deploy": "echo \"Deploying\"; redis-cli -x TFUNCTION LOAD REPLACE < ./src/limiter.js" + } ``` -redis-cli -x TFUNCTION LOAD REPLACE < ./limiter.js + +To deploy the Triggers and Functions: + +``` +npm run deploy ``` A load generator is provided and written in Python. Now prepare the Python environment: @@ -71,6 +88,48 @@ Options to configure the load generator are: ![demo](limiter.gif) +## Limiter.js + +The `limiter.js` file contains the trigger registration and the callback function that is executed on each event for the keys starting with the `{api:` prefix. + +The trigger registration +```JavaScript +redis.registerKeySpaceTrigger('limiter', '{api:', function(client, data){ ... }); +``` + +- `limiter`: The name of the trigger. +- `{api:`: Key prefix where to listen to changes. +- `function(client, data){...}`: The callback signature containing a client to execute Redis commands and the data we receive from the trigger. + +Within the callback function we define what events to react to. The `redis` object can be used for registering functions, triggers and logging. + +```JavaScript +redis.registerKeySpaceTrigger('limiter', '{api:', function(client, data){ + if((data.event == 'incrby') || (data.event == 'incr')){ + + // log the event + redis.log(JSON.stringify(data)); + } +}); +``` + +If the event is generated because of a `incrby` or `incr` event, the business logic is applied and the `client` object is used to `call()` a command on the Redis Server. + +```JavaScript +// get the token identifier +const tokenApi = data.key.split(":").slice(0, -1).join(":"); + +// build the Hash name, e.g.: {api:5I68T5910K}:data +// the use of curly brackets is to co-locate the data with the counter +const tokenApiData = `${tokenApi}:data`; + +// get the current timestamp +var curr_time = client.call("time")[0]; + +// add data to the Hash +client.call('hset', tokenApiData, 'last', curr_time); +client.call('hincrby', tokenApiData, 'ops', '1'); +``` ## Further developments @@ -78,5 +137,3 @@ The same keyspace trigger can enhance the rate limiter with additional functiona - Data in Hash data structures can be indexed, and querying the user metadata space with `FT.SEARCH` can provide useful information such as the oldest and newest token usage, sorting tokens by usage, sorting tokens by the total number of requests, and more. - storing all the requests in a per-token time series, enabling token usage analytics (usage during a time window, averages, etc.). - - diff --git a/generator.py b/generator/generator.py similarity index 100% rename from generator.py rename to generator/generator.py diff --git a/package-lock.json b/package-lock.json new file mode 100644 index 0000000..1235014 --- /dev/null +++ b/package-lock.json @@ -0,0 +1,13 @@ +{ + "name": "redis-limiter", + "version": "1.0.0", + "lockfileVersion": 2, + "requires": true, + "packages": { + "": { + "name": "redis-limiter", + "version": "1.0.0", + "license": "MIT" + } + } +} diff --git a/package.json b/package.json new file mode 100644 index 0000000..553d5a3 --- /dev/null +++ b/package.json @@ -0,0 +1,19 @@ +{ + "name": "redis-limiter", + "version": "1.0.0", + "description": "In the rate-limiting use case, each user token is allowed a certain number of requests in a limited amount of time. One of the simplest algorithms that solve this problem is the \"Fixed window counter\" algorithm which checks the number of calls in a specific time interval. Implementing this algorithm with Redis is straightforward and is based on counters. The traditional logic of a rate limiter implies that the API gateway checks the current calls in a specific minute.", + "main": "limiter.js", + "scripts": { + "deploy": "echo \"Deploying\";redis-cli -x TFUNCTION LOAD REPLACE < ./src/limiter.js" + }, + "repository": { + "type": "git", + "url": "git+https://github.com/redislabs-training/redis-limiter.git" + }, + "keywords": [], + "license": "MIT", + "bugs": { + "url": "https://github.com/redislabs-training/redis-limiter/issues" + }, + "homepage": "https://github.com/redislabs-training/redis-limiter#readme" +} diff --git a/limiter.js b/src/limiter.js similarity index 100% rename from limiter.js rename to src/limiter.js