Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add npm for Triggers and Functions deployment #1

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
/dist
/node_modules

.DS_Store
65 changes: 61 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,14 @@ EXPIRE {[user-api-key]}:[current minute number] 59
EXEC
```

## Prerequisites

To run the project yourself, you will need:
- [Redis](https://redis.io/) with the [Triggers and Functions](https://github.com/RedisGears/RedisGears#run-using-docker) module.
- Redis-cli
- [git](https://git-scm.com/download) command line tools.
- [Node.js](https://nodejs.org/) and [npm](https://www.npmjs.com)
- [Python](https://www.python.org/)

## An enhanced Rate Limiter

Expand All @@ -37,10 +45,19 @@ You can test this proof-of-concept using the latest Docker image including the "
docker run -p 6379:6379 redislabs/redisgears:edge
```

Clone this repository and import the Javascript library into the Redis Server:
Clone this repository and import the Javascript library into the Redis Server. Npm is used to automate the deployment of the Triggers and Functions library.

In the package.json use the scripts section to add:
```Json
"scripts": {
"deploy": "echo \"Deploying\"; redis-cli -x TFUNCTION LOAD REPLACE < ./src/limiter.js"
}
```
redis-cli -x TFUNCTION LOAD REPLACE < ./limiter.js

To deploy the Triggers and Functions:

```
npm run deploy
```

A load generator is provided and written in Python. Now prepare the Python environment:
Expand Down Expand Up @@ -71,12 +88,52 @@ Options to configure the load generator are:

![demo](limiter.gif)

## Limiter.js

The `limiter.js` file contains the trigger registration and the callback function that is executed on each event for the keys starting with the `{api:` prefix.

The trigger registration
```JavaScript
redis.registerKeySpaceTrigger('limiter', '{api:', function(client, data){ ... });
```

- `limiter`: The name of the trigger.
- `{api:`: Key prefix where to listen to changes.
- `function(client, data){...}`: The callback signature containing a client to execute Redis commands and the data we receive from the trigger.

Within the callback function we define what events to react to. The `redis` object can be used for registering functions, triggers and logging.

```JavaScript
redis.registerKeySpaceTrigger('limiter', '{api:', function(client, data){
if((data.event == 'incrby') || (data.event == 'incr')){

// log the event
redis.log(JSON.stringify(data));
}
});
```

If the event is generated because of a `incrby` or `incr` event, the business logic is applied and the `client` object is used to `call()` a command on the Redis Server.

```JavaScript
// get the token identifier
const tokenApi = data.key.split(":").slice(0, -1).join(":");

// build the Hash name, e.g.: {api:5I68T5910K}:data
// the use of curly brackets is to co-locate the data with the counter
const tokenApiData = `${tokenApi}:data`;

// get the current timestamp
var curr_time = client.call("time")[0];

// add data to the Hash
client.call('hset', tokenApiData, 'last', curr_time);
client.call('hincrby', tokenApiData, 'ops', '1');
```

## Further developments

The same keyspace trigger can enhance the rate limiter with additional functionality, such as:

- Data in Hash data structures can be indexed, and querying the user metadata space with `FT.SEARCH` can provide useful information such as the oldest and newest token usage, sorting tokens by usage, sorting tokens by the total number of requests, and more.
- storing all the requests in a per-token time series, enabling token usage analytics (usage during a time window, averages, etc.).


File renamed without changes.
13 changes: 13 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

19 changes: 19 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"name": "redis-limiter",
"version": "1.0.0",
"description": "In the rate-limiting use case, each user token is allowed a certain number of requests in a limited amount of time. One of the simplest algorithms that solve this problem is the \"Fixed window counter\" algorithm which checks the number of calls in a specific time interval. Implementing this algorithm with Redis is straightforward and is based on counters. The traditional logic of a rate limiter implies that the API gateway checks the current calls in a specific minute.",
"main": "limiter.js",
"scripts": {
"deploy": "echo \"Deploying\";redis-cli -x TFUNCTION LOAD REPLACE < ./src/limiter.js"
},
"repository": {
"type": "git",
"url": "git+https://github.com/redislabs-training/redis-limiter.git"
},
"keywords": [],
"license": "MIT",
"bugs": {
"url": "https://github.com/redislabs-training/redis-limiter/issues"
},
"homepage": "https://github.com/redislabs-training/redis-limiter#readme"
}
File renamed without changes.