Proof-of-Concept Redis/FastApi Service integration with ChEMBL data

This project has the following goals:

• Use FastAPI to build a REST API in front of a Redis database to read data.
  • Use the Python Redis package to interact with the database.
  • Use the python-redis-om package to interact with and create object models.
• Ingest ChEMBL data data & enable for smiles string full-text search.
• Integrate Poetry & pyenv. Here is an example project using FastAPI & Poetry.
• Integrate with docker-compose.

Table of Contents

• Proof-of-Concept Redis/FastApi Service integration with ChEMBL data
  • Table of Contents
  • Installation
    • Pyenv
    • Poetry
    • Redis
    • Fast Api
    • Docker & Docker Compose
  • Development
    • Run Locally
    • Run the Docker Container Locally
    • Retrieve the Open API Documentation
  • Example Use
  • Future Work

Installation

The following are dependencies for running the project. Install each. It is advised to install them in the following order.

Pyenv

pyenv lets you easily switch between multiple versions of Python. It's simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well. Install pyenv first. Then create the python environment using the .python-version file.

cd <project>                            # Change to the project directory
pyenv install $(cat .python-version)    # Install the version

Poetry

Poetry is a tool for dependency management and packaging in Python. It allows you to declare the libraries your project depends on and manages (install/update) them for you.

cd <project>                            # Change to the project directory
poetry init                             # Initialize the existing project
poetry shell                            # Activate the virtual environment
exit                                    # Deactivate the virtual environment, if desired

Redis

Redis is an open source (BSD licensed), in-memory data structure store, used as a database, cache, and message broker. Install Redis first, then start the Redis server using a docker container, locally. Note, docker must be installed first. See below for more information on docker and docker-compose.

docker run -it --rm --name redis-stack-latest \         # Run the Redis container locally
   -p 6379:6379 \                                       # Expose the Redis port
   redis/redis-stack:latest                             # Use the latest version

Fast Api

Fast Api is a modern, fast (high-performance), web framework for building APIs with Python 3.6+ based on standard Python type hints. Install fastapi (with uvicorn & gunicorn) first, then start the application. Note, the redis server must be running to use the app.

cd <project>                                            # Change to the project directory
poetry shell                                            # Activate the virtual environment
uvicorn poc_redis_fastapi_chemblntd.main:app --reload   # Run the FastAPI server

Docker & Docker Compose

Docker is a set of platform as a service (PaaS) products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels. Docker Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application's services. Then, with a single command, you create and start all the services from your configuration.

cd <project>                            # Change to the project directory
docker-compose up                       # Run the docker container locally

Development

Run Locally

1. Perform the installation steps.
2. Start the Redis server.
3. Start the FastAPI server with reload.
4. Perform any changes to the code.
5. When finished, shut down the FastAPI server with Ctrl+C. Then shut down the Redis server with Ctrl+C.

Run the Docker Container Locally

1. Perform the installation steps.
2. Run the docker container locally.

Retrieve the Open API Documentation

Navigate to http://localhost:8000/redoc to view the Open API documentation while the FastAPI server is running.

Example Use

Ideally, one would refresh the data in redis or load it. As of 7/19/2023, the refresh process pulls down the ChEMBL-ntd data to a temp file and loads it into the running redis server. It flushes the database before loading each time. To refresh the data after starting the FastAPI server, run the following:

curl -X POST "http://localhost:8000/refresh"

Then, one can search for a smiles string in the ChEMBL-ntd data with the following:

curl -X GET "http://localhost:8000/chemblntd/search/smiles/NC(=O)" -H  "accept: application/json"

Future Work

• Integrate more caching with Redis & add the ability to add data.
• Modify the refresh process to be more dynamic.
• Create tests and test coverage.
• Update the dosstrings and OpenApi documentation.
• Add pre-commit hooks.
• Integrate different data sources & models.
• Modify the indexing model and REST methods for more robust searching.