Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add perpetual deep_searcher scheduler #53

Draft
wants to merge 32 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
32 commits
Select commit Hold shift + click to select a range
3ee24e1
add initial deepsearcher project
latekvo Jul 3, 2024
993514b
add completion task getter function
latekvo Jul 3, 2024
c979613
add troubleshooting note to README
latekvo Jul 3, 2024
4a81b84
remove old scheduler.py
latekvo Jul 4, 2024
07b3bf6
add new structured query prompts
latekvo Jul 6, 2024
4623529
add functional model loader
latekvo Jul 6, 2024
887f7be
main loop and bug fixes
latekvo Jul 10, 2024
07edde0
implement initial core functions
latekvo Jul 14, 2024
0e60669
add deep_searcher as an optional service to docker compose
latekvo Jul 15, 2024
cc92986
finish all initial deep searcher flow functions
latekvo Jul 15, 2024
bae8aa4
black reformat
latekvo Jul 15, 2024
8eeceb7
update misconfiguration
latekvo Jul 15, 2024
23ac042
update conda env
latekvo Jul 15, 2024
8a71e42
add TEMPORARY worker entry for ds
latekvo Jul 15, 2024
c2c0d21
remove all remaining runtime crashes and bugs from ds
latekvo Jul 15, 2024
6574008
refactor small portion of summarizer
latekvo Jul 15, 2024
a92ae9e
add missing llm initialization and missing context compiler
latekvo Jul 15, 2024
55229b7
bugs, improvements and reformatting
latekvo Jul 15, 2024
8d80708
remove resolved fixme
latekvo Jul 15, 2024
b3daa0a
remove unnecessary config line
latekvo Jul 16, 2024
2ed5470
add basic completion prompt
latekvo Jul 23, 2024
2ca5d72
add basic query to summarizer workflow
latekvo Jul 24, 2024
2e135b9
add readme entry on running locally
latekvo Jul 25, 2024
1b44c62
fix crashes on start
latekvo Jul 25, 2024
1ae5daa
fix license
latekvo Jul 26, 2024
d7ab5c8
isolate schedulers from workers, add workaround for schedulers crashi…
latekvo Jul 26, 2024
e6eaea0
fix bug and change naming
latekvo Jul 26, 2024
7dc8362
update scheduler dockerfiles
latekvo Jul 27, 2024
dd88785
make frontend an optional container
latekvo Jul 27, 2024
50287b4
make pgadmin an optional container
latekvo Jul 27, 2024
2e74102
add basic task as an available mode on frontend
latekvo Jul 27, 2024
42e504a
add database execution timeout checking for crawls and summaries
latekvo Jul 27, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 0 additions & 4 deletions .github/workflows/black.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,3 @@ jobs:
uses: cytopia/[email protected]
with:
path: 'workers/embedder.py'
- name: Python Black (scheduler)
uses: cytopia/[email protected]
with:
path: 'workers/scheduler.py'
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@
same "printed page" as the copyright notice for easier
identification within third-party archives.

Copyright [yyyy] [name of copyright owner]
Copyright 2024 Research Chain Team

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
Expand Down
21 changes: 21 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,27 @@ Frontend is launched separately to back end, run the following command to start
- `environment.yml` is the linux env, but for macOS (silicon) and windows there are other available
- Apple intel is not supported anymore, but you can still get it working by manually installing
any missing package that comes up during the program execution.
- `pull access denied for X` error: The connection may occasionally get throttled, resulting in this error.
To solve this issue, let all the current downloads finish downloading, and restart the program.
Repeat until every file is downloaded.

#### Running locally

If you intend on running this project locally, whether for development, debugging or performance purposes,
you'll still need to have these three docker containers launched somewhere in the background: `ollama`, `postgres` and `rabbitmq`.

Here's a full command on how to initialize the conda environment run them all at once:

- `conda env create -f environment.yml`
- `sudo docker-compose -f docker/docker-compose.yml up ollama postgres rabbitmq`

Rest of the workers and services can now be launched directly via `main.py`:

- `conda activate ResearchChain`
- `python main.py -w crawler`

For list of all available workers and services see:
- `python main.py -h`

### This is a monorepo for both a tool, and apps for it:

Expand Down
5 changes: 5 additions & 0 deletions configs/none.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
{
"worker_type": "none",
"llm_config_name": "none",
"embedder_config_name": "none"
}
19 changes: 15 additions & 4 deletions configurator.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,25 @@
type=str,
dest="worker_type",
choices=[
"webui",
"crawler",
"embedder",
"summarizer",
],
default="none",
help="Select one of the ready worker configs to be used",
)
parser.add_argument(
"-s",
"--run-scheduler",
type=str,
dest="scheduler_type",
choices=[
"webui",
"deep_searcher",
],
default="none",
help="Select one of the available schedulers",
)
parser.add_argument(
"-c",
"--custom-worker-path",
Expand Down Expand Up @@ -78,11 +89,11 @@
def get_runtime_config():
global runtime_config

fallback_config_path = "configs/crawler.json"
empty_config_path = "configs/none.json"

if args.worker_type == "webui":
if args.worker_type == "none":
# fixme: this is a workaround, webui should be started from it's folder
return load_runtime_config_from_file(fallback_config_path)
return load_runtime_config_from_file(empty_config_path)

# fetch cache
if runtime_config:
Expand Down
32 changes: 32 additions & 0 deletions core/chainables/web.py
Original file line number Diff line number Diff line change
Expand Up @@ -77,3 +77,35 @@ def web_news_lookup_prompt():
),
]
)


def basic_query_prompt():
return ChatPromptTemplate.from_messages(
[
(
"system",
"You are a personal assistant. "
"Your job is to respond to the requests given to you by the user. "
"You are to follow the requests given precisely and intelligently. "
"Answer or complete the request to the best of your abilities. ",
),
("user", "{user_request}"),
]
)


# some schedulers may require data-extraction capabilities outside data-gathering
def structured_extraction_prompt():
return ChatPromptTemplate.from_messages(
[
(
"system",
"You are a data extraction and analysis specialist. "
"Your job is to respond in a structured way to the question you were given. "
"You are to follow the orders given precisely and intelligently. "
"You are provided with data chunk, use it, to fulfill user's request. "
"Satisfy the requested task to the best of your abilities. ",
),
("user", "Data: ```{data}``` User request: '{user_request}'"),
]
)
33 changes: 32 additions & 1 deletion core/databases/db_completion_tasks.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
from typing import Optional
from sqlalchemy import String, TEXT, Integer, Boolean, select, update
from sqlalchemy.orm import Mapped, mapped_column, Session, relationship

Expand Down Expand Up @@ -106,6 +105,20 @@ def db_get_incomplete_completion_tasks(amount: int = 1):
return results


def db_get_complete_completion_tasks(amount: int = 1):
with Session(engine) as session:
session.expire_on_commit = False

query = (
select(CompletionTask).where(CompletionTask.completed == True).limit(amount)
)

results = list(session.scalars(query).all())
session.expunge_all()

return results


def db_release_executing_tasks(uuid_list: list[str]):
with Session(engine) as session:
session.execute(
Expand Down Expand Up @@ -149,3 +162,21 @@ def db_update_completion_task_after_summarizing(summary: str, uuid: str):
)

session.commit()


def db_refresh_completion_tasks(timeout_seconds: int = 600):
# find completion tasks with timed-out execution and restart them to the awaiting state
# timing out after 600 seconds = 10 minutes by default
timeout_date = utils.gen_unix_time() + timeout_seconds
with Session(engine) as session:
session.execute(
update(CompletionTask)
.where(CompletionTask.executing == True)
.where(CompletionTask.completed == False)
.where(CompletionTask.execution_date > timeout_date)
.values(
executing=False,
)
)

session.commit()
18 changes: 18 additions & 0 deletions core/databases/db_crawl_tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -225,3 +225,21 @@ def db_increment_task_embedding_progression(uuid: str, model_name: str):
)

session.commit()


def db_refresh_crawl_tasks(timeout_seconds: int = 600):
# find completion tasks with timed-out execution and restart them to the awaiting state
# timing out after 600 seconds = 10 minutes by default
timeout_date = utils.gen_unix_time() + timeout_seconds
with Session(engine) as session:
session.execute(
update(CrawlTask)
.where(CrawlTask.executing == True)
.where(CrawlTask.completed == False)
.where(CrawlTask.execution_date > timeout_date)
.values(
executing=False,
)
)

session.commit()
26 changes: 26 additions & 0 deletions core/tools/model_loader.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
from langchain_community.llms.ollama import Ollama
from huggingface_hub import hf_hub_download
from llama_cpp import Llama
from langchain_experimental.llms.ollama_functions import OllamaFunctions

from configurator import get_runtime_config
from core.tools import errorlib
Expand All @@ -11,6 +12,7 @@
llm_config = runtime_configuration.llm_config
embedder_config = runtime_configuration.embedder_config


# problem with the current caching: we have to share those singletons across instances

# fixme: n_gpu_layers=-1 is a poor approach, it can and will cause crashes.
Expand All @@ -27,6 +29,11 @@ def load_ollama_llm() -> Ollama:
return llm


def load_ollama_functional_llm() -> OllamaFunctions:
# todo: As far as i see OllamaFunctions could be used by default, but old code has to be adapted
return OllamaFunctions(model=llm_config.model_name, base_url="http://ollama:11434")


def load_ollama_embedder() -> OllamaEmbeddings:
cached_embedder = runtime_configuration.embedder_object
if cached_embedder:
Expand Down Expand Up @@ -89,6 +96,25 @@ def load_llm():
return load_ollama_llm()


def load_functional_llm():
# EXPERIMENTAL
if llm_config is None:
errorlib.pretty_error(
title="Tried loading functional LLM without a valid configuration",
advice=f"Your worker configuration file is likely missing "
f"a valid {Fore.CYAN}llm_config_name{Fore.RESET} variable",
)

if llm_config.supplier == "hugging_face":
errorlib.pretty_error(
title="Tried running functional model with a HF configuration.",
advice=f"Functional models are not yet supported with llama.cpp loaders. "
f"Please switch to {Fore.CYAN}Ollama{Fore.RESET} or stop using functional models.",
)
else:
return load_ollama_functional_llm()


def load_embedder():
if embedder_config is None:
errorlib.pretty_error(
Expand Down
16 changes: 16 additions & 0 deletions docker/deep_searcher/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
FROM condaforge/miniforge3

WORKDIR /app

COPY . /app

RUN apt-get update && apt-get install -y \
build-essential \
g++ \
&& rm -rf /var/lib/apt/lists/*

RUN conda env create -f environment.yml

SHELL ["conda", "run", "-n", "ResearchChain", "/bin/bash", "-c"]

ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "ResearchChain", "python3", "main.py", "-s", "deep_searcher"]
24 changes: 22 additions & 2 deletions docker/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ services:
- app-network

pgadmin:
profiles:
- pgadmin
image: dpage/pgadmin4
depends_on:
- postgres
Expand All @@ -39,8 +41,6 @@ services:

ollama:
image: ollama/ollama
expose:
- 11434
ports:
- 11434:11434
pull_policy: always
Expand Down Expand Up @@ -113,6 +113,8 @@ services:
- app-network

frontend:
profiles:
- frontend
image: frontend
depends_on:
- postgres
Expand All @@ -125,6 +127,24 @@ services:
networks:
- app-network

deep_searcher:
profiles:
- deep_searcher
image: deep_searcher
depends_on:
postgres:
condition: service_started
rabbitmq:
condition: service_healthy
ollama:
condition: service_started
build:
context: ../.
dockerfile: ./docker/deep_searcher/Dockerfile
networks:
- app-network
- ollama-network

volumes:
ollama:
pgdata:
Expand Down
2 changes: 1 addition & 1 deletion docker/webui/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,4 @@ EXPOSE 8000

SHELL ["conda", "run", "-n", "ResearchChain", "/bin/bash", "-c"]

ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "ResearchChain", "python3", "main.py", "-w", "webui"]
ENTRYPOINT ["conda", "run", "--no-capture-output", "-n", "ResearchChain", "python3", "main.py", "-s", "webui"]
Loading
Loading