LawSuite Ug

AI-powered search and chat for Uganda's Law Corpus

All code & data used is 100% open-source.

law vault

This is the website with the law material that is scrapped. I built it with Nextra.

Dataset

The dataset is a json file containing all text.

View it here.

I recommend getting familiar with fetching, cleaning, and storing data as outlined in the scraping and embedding scripts below, but feel free to skip those steps and just use the dataset.

How It Works

LawSuite provides 2 things:

1. A search interface for Uganda's law corpus e.g constitution, penal code.
2. A chat interface for the information.

Search

Search was created with OpenAI Embeddings (text-embedding-ada-002).

First, we loop over our website and generate embeddings for each chunk of text.

Then in the app we take the user's search query, generate an embedding, and use the result to find the most similar passages from the book.

The comparison is done using cosine similarity across our database of vectors.

Our database is a Postgres database with the pgvector extension hosted on Supabase.

Results are ranked by similarity score and returned to the user.

Chat

Chat builds on top of search. It uses search results to create a prompt that is fed into GPT-3.

This allows for a chat-like experience where the user can ask questions about the book and get answers.

Running Locally

Here's a quick overview of how to run it locally.

Requirements

1. Set up OpenAI

You'll need an OpenAI API key to generate embeddings.

2. Set up Supabase

There is a schema.sql file in the root of the repo that you can use to set up the database.

Run that in the SQL editor in Supabase.

I recommend turning on Row Level Security and setting up a service role to use with the app.

Note: You don't have to use Supabase. Use whatever method you prefer to store your data. But I like Supabase and think it's easy to use.

Repo Setup

3. Clone repo

git clone https://github.com/greatsage-raphael/lawSuite.git

4. Install dependencies

npm i

5. Set up environment variables

Create a .env.local file in the root of the repo with the following variables:

OPENAI_API_KEY=

NEXT_PUBLIC_SUPABASE_URL=
SUPABASE_SERVICE_ROLE_KEY=

Dataset

6. Run scraping script

npm run scrape

This scrapes the text of the book from The Network State website and saves it to a json file.

7. Run embedding script

npm run embed

This reads the json file, generates embeddings for each chunk of text, and saves the results to your database.

App

8. Run app

npm run dev

Credits

Thanks to Mckay Wrigley for his work on The Network State GPT. This project would not be possible if he didn't make the project open-source. I highly recommend you check it out.

I would also like to shout out the open source technologies without which this project wouldn't have been possible: Supabase, Nextjs, Nextra. I highly recommend you check them out.

Contact

If you have any questions, feel free to reach out to me on Gmail.

Notes

New feature incoming: Ability to upload a pdf and have questions about the pdf answered.