update readme and env keys

Author: chiral-carbon

.gitignore

@@ -165,7 +165,7 @@ requirements.txt
 wandb/
 slurm_logs/
 notebooks/
-misc/polymathic_data_files
-misc/notes
-misc/test.ipynb
+assets/polymathic_data_files
+assets/notes
+assets/test.ipynb
 Meta-Llama-3-70B-Instruct/

README.md

@@ -21,7 +21,7 @@ We use the Llama-3-70B-Instruct model with 2 A100 80GB GPUs for structured infor
 <table>
   <tr>
     <td width="50%" valign="top">
-      <img src="misc/eval_pipeline.png" alt="Fig 1" width="100%">
+      <img src="assets/eval_pipeline.png" alt="Fig 1" width="100%">
       <p align="center">
         <em>Fig 1: Prompt optimization pipeline to maximize precision of the model annotated 
         predictions by running on manually annotated subset of scientific corpora. The 
@@ -30,7 +30,7 @@ We use the Llama-3-70B-Instruct model with 2 A100 80GB GPUs for structured infor
       </p>
     </td>
     <td width="50%" valign="top">
-      <img src="misc/pipeline.png" alt="Fig 2" width="100%">
+      <img src="assets/pipeline.png" alt="Fig 2" width="100%">
       <p align="center">
         <em>Fig 2: Illustration of the structured prediction pipeline on the full corpus of 
         scientific papers, which runs optimized prompts and stores the model's outputs in 
@@ -59,10 +59,26 @@ Set up code formatting and pre-commit hooks:
 ```
 pre-commit install
 ```
+## Quickstart
 
-## Running the tool
+### Run an existing DB
 
-### On new data: Download raw data from arXiv
+To run an existing database in the `databases` directory:
+
+```
+sqlite3 databases/<table_name>
+```
+
+### Launch a Gradio interface for SQL query search over the created databases
+```
+gradio scripts/run_db_interface.py
+```
+The interface shows all the created databases in the `data/databases` directory which can be loaded and queried.
+
+
+## Running the tool on new data 
+
+### Download raw data from arXiv
 
 Run `scripts/collect_data.py` to download papers for arXiv:
 ```
@@ -150,20 +166,6 @@ Options:
 
 All current databases are in the ```data/databases``` directory which can be downloaded and loaded with ```sqlite3``` to run queries on your own terminal. Refer to the [databases README](data/databases/README.md) for information on the tables that constitute each of the databases.
 
-## Run an existing DB
-
-To run an existing database in the `databases` directory:
-
-```
-sqlite3 databases/<table_name>
-```
-
-## Launch a Gradio interface for SQL query search over the created databases
-```
-gradio scripts/run_db_interface.py
-```
-The interface shows all the created databases in the `data/databases` directory which can be loaded and queried.
-
 
 ## Relevant Resources for Reference
 ### Tools

access_keys.json

@@ -0,0 +1 @@
+{"openai_api_key": "", "openai_org_id": "", "hf_token": "", "hf_token_write": ""}

misc/eval_pipeline.png assets/eval_pipeline.png

@@ -198,8 +198,6 @@ def check_db_exists(db_path):
     "--force", is_flag=True, help="Force overwrite if database already exists"
 )
 def main(data_path, pred_path, db_name, force):
-    set_env_vars()
-
     ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), ".."))
     tables_dir = os.path.join(ROOT, DEFAULT_TABLES_DIR)
     os.makedirs(tables_dir, exist_ok=True)

misc/graph-prev.mp4 assets/graph-prev.mp4

@@ -281,7 +281,6 @@ def submit_canned_query(query_description, limit, wrap):
     )
 
 if __name__ == "__main__":
-    set_env_vars()
     demo.launch(share=True)
 
 demo.launch()

misc/graph.mp4 assets/graph.mp4

@@ -94,7 +94,7 @@ def save_best_config(metrics, config):
         json.dump(best_config, f, indent=4)
 
 
-def set_env_vars(fname="../access_keys.json"):
+def set_env_vars(fname="access_keys.json"):
     with open(fname) as f:
         keys = json.load(f)
         for key in keys: