Merge branch 'main' into function-debug

merging from main

Author: Majdoddin

.github/workflows/ci.yml

@@ -0,0 +1,30 @@
+name: docs-ci 
+on:
+  push:
+    branches:
+      - main
+      - add-docs-portal
+permissions:
+  contents: write
+jobs:
+  deploy:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - name: Configure Git Credentials
+        run: |
+          git config user.name github-actions[bot]
+          git config user.email 41898282+github-actions[bot]@users.noreply.github.com
+      - uses: actions/setup-python@v5
+        with:
+          python-version: 3.x
+      - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV 
+      - uses: actions/cache@v4
+        with:
+          key: mkdocs-material-${{ env.cache_id }}
+          path: .cache
+          restore-keys: |
+            mkdocs-material-
+      - run: pip install mkdocs-material 
+      - run: pip install "mkdocs-material[imaging]"
+      - run: mkdocs gh-deploy -f docs/mkdocs.yml --force

.gitignore

@@ -9,3 +9,4 @@ valid_and_test_processed/
 
 # Python cache files
 **/__pycache__/
+docs/.cache/

README.md

@@ -57,7 +57,7 @@ key = "..."
 The file: `alpha_codium/settings/configuration.toml` contains the configuration for the project.
 In the `config` section you can choose the model you want to use ("gpt-4", "gpt-3.5-turbo-16k", or others).
 
-### Solving a specific problem
+### Solving a specific problem from CodeContest
 To solve a specific problem with AlphaCodium, from the root folder run:
 ```
 python -m alpha_codium.solve_problem \
@@ -82,7 +82,7 @@ Example problem (test set, problem number 12):
 </table>
 </p>
 
-### Solving the entire dataset
+### Solving an entire CodeContest dataset split
 to solve the entire dataset with AlphaCodium, from the root folder run:
 ```
 python -m alpha_codium.solve_dataset \
@@ -107,6 +107,24 @@ python -m alpha_codium.evaluate_dataset\
 --database_solution_path /path/to/output/dir/dataset_output.json
 ```
 
+### Solving a new problem (CodeContest format)
+To solve a custom problem with AlphaCodium, first create a json file that includes the CodeContest problem fields, and then from the root folder run:
+```
+python -m alpha_codium.solve_my_problem \
+--my_problem_json_file /path/to/my_problem.json
+```
+- The `my_problem_json_file` is the path to to the custom problem json file.
+
+See the `my_problem_example.json` to see an example of a custom problem. The json file should include the following fields:
+- `name` is the name of the problem.
+- `description` is a description of the problem.
+- (optional) `public_tests` with the following fields:
+  - `input` is a list of strings that represent the input.
+  - `output` is a list of strings that represent the output.
+- (optional) `private_tests`, that follows the same structure as `public_tests`
+- (optional) `generated_tests`, that follows the same structure as `public_tests`
+
+
 ## Technical Q&A
 Aggregating some technical questions we received about this project:
 ___

alpha_codium/code_contests/eval/pass_at_k_evaluator.py

@@ -3,7 +3,7 @@
 from evaluate import load as load_metric
 
 from alpha_codium.code_contests.data.provider import CodeContestDataProvider
-from alpha_codium.config_loader import get_settings
+from alpha_codium.settings.config_loader import get_settings
 
 
 def calculate_metrics(ds, k_values=[1, 10, 100]):  # noqa: B006

alpha_codium/gen/coding_competitor.py

@@ -43,12 +43,15 @@ def render(self, problem_json, prompt: str):
         if hasattr(self.prompt[prompt], 'frequency_penalty'):
             frequency_penalty = self.prompt[prompt].frequency_penalty
         else:
-            frequency_penalty = 0
+            frequency_penalty = None
         return sys_prompt, usr_prompt, temperature, frequency_penalty
 
     async def _run(self, model, problem, prompt:str = "code_contests_prompt_reflect"):
         system_prompt, user_prompt, temperature, frequency_penalty = self.render(problem, prompt)
 
+        if frequency_penalty == None:
+            frequency_penalty = get_settings().get("config.frequency_penalty")
+
         response, finish_reason = await self.ai_handler.chat_completion(
             model=model, system=system_prompt, user=user_prompt,
             temperature=temperature, frequency_penalty=frequency_penalty,
@@ -107,7 +110,6 @@ def solve_problem(dataset_name,
                   problem_number=0):
 
     # load dataset
-    base_path = os.getcwd()
     logger = get_logger(__name__)
     data_provider = CodeContestDataProvider(dataset_location=dataset_name)
     if problem_number and problem_name:
@@ -156,6 +158,14 @@ def solve_problem(dataset_name,
             pass
 
 
+    return solve_my_problem(problem)
+
+
+def solve_my_problem(problem):
+
+    base_path = os.getcwd()
+    logger = get_logger(__name__)
+
     solver = CodeContestsCompetitor()
     os.chdir(base_path)
     solution = solver.solve_problem_in_dataset(problem)

alpha_codium/llm/token_handler.py

@@ -1,7 +1,7 @@
 from jinja2 import Environment, StrictUndefined
 from tiktoken import encoding_for_model, get_encoding
 
-from alpha_codium.config_loader import get_settings
+from alpha_codium.settings.config_loader import get_settings
 
 
 def get_token_encoder():

alpha_codium/settings/choose_best_solution_direct.toml

@@ -1,6 +1,5 @@
 [code_contests_prompts_choose_best_solution_direct]
 temperature = 0.3
-frequency_penalty = 0.1
 system = """\
 To solve the problem, utilize an exhaustive high-computational approach like simulation, recursion, brute-force or other direct approach. Ignore the problem constraints regarding large input size.
 """

alpha_codium/settings/code_contests_prompt_analyze_and_fix.toml

@@ -1,6 +1,5 @@
 [code_contests_prompt_analyze_and_fix]
 temperature = 0.2
-frequency_penalty = 0.1
 system = """\
 - You must divide the fixed code into small sub-functions, with meaningful names and functionality. Each function should be no longer than 10 lines of code.
 - The fixed code should be robust and general, and work for other input examples as well.

alpha_codium/settings/code_contests_prompt_analyze_and_fix_direct.toml

@@ -1,6 +1,5 @@
 [code_contests_prompt_analyze_and_fix_direct]
 temperature = 0.2
-frequency_penalty = 0.1
 system = """\
 - You must divide the new code into small sub-functions, with meaningful names and functionality. Variable names should also be meaningful.
 - The new code still needs to utilize an exhaustive approach like simulation, recursion, brute-force or direct solution. Ignore the problem constraints regarding large input size.

alpha_codium/settings/code_contests_prompt_analyze_failure.toml

@@ -1,6 +1,5 @@
 [code_contests_prompt_analyze_failure]
 temperature = 0.3
-frequency_penalty = 0.1
 system = """\
 """
 user="""\

alpha_codium/settings/code_contests_prompts_baseline.toml

@@ -1,6 +1,5 @@
 [code_contests_prompts_baseline]
 temperature = 0.3
-frequency_penalty = 0.1
 system= """\
 """
 user="""

alpha_codium/settings/code_contests_prompts_choose_best_solution.toml

@@ -1,6 +1,5 @@
 [code_contests_prompts_choose_best_solution]
 temperature = 0.2
-frequency_penalty = 0.1
 system = """\
 - As a seasoned programmer, choose the best solution.
 - Output nothing other that the YAML.

alpha_codium/settings/code_contests_prompts_fix_solution.toml

@@ -1,6 +1,5 @@
 [code_contests_prompt_fix_solution]
 temperature = 0.3
-frequency_penalty = 0.1
 system="""\
 - You must divide the fixed code into small sub-functions, with meaningful names and functionality. Each function should be no longer than 10 lines of code.
 - The fixed code should be robust and general, and work for other input examples as well.

alpha_codium/settings/code_contests_prompts_generate_ai_tests.toml

@@ -1,6 +1,5 @@
 [code_contests_prompts_generate_ai_tests]
 temperature = 0.2
-frequency_penalty = 0.1
 system = """\
 """
 

alpha_codium/settings/code_contests_prompts_generate_possible_solutions.toml

@@ -1,6 +1,5 @@
 [code_contests_prompt_generate_possible_solutions]
 temperature = 0.3
-frequency_penalty = 0.1
 system= """\
 Pay attention to small details and nuances in the problem description.
 """

alpha_codium/settings/code_contests_prompts_reflect.toml

@@ -1,6 +1,5 @@
 [code_contests_prompt_reflect]
 temperature = 0.2
-frequency_penalty = 0.1
 system= """\
 The self-reflection must cover every aspect of the problem. Pay attention to small details and nuances in the problem description.
 """

alpha_codium/settings/code_contests_prompts_solve.toml

@@ -1,6 +1,5 @@
 [code_contests_prompts_solve]
 temperature = 0.3
-frequency_penalty = 0.1
 system= """\
 - You must divide the generated code into small sub-functions, with meaningful names and functionality. Each function should be no longer than 10 lines of code.
 - Double-check the solution code. The generated solution must generalize to any valid input, and not just the provided examples.

alpha_codium/settings/code_contests_prompts_solve_direct.toml

@@ -1,6 +1,5 @@
 [code_contests_prompts_solve_direct]
 temperature = 0.3
-frequency_penalty = 0.1
 system= """\
 - You must divide the generated code into small sub-functions, with meaningful names and functionality. Each function should be no longer than 10 lines of code.
 - The code should ignore the problem constraints for large inputs.

alpha_codium/settings/code_contests_prompts_validate_ai_tests.toml

@@ -1,6 +1,5 @@
 [code_contests_prompts_validate_ai_tests]
 temperature = 0.2
-frequency_penalty = 0.1
 system = """\
 Your goal is to consider each AI-generated test, and make sure its output and its explanation are correct. Be critical - they could be wrong.
 guidelines:

alpha_codium/settings/code_contests_prompts_validate_reflection.toml

@@ -1,6 +1,5 @@
 [code_contests_prompts_validate_reflection]
 temperature = 0.2
-frequency_penalty = 0.1
 system = """\
 """
 

alpha_codium/settings/configuration.toml

@@ -2,13 +2,13 @@
 model="gpt-4-0125-preview"
 #model="gpt-4-0613"
 #model = "gpt-3.5-turbo-16k"
+frequency_penalty=0.1
 ai_timeout=90 # seconds
 fallback_models =[]
 verbosity_level=2 # 0,1,2
 private_dataset_cache_dir="~/ai/alphacodium"
 max_requests_per_minute=60
 
-
 [dataset]
 evaluate_prev_solutions=false
 num_iterations=1 # X iterations to try to solve the problem

alpha_codium/solve_my_problem.py

@@ -0,0 +1,16 @@
+import argparse
+import json
+
+from alpha_codium.gen.coding_competitor import solve_problem, solve_my_problem
+from alpha_codium.log import setup_logger
+from alpha_codium.settings.config_loader import get_settings
+
+parser = argparse.ArgumentParser()
+parser.add_argument("--my_problem_json_file", type=str, default="my_problem_example.json")
+
+if __name__ == "__main__":
+    args = parser.parse_args()
+    setup_logger()
+    
+    with open(args.my_problem_json_file, "r") as my_problem:
+        solve_my_problem(json.load(my_problem))

docs/docs/CNAME

@@ -0,0 +1 @@
+alpha-codium-docs.codium.ai

docs/docs/assets/favicon.ico

@@ -0,0 +1,15 @@
+:root {
+    --md-primary-fg-color: #765bfa;
+    --md-accent-fg-color: #AEA1F1;
+  }
+.md-nav__title, .md-nav__link  {
+    font-size: 16px; /* Adjust the font size as needed */
+}
+
+.md-tabs__link  {
+    font-size: 16px; /* Adjust the font size as needed */
+}
+
+.md-header__title {
+    font-size: 20px; /* Adjust the font size as needed */
+}