Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pydantic config #189

Merged
merged 53 commits into from
Sep 19, 2024
Merged
Changes from 1 commit
Commits
Show all changes
53 commits
Select commit Hold shift + click to select a range
362b8d5
config is now backed by pydantic (WIP)
SecretiveShell Sep 5, 2024
36e991c
automate arg parse
SecretiveShell Sep 5, 2024
8e93446
patch pydantic config into old config
SecretiveShell Sep 6, 2024
420fd84
add env var loading automation
SecretiveShell Sep 6, 2024
e8fcecd
Merge remote-tracking branch 'upstream/main' into HEAD
SecretiveShell Sep 11, 2024
0d74591
fix arg parser for dict types
SecretiveShell Sep 11, 2024
c6f9806
remove unused imports
SecretiveShell Sep 11, 2024
05f1c3e
fix line lengths
SecretiveShell Sep 11, 2024
8b48f00
fix model names
SecretiveShell Sep 12, 2024
e11d80b
fix missing rename
SecretiveShell Sep 12, 2024
eb5f42c
add error message for invalid use_as_default
SecretiveShell Sep 12, 2024
6e935c5
remove private attributes in args
SecretiveShell Sep 12, 2024
21747bf
Args: Switch to use model_field for everything
bdashore3 Sep 13, 2024
d5b3fde
Config: Fix descriptions
bdashore3 Sep 13, 2024
dc4946b
make pydantic do all the validation
SecretiveShell Sep 13, 2024
533e7c9
remove unnecessary code
SecretiveShell Sep 14, 2024
0903f85
add export openAPI to config
SecretiveShell Sep 14, 2024
a09dd80
Config: Cleanup and organize functions
bdashore3 Sep 15, 2024
6f28cfe
Logging: Remove preferences global
bdashore3 Sep 15, 2024
d013729
Config: Add aliases for logging config
bdashore3 Sep 15, 2024
5bfa952
Actions: Format
bdashore3 Sep 15, 2024
f05229b
Merge branch 'main' into pydantic-config
bdashore3 Sep 15, 2024
92af656
improve config generation action
SecretiveShell Sep 15, 2024
250d76f
Config: Alter YAML generator function
bdashore3 Sep 16, 2024
8ff9f2c
Config: Rewrite docstrings for models
bdashore3 Sep 16, 2024
4c8bb42
Config: Reorder models
bdashore3 Sep 16, 2024
3340c3b
Config: Rewrite descriptions
bdashore3 Sep 16, 2024
b6dd21f
Config: Handle default factories in config generation
bdashore3 Sep 16, 2024
564bdcf
add legacy config converter
SecretiveShell Sep 16, 2024
7f03003
rephrase info message
SecretiveShell Sep 16, 2024
81ae461
Config: Allow existing values to get included in generated file
bdashore3 Sep 16, 2024
c715094
Config: Add logging config to migration checks
bdashore3 Sep 16, 2024
e60c4ba
Config: Fix existing value check
bdashore3 Sep 16, 2024
ebe7f35
Config: Alter migration error handling and cleanup
bdashore3 Sep 16, 2024
d2d07ed
Config: Update auto-migration flow
bdashore3 Sep 16, 2024
46f9fff
Config: Move config file generation to tabby_config
bdashore3 Sep 17, 2024
06a798d
Main: Remove debug print statement for config object
bdashore3 Sep 17, 2024
26ad0ef
API: Fix model info reporting
bdashore3 Sep 17, 2024
8e6b8bd
Update .gitignore
bdashore3 Sep 17, 2024
f6fb60a
Config: Inline model loading is False
bdashore3 Sep 17, 2024
ececce1
Config: Fix addition of preamble
bdashore3 Sep 17, 2024
852ea8f
Config: Don't load from file if actions present
bdashore3 Sep 17, 2024
63f8c46
Config: Make a better description for lora config
bdashore3 Sep 17, 2024
7fe0dbd
Tree: Update config_sample
bdashore3 Sep 17, 2024
daa57ce
API: Upgrade config declarations
bdashore3 Sep 17, 2024
bb4dd72
fix defaults for api_servers
SecretiveShell Sep 17, 2024
948fcb7
migrate to ruamel.yaml
SecretiveShell Sep 18, 2024
a34bd9a
Config: Alter YAML generation script for formatting adherence
bdashore3 Sep 18, 2024
754fb15
Config: Fix draft model migration and loading
bdashore3 Sep 18, 2024
63634be
Config: Clarify Rope alpha options
bdashore3 Sep 18, 2024
6c7542d
migrate all yaml loaders to ruamel.yaml
SecretiveShell Sep 18, 2024
24ea85b
Tree: Use safe loader for YAML
bdashore3 Sep 18, 2024
4cf8551
Tree: Format
bdashore3 Sep 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
remove private attributes in args
  • Loading branch information
SecretiveShell committed Sep 12, 2024
commit 6e935c565e642bbeabe8e776a1cb277e314a3909
28 changes: 18 additions & 10 deletions common/args.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
"""Argparser for overriding config values"""

import argparse
from typing import get_origin, get_args, Union, List
from common.tabby_config import config
from typing import Any, Type, get_origin, get_args, Union, List
from inspect import get_annotations, isclass

from pydantic import BaseModel

from common.config_models import TabbyConfigModel


def str_to_bool(value):
Expand Down Expand Up @@ -33,7 +37,7 @@ def argument_with_auto(value):
) from ex


def map_pydantic_type_to_argparse(pydantic_type):
def map_pydantic_type_to_argparse(pydantic_type: Any):
"""
Maps Pydantic types to argparse compatible types.
Handles special cases like Union and List.
Expand All @@ -57,7 +61,7 @@ def map_pydantic_type_to_argparse(pydantic_type):
return str


def add_field_to_group(group, field_name, field_type, field):
def add_field_to_group(group, field_name, field_type, field) -> None:
"""
Adds a Pydantic field to an argparse argument group.
"""
Expand All @@ -67,22 +71,24 @@ def add_field_to_group(group, field_name, field_type, field):
group.add_argument(f"--{field_name}", type=arg_type, help=help_text)


def init_argparser():
def init_argparser() -> argparse.ArgumentParser:
"""
Initializes an argparse parser based on a Pydantic config schema.
"""
parser = argparse.ArgumentParser(description="TabbyAPI server")

field_type: Union[Type[BaseModel], Any]

# Loop through each top-level field in the config
for field_name, field_type in config.__annotations__.items():
for field_name, field_type in get_annotations(TabbyConfigModel).items():
group = parser.add_argument_group(
field_name, description=f"Arguments for {field_name}"
)

# Check if the field_type is a Pydantic model
if hasattr(field_type, "__annotations__"):
for sub_field_name, sub_field_type in field_type.__annotations__.items():
field = field_type.__fields__[sub_field_name]
if isclass(field_type):
for sub_field_name, sub_field_type in get_annotations(field_type).items():
field = field_type.model_fields[sub_field_name]
add_field_to_group(group, sub_field_name, sub_field_type, field)
else:
# Handle cases where the field_type is not a Pydantic mode
Expand All @@ -94,7 +100,9 @@ def init_argparser():
return parser


def convert_args_to_dict(args: argparse.Namespace, parser: argparse.ArgumentParser):
def convert_args_to_dict(
args: argparse.Namespace, parser: argparse.ArgumentParser
) -> dict[str, dict[str, Any]]:
"""Broad conversion of surface level arg groups to dictionaries"""

arg_groups = {}
Expand Down
Loading