Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Explicit config #2294

Merged
merged 150 commits into from
Jun 17, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
150 commits
Select commit Hold shift + click to select a range
14125a3
upgrade pydantic
fangchenli Aug 22, 2023
48f26ba
run bump-pydantic
fangchenli Aug 23, 2023
b57c75f
uncomment Werror
fangchenli Aug 23, 2023
250be86
Merge branch 'develop' into pydantic-v2
fangchenli Aug 23, 2023
7912e37
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Aug 23, 2023
4eb1e08
Merge branch 'develop' into pydantic-v2
fangchenli Aug 23, 2023
553d021
update dependency in pyproject
fangchenli Aug 23, 2023
9febc8a
Merge branch 'develop' into pydantic-v2
fangchenli Aug 23, 2023
8e33010
Merge remote-tracking branch 'upstream/develop' into pydantic-v2
fangchenli Aug 23, 2023
99cb520
Merge remote-tracking branch 'origin/pydantic-v2' into pydantic-v2
fangchenli Aug 23, 2023
8fb92ff
fix typo
fangchenli Aug 23, 2023
0967d52
fix cpu_guarantee type
fangchenli Aug 23, 2023
1692797
fix typo
fangchenli Aug 23, 2023
bd76c4d
Merge branch 'develop' into pydantic-v2
fangchenli Aug 24, 2023
708d165
Merge remote-tracking branch 'upstream/develop' into pydantic-v2
fangchenli Aug 24, 2023
2f99cac
Merge remote-tracking branch 'origin/pydantic-v2' into pydantic-v2
fangchenli Aug 24, 2023
82ec511
fix more validation errors
fangchenli Aug 24, 2023
aba88ec
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Aug 24, 2023
3e645b4
fix more validator errors
fangchenli Aug 24, 2023
ff0682e
Merge remote-tracking branch 'origin/pydantic-v2' into pydantic-v2
fangchenli Aug 24, 2023
eaab189
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Aug 24, 2023
fb90cc6
Merge remote-tracking branch 'upstream/develop' into pydantic-v2
fangchenli Aug 24, 2023
b085e49
resolve conflict
fangchenli Aug 24, 2023
e520dcc
resolve conflict
fangchenli Aug 24, 2023
91fb5b4
resolve conflict
fangchenli Aug 24, 2023
5d0fca4
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Aug 24, 2023
2935c1f
fix monkeypatch
fangchenli Aug 25, 2023
961a278
revert printout
fangchenli Aug 25, 2023
f725534
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Aug 25, 2023
c543bdd
fix validation error
fangchenli Aug 25, 2023
6b98638
set none
fangchenli Aug 25, 2023
2f3bbae
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Aug 25, 2023
ef8dfb4
revert change
fangchenli Aug 26, 2023
a9d8d71
Merge branch 'develop' into pydantic-v2
fangchenli Aug 29, 2023
e920e5b
rebase
fangchenli Aug 29, 2023
19af132
fix cli error test
fangchenli Aug 29, 2023
819abe9
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Aug 29, 2023
d0c07e8
Merge branch 'develop' into pydantic-v2
fangchenli Aug 30, 2023
f3c518e
Merge remote-tracking branch 'upstream/develop' into pydantic-v2
fangchenli Aug 30, 2023
bfba588
Merge remote-tracking branch 'origin/pydantic-v2' into pydantic-v2
fangchenli Aug 30, 2023
70a5803
Merge branch 'develop' into pydantic-v2
fangchenli Sep 5, 2023
afaf06a
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Sep 5, 2023
f6c8468
Merge branch 'develop' into pydantic-v2
fangchenli Sep 11, 2023
eb5afa7
resolve conflict
fangchenli Sep 11, 2023
292087a
resolve conflict
fangchenli Sep 11, 2023
ec2417c
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Sep 11, 2023
f4aeef2
Merge branch 'develop' into pydantic-v2
fangchenli Sep 11, 2023
41699ea
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Sep 11, 2023
27ca9ff
Merge branch 'develop' into pydantic-v2
fangchenli Sep 11, 2023
70c8f6c
Merge branch 'develop' into pydantic-v2
fangchenli Sep 24, 2023
7b695f0
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Sep 24, 2023
dbf5157
resolve conflict
fangchenli Sep 25, 2023
ac0b6ae
resolve conflict
fangchenli Sep 25, 2023
a770d2a
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Sep 25, 2023
5e57a3a
change varible name
fangchenli Sep 25, 2023
7481469
refactor model validation
fangchenli Sep 25, 2023
bc3f5f6
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Sep 25, 2023
4ff3236
Merge branch 'develop' into pydantic-v2
fangchenli Oct 9, 2023
22bef83
Merge branch 'develop' into pydantic-v2
fangchenli Oct 27, 2023
e41f3a7
resolve conflict, uddate pydantic
fangchenli Oct 29, 2023
2d0ee62
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Oct 29, 2023
7d42def
resolve conflict
fangchenli Oct 29, 2023
2f6cb7f
update
fangchenli Oct 30, 2023
bd50f0b
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Oct 30, 2023
a30760a
revert comment
fangchenli Oct 30, 2023
ba53843
update
fangchenli Nov 2, 2023
6532f6a
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Nov 2, 2023
8949cfe
update
fangchenli Nov 4, 2023
64d5943
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Nov 4, 2023
6c166cd
fix name
fangchenli Nov 4, 2023
acc7ebd
revert change
fangchenli Nov 4, 2023
4dfd46c
debug
fangchenli Nov 4, 2023
f3775d7
Merge remote-tracking branch 'upstream/develop' into pydantic-v2
fangchenli Nov 4, 2023
842de7b
update
fangchenli Nov 5, 2023
e4b458c
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Nov 5, 2023
a443deb
Merge branch 'develop' into pydantic-v2
fangchenli Nov 8, 2023
69ea483
resolve conflict
fangchenli Nov 8, 2023
bc79fd6
unskip test
fangchenli Nov 8, 2023
2da0b89
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Nov 8, 2023
ed1329d
uncomment
fangchenli Nov 9, 2023
9f648c0
Merge remote-tracking branch 'origin/pydantic-v2' into pydantic-v2
fangchenli Nov 9, 2023
8236673
remove fixture typing
fangchenli Nov 9, 2023
ae7d918
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Nov 9, 2023
b141ff3
resolve confilct
fangchenli Nov 9, 2023
deada6e
Merge remote-tracking branch 'origin/pydantic-v2' into pydantic-v2
fangchenli Nov 9, 2023
b3b5268
avoid import typing
fangchenli Nov 9, 2023
3831b51
use fixture for cli
fangchenli Nov 9, 2023
b77a59a
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Nov 9, 2023
f14529a
debug conda build
fangchenli Nov 9, 2023
3e46486
Merge remote-tracking branch 'upstream/develop' into pydantic-v2
fangchenli Nov 11, 2023
33fde03
fix typing import in init
fangchenli Nov 11, 2023
ec07870
Merge remote-tracking branch 'upstream/develop' into pydantic-v2
fangchenli Nov 13, 2023
5c50185
refactor env variable check
fangchenli Nov 13, 2023
47b86eb
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Nov 13, 2023
5cef9ce
Merge branch 'develop' into pydantic-v2
fangchenli Dec 26, 2023
7c5b5cc
Merge branch 'develop' into pydantic-v2
aktech Feb 14, 2024
d74d69d
render all config before writing
Adam-D-Lewis Mar 6, 2024
44c9f71
update
Adam-D-Lewis Mar 6, 2024
12f5288
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Mar 6, 2024
88d8bc4
split certs into different pydantic models
Adam-D-Lewis Mar 7, 2024
bbdf6f2
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Mar 7, 2024
b2891d1
allow each InputSchema to exclude certain parts of itself from the co…
Adam-D-Lewis Mar 9, 2024
0e4d7fd
allow each InputSchema to exclude certain parts of itself from the co…
Adam-D-Lewis Mar 9, 2024
845becb
import future annotations
Adam-D-Lewis Mar 11, 2024
b01446e
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Mar 11, 2024
0df2e64
switch typing annotation to fix tests
Adam-D-Lewis Mar 11, 2024
33f4712
switch typing annotation to fix tests
Adam-D-Lewis Mar 11, 2024
4a80f23
Merge branch 'develop' into explicit_config
viniciusdc Mar 11, 2024
4d824be
fix tests
Adam-D-Lewis Mar 12, 2024
6e330e5
[pre-commit.ci] Apply automatic pre-commit fixes
pre-commit-ci[bot] Mar 12, 2024
88b46a8
Merge branch 'develop' into pydantic2
Adam-D-Lewis Mar 19, 2024
924659d
merge develop
Adam-D-Lewis Mar 19, 2024
75d8e70
small fixes
Adam-D-Lewis Mar 19, 2024
a8bab5c
Merge branch 'develop' into pydantic2
Adam-D-Lewis Mar 27, 2024
35252ef
fix arg of classmethod
Adam-D-Lewis Mar 29, 2024
436dab7
fix req'd vars
Adam-D-Lewis Mar 29, 2024
283150f
fix availability zones
Adam-D-Lewis Mar 29, 2024
b2dbbd9
undo signature change
Adam-D-Lewis Mar 29, 2024
082cc41
fix fn call
Adam-D-Lewis Mar 29, 2024
3d6726f
remove unused var in fn signature
Adam-D-Lewis Mar 29, 2024
7e5891f
update cpu_limit dtype
Adam-D-Lewis Apr 1, 2024
32ee7ba
update cpu_limit dtype
Adam-D-Lewis Apr 1, 2024
e1030d8
revert test changes unrelated to pydantic v2
Adam-D-Lewis Apr 3, 2024
850de95
update for pydantic2
Adam-D-Lewis Apr 3, 2024
6b2b629
fix tests
Adam-D-Lewis Apr 3, 2024
473a1ee
Merge branch 'develop' into pydantic2
Adam-D-Lewis Apr 4, 2024
9d9fd49
replace .dict( with .model_dump(
Adam-D-Lewis Apr 4, 2024
c4b8c6e
Merge branch 'develop' into pydantic2
Adam-D-Lewis Apr 4, 2024
89e73aa
Merge branch 'develop' into pydantic2
Adam-D-Lewis Apr 10, 2024
680ce25
merge with pydantic2
Adam-D-Lewis Apr 11, 2024
c8feabc
reverse base class order
Adam-D-Lewis Apr 15, 2024
52b4ea8
merge with develop
Adam-D-Lewis Apr 15, 2024
2b38f46
make fields optional
Adam-D-Lewis Apr 15, 2024
a46cf1d
add default values
Adam-D-Lewis Apr 15, 2024
d4f301d
merge develop
Adam-D-Lewis May 6, 2024
8881d7c
merge develop
Adam-D-Lewis May 6, 2024
b88aa1b
remove validate default
Adam-D-Lewis May 6, 2024
992ae28
make verbose not the default
Adam-D-Lewis May 6, 2024
e904428
fix tests
Adam-D-Lewis May 6, 2024
7c90e58
Merge branch 'develop' into explicit_config
Adam-D-Lewis Jun 4, 2024
f4508cb
convert verbose from bool to count
Adam-D-Lewis Jun 4, 2024
e771897
revert cert changes
Adam-D-Lewis Jun 4, 2024
ef6caad
revert exclude from config
Adam-D-Lewis Jun 4, 2024
1ec6087
update default value on verbose cli arg
Adam-D-Lewis Jun 4, 2024
f39e3aa
readd a bit from exclude_from_config
Adam-D-Lewis Jun 4, 2024
aa7426f
add advanced users only warning
Adam-D-Lewis Jun 4, 2024
1d60cda
Merge branch 'develop' into explicit_config
Adam-D-Lewis Jun 13, 2024
31cf77b
change verbose to explicit
Adam-D-Lewis Jun 13, 2024
28bc17f
test explicit config
Adam-D-Lewis Jun 13, 2024
aaa7682
fix comment
Adam-D-Lewis Jun 13, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion src/_nebari/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -103,7 +103,8 @@ def write_configuration(
"""Write the nebari configuration file to disk"""
with config_filename.open(mode) as f:
if isinstance(config, pydantic.BaseModel):
yaml.dump(config.model_dump(), f)
config_dict = config.model_dump()
yaml.dump(config_dict, f)
else:
config = dump_nested_model(config)
yaml.dump(config, f)
Expand Down
19 changes: 18 additions & 1 deletion src/_nebari/subcommands/init.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ class InitInputs(schema.Base):
ssl_cert_email: Optional[schema.email_pydantic] = None
disable_prompt: bool = False
output: pathlib.Path = pathlib.Path("nebari-config.yaml")
explicit: int = 0


def enum_to_list(enum_cls):
Expand Down Expand Up @@ -152,7 +153,7 @@ def handle_init(inputs: InitInputs, config_schema: BaseModel):
try:
write_configuration(
inputs.output,
config,
config if not inputs.explicit else config_schema(**config),
mode="x",
)
except FileExistsError:
Expand Down Expand Up @@ -565,6 +566,13 @@ def init(
"-o",
help="Output file path for the rendered config file.",
),
explicit: int = typer.Option(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Adam-D-Lewis is there any reason to have int over bool here? I know either of those will work but I'm wondering if there is any advantage.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making it an int allows you to specify --explicit --explicit or -ee to have levels of explicitness. We aren't using the levels yet, but may in the future.

0,
"--explicit",
"-e",
count=True,
help="Write explicit nebari config file (advanced users only).",
),
):
"""
Create and initialize your [purple]nebari-config.yaml[/purple] file.
Expand Down Expand Up @@ -604,6 +612,7 @@ def init(
inputs.ssl_cert_email = ssl_cert_email
inputs.disable_prompt = disable_prompt
inputs.output = output
inputs.explicit = explicit

from nebari.plugins import nebari_plugin_manager

Expand Down Expand Up @@ -894,6 +903,14 @@ def guided_init_wizard(ctx: typer.Context, guided_init: str):
)
inputs.kubernetes_version = kubernetes_version

# EXPLICIT CONFIG
inputs.explicit = questionary.confirm(
"Would you like the nebari config to show all available options? (recommended for advanced users only)",
default=False,
qmark=qmark,
auto_enter=False,
).unsafe_ask()

from nebari.plugins import nebari_plugin_manager

config_schema = nebari_plugin_manager.config_schema
Expand Down
2 changes: 1 addition & 1 deletion src/nebari/plugins.py
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ def config_schema(self):
classes = [schema.Main] + [
_.input_schema for _ in self.ordered_stages if _.input_schema is not None
]
return type("ConfigSchema", tuple(classes), {})
return type("ConfigSchema", tuple(classes[::-1]), {})


nebari_plugin_manager = NebariPluginManager()
4 changes: 3 additions & 1 deletion src/nebari/schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@

class Base(pydantic.BaseModel):
model_config = ConfigDict(
extra="forbid", validate_assignment=True, populate_by_name=True
extra="forbid",
validate_assignment=True,
populate_by_name=True,
)


Expand Down
34 changes: 21 additions & 13 deletions tests/tests_unit/test_cli_init.py
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@
(["--ssl-cert-email"], 2, ["requires an argument"]),
(["--output"], 2, ["requires an argument"]),
(["-o"], 2, ["requires an argument"]),
(["--explicit"], 2, ["Missing option"]),
(["-e"], 2, ["Missing option"]),
],
)
def test_cli_init_stdout(args: List[str], exit_code: int, content: List[str]):
Expand Down Expand Up @@ -90,20 +92,22 @@ def generate_test_data_test_cli_init_happy_path():
) in get_kubernetes_versions(provider) + [
"latest"
]:
test_data.append(
(
provider,
region,
project_name,
domain_name,
namespace,
auth_provider,
ci_provider,
terraform_state,
email,
kubernetes_version,
for explicit in [True, False]:
test_data.append(
(
provider,
region,
project_name,
domain_name,
namespace,
auth_provider,
ci_provider,
terraform_state,
email,
kubernetes_version,
explicit,
)
)
)

keys = [
"provider",
Expand All @@ -116,6 +120,7 @@ def generate_test_data_test_cli_init_happy_path():
"terraform_state",
"email",
"kubernetes_version",
"explicit",
]
return {"keys": keys, "test_data": test_data}

Expand All @@ -131,6 +136,7 @@ def test_cli_init_happy_path(
terraform_state: str,
email: str,
kubernetes_version: str,
explicit: bool,
):
app = create_cli()
args = [
Expand Down Expand Up @@ -159,6 +165,8 @@ def test_cli_init_happy_path(
"--region",
region,
]
if explicit:
args += ["--explicit"]

expected_yaml = f"""
provider: {provider}
Expand Down
Loading