Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to pydantic #214

Merged
merged 99 commits into from
Oct 8, 2024
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
99 commits
Select commit Hold shift + click to select a range
368a6aa
Initial pydantic testing.
terjekv Jan 29, 2024
faeb23c
Implement `host find`.
terjekv Apr 9, 2024
9718859
Clean up authentication mess. (#206)
terjekv Jan 30, 2024
51528cd
Fix `host find -comment` metavar (#207)
pederhan Feb 7, 2024
8d0b673
Allow oneshot commands from the terminal. (#208)
terjekv Feb 12, 2024
6c66d5f
Improve connection errors. (#209)
terjekv Feb 19, 2024
1964597
Fix a bug where not having any default URL breaks application start. …
terjekv Feb 20, 2024
6215a6c
Use corrolation_id for logging across commands. (#213)
terjekv Feb 21, 2024
3838470
Make `host remove` a bit safer to use. (#211)
terjekv Feb 29, 2024
683379a
Merge branch 'master' into migrate_to_pydantic
terjekv Apr 9, 2024
5457b3d
Add Endpoint enum, migrate add_user, refactor MacAddress.
terjekv Apr 9, 2024
28c696d
Make api.get_host generic.
terjekv Apr 15, 2024
181e811
Refactoring for readability.
terjekv Apr 15, 2024
100fdc8
Remove redudant docstring.
terjekv Apr 15, 2024
ad8a341
Rename HostModel to Host.
terjekv Apr 15, 2024
071e69f
Refactoring and cleaning up host processing.
terjekv Apr 17, 2024
ce243bd
Typing, Zones and some more RRs.
terjekv Apr 18, 2024
dd6f21c
Use class methods for lookups, object methods for API operations.
terjekv Apr 20, 2024
d6708f6
Fix host comparison.
terjekv Apr 20, 2024
684ace3
Type fixes.
terjekv Apr 20, 2024
0cc061d
Move knowledge of endpoint ID mapping peculiarities to Endpoints only…
terjekv Apr 20, 2024
a66a256
Allow for manually setting ip or pass network to host add.
terjekv Apr 22, 2024
ccd6f6c
Look up field names by alias
pederhan Apr 22, 2024
b8fbce5
Use dict.get() with no default
pederhan Apr 22, 2024
eb37282
Make pylance happy with explicit type
terjekv Apr 22, 2024
f0eb55c
Split the models.py into smaller files.
terjekv Apr 23, 2024
2f2cd6c
Add a run script for pyinstaller.
terjekv Apr 23, 2024
001b980
Either raise an exception *or* use cli_{warning,error}.
terjekv Apr 23, 2024
1cf92ae
Fix host add (again).
terjekv Apr 23, 2024
c1eee08
Upgrade models.py to Pydantic V2 semantics (#215)
pederhan Apr 23, 2024
1066e98
Migrate to python 3.11.
terjekv Apr 23, 2024
0fc7df5
Require setuptools due to python 3.12+
terjekv Apr 23, 2024
b753df4
Require a newer version of setuptools.
terjekv Apr 23, 2024
af5d22e
Newer image, force upgrade of pip, and drop caches,
terjekv Apr 23, 2024
8c1ca58
Use tox to test pyinstaller for 3.11 and 3.12.
terjekv Apr 23, 2024
0af325a
Add `from __future__ import annotations` lint rule (#216)
pederhan Apr 24, 2024
3fb1b1a
Finish up the last bits of core host commands...
terjekv Apr 24, 2024
58edfe1
Add Host.get_by_any_means_or_raise and use it wisely. :)
terjekv Apr 24, 2024
8080aff
Acutally support multi-host lookups with info and fix bacnet display.
terjekv Apr 24, 2024
f95ff73
Add group lookup support to host info.
terjekv Apr 24, 2024
0c558b1
Refactor hostgroup parent parsing.
terjekv Apr 24, 2024
102c345
Refactor and add group output support.
terjekv Apr 25, 2024
4073a62
Refactor away the function-based API and migrated a/aaaa commands.
terjekv Apr 26, 2024
a1d4f6c
host cname commands migration.
terjekv Apr 27, 2024
3523122
Hinfo support.
terjekv Apr 27, 2024
9f892b6
Well, that was effin' awful.
terjekv Apr 27, 2024
9e73353
Add RR MX support.
terjekv Apr 29, 2024
13eae87
Add `get_typed()` utility function
pederhan May 2, 2024
aed9364
Add NAPTR and PTR support.
terjekv May 3, 2024
2914d45
Add `Atom` & `Label` models + base Host Policy model (#217)
pederhan May 3, 2024
be67d3e
Add SRV support.
terjekv May 3, 2024
fdf42cc
Finalize RR support.
terjekv May 3, 2024
38f7353
Add inline comments for `HostPolicy` validator
pederhan May 5, 2024
34f0bdc
Add `WithName` mixin (#218)
pederhan May 7, 2024
ad4e1b5
Better exception modelling. (#219)
terjekv May 8, 2024
71dddd1
Use `Self` type annotation for `APIMixin` (#222)
pederhan May 8, 2024
98ea140
dhcp commands and exception usage. (#223)
terjekv May 10, 2024
7123311
Uniform exception names (#225)
terjekv May 12, 2024
da3335a
Add policy commands (#220)
pederhan May 13, 2024
8d711f9
Pydantic group commands (#224)
terjekv May 13, 2024
b7c7370
Labels and some permission work. (#226)
terjekv May 14, 2024
b79a1b0
Permission commands migrated. (#228)
terjekv May 21, 2024
4877e83
Zone commands (#227)
pederhan May 21, 2024
dd51e64
Fix incorrect method used in `host sshfp_remove` (#233)
pederhan May 21, 2024
f08aaca
Support segmenting tokens per user and url. (#229)
terjekv May 21, 2024
d482bde
Set prompt to default to the server one is connected to. (#234)
terjekv May 21, 2024
4a6f262
Remove `use_json` (#235)
pederhan May 21, 2024
29d6d81
Fix/standardize some patch commands (#236)
pederhan May 22, 2024
cd156d1
Add `WithHistory` + fetching history for deleted objects (#237)
pederhan May 22, 2024
a2fb43b
Network commands (#238)
pederhan May 24, 2024
2de44b6
Add pagination support for `get_typed` (#240)
pederhan May 27, 2024
bf8c1d3
Define metadata and requirements in pyproject.toml (#239)
pederhan May 27, 2024
248a19e
Fix broken command(s) in CI (#243)
pederhan May 27, 2024
db17b84
Fix missing pyinstaller dev dependency (#246)
pederhan May 28, 2024
07cb471
Fix DHCP commands (#245)
pederhan May 28, 2024
7bbc3da
Add period when finding subzones (#248)
pederhan May 29, 2024
e1c333e
Validate GET responses with Pydantic (#241)
pederhan May 29, 2024
b5fac79
Fix typo
pederhan May 29, 2024
0c198c5
Strip None values when sending JSON (#249)
pederhan May 29, 2024
d234c36
Require SOCKS dependencies (#244)
pederhan May 30, 2024
4a35efe
Add `--version` option (#252)
pederhan May 30, 2024
b922b50
Show git commit in `--version` (#253)
pederhan May 31, 2024
02681f0
Copy policies from one host to another. (#257)
terjekv Jun 4, 2024
5dabdac
Support multiple destinations in host_copy. (#259)
terjekv Jun 6, 2024
54a3c09
Instantiate JSON mapping validator at top level (#260)
pederhan Jun 10, 2024
bb27ad1
Import host_submodules explicitly. (#262)
pederhan Jun 10, 2024
98cf819
Add publishing workflow and contribution instructions (#254)
pederhan Jun 12, 2024
7485ec1
Add authors to pyproject.toml (#263)
pederhan Jun 12, 2024
c5f46ca
Make testsuite work again (#255)
oyvindhagberg Sep 17, 2024
2729de6
Build Linux PyInstaller binary in CentOS 8 container (#293)
pederhan Sep 17, 2024
84aa6b8
Change publish tag pattern (#294)
pederhan Sep 17, 2024
0ba648c
Update contributing instructions
pederhan Sep 17, 2024
2853ce7
Add `getattr` default in `APIMixin.refetch` (#295)
pederhan Oct 2, 2024
b65ad63
Don't pass params for pagination requests (#296)
pederhan Oct 2, 2024
c36ad96
Reorder overloads (#297)
pederhan Oct 2, 2024
8d23297
Fix cli.py type checking errors (#299)
pederhan Oct 4, 2024
4e7b19a
Fix unquoted regex error handling (#298)
pederhan Oct 4, 2024
95356b2
Add error msg builder with underlined suggestions (#300)
pederhan Oct 7, 2024
b7d1a6c
Add defaults for url and domain config options (#301)
pederhan Oct 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
157 changes: 157 additions & 0 deletions mreg_cli/api/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,157 @@
"""API glue code for the mreg_cli package.

Originally the API code took whatever JSON data it received and returned it as a dictionary.
This led to horrible code that was hard to maintain and debug. This module is an attempt to
fix that by using pydantic models to validate incoming data so the client code has
guarantees about the data it is working with.
"""

import re
from ipaddress import ip_address
from typing import Dict, Union

from mreg_cli.api.endpoints import Endpoint
from mreg_cli.api.models import Host, HostList, MACAddressField
from mreg_cli.config import MregCliConfig
from mreg_cli.log import cli_warning
from mreg_cli.outputmanager import OutputManager
from mreg_cli.types import IP_AddressT
from mreg_cli.utilities.api import get, get_item_by_key_value


def get_host(
identifier: str, ok404: bool = False, inform_as_cname: bool = False
) -> Union[None, Host]:
"""Get a host by the given identifier.

- If the identifier is numeric, it will be treated as an ID.
- If the identifier is an IP address, it will be treated as an IP address (v4 or v6).
- If the identifier is a MAC address, it will be treated as a MAC address.
- Otherwise, it will be treated as a hostname. If the hostname is a CNAME,
the host it points to will be returned.

To check if a returned host is a cname, one can do the following:

```python
hostname = "host.example.com"
host = get_host(hostname, ok404=True)
if host is None:
print("Host not found.")
elif host.name != hostname:
print(f"{hostname} is a CNAME pointing to {host.name}")
else:
print(f"{host.name} is a host.")
```

Note that get_host will perform a case-insensitive search for a fully qualified version
of the hostname, so the comparison above may fail.

:param identifier: The identifier to search for.
:param ok404: If True, don't raise a CliWarning if the host is not found.
:param inform_as_cname: If True, inform the user if the host is a CNAME.

:raises CliWarning: If we don't find the host and `ok404` is False.

:returns: A Host object if the host was found, otherwise None.
"""
data = None
if identifier.isdigit():
data = get_item_by_key_value(Endpoint.Hosts, "id", identifier, ok404=ok404)
else:
try:
ip_address(identifier)
data = get_item_by_key_value(Endpoint.Hosts, "ipaddress", identifier, ok404=ok404)
except ValueError:
pass

try:
mac = MACAddressField(address=identifier)
data = get_item_by_key_value(Endpoint.Hosts, "macaddress", mac.address, ok404=ok404)
except ValueError:
pass

hostname = clean_hostname(identifier)

data = get(Endpoint.Hosts.with_id(hostname), ok404=True)

if data:
data = data.json()
else:
data = get_item_by_key_value(Endpoint.Cnames, "name", hostname, ok404=ok404)
# If we found a CNAME, get the host it points to. We're not interested in the CNAME
# itself. Also, no point in pydantic-ifying the CNAME data since we're not using it.
if data is not None:
data = get_item_by_key_value(Endpoint.Hosts, "id", data["host"], ok404=ok404)

if data and inform_as_cname:
OutputManager().add_line(f"{hostname} is a CNAME for {data['name']}")

if data is None:
if not ok404:
cli_warning(f"Host {identifier} not found.")

return None

return Host(**data)


def delete_host(identifier: str) -> bool:
"""Delete a host by the given identifier."""
host = get_host(identifier)
if host is None:
cli_warning(f"Host {identifier} not found.")

return host.delete()


def get_hosts(params: Dict[str, Union[str, int]]) -> HostList:
"""Get a list of hosts."""
return HostList.get(params=params)


def add_host(data: Dict[str, Union[str, None]]) -> Union[Host, None]:
"""Add a host."""
return Host.create(kwargs=data)


def get_network_by_ip(ip: IP_AddressT) -> Union[None, Dict[str, Union[str, int]]]:
"""Return a network associated with given IP."""
return get(Endpoint.NetworksByIP.with_id(str(ip))).json()


def clean_hostname(name: Union[str, bytes]) -> str:
"""Ensure hostname is fully qualified, lowercase, and has valid characters.

:param name: The hostname to clean.

:raises CliWarning: If the hostname is invalid.

:returns: The cleaned hostname.
"""
# bytes?
if not isinstance(name, (str, bytes)):
cli_warning("Invalid input for hostname: {}".format(name))

if isinstance(name, bytes):
name = name.decode()

name = name.lower()

# invalid characters?
if re.search(r"^(\*\.)?([a-z0-9_][a-z0-9\-]*\.?)+$", name) is None:
cli_warning("Invalid input for hostname: {}".format(name))

# Assume user is happy with domain, but strip the dot.
if name.endswith("."):
return name[:-1]

# If a dot in name, assume long name.
if "." in name:
return name

config = MregCliConfig()
default_domain = config.get("domain")
# Append domain name if in config and it does not end with it
if default_domain and not name.endswith(default_domain):
return "{}.{}".format(name, default_domain)
return name
Loading
Loading