Skip to content

Commit

Permalink
Merge back stable into develop with conflict resolutions (#3259)
Browse files Browse the repository at this point in the history
* update layout for small screens

* update layout for small screens

* Change version to 0.12.1

* Add plausible analytics in infrahub docs

* cherry pick new docker-compose.yml (#3018)

* fix(ci): disable flaky/failing job/test

Signed-off-by: Fatih Acar <[email protected]>

* Fix IPAM prefix/address upsert null reference (#3121)

* remove is_inheritied metadata (#3118)

* Ple prefix pagination (#3119)

* fix pagination for prefix children

* minor ui improvement for scrolling table

* Columns update for IP details list view (#3134)

* display prefix column only in global list view + remove interface column

* remove log

* Prefix summary updates (#3122)

* remove log

* rename constant

* bold for current prefix in prefix details and list

* display progress bar value

* switch to progress bar in prefix summary

* rename props

* update skeleton

* 🔧

* Display profiles schema in Schema visualizer

* fix e2e test

* fix typo -> optional

* fix edit on profile

* Url on meta tooltips uses profile's kind instead of profile generic

* update get atributes for queries function to exclude reead only fields + rename parameters (#3137)

* Raise error on RelationshipAdd to busy peer

* fix TS type on ip-details-card

* on ipam summary, utilization show a progress bar

* remove chart card in IP summary

* Python SDK 0.10.0 first release candidate (#3141)

* Prefix columns and icons updates (#3149)

* lint

* update colomns and icons

* improve icon test for member type

* hierarchy update fix (#3143)

* test for hierarchy update failure

* set hierarchy when creating relationship

* add fix for all relationship update queries

* one more test

* Added a visual difference between "select a kind" and "select a profile"

* ProgressBar adheres to the  progressbar role requirements + redesign

* renamed component

* rename function to make it more explicit on its action

* fix relationship filter on IP address/prefix summary

* removed chart skeleton

* Fix flaky on profiles spect e2e test

* Add IPAM reconciliation to merge (#3135)

* add IpamReconcile to merge mutation

* Ipam reconcile in proposed change merge

* move Ipam reconciliation up earlier in message chain

* Raise correct exception when getting wrong type of node (#3140)

* Fix IPAM builtin generics read-only relationships (#3157)

* on sidebar footer, docs link redirects to local documentation

* Force Ip in IP in BuiltinIPAM nodes (#3166)

* Force IP instead of Ip in Builtin with Label

* rework a bit the base schema to link circuit<->bgp session

* No focus ring on button when clicking with mouse (#3160)

* removed focus ring on mouse click on button

* no focus with mouse on wayback machine + improved animation

* revert hover bg on button variant ghost

* Metadata popover in relationships list (#3167)

* add again popover for metadata without edit button

* get properties from relationships

* fix function to get mutation for relationships

* minor ui alignement

* fix mock query

* Strip trailing slashes from address in config (#3183)

* fix for stale relationships being included in NodeGetListQuery filtering and ordering results (#3165)

* failing tests for issue in node get list query

* possible fix for missing active filter in subquery filter

* test and fix for inactive relationships included in NodeGetList filter

* fix for ordering subquery too

* profile update fix (#3147)

* add failing test for profile update

* a bunch of tests and some bug fixes

* fix source clearing

* order profiles deterministically for filter,order, and application

* IPAM edit for prefixes and adresses (#3189)

* update card to display edit button and panel

* profive refetch for edit

* provide refetch for ip adress

* fix edit for prefix and attributes

* 🔧

* Allow null values for optional enums

Fixes #3182

* exclude mypy cache in subdirectories from docker

* fix flaky e2e test for search anywhere

* removes pagination for groups selector (#3198)

* send null for dropdown + enum (#3199)

* fix for hierarchy cypher query (#3194)

* fix for hierarchy cypher query

* fix for neo4j

* remove DISTINCT for neo4j

* Guide - Jinja Transform: Update Jinja2 template to iterate over GQL data at proper level.

* remove initial value for filters (#3206)

* replace prefix by uuid on IPAM url + E2E Testing on IPAM (#3161)

* removed unused files

* Added e2e test for ipam tree on expand on toggle

* link in ipam tree uses uuid

* e2e test for prefix summary

* link in prefix list use uuid

* fix test

* use uuid on prefix details tab

* add e2e test on permission and create/delete prefix

* use new button primitive on table component

* use uuid on Ipam tab ip address

* Add test on ip address create and delete

* fix test

* improve ipam test reliability

* improve ipam test consistency

* added e2e test for edit  on prefix and ip address

* Fixed ipam tree expansion on 1st load

* cleaning graphql request

* Keep all initial url query params after login success

* supports adding a profile to a group on UI

* fix create schema guide

relationships of kind Parent are now required

* only optional attrs on profiles (#3177)

* only optional attrs on profiles

* try adding groups back in to profiles

* hopefully fixes tests and doesn't break any

* get attributes for profiles in create and edit forms

---------

Co-authored-by: pa-lem <[email protected]>

* Fix crash on IPAM when prefix id is not found

* fix crash on prefix details tab when prefix id is not found

* put loading before checking if prefix data is present

* Keep QSP when navigating with tabs on IPAM

* Display prefix value instead of id in ip address breadcrumb

* add more nodes to CodeNode.used_by (#3218)

* add more nodes to CodeNode.used_by

* add CoreNode to InfrahubKind

* don't include CoreNode in diff

* use version of schema for correct branch in uniqueness checks (#3219)

* get the schema version on this branch for uniqueness check

* update some unit tests

* use the branch in the right place instead of overriding the input schema

* Improve diff print to include lower level

* Update _diff_element to add element when the id is missing on both side

* Add migrations 5 6 & 7 related to new fields in the schema

* Add deprecated models structure in schema migration

* Typos and spelling covering those reported from @hbled: #3152

* Release node for 0.13

* Update NodeAttributeAddMigrationQuery01 to add is_default

* Fix for pylint

* Add method function to generate uuid

* Add load_schema_to_db to ensure all nodes have a uuid

* Leverage generate_uuid in tests to similate real data

* Add generate_uuid to similar real data

* Revert changes to TestInfrahubApp and add PYTEST_RUNNING var as a tracker

* Remove readme page in docs for topics and guides

* Update link in sidebars

* Add 0.13 to menu

* Remove retype

* Updatel links, remove section about config changes

* Update versions to 0.13 and 0.10

* Update flask-cors to fix security vulnerability

Flask-Cors is only used from Locust and doesn't impact the security of
Infrahub, still it looks better to have it patched for the release.

* Remove known IPAM limitations that where fixed (#3231)

* Flaky test fix for select 2 steps and options loading time (#3229)

* start fix flaky select

* rename operation name

* Fix for 3227, add branch to get_unique_constraint_schema_attribute_paths

* Add integration test to validate loading new schema in branch

* Fix PYTEST_RUNNING check in _diff_element

* Cleanup integration test

* Add PYTEST_RUNNING env var for integration tests for the SDK

* Include feedback

* rename read_only

* Fix typo, update gitignore

* Fix IP namespace being reset on mutation (#3242)

* add neo4j storage requirements to hardware requirements

* run site generation outside of batch

* Fix typo

* Adds advanced test for profile for interface l2 (#3239)

* add advanced test for profile for interface l2

* add steps

* add describe in previous e2e test for interface l2

* update steps

* Update Jinja2 for vulnerability fix (#3245)

* Fix hot reload on IPAM tree (#3247)

* Only show IPAM menu if proper models exist.

* Refresh for version 0.13.0

* Modify how the schema is loaded for version 0.13.0

* Switch to neo4j community and make neo4j te default in codespace

* Remove wait and re-add restart

* Switch to neo4j community and make neo4j te default in codespace

---------

Signed-off-by: Fatih Acar <[email protected]>
Co-authored-by: Damien Garros <[email protected]>
Co-authored-by: Bilal <[email protected]>
Co-authored-by: Benoit Kohler <[email protected]>
Co-authored-by: Fatih Acar <[email protected]>
Co-authored-by: fatih-acar <[email protected]>
Co-authored-by: Guillaume Mazoyer <[email protected]>
Co-authored-by: Patrick Ogenstad <[email protected]>
Co-authored-by: Aaron McCarty <[email protected]>
Co-authored-by: Benoit Kohler <[email protected]>
Co-authored-by: Mikhail Yohman <[email protected]>
Co-authored-by: Wim Van Deun <[email protected]>
Co-authored-by: Pete Crocker <[email protected]>
Co-authored-by: Aaron McCarty <[email protected]>
  • Loading branch information
14 people authored May 7, 2024
1 parent 7f00d36 commit 2f6dc5b
Show file tree
Hide file tree
Showing 55 changed files with 655 additions and 161 deletions.
57 changes: 31 additions & 26 deletions .devcontainer/demo-container/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,43 +5,48 @@
"context": ".."
},
"customizations": {
"settings": {
"python.pythonPath": "/workspace/infrahub/.venv/bin/python",
"terminal.integrated.shell.linux": "/bin/bash",
"python.venvPath": "/workspace/infrahub/.venv",
"python.linting.enabled": true
},
"vscode": {
"settings": {
"python.pythonPath": "/workspace/infrahub/.venv/bin/python",
"terminal.integrated.shell.linux": "/bin/bash",
"python.venvPath": "/workspace/infrahub/.venv",
"python.linting.enabled": true
},
"vscode": {
"extensions": [
"pomdtr.excalidraw-editor",
"wholroyd.jinja",
"ms-python.vscode-pylance",
"shardulm94.trailing-spaces",
"nickmillerdev.pytest-fixtures",
"yzhang.markdown-all-in-one",
"GraphQL.vscode-graphql-syntax",
"MS-vsliveshare.vsliveshare"
"pomdtr.excalidraw-editor",
"wholroyd.jinja",
"ms-python.vscode-pylance",
"shardulm94.trailing-spaces",
"nickmillerdev.pytest-fixtures",
"yzhang.markdown-all-in-one",
"GraphQL.vscode-graphql-syntax",
"MS-vsliveshare.vsliveshare"
]
}
}
},
"remoteUser": "vscode",
"remoteEnv": {
"INFRAHUB_API_TOKEN": "06438eb2-8019-4776-878c-0941b1f1d1ec",
"INFRAHUB_DB_TYPE": "memgraph",
"INFRAHUB_IMAGE_VER": "local"
"INFRAHUB_SDK_API_TOKEN": "06438eb2-8019-4776-878c-0941b1f1d1ec",
"INFRAHUB_DB_TYPE": "neo4j",
"INFRAHUB_IMAGE_VER": "local"
},
"forwardPorts": [8000],
"runArgs": ["--volume=/var/lib/docker", "--privileged"],
"forwardPorts": [
8000
],
"runArgs": [
"--volume=/var/lib/docker",
"--privileged"
],
"onCreateCommand": "./.devcontainer/onCreateCommand.sh",
"postCreateCommand": "./.devcontainer/postCreateCommand.sh",
"postStartCommand": "./.devcontainer/postStartCommand.sh",
"postAttachCommand": "./.devcontainer/postAttachCommand.sh",
"features": {
"docker-in-docker": "latest",
"github-cli": "latest"
"docker-in-docker": "latest",
"github-cli": "latest"
},
"hostRequirements": {
"cpus": 8,
"memory": "16gb"
"cpus": 8,
"memory": "16gb"
}
}
}
91 changes: 48 additions & 43 deletions .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
@@ -1,46 +1,51 @@
{
"name": "infrahub",
"build": {
"dockerfile": "./Dockerfile",
"context": "."
"name": "infrahub",
"build": {
"dockerfile": "./Dockerfile",
"context": "."
},
"customizations": {
"settings": {
"python.pythonPath": "/workspace/infrahub/.venv/bin/python",
"terminal.integrated.shell.linux": "/bin/bash",
"python.venvPath": "/workspace/infrahub/.venv",
"python.linting.enabled": true
},
"customizations": {
"settings": {
"python.pythonPath": "/workspace/infrahub/.venv/bin/python",
"terminal.integrated.shell.linux": "/bin/bash",
"python.venvPath": "/workspace/infrahub/.venv",
"python.linting.enabled": true
},
"vscode": {
"extensions": [
"pomdtr.excalidraw-editor",
"wholroyd.jinja",
"ms-python.vscode-pylance",
"shardulm94.trailing-spaces",
"nickmillerdev.pytest-fixtures",
"yzhang.markdown-all-in-one",
"GraphQL.vscode-graphql-syntax",
"MS-vsliveshare.vsliveshare"
]
}
},
"remoteUser": "vscode",
"remoteEnv": {
"INFRAHUB_API_TOKEN": "06438eb2-8019-4776-878c-0941b1f1d1ec",
"INFRAHUB_DB_TYPE": "memgraph",
"INFRAHUB_IMAGE_VER": "local"
},
"forwardPorts": [8000],
"runArgs": ["--volume=/var/lib/docker", "--privileged"],
"onCreateCommand": "./.devcontainer/onCreateCommand.sh",
"postCreateCommand": "./.devcontainer/postCreateCommand.sh",
"postAttachCommand": "./.devcontainer/postAttachCommand.sh",
"features": {
"docker-in-docker": "latest",
"github-cli": "latest"
},
"hostRequirements": {
"cpus": 8,
"memory": "16gb"
"vscode": {
"extensions": [
"pomdtr.excalidraw-editor",
"wholroyd.jinja",
"ms-python.vscode-pylance",
"shardulm94.trailing-spaces",
"nickmillerdev.pytest-fixtures",
"yzhang.markdown-all-in-one",
"GraphQL.vscode-graphql-syntax",
"MS-vsliveshare.vsliveshare"
]
}
}
},
"remoteUser": "vscode",
"remoteEnv": {
"INFRAHUB_SDK_API_TOKEN": "06438eb2-8019-4776-878c-0941b1f1d1ec",
"INFRAHUB_DB_TYPE": "neo4j",
"INFRAHUB_IMAGE_VER": "local"
},
"forwardPorts": [
8000
],
"runArgs": [
"--volume=/var/lib/docker",
"--privileged"
],
"onCreateCommand": "./.devcontainer/onCreateCommand.sh",
"postCreateCommand": "./.devcontainer/postCreateCommand.sh",
"postAttachCommand": "./.devcontainer/postAttachCommand.sh",
"features": {
"docker-in-docker": "latest",
"github-cli": "latest"
},
"hostRequirements": {
"cpus": 8,
"memory": "16gb"
}
}
7 changes: 1 addition & 6 deletions .github/build-docs.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,3 @@
#!/bin/sh

if [ -f docs/docusaurus.config.ts ]; then
(cd docs && npm install && npm run build)
else
npx retypeapp build docs
fi

cd docs && npm install && npm run build
4 changes: 0 additions & 4 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,7 @@ development/docker-compose.dev-override.yml
.direnv/
.envrc

# Retype App
.retype/*
docs/.retype
docs/build
retype.manifest

storage/*
.coverage.*
Expand Down
1 change: 1 addition & 0 deletions .vale/styles/spelling-exceptions.txt
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ kbps
markdownlint
memgraph
menu_placement
modularization
namespace
namespaces
Nautobot
Expand Down
15 changes: 14 additions & 1 deletion backend/infrahub/api/menu.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
from infrahub.core import registry
from infrahub.core.branch import Branch # noqa: TCH001
from infrahub.core.constants import InfrahubKind
from infrahub.core.schema import NodeSchema
from infrahub.log import get_logger

if TYPE_CHECKING:
Expand Down Expand Up @@ -85,9 +86,16 @@ async def get_menu(
],
)

has_ipam = False

for key in full_schema.keys():
model = full_schema[key]

if isinstance(model, NodeSchema) and (
InfrahubKind.IPADDRESS in model.inherit_from or InfrahubKind.IPPREFIX in model.inherit_from
):
has_ipam = True

if not model.include_in_menu:
continue

Expand Down Expand Up @@ -215,4 +223,9 @@ async def get_menu(
],
)

return [objects, ipam, groups, unified_storage, change_control, deployment, admin]
menu_items = [objects]
if has_ipam:
menu_items.append(ipam)
menu_items.extend([groups, unified_storage, change_control, deployment, admin])

return menu_items
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ async def _check_one_schema(
at: Optional[Timestamp] = None,
filters: Optional[List[str]] = None,
) -> None:
path_groups = node_schema.get_unique_constraint_schema_attribute_paths()
path_groups = node_schema.get_unique_constraint_schema_attribute_paths(branch=self.branch)
query_request = self._build_query_request(
updated_node=node, node_schema=node_schema, path_groups=path_groups, filters=filters
)
Expand Down
3 changes: 1 addition & 2 deletions backend/infrahub/core/schema/basenode_schema.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ def _diff_element(
for name in sorted(present_both):
# If the element doesn't have an ID on either side
# this most likely means it was added recently from the internal schema.
if os.environ.get("PYTEST_RUNNING", "") == "true" and local_map[name] is None and other_map[name] is None:
if os.environ.get("PYTEST_RUNNING", "") != "true" and local_map[name] is None and other_map[name] is None:
elements_diff.added[name] = None
continue
local_element: obj_type = get_func(self, name=name)
Expand Down Expand Up @@ -402,7 +402,6 @@ def get_unique_constraint_schema_attribute_paths(
self, include_unique_attributes: bool = False, branch: Optional[Branch] = None
) -> List[List[SchemaAttributePath]]:
constraint_paths_groups = []

if include_unique_attributes:
for attribute_schema in self.unique_attributes:
constraint_paths_groups.append(
Expand Down
2 changes: 1 addition & 1 deletion backend/infrahub/graphql/mutations/ipam.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,8 @@ async def validate_namespace(
"""Validate or set (if not present) the namespace to pass to the mutation and return its ID."""
namespace_id: Optional[str] = None
if "ip_namespace" not in data or not data["ip_namespace"]:
data["ip_namespace"] = {"id": registry.default_ipnamespace}
namespace_id = existing_namespace_id or registry.default_ipnamespace
data["ip_namespace"] = {"id": namespace_id}
elif "id" in data["ip_namespace"]:
namespace = await registry.manager.get_one(db=db, kind=InfrahubKind.IPNAMESPACE, id=data["ip_namespace"]["id"])
namespace_id = namespace.id
Expand Down
62 changes: 62 additions & 0 deletions backend/tests/integration/schema_lifecycle/test_schema_update.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
from typing import Any, Dict

import pytest
from infrahub_sdk import InfrahubClient

from .shared import (
TestSchemaLifecycleBase,
)

# pylint: disable=unused-argument
ACCORD_COLOR = "#3443eb"


class TestSchemaLifecycleValidatorMain(TestSchemaLifecycleBase):
@pytest.fixture(scope="class")
def schema_network(
self,
) -> Dict[str, Any]:
return {
"version": "1.0",
"nodes": [
{
"name": "Device",
"namespace": "Network",
"default_filter": "hostname__value",
"attributes": [{"name": "hostname", "kind": "Text"}, {"name": "model", "kind": "Text"}],
},
{
"name": "Interface",
"namespace": "Network",
"uniqueness_constraints": [["device", "name__value"]],
"attributes": [{"name": "name", "kind": "Text", "optional": False}],
"relationships": [
{
"name": "device",
"cardinality": "one",
"kind": "Parent",
"peer": "NetworkDevice",
"optional": False,
}
],
},
],
}

async def test_step_01_create_branch(self, client: InfrahubClient):
branch = await client.branch.create(branch_name="test", sync_with_git=False)
assert branch

async def test_step_02_load_schema(self, client: InfrahubClient, schema_network):
# Load the new schema and apply the migrations
response = await client.schema.load(schemas=[schema_network], branch="test")
assert not response.errors

async def test_step_03_load_data(self, client: InfrahubClient, schema_network):
dev1 = await client.create(kind="NetworkDevice", hostname="device", model="switch", branch="test")
await dev1.save()
assert dev1.id

intf1 = await client.create(kind="NetworkInterface", name="interface1", device=dev1.id, branch="test")
await intf1.save()
assert intf1.id
Loading

0 comments on commit 2f6dc5b

Please sign in to comment.