Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

IFC-1245 Initial implementation for object templates #5610

Merged
merged 40 commits into from
Feb 17, 2025

Conversation

gmazoyer
Copy link
Contributor

@gmazoyer gmazoyer commented Jan 29, 2025

This is the initial implementation of object templates.

An object template is a schema node that is generated on the fly when a node defines generate_template: true. When generating object template schema, the schema manager will traverse parent/component relationships in order to determine which templates are required.

Nodes for which templates are generated will automatically get a object_template relationship pointing to the template that was used to create the objects. Templates will also have a related_nodes relationship with references to objects created by templates.

Templates have their own namespace and are published in the REST API schema under the templates key.

When running a GraphQL mutation, if an object_template relationship is specified, attributes from the template will be applied as the ones of the object to create. Attributes with values given in the mutation input will still prevail over the ones of the template.

When creating an object, parent/component relationships are resolved and objects will also be created. This is done in 2 steps: it first creates the initial object which is based on the given template, then it goes through relationships and populates them.

Relationships of other kinds are not yet supported but can be implemented at the next step of this feature implementation.

@github-actions github-actions bot added the group/backend Issue related to the backend (API Server, Git Agent) label Jan 29, 2025
Copy link

codspeed-hq bot commented Jan 29, 2025

CodSpeed Performance Report

Merging #5610 will not alter performance

Comparing gma-20250129-ifc1177 (f9aa039) with develop (6244a9d)

Summary

✅ 11 untouched benchmarks

@gmazoyer gmazoyer linked an issue Jan 29, 2025 that may be closed by this pull request
5 tasks
@gmazoyer gmazoyer force-pushed the gma-20250129-ifc1177 branch from 18008c2 to f036a13 Compare January 30, 2025 10:55
@github-actions github-actions bot added the type/documentation Improvements or additions to documentation label Jan 30, 2025
@gmazoyer gmazoyer force-pushed the gma-20250129-ifc1177 branch 4 times, most recently from ef6cf14 to 954f385 Compare February 7, 2025 12:00
@gmazoyer gmazoyer force-pushed the gma-20250129-ifc1177 branch 5 times, most recently from 8ec5689 to 9833cd0 Compare February 13, 2025 10:11
@gmazoyer gmazoyer marked this pull request as ready for review February 13, 2025 10:30
@gmazoyer gmazoyer requested review from a team as code owners February 13, 2025 10:30
Copy link
Contributor

@ogenstad ogenstad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding some minor comments for now, will look more later.

backend/tests/unit/core/test_node.py Outdated Show resolved Hide resolved
@@ -313,6 +314,7 @@ def to_int(self) -> int:
"Lineage",
"Schema",
"Profile",
"Template",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add this to the release notes to indicate that we're adding a new namespace to the list of restricted ones.

@@ -265,7 +272,7 @@ async def query(
async def count(
cls,
db: InfrahubDatabase,
schema: Union[type[SchemaProtocol], NodeSchema, GenericSchema, ProfileSchema, str],
schema: Union[type[SchemaProtocol], NodeSchema, GenericSchema, ProfileSchema, TemplateSchema, str],
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpicking here but for the count and query methods when we are updating them we might as well get rid of the Union.

@@ -327,13 +341,22 @@ def get_profile(self, name: str, duplicate: bool = True) -> ProfileSchema:
raise ValueError(f"{name!r} is not of type ProfileSchema")
return item

def get_template(self, name: str, duplicate: bool = True) -> TemplateSchema:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be kind instead of name? Same for SchemaBranch.get?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used the same parameter name as all the methods that perform the same kind of operations. I agree that name is not the best name for it, I used it for consistency reasons. I guess we could rename that parameter for all the methods in another PR.

Copy link
Contributor

@ogenstad ogenstad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work! Looks good to me overall. I left some comments, the most important one would be related to the scenario if we can't find the template during the processing of a node.

@@ -546,7 +547,7 @@ def generate_graphql_object(self, schema: MainSchemaTypes, populate_cache: bool

interfaces: set[type[InfrahubObject]] = set()

if isinstance(schema, NodeSchema | ProfileSchema) and schema.inherit_from:
if isinstance(schema, NodeSchema | ProfileSchema | TemplateSchema) and schema.inherit_from:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should have a common name for NodeSchema | ProfileSchema | TemplateSchema in the same we we have for MainSchemaTypes.

backend/infrahub/graphql/manager.py Outdated Show resolved Hide resolved

await obj_peer.new(db=db, **obj_peer_data)
await node_constraint_runner.check(node=obj_peer, field_filters=list(obj_peer_data))
await obj_peer.save(db=db)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When saving the peer here we would have a changelog for that peer under obj_peer.node_changelog I think we'd want the .handle_relationships() method to send back a list[NodeChangelog] to the caller in order to get a full picture of what has changed.

backend/infrahub/core/node/__init__.py Show resolved Hide resolved
backend/infrahub/core/node/__init__.py Show resolved Hide resolved
backend/infrahub/core/node/__init__.py Outdated Show resolved Hide resolved
backend/infrahub/core/node/__init__.py Outdated Show resolved Hide resolved
backend/infrahub/core/schema/definitions/internal.py Outdated Show resolved Hide resolved
backend/infrahub/core/schema/manager.py Outdated Show resolved Hide resolved
backend/infrahub/core/schema/schema_branch.py Outdated Show resolved Hide resolved
backend/infrahub/graphql/manager.py Show resolved Hide resolved
@@ -197,13 +282,14 @@ async def mutate_create_object(
await obj.new(db=db, **data)
await node_constraint_runner.check(node=obj, field_filters=fields_to_validate)
await obj.save(db=db)
await cls._handle_object_template(db=db, branch=branch, obj=obj, data=data)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you do something like this instead

object_template = await cls.get_object_template(db=db, branch=branch, obj=obj, data=data)
if object_template:
    await cls._handle_template_relationships(
        db=db,
        branch=branch,
        constraint_runner=node_constraint_runner,
        template=object_template,
        obj=obj
    )

seems like _handle_object_template is basically getting an object template and then running it through _handle_relationships and _handle_relationships doesn't need to get its own version of NodeConstraintRunner this way

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That node constraint runner setup bugs me as well. When I tried to use the existing one, built by the mutation code, it failed at identifying the original node (Unable to find the node X / KIND error). My guess is that it does not play very well with a performance fix we pushed earlier this week.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After giving this some thoughts I wonder if it is going to be an issue with relationships allocated from pool, if they are also used in uniqueness constraints.

backend/infrahub/graphql/mutations/main.py Show resolved Hide resolved
backend/infrahub/graphql/mutations/main.py Outdated Show resolved Hide resolved
@gmazoyer gmazoyer changed the title IFC-1177 Initial implementation for object templates IFC-1245 Initial implementation for object templates Feb 14, 2025
@gmazoyer gmazoyer force-pushed the gma-20250129-ifc1177 branch from ece08c3 to aadb959 Compare February 14, 2025 15:11
@gmazoyer gmazoyer force-pushed the gma-20250129-ifc1177 branch 3 times, most recently from e96db49 to 50a6eaf Compare February 17, 2025 07:33
@gmazoyer gmazoyer force-pushed the gma-20250129-ifc1177 branch from e2846fb to f9aa039 Compare February 17, 2025 17:43
@gmazoyer gmazoyer merged commit 5544ada into develop Feb 17, 2025
35 checks passed
@gmazoyer gmazoyer deleted the gma-20250129-ifc1177 branch February 17, 2025 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
group/backend Issue related to the backend (API Server, Git Agent) type/documentation Improvements or additions to documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

feature: Object Templates
4 participants