bug: 500 error after upgrading from 1.0.8 to 1.0.9

[FragmentedPacket]

Component

API Server / GraphQL

Infrahub version

1.0.9

Current Behavior

500 Error when attempting to load a schema via infrahubctl

This most likely happens on upgrades from previous versions as well, I believe 1.0.7.

Expected Behavior

Ability to load a schema after upgrading to 1.0.9

Steps to Reproduce

1. spin up 1.0.8
2. load the base/ and extensions/location_minimal/ from https://github.com/opsmill/schema-library
3. create location, platform, device type, device, and interfaces (one L2, one L3)
4. upgrade 1.0.9
5. Run migration commands (but they fail and might be root cause?)

Core Schema has diff, will need to be updated
Traceback (most recent call last):
  File "/usr/local/bin/infrahub", line 6, in <module>
    sys.exit(app())
             ^^^^^
  File "/usr/local/lib/python3.12/site-packages/typer/main.py", line 338, in __call__
    raise e
  File "/usr/local/lib/python3.12/site-packages/typer/main.py", line 321, in __call__
    return get_command(self)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/typer/core.py", line 728, in main
    return _main(
           ^^^^^^
  File "/usr/local/lib/python3.12/site-packages/typer/core.py", line 197, in _main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/typer/main.py", line 703, in wrapper
    return callback(**use_params)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/source/python_sdk/infrahub_sdk/async_typer.py", line 18, in runner
    return asyncio.run(func(*args, **kwargs))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/asyncio/base_events.py", line 686, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/source/backend/infrahub/cli/db.py", line 289, in update_core_schema
    await registry.schema.update_schema_branch(
  File "/source/backend/infrahub/core/schema/manager.py", line 157, in update_schema_branch
    schema_diff = await self.update_schema_to_db(schema=schema, db=db, branch=branch, diff=diff)
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/source/backend/infrahub/core/schema/manager.py", line 211, in update_schema_to_db
    node = await self.update_node_in_db_based_on_diff(node=item, branch=branch, db=db, diff=item_diff)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/source/backend/infrahub/core/schema/manager.py", line 385, in update_node_in_db_based_on_diff
    obj = await self.get_one(id=node.get_id(), branch=branch, db=db)
                                ^^^^^^^^^^^^^
  File "/source/backend/infrahub/core/schema/basenode_schema.py", line 60, in get_id
    raise ValueError(f"id is not defined on {self.kind}")
ValueError: id is not defined on CoreProfile

7. load the cable.yml schema from https://github.com/opsmill/schema-library

❯ infrahubctl schema load ~/cloned/schema-library/extensions/cable/cable.yml --branch branch2
HTTP communication failure: Server error '500 Internal Server Error' for url 'http://localhost:8000/api/schema/load?branch=branch2'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500 on POST to http://localhost:8000/api/schema/load?branch=branch2

❯ infrahubctl schema load ~/cloned/schema-library/extensions/cable/cable.yml
HTTP communication failure: Server error '500 Internal Server Error' for url 'http://localhost:8000/api/schema/load?branch=main'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/500 on POST to http://localhost:8000/api/schema/load?branch=main

Additional Information

1.0.9-upgrade-server.log
1.0.9-upgrade-worker.log

The worker logs are pretty informative. I think we need to one catch the exception at

infrahub/backend/infrahub/core/validators/tasks.py

Line 43 in 3493c8b

for violation in result.violations:

and then fix the root issue causing the exception to be added the violations attribute.

This is Line 1986-2078 in the worker logs.

│ /source/backend/infrahub/core/schema/basenode_schema.py:214 in get_relationship                  │
│                                                                                                  │
│   211 │   │   for item in self.relationships:                                                    │
│   212 │   │   │   if item.name == name:                                                          │
│   213 │   │   │   │   return item                                                                │
│ ❱ 214 │   │   raise ValueError(f"Unable to find the relationship {name}")                        │
│   215 │                                                                                          │
│   216 │   def get_relationship_by_id(self, id: str) -> RelationshipSchema:                       │
│   217 │   │   for item in self.relationships:                                                    │
│                                                                                                  │
│ ╭───────────────────────────── locals ──────────────────────────────╮                            │
│ │ name = 'subscriber_of_groups'                                     │                            │
│ │ self = GenericSchema(                                             │                            │
│ │        │   id=None,                                               │                            │
│ │        │   state=<HashableModelState.PRESENT: 'present'>,         │                            │
│ │        │   name='Profile',                                        │                            │
│ │        │   namespace='Core',                                      │                            │
│ │        │   description='Base Profile in Infrahub.',               │                            │
│ │        │   label='Profile',                                       │                            │
│ │        │   branch=<BranchSupportType.AWARE: 'aware'>,             │                            │
│ │        │   default_filter='profile_name__value',                  │                            │
│ │        │   human_friendly_id=['profile_name__value'],             │                            │
│ │        │   display_labels=['profile_name__value'],                │                            │
│ │        │   include_in_menu=False,                                 │                            │
│ │        │   menu_placement=None,                                   │                            │
│ │        │   icon='mdi:shape-plus-outline',                         │                            │
│ │        │   order_by=None,                                         │                            │
│ │        │   uniqueness_constraints=[['profile_name__value']],      │                            │
│ │        │   documentation=None,                                    │                            │
│ │        │   attributes=[                                           │                            │
│ │        │   │   AttributeSchema(                                   │                            │
│ │        │   │   │   id=None,                                       │                            │
│ │        │   │   │   state=<HashableModelState.PRESENT: 'present'>, │                            │
│ │        │   │   │   name='profile_name',                           │                            │
│ │        │   │   │   kind='Text',                                   │                            │
│ │        │   │   │   enum=None,                                     │                            │
│ │        │   │   │   choices=None,                                  │                            │
│ │        │   │   │   regex=None,                                    │                            │
│ │        │   │   │   max_length=32,                                 │                            │
│ │        │   │   │   min_length=3,                                  │                            │
│ │        │   │   │   label='Profile Name',                          │                            │
│ │        │   │   │   description=None,                              │                            │
│ │        │   │   │   read_only=False,                               │                            │
│ │        │   │   │   unique=True,                                   │                            │
│ │        │   │   │   optional=False,                                │                            │
│ │        │   │   │   branch=<BranchSupportType.AWARE: 'aware'>,     │                            │
│ │        │   │   │   order_weight=1000,                             │                            │
│ │        │   │   │   default_value=None,                            │                            │
│ │        │   │   │   inherited=False,                               │                            │
│ │        │   │   │   allow_override=<AllowOverrideType.ANY: 'any'>  │                            │
│ │        │   │   ),                                                 │                            │
│ │        │   │   AttributeSchema(                                   │                            │
│ │        │   │   │   id=None,                                       │                            │
│ │        │   │   │   state=<HashableModelState.PRESENT: 'present'>, │                            │
│ │        │   │   │   name='profile_priority',                       │                            │
│ │        │   │   │   kind='Number',                                 │                            │
│ │        │   │   │   enum=None,                                     │                            │
│ │        │   │   │   choices=None,                                  │                            │
│ │        │   │   │   regex=None,                                    │                            │
│ │        │   │   │   max_length=None,                               │                            │
│ │        │   │   │   min_length=None,                               │                            │
│ │        │   │   │   label='Profile Priority',                      │                            │
│ │        │   │   │   description=None,                              │                            │
│ │        │   │   │   read_only=False,                               │                            │
│ │        │   │   │   unique=False,                                  │                            │
│ │        │   │   │   optional=True,                                 │                            │
│ │        │   │   │   branch=<BranchSupportType.AWARE: 'aware'>,     │                            │
│ │        │   │   │   order_weight=2000,                             │                            │
│ │        │   │   │   default_value=1000,                            │                            │
│ │        │   │   │   inherited=False,                               │                            │
│ │        │   │   │   allow_override=<AllowOverrideType.ANY: 'any'>  │                            │
│ │        │   │   )                                                  │                            │
│ │        │   ],                                                     │                            │
│ │        │   relationships=[],                                      │                            │
│ │        │   hierarchical=False,                                    │                            │
│ │        │   generate_profile=True,                                 │                            │
│ │        │   used_by=[                                              │                            │
│ │        │   │   'ProfileBuiltinIPAddress',                         │                            │
│ │        │   │   'ProfileBuiltinIPPrefix',                          │                            │
│ │        │   │   'ProfileBuiltinTag',                               │                            │
│ │        │   │   'ProfileDcimConnector',                            │                            │
│ │        │   │   'ProfileDcimDevice',                               │                            │
│ │        │   │   'ProfileDcimDeviceType',                           │                            │
│ │        │   │   'ProfileDcimEndpoint',                             │                            │
│ │        │   │   'ProfileDcimGenericDevice',                        │                            │
│ │        │   │   'ProfileDcimInterface',                            │                            │
│ │        │   │   'ProfileDcimInterfaceL2',                          │                            │
│ │        │   │   ... +15                                            │                            │
│ │        │   ]                                                      │                            │
│ │        )                                                          │                            │
│ ╰───────────────────────────────────────────────────────────────────╯                            │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: Unable to find the relationship subscriber_of_groups