Skip to content

⚡️ Speed up method ModelManager.add_model by 11% in PR #1175 (depth-estimation-workflow-block) #1194

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

⚡️ Speed up method ModelManager.add_model by 11% in PR #1175 (depth-estimation-workflow-block) #1194

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions inference/core/managers/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,25 +48,26 @@ def add_model(
model_id (str): The identifier of the model.
model (Model): The model instance.
"""
logger.debug(
f"ModelManager - Adding model with model_id={model_id}, model_id_alias={model_id_alias}"
)
# Minimize logging overhead by combining log messages
resolved_identifier = model_id if model_id_alias is None else model_id_alias
if resolved_identifier in self._models:
logger.debug(
f"ModelManager - model with model_id={resolved_identifier} is already loaded."
)
return

logger.debug("ModelManager - model initialisation...")
logger.debug(
f"ModelManager - Adding and initializing model with model_id={model_id}, model_id_alias={model_id_alias}"
)

try:
# Call the model constructor directly
model = self.model_registry.get_model(resolved_identifier, api_key)(
model_id=model_id,
api_key=api_key,
)
logger.debug("ModelManager - model successfully loaded.")
self._models[resolved_identifier] = model
logger.debug("ModelManager - model successfully loaded.")
except Exception as e:
raise

Expand Down
6 changes: 4 additions & 2 deletions inference/core/registries/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,10 @@ def get_model(self, model_type: str, model_id: str) -> Model:
Raises:
ModelNotRecognisedError: If the model_type is not found in the registry_dict.
"""
if model_type not in self.registry_dict:
# Combining lookup and retrieval into a single step
try:
return self.registry_dict[model_type]
except KeyError:
raise ModelNotRecognisedError(
f"Could not find model of type: {model_type} in configured registry."
)
return self.registry_dict[model_type]