Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor/solver_decorators #244

Merged
merged 5 commits into from
Jul 25, 2024
Merged

refactor/solver_decorators #244

merged 5 commits into from
Jul 25, 2024

Conversation

JarbasAl
Copy link
Member

@JarbasAl JarbasAl commented Jul 23, 2024

  • move class properties to init method
  • move autotranslation logic to @auto_translate() decorator (easier to maintain, less convoluted code)
  • add language detector plugin
  • add util methods self.translate and self.detect_language
  • deprecate context kwarg, new @_deprecate_context2lang decorator handles migration into new kwarg lang if needed
  • new util method ensure plugins with old signatures keep working _call_with_sanitized_kwargs
  • improve docstrs and typing for solver and translations plugins

may fix issues around the language kwarg, i am not sure exactly which issues but we have intermittent reports seemingly related to this

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Enhanced multilingual support with new parameters for language detection and translation in solver classes.
    • Automatic translation of inputs and outputs for various methods, improving user experience in multilingual contexts.
  • Improvements

    • Deprecated the context parameter in favor of a clearer lang parameter across multiple methods, providing better guidance for users.
  • Documentation

    • Expanded docstrings for methods to clarify their purpose and parameters, aiding user understanding.
  • Tests

    • Updated test cases to accommodate the new language parameter in method calls, enhancing the clarity and maintainability of tests.

@JarbasAl JarbasAl added bug Something isn't working refactor code improvements with no functional changes labels Jul 23, 2024
Copy link

coderabbitai bot commented Jul 23, 2024

Walkthrough

The recent enhancements to the AbstractSolver class and its derived classes significantly improve multilingual functionality by introducing decorators for translation management and replacing the deprecated context parameter with a clearer lang parameter. These updates foster greater clarity and flexibility, enabling effective language detection and translation, thereby enriching the user experience for multilingual tasks.

Changes

Files Change Summary
ovos_plugin_manager/templates/solvers.py Enhanced AbstractSolver with detector and translator parameters. Updated methods to replace context with lang for improved language handling. Implemented decorators for automatic translation, refining method signatures.
requirements/test.txt Added ovos-classifiers dependency to enhance testing capabilities related to classification tasks.
test/unittests/test_solver.py Updated test_translation to reflect changes in method calls for spoken_answer, now including a lang parameter. Introduced new test classes for managing translation and language detection.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant AbstractSolver
    participant Translator
    participant Detector

    User->>AbstractSolver: query
    AbstractSolver->>Detector: detect language
    Detector-->>AbstractSolver: detected language
    AbstractSolver->>Translator: translate input to default language
    Translator-->>AbstractSolver: translated input
    AbstractSolver->>Translator: translate output to original language
    Translator-->>AbstractSolver: translated output
    AbstractSolver->>User: return results
Loading

🐇 In a world of words, both vast and bright,
A solver hops forth, ready to write.
With languages dancing, a joyful cheer,
Translating our thoughts, making all clear!
So here's to the changes, oh what a delight,
A multilingual journey, all set to take flight! 🌍✨


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@JarbasAl JarbasAl changed the title refactor/decorators refactor/solver_decorators Jul 23, 2024
@JarbasAl JarbasAl marked this pull request as ready for review July 23, 2024 18:29
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between d3bbb47 and 2ed15e3.

Files selected for processing (1)
  • ovos_plugin_manager/templates/solvers.py (2 hunks)
Additional context used
Ruff
ovos_plugin_manager/templates/solvers.py

341-341: Do not use bare except

(E722)

Additional comments not posted (15)
ovos_plugin_manager/templates/solvers.py (15)

111-128: Enhance type annotations and initialization clarity.

The __init__ method has been updated with type annotations for translator and detector. This improves readability and maintainability. The initialization logic appears correct.


209-230: Enhance type annotations and initialization clarity.

The __init__ method has been updated with type annotations for translator and detector. This improves readability and maintainability. The initialization logic appears correct.


259-271: Ensure proper deprecation handling in stream_utterances method.

The _deprecate_context2lang decorator is used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


276-284: Ensure proper deprecation handling in get_data method.

The _deprecate_context2lang decorator is used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


289-297: Ensure proper deprecation handling in get_image method.

The _deprecate_context2lang decorator is used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


302-310: Ensure proper deprecation handling in get_expanded_answer method.

The _deprecate_context2lang decorator is used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


318-331: Ensure proper deprecation handling and translation in search method.

The _deprecate_context2lang and auto_translate_inputs decorators are used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


350-364: Ensure proper deprecation handling and translation in visual_answer method.

The _deprecate_context2lang and auto_translate_inputs decorators are used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


370-383: Ensure proper deprecation handling and translation in spoken_answer method.

The _deprecate_context2lang and auto_translate_inputs decorators are used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


400-414: Ensure proper deprecation handling and translation in long_answer method.

The _deprecate_context2lang and auto_translate_inputs decorators are used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


454-474: Ensure proper deprecation handling and translation in tldr method.

The _deprecate_context2lang and auto_translate_inputs decorators are used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


499-520: Ensure proper deprecation handling and translation in extract_answer method.

The _deprecate_context2lang and auto_translate_inputs decorators are used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


550-574: Ensure proper deprecation handling and translation in select_answer method.

The _deprecate_context2lang and auto_translate_inputs decorators are used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


578-596: Ensure proper deprecation handling and translation in solve method.

The _deprecate_context2lang and auto_translate_inputs decorators are used correctly. Ensure that users are aware of the deprecation and update their code accordingly.


623-643: Ensure proper deprecation handling and translation in entails method.

The _deprecate_context2lang and auto_translate_inputs decorators are used correctly. Ensure that users are aware of the deprecation and update their code accordingly.

ovos_plugin_manager/templates/solvers.py Outdated Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 2ed15e3 and 12881cc.

Files selected for processing (1)
  • ovos_plugin_manager/templates/language.py (4 hunks)
Additional comments not posted (12)
ovos_plugin_manager/templates/language.py (12)

10-17: Enhance constructor documentation.

The constructor's docstring is clear and provides detailed information about the configuration dictionary. This improves readability and maintainability.


24-38: Good use of type annotations and docstrings.

The runtime_requirements method has been updated to include type annotations and a detailed docstring, which enhances clarity and type safety.


40-50: Enforce implementation of detect method in subclasses.

The detect method is now abstract, ensuring that subclasses must implement this method. The detailed docstring improves usability.


52-62: Enforce implementation of detect_probs method in subclasses.

The detect_probs method is now abstract, ensuring that subclasses must implement this method. The detailed docstring improves usability.


64-72: Consider making available_languages an abstract method.

The available_languages property is currently not abstract but includes a TODO comment suggesting it should be made abstract in future releases. This will enforce implementation in subclasses.

Ensure that the change to make this method abstract is tracked and implemented in future releases.


78-85: Enhance constructor documentation.

The constructor's docstring is clear and provides detailed information about the configuration dictionary. This improves readability and maintainability.


95-109: Good use of type annotations and docstrings.

The runtime_requirements method has been updated to include type annotations and a detailed docstring, which enhances clarity and type safety.


111-123: Enforce implementation of translate method in subclasses.

The translate method is now abstract, ensuring that subclasses must implement this method. The detailed docstring improves usability.


125-136: Enhance translate_dict method documentation.

The translate_dict method's docstring is clear and provides detailed information about the expected argument types and return types. This improves usability and maintainability.


146-157: Enhance translate_list method documentation.

The translate_list method's docstring is clear and provides detailed information about the expected argument types and return types. This improves usability and maintainability.


167-175: Consider making available_languages an abstract method.

The available_languages property is currently not abstract but includes a TODO comment suggesting it should be made abstract in future releases. This will enforce implementation in subclasses.

Ensure that the change to make this method abstract is tracked and implemented in future releases.


179-188: Consider making supported_translations an abstract method.

The supported_translations method is currently not abstract but includes a TODO comment suggesting it should be made abstract in future releases. This will enforce implementation in subclasses.

Ensure that the change to make this method abstract is tracked and implemented in future releases.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 12881cc and 74cc917.

Files selected for processing (1)
  • ovos_plugin_manager/templates/solvers.py (2 hunks)
Additional context used
Ruff
ovos_plugin_manager/templates/solvers.py

160-160: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


170-170: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


307-307: Undefined name kwargs

(F821)


364-364: Do not use bare except

(E722)

Additional comments not posted (13)
ovos_plugin_manager/templates/solvers.py (13)

19-49: LGTM! The _deprecate_context2lang decorator is well-implemented.

The function correctly handles the deprecation of the 'context' kwarg and replaces it with 'lang'.


52-118: LGTM! The auto_translate_inputs decorator is well-implemented.

The function correctly handles the translation of specified keyword arguments to the solver's default language.


124-141: LGTM! The __init__ method in AbstractSolver is well-implemented.

The method correctly initializes the class with the new translator and detector parameters.


144-159: LGTM! The sentence_split method is well-implemented.

The method correctly handles errors and includes a detailed docstring.


230-249: LGTM! The __init__ method in QuestionSolver is well-implemented.

The method correctly initializes the class with the new translator and detector parameters.


266-274: LGTM! The get_spoken_answer method is well-implemented.

The method correctly handles the new lang parameter.


278-291: LGTM! The stream_utterances method is well-implemented.

The method correctly handles the deprecation of the context parameter and includes the new lang parameter.


310-318: LGTM! The get_image method is well-implemented.

The method correctly handles the deprecation of the context parameter and includes the new lang parameter.


323-337: LGTM! The get_expanded_answer method is well-implemented.

The method correctly handles the deprecation of the context parameter and includes the new lang parameter.


374-390: LGTM! The visual_answer method is well-implemented.

The method correctly handles the deprecation of the context parameter, includes the new lang parameter, and integrates the translation logic.


393-420: LGTM! The spoken_answer method is well-implemented.

The method correctly handles the deprecation of the context parameter, includes the new lang parameter, and integrates the translation logic.


424-447: LGTM! The long_answer method is well-implemented.

The method correctly handles the deprecation of the context parameter, includes the new lang parameter, and integrates the translation logic.


461-469: **LGTM! The get_tldr method is

ovos_plugin_manager/templates/solvers.py Outdated Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Outdated Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Outdated Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Outdated Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Outdated Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 6

Outside diff range, codebase verification and nitpick comments (2)
ovos_plugin_manager/templates/solvers.py (2)

278-284: Add docstring for get_spoken_answer method.

The get_spoken_answer method lacks a detailed docstring. Consider adding a docstring to describe the method's functionality, parameters, and return value.

@abc.abstractmethod
def get_spoken_answer(self, query: str, lang: Optional[str] = None) -> str:
    """
    Obtain the spoken answer for a given query.

    :param query: The query text.
    :param lang: Optional language code.
    :return: The spoken answer as a text response.
    """
    raise NotImplementedError

313-319: Add docstring for get_image method.

The get_image method lacks a detailed docstring. Consider adding a docstring to describe the method's functionality, parameters, and return value.

@_deprecate_context2lang()
def get_image(self, query: str, lang: Optional[str] = None) -> Optional[str]:
    """
    Get the path or URL to an image associated with the query.

    :param query: The query text
    :param lang: Optional language code.
    :return: The path or URL to a single image.
    """
    return None
Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 74cc917 and 9ef365d.

Files selected for processing (1)
  • ovos_plugin_manager/templates/solvers.py (2 hunks)
Additional context used
Ruff
ovos_plugin_manager/templates/solvers.py

173-173: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


183-183: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


359-359: Do not use bare except

(E722)

Additional comments not posted (8)
ovos_plugin_manager/templates/solvers.py (8)

157-172: Add error handling for sentence tokenization.

The sentence_split method lacks error handling. Consider adding a try-except block to handle potential errors during sentence tokenization.

def sentence_split(text: str, max_sentences: int = 25) -> List[str]:
    """
    Split text into sentences.

    :param text: Input text.
    :param max_sentences: Maximum number of sentences to return.
    :return: List of sentences.
    """
    try:
        return flatten_list([sentence_tokenize(t)
                             for t in text.split("\n")])[:max_sentences]
    except Exception as e:
        LOG.exception(f"Error in sentence_split: {e}")
        return [text]

173-182: Enhance error handling in detect_language method.

The detect_language method lacks error handling. Consider adding a try-except block to handle potential errors during language detection.

def detect_language(self, text: str) -> str:
    """
    Detect the language of the input text.

    :param text: Input text.
    :return: Detected language code.
    """
    try:
        return self.detector.detect(text)
    except Exception as e:
        LOG.error(f"Error in detect_language: {e}")
        return self.default_lang
Tools
Ruff

173-173: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


183-201: Enhance error handling in translate method.

The translate method lacks error handling. Consider adding a try-except block to handle potential errors during translation.

def translate(self, text: str,
              target_lang: Optional[str] = None,
              source_lang: Optional[str] = None) -> str:
    """
    Translate text from source_lang to target_lang.

    :param text: Input text.
    :param target_lang: Target language code.
    :param source_lang: Source language code.
    :return: Translated text.
    """
    source_lang = source_lang or self.detect_language(text)
    target_lang = target_lang or self.default_lang
    if source_lang.split("-")[0] == target_lang.split("-")[0]:
        return text  # skip translation
    try:
        return self.translator.translate(text,
                                         target=target_lang,
                                         source=source_lang)
    except Exception as e:
        LOG.error(f"Error in translate: {e}")
        return text
Tools
Ruff

183-183: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


203-216: Enhance error handling in translate_list method.

The translate_list method lacks error handling. Consider adding a try-except block to handle potential errors during translation.

def translate_list(self, data: List[str],
                   target_lang: Optional[str] = None,
                   source_lang: Optional[str] = None) -> List[str]:
    """
    Translate a list of strings from source_lang to target_lang.

    :param data: List of strings.
    :param target_lang: Target language code.
    :param source_lang: Source language code.
    :return: List of translated strings.
    """
    try:
        return self.translator.translate_list(data,
                                              lang_tgt=target_lang,
                                              lang_src=source_lang)
    except Exception as e:
        LOG.error(f"Error in translate_list: {e}")
        return data

218-231: Enhance error handling in translate_dict method.

The translate_dict method lacks error handling. Consider adding a try-except block to handle potential errors during translation.

def translate_dict(self, data: Dict[str, str],
                   target_lang: Optional[str] = None,
                   source_lang: Optional[str] = None) -> Dict[str, str]:
    """
    Translate a dictionary of strings from source_lang to target_lang.

    :param data: Dictionary of strings.
    :param target_lang: Target language code.
    :param source_lang: Source language code.
    :return: Dictionary of translated strings.
    """
    try:
        return self.translator.translate_dict(data,
                                              lang_tgt=target_lang,
                                              lang_src=source_lang)
    except Exception as e:
        LOG.error(f"Error in translate_dict: {e}")
        return data

302-310: Fix the undefined kwargs variable.

The get_data method references an undefined kwargs variable.

- return {"answer": _call_with_sanitized_kwargs(self.get_spoken_answer, query,
-                                              lang=lang, context=context)(query, **kwargs)}
+ return {"answer": _call_with_sanitized_kwargs(self.get_spoken_answer, query,
+                                              lang=lang, context=context)}

338-367: Fix the bare except block in search method.

The search method contains a bare except block, which is not recommended. Specify the exception type to ensure proper error handling.

- try:
-     data = _call_with_sanitized_kwargs(self.get_data, query,
-                                        lang=lang, context=context)
- except:
-     return {}
+ try:
+     data = _call_with_sanitized_kwargs(self.get_data, query,
+                                        lang=lang, context=context)
+ except Exception as e:
+     LOG.error(f"Error in search: {e}")
+     return {}
Tools
Ruff

359-359: Do not use bare except

(E722)


414-434: Add error handling in long_answer method.

The long_answer method lacks error handling. Consider adding a try-except block to handle potential errors during retrieval of long answers.

@_deprecate_context2lang()
@auto_translate(translate_keys=["query"])
def long_answer(self, query: str, lang: Optional[str] = None) -> List[dict]:
    """
    Retrieve a detailed list of steps to expand the answer.

    NOTE: "lang" assured to be in self.supported_langs,
        otherwise "query"  automatically translated to self.default_lang.
        If translations happens, the returned value of this method will also
        be automatically translated back

    :param query: The query text.
    :param lang: Optional language code.
    :return: A list of steps to elaborate on the answer, with each step containing a title, summary, and optional image.
    """
    try:
        steps = _call_with_sanitized_kwargs(self.get_expanded_answer, query, lang=lang)
        # use spoken_answer as last resort
        if not steps:
            summary = _call_with_sanitized_kwargs(self.get_spoken_answer, query, lang=lang)
            if summary:
                img = _call_with_sanitized_kwargs(self.get_image, query, lang=lang)

</blockquote></details>

</blockquote></details>

</details>

<!-- This is an auto-generated comment by CodeRabbit for review status -->

ovos_plugin_manager/templates/solvers.py Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Outdated Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Outdated Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Outdated Show resolved Hide resolved
make code more portable and easier to understand and maintain by using decorators for the autotranslation steps

move class properties to init method

add configurable language detector plugin and new util methods for translation and detection
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 9ef365d and 9155fe1.

Files selected for processing (2)
  • ovos_plugin_manager/templates/language.py (4 hunks)
  • ovos_plugin_manager/templates/solvers.py (2 hunks)
Files skipped from review as they are similar to previous changes (1)
  • ovos_plugin_manager/templates/language.py
Additional context used
Ruff
ovos_plugin_manager/templates/solvers.py

173-173: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


183-183: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


359-359: Do not use bare except

(E722)

Additional comments not posted (14)
ovos_plugin_manager/templates/solvers.py (14)

19-49: LGTM!

The _deprecate_context2lang decorator effectively handles the deprecation of the context kwarg and replaces it with lang. The changes are clear and maintain backward compatibility.


52-118: LGTM!

The auto_translate decorator is well-implemented, ensuring that specified kwargs are translated to the default language and handling different data types effectively. The changes enhance the modularity and usability of the code.


121-132: LGTM!

The _call_with_sanitized_kwargs function effectively ensures compatibility for plugins with old signatures by inspecting function parameters and adjusting kwargs accordingly. The changes are clear and maintain backward compatibility.


137-154: LGTM!

The updates to the __init__ method in AbstractSolver improve type safety and flexibility in language processing by including type annotations for translator and detector. The changes enhance the overall robustness of the class.


157-172: LGTM!

The sentence_split method effectively splits text into sentences and includes error handling to manage potential issues. The changes enhance the reliability of the method.


173-182: Enhance error handling in detect_language method.

The detect_language method lacks error handling. Consider adding a try-except block to handle potential errors during language detection.

def detect_language(self, text: str) -> str:
    """
    Detect the language of the input text.

    :param text: Input text.
    :return: Detected language code.
    """
    try:
        return self.detector.detect(text)
    except Exception as e:
        LOG.error(f"Error in detect_language: {e}")
        return self.default_lang
Tools
Ruff

173-173: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


183-201: Enhance error handling in translate method.

The translate method lacks error handling. Consider adding a try-except block to handle potential errors during translation.

def translate(self, text: str,
              target_lang: Optional[str] = None,
              source_lang: Optional[str] = None) -> str:
    """
    Translate text from source_lang to target_lang.

    :param text: Input text.
    :param target_lang: Target language code.
    :param source_lang: Source language code.
    :return: Translated text.
    """
    source_lang = source_lang or self.detect_language(text)
    target_lang = target_lang or self.default_lang
    if source_lang.split("-")[0] == target_lang.split("-")[0]:
        return text  # skip translation
    try:
        return self.translator.translate(text,
                                         target=target_lang,
                                         source=source_lang)
    except Exception as e:
        LOG.error(f"Error in translate: {e}")
        return text
Tools
Ruff

183-183: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


203-216: Enhance error handling in translate_list method.

The translate_list method lacks error handling. Consider adding a try-except block to handle potential errors during translation.

def translate_list(self, data: List[str],
                   target_lang: Optional[str] = None,
                   source_lang: Optional[str] = None) -> List[str]:
    """
    Translate a list of strings from source_lang to target_lang.

    :param data: List of strings.
    :param target_lang: Target language code.
    :param source_lang: Source language code.
    :return: List of translated strings.
    """
    try:
        return self.translator.translate_list(data,
                                              lang_tgt=target_lang,
                                              lang_src=source_lang)
    except Exception as e:
        LOG.error(f"Error in translate_list: {e}")
        return data

218-231: Enhance error handling in translate_dict method.

The translate_dict method lacks error handling. Consider adding a try-except block to handle potential errors during translation.

def translate_dict(self, data: Dict[str, str],
                   target_lang: Optional[str] = None,
                   source_lang: Optional[str] = None) -> Dict[str, str]:
    """
    Translate a dictionary of strings from source_lang to target_lang.

    :param data: Dictionary of strings.
    :param target_lang: Target language code.
    :param source_lang: Source language code.
    :return: Dictionary of translated strings.
    """
    try:
        return self.translator.translate_dict(data,
                                              lang_tgt=target_lang,
                                              lang_src=source_lang)
    except Exception as e:
        LOG.error(f"Error in translate_dict: {e}")
        return data

243-272: LGTM!

The updates to the __init__ method in QuestionSolver improve type safety and flexibility in language processing by including type annotations for translator and detector. The changes also handle caching effectively.


288-297: Add error handling in stream_utterances method.

The stream_utterances method lacks error handling. Consider adding a try-except block to handle potential errors during streaming.

@_deprecate_context2lang()
def stream_utterances(self, query: str, lang: Optional[str] = None) -> Iterable[str]:
    """
    Stream utterances for the given query as they become available.

    :param query: The query text.
    :param lang: Optional language code.
    :return: An iterable of utterances.
    """
    try:
        ans = _call_with_sanitized_kwargs(self.get_spoken_answer, query, lang=lang)
        for utt in self.sentence_split(ans):
            yield utt
    except Exception as e:
        LOG.error(f"Error in stream_utterances: {e}")
        yield from []

323-334: Add error handling in get_expanded_answer method.

The get_expanded_answer method lacks error handling. Consider adding a try-except block to handle potential errors during retrieval of expanded answers.

@_deprecate_context2lang()
def get_expanded_answer(self, query: str, lang: Optional[str] = None) -> List[dict]:
    """
    Get an expanded list of steps to elaborate on the answer.

    :param query: The query text
    :param lang: Optional language code.
    :return: A list of dictionaries with each step containing a title, summary, and optional image.
    """
    try:
        return [{"title": query,
                 "summary": _call_with_sanitized_kwargs(self.get_spoken_answer, query, lang=lang),
                 "img": _call_with_sanitized_kwargs(self.get_image, query, lang=lang)}]
    except Exception as e:
        LOG.error(f"Error in get_expanded_answer: {e}")
        return []

370-384: Add error handling in visual_answer method.

The visual_answer method lacks error handling. Consider adding a try-except block to handle potential errors during retrieval of visual answers.

@_deprecate_context2lang()
@auto_translate(translate_keys=["query"])
def visual_answer(self, query: str, lang: Optional[str] = None) -> str:
    """
    Retrieve the image associated with the query with automatic translation and caching.

    NOTE: "lang" assured to be in self.supported_langs,
        otherwise "query"  automatically translated to self.default_lang.
        If translations happens, the returned value of this method will also
        be automatically translated back

    :param query: The query text.
    :param lang: Optional language code.
    :return: The path or URL to the image.
    """
    try:
        return _call_with_sanitized_kwargs(self.get_image, query, lang=lang)
    except Exception as e:
        LOG.error(f"Error in visual_answer: {e}")
        return ""

387-411: Add error handling in spoken_answer method.

The spoken_answer method lacks error handling. Consider adding a try-except block to handle potential errors during retrieval of spoken answers.

@_deprecate_context2lang()
@auto_translate(translate_keys=["query"])
def spoken_answer(self, query: str, lang: Optional[str] = None) -> str:
    """
    Retrieve the spoken answer for the query with automatic translation and caching.

    NOTE: "lang" assured to be in self.supported_langs,
        otherwise "query"  automatically translated to self.default_lang.
        If translations happens, the returned value of this method will also
        be automatically translated back

    :param query: The query text.
    :param lang: Optional language code.
    :return: The spoken answer as a text response.
    """
    # get answer
    if self.enable_cache and query in self.spoken_cache:
        # read from cache
        summary = self.spoken_cache[query]
    else:
        try:
            summary = _call_with_sanitized_kwargs(self.get_spoken_answer, query, lang=lang)
            # save to cache
            if self.enable_cache:
                self.spoken_cache[query] = summary
                self.spoken_cache.store()
        except Exception as e:
            LOG.error(f"Error in spoken_answer: {e}")
            summary = ""
    return summary

@JarbasAl JarbasAl marked this pull request as draft July 25, 2024 00:24
@JarbasAl JarbasAl marked this pull request as ready for review July 25, 2024 01:32
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Outside diff range, codebase verification and nitpick comments (1)
ovos_plugin_manager/templates/solvers.py (1)

240-240: Consider potential memory leaks with lru_cache.

The use of lru_cache on methods detect_language and translate in ovos_plugin_manager/templates/solvers.py can lead to memory leaks. Additionally, there is another occurrence of lru_cache on the load_stream_extractors function in ovos_plugin_manager/ocp.py which should also be reviewed. Ensure that the cache size is appropriate and consider adding cache invalidation logic if necessary.

  • ovos_plugin_manager/templates/solvers.py: detect_language method and translate method.
  • ovos_plugin_manager/ocp.py: load_stream_extractors function.
Analysis chain

Consider potential memory leaks with lru_cache.

The use of lru_cache on methods detect_language and translate can lead to memory leaks. Ensure that the cache size is appropriate and consider adding cache invalidation logic if necessary.

Also applies to: 250-250

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of lru_cache and check for potential memory leaks.

# Test: Search for the usage of lru_cache. Expect: Only occurances of lru_cache on methods detect_language and translate.
rg --type python '@lru_cache' -A 5

Length of output: 65


Script:

#!/bin/bash
# Description: Verify the usage of lru_cache and check for potential memory leaks.

# Test: Search for the usage of lru_cache. Expect: Only occurrences of lru_cache on methods detect_language and translate.
rg '@lru_cache' -A 5 --glob '*.py'

Length of output: 1258

Tools
Ruff

240-240: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 9155fe1 and 29869b0.

Files selected for processing (3)
  • ovos_plugin_manager/templates/solvers.py (1 hunks)
  • requirements/test.txt (1 hunks)
  • test/unittests/test_solver.py (1 hunks)
Additional context used
Ruff
test/unittests/test_solver.py

148-148: Local variable ans is assigned to but never used

Remove assignment to unused variable ans

(F841)

ovos_plugin_manager/templates/solvers.py

240-240: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


250-250: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


428-428: Do not use bare except

(E722)

Additional comments not posted (8)
requirements/test.txt (1)

5-5: Dependency addition looks good.

The addition of ovos-classifiers to the test requirements is appropriate.

test/unittests/test_solver.py (1)

144-144: LGTM!

The addition of the lang argument to the spoken_answer method call improves flexibility.

ovos_plugin_manager/templates/solvers.py (6)

19-49: LGTM!

The _deprecate_context2lang decorator is well-implemented and includes appropriate logging for deprecation warnings.


75-126: LGTM!

The auto_translate decorator is well-implemented and includes comprehensive logic to handle various data types for translation.


129-160: LGTM!

The auto_detect_lang decorator is well-implemented and includes comprehensive logic to detect language from both kwargs and args.


179-198: LGTM!

The enhancements to the __init__ method, including the addition of detector and translator parameters and improved error handling, are well-implemented.


310-344: LGTM!

The enhancements to the __init__ method and the addition of the lang parameter in various methods are well-implemented.


511-546: LGTM!

The enhancements to the CorpusSolver class, including the addition of the lang parameter and improved error handling, are well-implemented.

test/unittests/test_solver.py Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Show resolved Hide resolved
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 29869b0 and 8b1004e.

Files selected for processing (2)
  • ovos_plugin_manager/templates/solvers.py (1 hunks)
  • test/unittests/test_solver.py (3 hunks)
Additional context used
Ruff
test/unittests/test_solver.py

150-150: Local variable ans is assigned to but never used

Remove assignment to unused variable ans

(F841)

ovos_plugin_manager/templates/solvers.py

204-204: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


214-214: Use of functools.lru_cache or functools.cache on methods can lead to memory leaks

(B019)


392-392: Do not use bare except

(E722)

Additional comments not posted (5)
test/unittests/test_solver.py (4)

150-150: Remove unused variable assignment.

The ans variable is assigned but never used. This should be removed to clean up the code.

-        ans = solver.spoken_answer("not english", lang="unk")
+        solver.spoken_answer("not english", lang="unk")
Tools
Ruff

150-150: Local variable ans is assigned to but never used

Remove assignment to unused variable ans

(F841)


405-426: Tests for auto_translate decorator look good!

The tests for the auto_translate decorator cover both scenarios: with and without translation. The mock translation is correctly set up, and the assertions validate the expected behavior.


428-451: Tests for auto_detect_lang decorator look good!

The tests for the auto_detect_lang decorator cover both scenarios: with and without explicitly provided language. The mock language detection is correctly set up, and the assertions validate the expected behavior.


453-472: Tests for _deprecate_context2lang decorator look good!

The tests for the _deprecate_context2lang decorator cover both scenarios: with and without the context argument. The assertions validate the expected behavior of the decorator.

ovos_plugin_manager/templates/solvers.py (1)

609-628: LGTM!

The changes in the extract_answer method align with the new function signature and the decorators ensure proper handling of the lang argument.

ovos_plugin_manager/templates/solvers.py Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Show resolved Hide resolved
ovos_plugin_manager/templates/solvers.py Show resolved Hide resolved
@JarbasAl JarbasAl merged commit dfaf818 into dev Jul 25, 2024
8 checks passed
@JarbasAl JarbasAl deleted the refactor/decorators branch July 25, 2024 01:43
@github-actions github-actions bot mentioned this pull request Sep 2, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working refactor code improvements with no functional changes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant