Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add custom resize_token_embeddings method to OLMoForCausalLM (#491) #501

Merged
merged 18 commits into from
Apr 2, 2024

Commits on Mar 13, 2024

  1. adds custom resize_token_embeddings method

    This commit introduces the `resize_token_embeddings` function for the
    `OLMoForCausalLM` class. The function is based on the implementation
    from the Hugging Face Transformers library, with modifications to
    accommodate the specific requirements of the OLMo model.
    
    The `resize_token_embeddings` function allows resizing the input token
    embeddings matrix of the model when the number of tokens differs from
    the model's `embedding_size` configuration. It updates the `embedding_size`
    attribute in both the model configuration and the model itself, ensuring
    consistency after resizing.
    
    The function also handles tying the weights of the input and output
    embeddings if the model supports weight tying.
    
    Attribution:
    The implementation of this function is inspired by and adapted from
    the `resize_token_embeddings` function in the Hugging Face Transformers
    library (https://github.com/huggingface/transformers). The original code
    is licensed under the Apache License 2.0.
    
    Modifications:
    - Updated the function to use the `embedding_size` attribute instead of
      `vocab_size` to align with the OLMo model configuration.
    - Adjusted the docstring and comments to match the OLMo model's terminology
      and requirements.
    djliden committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    5b4e866 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    4e4ff1f View commit details
    Browse the repository at this point in the history
  3. fixes indentation

    djliden committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    cf774dc View commit details
    Browse the repository at this point in the history
  4. updates changelog

    djliden committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    17149cf View commit details
    Browse the repository at this point in the history
  5. Configuration menu
    Copy the full SHA
    8f8e8c2 View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. Configuration menu
    Copy the full SHA
    36140c4 View commit details
    Browse the repository at this point in the history
  2. fixes import ordering

    djliden committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    1dedce9 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    2da225e View commit details
    Browse the repository at this point in the history

Commits on Mar 16, 2024

  1. Configuration menu
    Copy the full SHA
    ba72c66 View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2024

  1. Configuration menu
    Copy the full SHA
    04065f0 View commit details
    Browse the repository at this point in the history
  2. updates comment

    clarifies difference between the updated method and the base class method
    djliden committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    f395a55 View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. Configuration menu
    Copy the full SHA
    ee9e9ff View commit details
    Browse the repository at this point in the history
  2. minor fixes

    switching warning to log.warning and fixes comment indent
    djliden committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    e1ec4b3 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. formats with ruff

    djliden committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    1595326 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e186f5b View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. Configuration menu
    Copy the full SHA
    aa5687d View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2024

  1. Configuration menu
    Copy the full SHA
    3cdfcde View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    81d30d4 View commit details
    Browse the repository at this point in the history