Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add block with Llama-like implementations #346

Merged
merged 13 commits into from
Nov 2, 2023
Merged

Add block with Llama-like implementations #346

merged 13 commits into from
Nov 2, 2023

Commits on Oct 25, 2023

  1. Add OlmoLlamaBlock

    @2015aroras
    2015aroras committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    a57a050 View commit details
    Browse the repository at this point in the history

  2. Add config for using the Llama block

    @2015aroras
    2015aroras committed Oct 25, 2023
    Configuration menu
    Copy the full SHA
    53d68c8 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. Fix inverted context mask in attention impl, further clean impl

    @2015aroras
    2015aroras committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    6c4b8e1 View commit details
    Browse the repository at this point in the history

  2. Run black

    @2015aroras
    2015aroras committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    7743b0f View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. Add rope precision

    @2015aroras
    2015aroras committed Oct 27, 2023
    Configuration menu
    Copy the full SHA
    07eb67c View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2023

  1. Add missing type cast in updated RoPE

    @2015aroras
    2015aroras committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    cebdbe5 View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2023

  1. Move some attention bias logic out of Olmo to use in OlmoLlamaBlock

    @2015aroras
    2015aroras committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    b8938d5 View commit details
    Browse the repository at this point in the history

  2. Run black

    @2015aroras
    2015aroras committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    2da1a0a View commit details
    Browse the repository at this point in the history

  3. Store buffer cache in OlmoLlamaBlock

    @2015aroras
    2015aroras committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    57caf05 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into shanea/llama-block

    @2015aroras
    2015aroras committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    9f0d165 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2023

  1. Merge branch 'main' into shanea/llama-block

    @2015aroras
    2015aroras committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    4b9aec6 View commit details
    Browse the repository at this point in the history

  2. Change RoPE precision to a boolean config

    @2015aroras
    2015aroras committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    7d443b6 View commit details
    Browse the repository at this point in the history

  3. Add activation checkpointing to Llama block

    @2015aroras
    2015aroras committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    6b1a77c View commit details
    Browse the repository at this point in the history