Code

Mccloskey/llama2 hf#17

Code

Merged

hunterjackson merged 18 commits intomainfacebookincubator/CP4M:mainfrom mccloskey/llama2-hffacebookincubator/CP4M:mccloskey/llama2-hfCopy head branch name to clipboardNov 6, 2023

+93,792-291

Commits

Commits on Oct 1, 2023

• Adding token counter and prompt builder class

  colinmccloskey

  committed
  6491ed2

Commits on Oct 3, 2023

• Local tokenizer

  colinmccloskey

  committed
  3ade3a0

• Adding token counter and prompt builder class

  colinmccloskey

  committed
  31dfcd5

• Local tokenizer

  colinmccloskey

  committed
  172c587

• Finishing token counting and adding tests

  colinmccloskey

  committed
  2a1fe09

• Merge branch 'mccloskey/llama2-hf' of https://github.com/facebookincubator/CP4M into mccloskey/llama2-hf

  colinmccloskey

  committed
  2c3fb97

• deleting old tokenizer.json

  colinmccloskey

  committed
  ab41ff3

Commits on Oct 4, 2023

• Fixing to truncate oldest messages not newest

  colinmccloskey

  committed
  c802798

• Formatting, removing comments, removing now-duplicated system message test

  colinmccloskey

  committed
  a70d5a3

Commits on Oct 25, 2023

• Fixing problem where new tokenizer, etc was created each time

  Colin1104

  committed
  d8441e1

• Removing unused files and functions

  Colin1104

  committed
  422367d

• Update OpenAIPluginTest to have default system message in ordered correctly test

  Colin1104

  committed
  e640b6a

• Deleting comments/unused imports

  Colin1104

  committed
  068752d

• Delete unused line

  Colin1104

  committed
  1bdd4a4

Commits on Nov 1, 2023

• Changing pruneMessages to not create middle array list and return optional

  Colin1104

  committed
  18541c7

• Removing comments

  colinmccloskey

  committed
  bea6e96

Commits on Nov 2, 2023

• Addressing comments

  colinmccloskey

  committed
  6342b3d

• Merge branch 'main' into mccloskey/llama2-hf

  colinmccloskey

  authored
  0154e86