Code

Add llama3#10

Code

Merged

arvid220u merged 6 commits intomainanysphere/tiktoken-rs:mainfrom add-llama3anysphere/tiktoken-rs:add-llama3Copy head branch name to clipboardOct 12, 2024

+128,126-2

Commits

Commits on Oct 10, 2024

• add llama3 to openai_public.rs

  arvid220u

  committed
  99f4f48

  Copy full SHA for 99f4f48

• add llama3

  arvid220u

  committed
  209c36d

  Copy full SHA for 209c36d

Commits on Oct 12, 2024

• ws

  tmm1

  committed
  0dd67a3

  Copy full SHA for 0dd67a3

• fix sha256

  tmm1

  committed
  e38c8ee

  Copy full SHA for e38c8ee

• fix mem test

  tmm1

  committed
  b67590d

  Copy full SHA for b67590d

• add llama tests (verified with official implementation)

  arvid220u

  committed
  ce07710

  Copy full SHA for ce07710