Code

TriePagedAttentionCache - 2#628

Code

Closed

renxida wants to merge 18 commits intonod-ai:mainnod-ai/shark-ai:mainfrom renxida:trie-2renxida/shark-ai:trie-2Copy head branch name to clipboard

+1,052-81

Commits

Commits on Dec 2, 2024

• initial PageAllocation handle implementation

  renxida

  committed
  b8864a7

• docstring udpates

  renxida

  committed
  947a7b5

• make pages a property

  renxida

  committed
  19aa8fd

• another small fix for efficiency. Thanks stephen!

  renxida

  committed
  af4d84b

• add unit tests

  renxida

  committed
  f184c32

• avoid magic numbers

  renxida

  committed
  13af913

• very extensive concurrency testing

  renxida

  committed
  c3c52d4

• prettify the other part too

  renxida

  committed
  cd923b5

• consistent naming for case_name

  renxida

  committed
  09d83e6

• add trie implementation

  renxida

  committed
  970dbc1

• add unit tests and fix numerous small problems

  renxida

  committed
  0503449

• all tests passing

  renxida

  committed
  558da42

• all but publishing working

  renxida

  committed
  99ff2c6

• naming consistency Page -> Paged

  renxida

  committed
  9e1f9c7

• passing all tests now

  renxida

  committed
  bf5c8ab

• name and documentation update publish_pages -> publish_pages_for_tokens

  renxida

  committed
  895e4d5

• undo accidental edit of messages.py

  renxida

  committed
  0caeb23

• remove not-very-useful test case

  renxida

  committed
  74adaab