Code

Token rotation#987

Code

Open

vshampor wants to merge 17 commits intoopenvinotoolkit:masteropenvinotoolkit/openvino.genai:masterfrom vshampor:token_rotationvshampor/openvino.genai:token_rotationCopy head branch name to clipboard

+1,079-151

Commits

Commits on Jan 10, 2025

• Implement token rotation

  vshampor

  committed
  2c54717

  Copy full SHA for 2c54717

• Add better test IDs

  vshampor

  committed
  09ab96f

  Copy full SHA for 09ab96f

• Remove debug print

  vshampor

  committed
  6436be0

  Copy full SHA for 6436be0

• Fill rotation coefficients before current forward step

  vshampor

  committed
  7d3b893

  Copy full SHA for 7d3b893

• Do not download whisper datasets during collection

  vshampor

  committed
  c4339c2

  Copy full SHA for c4339c2

• Refactor coefficient filling, do not evict/rotate at prefill

  vshampor

  committed
  518e4c8

  Copy full SHA for 518e4c8

• Fix long_prompts.txt

  vshampor

  committed
  2125f85

  Copy full SHA for 2125f85

• Refactor to deltas + LUT

  vshampor

  committed
  ded7fcf

  Copy full SHA for ded7fcf

• Make LUT smaller

  vshampor

  committed
  302406b

  Copy full SHA for 302406b

• Debug prints

  vshampor

  committed
  b6fa966

  Copy full SHA for b6fa966

• Fix bugs

  vshampor

  committed
  6f43d44

  Copy full SHA for 6f43d44

• Add shape to rotation deltas

  vshampor

  committed
  f72ad37

  Copy full SHA for f72ad37

• Remove debug prints, use per-block rotation

  vshampor

  committed
  70fa762

  Copy full SHA for 70fa762

• Add tests

  vshampor

  committed
  466a75c

  Copy full SHA for 466a75c

• Fix max seq len

  vshampor

  committed
  39be9d0

  Copy full SHA for 39be9d0

• Fix rebase

  vshampor

  committed
  0553946

  Copy full SHA for 0553946

• Fix test

  vshampor

  committed
  40b0225

  Copy full SHA for 40b0225