Code

Fix 8K decode latency jump issue. #55

Code

Merged

lcskrishna merged 1 commit intomainROCm/vllm:mainfrom cl/fix-8k-issueROCm/vllm:cl/fix-8k-issueCopy head branch name to clipboardJun 19, 2024

+2-2

Commits

Commits on Jun 19, 2024

• fix 8k issue by changing max-context/seq len to 32k

  lcskrishna

  committed
  3c86a03

  Copy full SHA for 3c86a03