Skip to content

Issues: flashinfer-ai/flashinfer

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

73 Open 137 Closed
73 Open 137 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Possible Bug in chain_speculative_sampling: output_emitted_token_num Exceeds Expected Limit
#879 opened Feb 19, 2025 by JaeminK
The current _append_paged_kv_cache_kernel does not support k_head_dim being different from v_head_dim
#877 opened Feb 19, 2025 by qiyuxinlin
1
Prebuilt kernels not found, using JIT backend
#876 opened Feb 19, 2025 by rangehow
1
BatchPrefillWithRaggedKVCacheWrapper custom_mask doesn't work for small batch sizes when head_dim_qk != head_dim_vo
#860 opened Feb 17, 2025 by pankajroark
@yzh119
1
[Feature] Can BatchDecodeWithPagedKVCacheWrapper return attention scores to all tokens, not just logsumexp?
#838 opened Feb 14, 2025 by yawnzh
1
PrefillPlan tries to allocate more memory than float_workspace_size_in_bytes passed in.
#809 opened Feb 12, 2025 by rchardx
1
[BUG] attention/prefill.cuh(138): error: expression must have a constant value
#806 opened Feb 11, 2025 by haohaibo
4
Numerical stability issue in recent commits since 0.2.0 bug Something isn't working priority: high
#805 opened Feb 11, 2025 by rchardx
5
[Docs] TypeError: single_prefill_with_kv_cache() got an unexpected keyword argument 'use_fp16_qk_reduction'
#802 opened Feb 10, 2025 by sleepwalker2017
[Tracing Issue] Multi-head Latent Attention
#792 opened Feb 6, 2025 by yzh119
3 of 4 tasks
[Feature] Reuse JIT code path for building AOT wheel enhancement New feature or request
#791 opened Feb 6, 2025 by abcdabcd987
@abcdabcd987
6
Question about cascade inference
#789 opened Feb 5, 2025 by sleepwalker2017
11
[Tracking Issue] Performance with CUDA 12.8
#780 opened Feb 4, 2025 by yzh119
6 tasks
PyTorch 2.6 wheels
#771 opened Feb 1, 2025 by yzh119
1
read cached content
#750 opened Jan 24, 2025 by Mrxiangli
4
[Feature] Llama3.1 RoPE on the fly enhancement New feature or request
#746 opened Jan 21, 2025 by turboderp
@yzh119
BatchPrefillWithPagedKVCacheDispatched for sm90
#745 opened Jan 21, 2025 by tangcy98
1
[Tracking Issue] Checklist for 8-bit KV-Cache improments
#742 opened Jan 17, 2025 by yzh119
5 tasks
AttributeError: module 'flashinfer._kernels' has no attribute 'apply_rope_pos_ids_cos_sin_cache'
#741 opened Jan 17, 2025 by fergusfinn
1
[Feature] MiniMax-01 Lightning Attention
#739 opened Jan 16, 2025 by yzh119
4
Build from source fails with older versions of setuptools (<72.2.0) because of PosixPath types returned from .glob
#738 opened Jan 16, 2025 by fergusfinn
Flashinfer==0.2.0 precision error when tested on vLLM unit tests
#736 opened Jan 13, 2025 by Dr-Left
@yzh119
C++ benchmarks CMake error caused by enable_fp16 option in generate.py
#734 opened Jan 13, 2025 by rtxxxpro
[RFC]: Introducing ReproSpec for Strong Reproducibility in LLM Inference RFC Request for Comments
#733 opened Jan 11, 2025 by yzh119
1
ROPE problem in MLA decode kernel
#730 opened Jan 9, 2025 by lsw825
2
ProTip! Add no:assignee to see everything that’s not assigned.