Skip to content

Issues: ReaLLMASIC/nanoGPT

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

87 Open 14 Closed
87 Open 14 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

Add Softcapping implementation
#397 opened Feb 13, 2025 by gkielian
Add visualizations for attention
#389 opened Feb 6, 2025 by gkielian
Implement Early Exit
#388 opened Feb 5, 2025 by gkielian
@gkielian
Add Delta Attention variation
#376 opened Jan 27, 2025 by gkielian
Add support for Maximal Update of Parameters - MUP
#372 opened Jan 23, 2025 by gkielian
@gkielian
Learned KV Cache Prefix
#369 opened Jan 21, 2025 by klei22
Add CJK IPA Tokenizer
#367 opened Jan 19, 2025 by gkielian
@gkielian @klei22
@Zhao-Yuting @xinyixuu
Create a variation with mixins of RNN / RWKV style components
#365 opened Jan 19, 2025 by gkielian
Add support for HookPoints
#364 opened Jan 19, 2025 by gkielian
Add Semantic Factorization
#363 opened Jan 19, 2025 by gkielian
10 tasks done
1
@gkielian
1
Add a Reweighted Cross-Entropy Loss for Improved Handling of Zipfian Distributions
#357 opened Jan 10, 2025 by gkielian
Add means for gauging average average loss per token and token subsets
#356 opened Jan 10, 2025 by gkielian
Add model variation for Gaussian Attention
#355 opened Jan 10, 2025 by gkielian
Add model variations for Megalodon innovations
#354 opened Jan 10, 2025 by gkielian
Add model variation for RPC attention
#353 opened Jan 10, 2025 by gkielian
Add BAdam optimizer option
#351 opened Jan 9, 2025 by gkielian
Add compatibility with CREMA-D Dataset Audio Dataset anything centered around new datasets
#349 opened Jan 6, 2025 by gkielian
@klei22
Add compatibility with Textrol Speech Dataset Audio Dataset anything centered around new datasets
#348 opened Jan 6, 2025 by gkielian
@klei22
Add compatibility with DailyTalk dataset Audio Dataset anything centered around new datasets
#347 opened Jan 6, 2025 by gkielian
@klei22
Add separate ckpt saving for tracking ICL accuracy
#346 opened Jan 6, 2025 by gkielian
Add exploration of reduced initialization stddev and increased weight decay -- see if this helps generalizablity
#345 opened Jan 6, 2025 by gkielian
1
Add visualization of Intrinsic Dimension Per Layer
#344 opened Jan 6, 2025 by gkielian
Add visualization of Concept Vectors
#343 opened Jan 6, 2025 by gkielian
Add higher dimensional Semantic Position Encodings
#342 opened Jan 5, 2025 by gkielian
Add means for learned semantic position embeddings
#341 opened Jan 5, 2025 by gkielian
ProTip! What’s not been updated in a month: updated:<2025-01-19.