Skip to content

Curation of resources for LLM research, screened by @tongyx361 to ensure high quality and accompanied with elaborately-written concise descriptions to help readers get the gist as quickly as possible.

License

Notifications You must be signed in to change notification settings

tongyx361/Awesome-LLM-Research

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

17 Commits
 
 
 
 

Repository files navigation

Awesome LLM Research

Curation of resources for LLM research, screened by @tongyx361 to ensure high quality and accompanied with elaborately-written concise descriptions to help readers get the gist as quickly as possible.

Awesome License: MIT

🐱 GitHub | 📝 Notion (Interactable) | 🐦 X(Twitter) | 🐶 Zhihu(知乎)

✨ Featured by:

  • Theory & practice comprehensive introductory materials.
  • Classic/high-quality information sources.
  • Latest hot-spot information sources.

📊 There is also an interactable (i.e. sort / filter / search) version of the following table.

📥 You can subscribe to our updates in the following ways:

📢 If you have any suggestions, please don't hesitate to

Link Abstract Description Language Modality Update Cycle Type
国立台湾大学: 李宏毅机器学习 - CS自学指南 Basic theory and fundamental works of Deep Learning Lectures from different years have different focuses, e.g. 2023 focuses on LLM. EN(Text) ZH(Speech) Speech Text Code Year Basic
Introduction - Hugging Face NLP Course Basic NLP practice (based on HuggingFace ecosystem) HuggingFace is so accessible that its success is a given (but this also comes with some hidden price for developers). EN ZH … Text Code Dynamic Basic
Yao Fu’s Blog Fundamental research topics walkthrough Such as emergent abilities, reasoning, long-context modeling. EN Text Months Fundamental
Transformer Math 101 | EleutherAI Blog Transformer-related math estimation - Basic Basic arithmetic about Transformer-based models. EN Text None Basic
分析transformer模型的参数量、计算量、中间激活、KV cache - 知乎 Transformer-related math estimation - Mediate Detailed analysis of calculations in Transformer-based model. ZH Text None Basic
紫气东来 - 知乎 Specific engineering details Such as inference and training frameworks. ZH Text Weeks Practical
GitHub - liguodongiot/llm-action Engineering detail summaries Summarizing AI engineering techniques, such as inference, parallel computing, etc. ZH Text Days Practical
微信公众号:大猿搬砖简记 Illustrated source code (e.g. vLLM, CUDA) and algorithms (e.g. FlashAttention) ZH Text Weeks Practical
游凯超 - 知乎 Infrastructure-level engineering details Such as CUDA, NCCL, torch.compile and other side infrastructures like Docker, etc. ZH Text Days Practical
Alignment Guidebook - Notion Introduction to LLM Alignment (SFT + RL) EN Text Dynamic Basic
Spinning Up in Deep RL! — Spinning Up documentation Basic Deep RL EN Text Code
None Basic
科学空间|Scientific Spaces Blogs combining graceful theories and solid experiments Blogs by Jianlin Su (苏剑林), the author of RoPE (de facto standard of positional encoding now), versed in math and ML theory while not unfamiliar with experiments and practice. ZH Text Weeks Fundamental
Research OpenAI research blogs “We keep re-discovering what OpenAI discovered five years ago.” EN Text Months Fundamental
Research \ Anthropic Anthropic research blogs EN Text Months Fundamental
Transformer Circuits Thread Amazingly insightful and open Anthropic interpretability team research blogs EN Text Month Fundamental
E.g. [2312.11805] Gemini: A Family of Highly Capable Multimodal Models LLM technical reports Such technical reports, while usually not very detailed, often do reveal some important details of SotA LLMs. EN Text Months Fundamental
Hazy Research Blogs of pioneer visions Blogs from Hazy Research led by Christopher Ré @ Stanford (one of the best NLP&AI research groups around the world). EN Text Months Fundamental
Ilya 30u30 Short reading list to understand the fundamentals of the AI today, said to be from Ilya. Not the most frontier and not the most suitable for research starters, but really fundamental for essential understanding. EN Text None Fundamental
FAI-Seminar High-quality talks (largely contributed by Yao class alumna) ZH Speech Text Week Trending
Cool Papers - Immersive Paper Discovery Daily arXiv paper & Kimi interaction EN Text Day Trending
Daily Papers - Hugging Face The most popular paper selection on Twitter. EN Text Day Trending
微信公众号: SparksofAGI Individual paper selection, some of which common popular paper collections might not notice Selected by Jianbo Dai (戴建波)* (senior researcher at Huawei). ZH Text Weeks Trending
微信公众号: AINLP Curations of other AI 微信公众号:s ZH Text Day Trending
中文 AI 媒体四大顶号:机器之心新智元量子位夕小瑶科技说 Popular paper selection ZH Text Day Trending
微信公众号: arXiv 每日学术速递 arXiv paper from broader domains ZH Text Day Auxiliary
微信公众号: AI 前线 Various AI news (not limited to research) ZH Text Day Auxiliary
Video channel Song Zhao (YouTube / BiliBili) Various practical academic-relevant affairs (e.g. paper submission, job choices) A little “abstract” though … ZH Speech Text Weeks Auxiliary

About

Curation of resources for LLM research, screened by @tongyx361 to ensure high quality and accompanied with elaborately-written concise descriptions to help readers get the gist as quickly as possible.

Topics

Resources

License

Stars

Watchers

Forks