Curation of resources for LLM research, screened by @tongyx361 to ensure high quality and accompanied with elaborately-written concise descriptions to help readers get the gist as quickly as possible.
🐱 GitHub | 📝 Notion (Interactable) | 🐦 X(Twitter) | 🐶 Zhihu(知乎)
✨ Featured by:
- Theory & practice comprehensive introductory materials.
- Classic/high-quality information sources.
- Latest hot-spot information sources.
📊 There is also an interactable (i.e. sort / filter / search) version of the following table.
📥 You can subscribe to our updates in the following ways:
- Follow the X(Twitter) account @tongyx361,
- Follow the Zhihu(知乎) account @天欲雪,
- Watch releases in this GitHub repository: upper right corner→Watch->Custom->Releases.
📢 If you have any suggestions, please don't hesitate to
- comment in the Notion page,
- reply to the X(Twitter) thread,
- post an issue in the GitHub repository,
- or E-mail Yuxuan Tong.
| Link | Abstract | Description | Language | Modality | Update Cycle | Type |
|---|---|---|---|---|---|---|
| 国立台湾大学: 李宏毅机器学习 - CS自学指南 | Basic theory and fundamental works of Deep Learning | Lectures from different years have different focuses, e.g. 2023 focuses on LLM. | EN(Text) ZH(Speech) | Speech Text Code | Year | Basic |
| Introduction - Hugging Face NLP Course | Basic NLP practice (based on HuggingFace ecosystem) | HuggingFace is so accessible that its success is a given (but this also comes with some hidden price for developers). | EN ZH … | Text Code | Dynamic | Basic |
| Yao Fu’s Blog | Fundamental research topics walkthrough | Such as emergent abilities, reasoning, long-context modeling. | EN | Text | Months | Fundamental |
| Transformer Math 101 | EleutherAI Blog | Transformer-related math estimation - Basic | Basic arithmetic about Transformer-based models. | EN | Text | None | Basic |
| 分析transformer模型的参数量、计算量、中间激活、KV cache - 知乎 | Transformer-related math estimation - Mediate | Detailed analysis of calculations in Transformer-based model. | ZH | Text | None | Basic |
| 紫气东来 - 知乎 | Specific engineering details | Such as inference and training frameworks. | ZH | Text | Weeks | Practical |
| GitHub - liguodongiot/llm-action | Engineering detail summaries | Summarizing AI engineering techniques, such as inference, parallel computing, etc. | ZH | Text | Days | Practical |
| 微信公众号：大猿搬砖简记 | Illustrated source code (e.g. vLLM, CUDA) and algorithms (e.g. FlashAttention) | ZH | Text | Weeks | Practical | |
| 游凯超 - 知乎 | Infrastructure-level engineering details | Such as CUDA, NCCL, torch.compile and other side infrastructures like Docker, etc. |
ZH | Text | Days | Practical |
| Alignment Guidebook - Notion | Introduction to LLM Alignment (SFT + RL) | EN | Text | Dynamic | Basic | |
| Spinning Up in Deep RL! — Spinning Up documentation | Basic Deep RL | EN | Text Code |
None | Basic | |
| 科学空间|Scientific Spaces | Blogs combining graceful theories and solid experiments | Blogs by Jianlin Su (苏剑林), the author of RoPE (de facto standard of positional encoding now), versed in math and ML theory while not unfamiliar with experiments and practice. | ZH | Text | Weeks | Fundamental |
| Research | OpenAI research blogs | “We keep re-discovering what OpenAI discovered five years ago.” | EN | Text | Months | Fundamental |
| Research \ Anthropic | Anthropic research blogs | EN | Text | Months | Fundamental | |
| Transformer Circuits Thread | Amazingly insightful and open Anthropic interpretability team research blogs | EN | Text | Month | Fundamental | |
| E.g. [2312.11805] Gemini: A Family of Highly Capable Multimodal Models | LLM technical reports | Such technical reports, while usually not very detailed, often do reveal some important details of SotA LLMs. | EN | Text | Months | Fundamental |
| Hazy Research | Blogs of pioneer visions | Blogs from Hazy Research led by Christopher Ré @ Stanford (one of the best NLP&AI research groups around the world). | EN | Text | Months | Fundamental |
| Ilya 30u30 | Short reading list to understand the fundamentals of the AI today, said to be from Ilya. | Not the most frontier and not the most suitable for research starters, but really fundamental for essential understanding. | EN | Text | None | Fundamental |
| FAI-Seminar | High-quality talks (largely contributed by Yao class alumna) | ZH | Speech Text | Week | Trending | |
| Cool Papers - Immersive Paper Discovery | Daily arXiv paper & Kimi interaction | EN | Text | Day | Trending | |
| Daily Papers - Hugging Face | The most popular paper selection on Twitter. | EN | Text | Day | Trending | |
| 微信公众号： SparksofAGI | Individual paper selection, some of which common popular paper collections might not notice | Selected by Jianbo Dai (戴建波)* (senior researcher at Huawei). | ZH | Text | Weeks | Trending |
| 微信公众号： AINLP | Curations of other AI 微信公众号：s | ZH | Text | Day | Trending | |
| 中文 AI 媒体四大顶号：机器之心、新智元、量子位、夕小瑶科技说 | Popular paper selection | ZH | Text | Day | Trending | |
| 微信公众号： arXiv 每日学术速递 | arXiv paper from broader domains | ZH | Text | Day | Auxiliary | |
| 微信公众号： AI 前线 | Various AI news (not limited to research) | ZH | Text | Day | Auxiliary | |
| Video channel Song Zhao (YouTube / BiliBili) | Various practical academic-relevant affairs (e.g. paper submission, job choices) | A little “abstract” though … | ZH | Speech Text | Weeks | Auxiliary |