README.md

awesome-llm-reasoning

Collections of resources for LLM reasoning, including papers, codes and data. There are numbers of awesome repos focus on reasoning of LLM, we try to summarize these methods in a different angle, their specific domain, such common reasoning, math, table related, KG relate or multimodal(welcome to name them more).

We try to give a comprehensive view of resources, thus papers here may not be well evaluated by now for the sake of time, any suggestion are well appreciated!

Tool-augmented/oriented Reasoning

Paper	code	data
Augmented Language Models: a Survey	/	/
Tool Learning with Foundation Models	project	ToolBench
Behavior Cloned Transformers are Neurosymbolic Reasoners	neurosymbolic	TWC/MapReader
RECITATION-AUGMENTED LANGUAGE MODELS	RECITE	NaturalQuestions/TriviaQA/hotpotqa
MIND’S EYE: GROUNDED LANGUAGE MODEL REASONING THROUGH SIMULATION	/	MuJoCo
Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks	Program-of-Thoughts	data
PAL: Program-aided Language Models	PAL	data
SHOW YOUR WORK: SCRATCHPADS FOR INTERMEDIATE COMPUTATION WITH LANGUAGE MODELS	/	/
GeneGPT: Augmenting Large Language Models with Domain Tools for Improved Access to Biomedical Information(Domain Tool)	/	GeneTuring

Table-based Reasoning

Paper	code	data
StructGPT: A General Framework for Large Language Model to Reason over Structured Data	code	data / raw
Large Language Models are Effective Table-to-Text Generators, Evaluators, and Feedback Providers	code	data
Large Language Models are few(1)-shot Table Reasoners	code	WikiTableQuestions / FetaQA / TabFact / FEVEROUS
OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering	code	data
REASTAP: Injecting Table Reasoning Skills During Pre-training via Synthetic Reasoning Examples	code	data
Large Language Models are Versatile Decomposers: Decompose Evidence and Questions for Table-based Reasoning	code	WikiTableQuestions / FetaQA
Evaluating and Enhancing Structural Understanding Capabilities of Large Language Models on Tables via Input Designs	code	HybridQA / ToTTo / TabFact / FEVEROUS
Enhancing Few-shot Text-to-SQL Capabilities of Large Language Models: A Study on Prompt Design Strategies	code	data
Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQL	BIRD	data

Related Fields

Hallucination Evaluation

Paper	code	data
Evaluating Object Hallucination in Large Vision-Language Models	code	MSCOCO / SQA / A-OKVQA / GQA
HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models	code	data

benchmark

Paper	code	data
Chain-of-Thought Hub: A Continuous Effort to Measure Large Language Models' Reasoning Performance	chain-of-thought-hub	chain-of-thought-hub

Refer to Awesome Lists

We are standing on the shoulders of giants, inspired and borrow a lot from them.

• Awesome-Multimodal-Reasoning Collection of papers and resources on Multimodal Reasoning, including Vision-Language Models, Multimodal Chain-of-Thought, Visual Inference, and others.
• Chain-of-ThoughtsPapers A trend starts from "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models".
• LM-reasoning Collection of papers and resources on Reasoning in Large Language Models.
• Prompt4ReasoningPapers Repository for the paper "Reasoning with Language Model Prompting: A Survey".
• ReasoningNLP Paper list on reasoning in NLP
• Instruction-Tuning-Papers Reading list of Instruction-tuning.
• Deep-Reasoning-Papers Recent Papers including Neural Symbolic Reasoning, Logical Reasoning, Visual Reasoning, planning and any other topics connecting deep learning and reasoning.
• Awesome-LLM Curated list of Large Language Model.
• Tool Learning Papers Awesome tool learning paper collection from THUNLP.