Plan-and-Solve-Prompting

Code for our ACL 2023 Paper "Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models".

🔥 We are honored to announce that Plan-and-Solve Prompting has been added to the core library of LangChain, that is, Plan-and-Execute. Find out what people are saying about it on Twitter and AI Daily Paper.

🤖 Run Plan-and-Solve-Prompting

Set an api-key of OpenAI API in the file apikeys.json

python main.py --prompt_id 201 \
--dataset SVAMP \
--engine text-davinci-003 \
--learning_type zero_shot

🤖 Run Plan-and-Solve-Prompting with threads

For faster inference, set 8 different api-keys of OpenAI API in the file apikeys.json

python main_threads.py --prompt_id 201 \
--dataset SVAMP \
--engine text-davinci-003 \
--learning_type zero_shot

🧑‍🍳 Prompts

Prompt_ID	Type	Trigger Sentence
101	CoT	Let's think step by step.
201	PS	Let's first understand the problem and devise a plan to solve the problem. Then, let's carry out the plan to solve the problem step by step.
301	PS+	Let's first understand the problem, extract relevant variables and their corresponding numerals, and devise a plan. Then, let's carry out the plan, calculate intermediate variables (pay attention to correct numeral calculation and commonsense), solve the problem step by step, and show the answer.
302	PS+	Let's first understand the problem, extract relevant variables and their corresponding numerals, and devise a complete plan. Then, let's carry out the plan, calculate intermediate variables (pay attention to correct numerical calculation and commonsense), solve the problem step by step, and show the answer.
303	PS+	Let's devise a plan and solve the problem step by step.
304	PS+	Let's first understand the problem and devise a complete plan. Then, let's carry out the plan and reason problem step by step. Every step answer the subquestion, "does the person flip and what is the coin's current state?". According to the coin's last state, give the final answer (pay attention to every flip and the coin’s turning state).
305	PS+	Let's first understand the problem, extract relevant variables and their corresponding numerals, and make a complete plan. Then, let's carry out the plan, calculate intermediate variables (pay attention to correct numerical calculation and commonsense), solve the problem step by step, and show the answer.
306	PS+	Let's first prepare relevant information and make a plan. Then, let's answer the question step by step (pay attention to commonsense and logical coherence).
307	PS+	Let's first understand the problem, extract relevant variables and their corresponding numerals, and make and devise a complete plan. Then, let's carry out the plan, calculate intermediate variables (pay attention to correct numerical calculation and commonsense), solve the problem step by step, and show the answer.

⭐ Star History

😸 Cite

If you find Plan-and-Solve-Prompting useful for your research and applications, please kindly cite using this BibTeX:

@article{wang2023plan,
  title={Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models},
  author={Wang, Lei and Xu, Wanyu and Lan, Yihuai and Hu, Zhiqiang and Lan, Yunshi and Lee, Roy Ka-Wei and Lim, Ee-Peng},
  journal={arXiv preprint arXiv:2305.04091},
  year={2023}
}