Refine prompts

[ccstan99]

Perform prompt engineering to ensure accuracy, minimize hallucinations, watch out for unnecessary jargon, adjust tone and level of depth to match user’s.

[ccstan99]

The goal is to get a reasonable set of answers judged by distillation fellows and/or Rob on a a benchmarked set of questions. Additional details in #6 .

[ccstan99]

Some things to try:

1. Sample prompt template used by langchain:
   """Given the following extracted parts of a long document and a question, create a final answer. If you don't know the answer, just say that you don't know. Don't try to make up an answer.
   {context}
   Question: {question}
   Helpful Answer:"""

2. To reduce hallucination, it's also suggested to try adding the phrase "not prior knowledge"
   """Given the following extracted parts of a long document, no other prior knowledge, and a question, create a final answer. If you don't know the answer, just say that you don't know. Don't try to make up an answer.

3. To avoid prompt injections, an OpenAI cookbook recommends using delimiters:
   """Given the following extracted parts of a long document, and a question which will be delimited by triple tackticks , create a final answer. If you don't know the answer, just say that you don't know. Don't try to make up an answer.
   {context}
   Question: ```{question}```
   Helpful Answer:"""

• Triple backticks: ```,
• Triple dashes: —-
• Angle brackets: < >
• XML tags: ‹tag›

4. Another way to prompt for citing source:
   """You are a world class algorithm to answer questions with correct and exact citations.
   Answer question using the following context
   {context}
   Question: {question}
   Tips: Make sure to cite your sources, and use the exact words from the context."
   Helpful Answer:"""

5. Other considerations:

• Assess the user's level of technical understanding based on the current and previous questions. Unless they have shown technical understanding, avoid using jargon and explain answers in plain language.
• Try to include analogies where helpful.
• Add some whimsical humor when appropriate.
• Answers in the style of aisafety.info, Stampy FAQ or Rob Miles if those are considered gold standards
• Give more weight to recent sources or those from {}
• Today's date is {}, avoid recommending any events from the past.
• Think through step by step before generating final answer?

Record generated answers in stampy-chat-test gsheet. 1 tab for each prompt plus any other ideas you want to try. Have distillation fellows review to comment on quality. Keep and combine any of the above prompt that show improvement. Other resources to investigate:

• https://github.com/openai/openai-cookbook
• https://github.com/openai/openai-cookbook/blob/main/techniques_to_improve_reliability.md#how-to-improve-reliability-on-complex-tasks
• https://github.com/dair-ai/Prompt-Engineering-Guide
• https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/
• https://www.deeplearning.ai/short-courses/chatgpt-prompt-engineering-for-developers/

[ccstan99]

Sample prompt from @markovial for a potential mode to help editors draft Stampy responses:

You are a copyeditor and proofreader. You edit manuscripts according to the Chicago Manual of Style. Focus on punctuation, grammar, syntax, typos, capitalization, formatting and consistency. Format all numbers according to the Chicago Manual of Style, spelling them out if necessary. Use italics and smart quotes and indent each paragraph.

You generate texts that are more professional in tone with no grammatical errors. After your editing process, the text also removes all instances of first person (I) and second person (you). You prefer the text to be strictly in third person, favoring an explanatory and descriptive tone which would fit well into an academic text. You do not use overly complicated language or words that would seem pretentious. Your editing process  manages to keep the text extremely engaging and lively by using active language. You also correct any factual errors you notice in the text.

Between --- is some text before having been through the copy editing process.

---
---
The following is the same text after having been put through your copy editing process:

[plexish]

Once we have a good way to assess prompts, we should for sure throw this task at the people on Cyborgism.

[FraserLee]

@plexish here's the documentation on the current prompt eng workflow I've created. #50 (comment)
It might be nice to get some expert experimenting with it. I'm already a programmer, so I'm a bit blind to the ways the current setup could be patched with additional documentation or tooling to make it accessible to more non-programmers.

This issue in particular is blocked until we've got the updated database fully working and integrated, since any specific findings before then will be pretty uninformative after we make the changes.

[ccstan99]

Try SmartGPT prompt:

• "Question." + "Answer: Let's work this out in a step by step way to be sure we have the right answer."
• You are a researcher tasked with investigating the X response options provided. List the flaws and faulty logic of each answer option. Let's work this out in a step by step way to be sure we have all the errors:
• You are a resolver tasked with I) finding which of the X answer options the researcher thought was best 2) improving that answer, and 3) Printing the improved answer in full. Let's work this out in a step by step way to be sure we have the right answer:

https://www.youtube.com/watch?v=wVzuvf9D9BU

[markovial]

I have been getting really amazing results with this prompt on base GPT-4. Some of the answers are more helpful than even the RAG'd chatbot. Here is the prompt:

You are a helpful professor. You are a polymath, and are considered a genius while still being one of the world's best communicators. You have expert level knowledge in mathematics, artificial intelligence, history and storytelling. You are able to provide both deep technical a wide breadth of information. Explain scientific concepts, theories, and phenomena in an engaging and accessible way. Use real-world examples and analogies to help people better understand and appreciate the wonders of artificial intelligence.

[ccstan99]

• Nemo set up a nice playground for testing these.
• Archana's added validation using tonic validate #141 should be helpful to objectively measure how changes impact the metrics.