ConversationalRetrievalQAChain answering general questions

[jteso]

I am using a ConversationalRetrievalQAChain that uses a vector store as a retriever to answer questions about some documents. However the chain always responds with "I'm sorry, I don't know" if the answer to the question cannot be found in the vector store.

My workaround looks pretty ugly at the moment, any body can point me to the right way to achieve this?

const llm = new OpenAI({temperature: 0});
const chain = ConversationalRetrievalQAChain.fromLLM(
    llm, 
    vectorStore.asRetriever(), 
 );
const question = "Hi. How are you?";
let res = await chain.call({ question, chat_history: chat_history });
if (res.text === "I'm sorry, I don't know.") {  // <-- workaround
let resp = await llm.call(question);

[markusobermaier96]

I think the right way to do it is to define a custom qaTemplate like so:

const QA_PROMPT = PromptTemplate.fromTemplate(
`You are an AI assistant providing helpful advice. You are given the following extracted parts of a long document and a question. Provide a conversational answer based on the context provided.
You should only provide hyperlinks that reference the context below. Do NOT make up hyperlinks.
If you can't find the answer in the context below, just say "The document does not provide information to answer your question." Don't try to make up an answer.
If the question is not related to the context, politely respond that you are tuned to only answer questions that are related to the context.

Question: {question}
=========
{context}
=========
Answer in Markdown:`
);

then you can overwrite the default prompt by setting
const chain = ConversationalRetrievalQAChain.fromLLM( llm, vectorStore.asRetriever(), { qaTemplate: QA_PROMPT.template } );

you can customize the template to your needs ofc, but should contain basic instructions for the model to fullfill your task correctly