A wrong answer from Cache record #385
-
|
Hi My script contains the follow questions, and it will run several loop within one session. question 1~3 are all about TV, so cache answer are linked to the same answer of the first question in the first loop, but from second loop the second question will linked with the forth question of " what is the function of money". I guess there are both the key prompt word of function. Is there parameters can control the weight of vector calculation for different key words? The first loop: Question: can you explain what function of TV is ? Question: can you tell me more about TV ? Question: what is the function of money ? Second Loop Question: what is TV ? Question: can you explain what function of TV is ? Question: can you tell me more about TV ? Question: what is the function of money ? |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 2 replies
-
|
@terryweijian there is no idea to control the weight of vector calculation for different key words. You can choose to skip cache searching when you think the cached answer doesn't meet the requirements, but save the llm result to the cache this time. The next time you ask the same question, you will be able to get an accurate answer.
openai.ChatCompletion.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "what's github"}],
cache_skip=True,
) |
Beta Was this translation helpful? Give feedback.
-
|
Not quite sure if the following step can fix the problem, I tried it and didn't find the problem in my environment again. use embedding_func=onnx.to_embeddings to replace my original huggingface.to_embedding in the llm_cache.init () Thanks |
Beta Was this translation helpful? Give feedback.
-
|
After more testing, and confirmed that the Huggingface embedding module is the root cause, after switch to onnx embedding module, didn't find the similar issue with the English questions Thanks |
Beta Was this translation helpful? Give feedback.
@terryweijian there is no idea to control the weight of vector calculation for different key words. You can choose to skip cache searching when you think the cached answer doesn't meet the requirements, but save the llm result to the cache this time. The next time you ask the same question, you will be able to get an accurate answer.
cache_skipparam usage