Feat/link suggest #253
Feat/link suggest #253
Conversation
jacobryas4
commented
Sep 15, 2023
jacobryas4
commented
- new schema, models for database
- scraper used to retrieve CIDs for IPFS retrieval
- slightly modified prompt
- update widget to use new index
tools/index_widget.py
Outdated
| @@ -355,13 +355,16 @@ def fn(token_handler): | |||
| @error_wrap | |||
| def fetch_scraped_sites(query: str) -> Callable: | |||
| def fn(token_handler): | |||
| scraped_sites_index = config.initialize(config.scraped_sites_index) | |||
| # below is the old index used previously for scraped sites - do we still use this? | |||
There was a problem hiding this comment.
I think there is some overlap between both link suggestion and scraped sites modules as both seek to answer generic questions about the ecosystem.
If we use both of them, I suspect the model may not be able to reliably discern which one to use appropriately
Can you test this by having both enabled and see how the system performs? To test both of them, you will need to a have a separate widget command for each in the yaml
You can also remove your comments.
tools/index_answer.py
Outdated
| @@ -12,7 +12,7 @@ | |||
|
|
|||
| TEMPLATE = '''You are a web3 assistant. You help users with answering web3-related questions. Your responses should sound natural, helpful, cheerful, and engaging, and you should use easy to understand language with explanations for jargon. | |||
|
|
|||
| Information to help complete your task is below. Only use information below to answer the question, and create a final answer with inline citations linked to the provided source URLs. If you don't know the answer, just say that you don't know. Don't try to make up an answer. ALWAYS return a "SOURCES" part in your answer corresponding to the numbered inline citations. | |||
| Information to help complete your task is below. Only use information below to answer the question, and create a final answer with inline citations linked to the provided source URLs. If you don't know the answer, just say that you don't know. Don't try to make up an answer. ALWAYS return a "SOURCES" part in your answer corresponding to the numbered inline citations. ALWAYS provide a link to each citation in SOURCES. | |||
There was a problem hiding this comment.
-
Similar to having a separate function handler for link suggestion different from scraped sites, maybe better to also have a separate tools module like
index_dapp_suggesstionor something -
Trying to figure out how consistently the LLM is able to cite SOURCES in a reliable format, any insights on that?
We could also explore a more structured format for the results like a list of JSON objects with fields such asdapp_name,dapp_desc,dapp_urletc and then post-process it or send it to the FE as JSON to render it consistently
| {"name": DAPP_NAME, "dataType": ["text"]}, | ||
| {"name": DAPP_DESCRIPTION, "dataType": ["text"]}, | ||
| {"name": DAPP_URL, "dataType": ["text"]}, | ||
| { |
There was a problem hiding this comment.
- let's just remove the social media links as they will not be used and unnecessarily will occupy space
- DAPP_URL does not need to be vectorized so can be skipped
