RAGFlow-0.9.0

Release Summary

1. Support GraphRAG
2. Add more Agent operators: keyword extraction, Baidu/duckduckgo/pubmed/wikipedia/bing/google search operators.
3. Add support for audio files (speech-to-text recognition).
4. Add support for Gemini, Groq and other LLMs.
5. Support for reranker provided by xinference.
6. Add support for LM studio/OpenRouter/LocalAI/Nvidia api and other inference frameworks/engines/services.

What's Changed

• feat: translate the to field #918 by @cike8899 in #1435
• feat: translate graph and chat text by @netyum in #1433
• Update README by @JinHai-CN in #1438
• feat: translate name of operator #918 by @cike8899 in #1437
• feat: display the debugging results of each operator in a pop-up window #918 by @cike8899 in #1445
• fix no resualt bugs by @KevinHuSh in #1448
• fix no resualt bugs by @KevinHuSh in #1449
• API doc: Correct Base URL and Dataset URL. by @cyhasuka in #1444
• fix pdf_parser content confusion by @guoyuhao2330 in #1458
• fix pdf_paser char content confusion by @guoyuhao2330 in #1462
• add support for Gemini by @hangters in #1465
• fix: Delete hardcode by @cyhasuka in #1464
• fix: monitor changes in the data.form field of the categorize and relevant operators and then synchronize them to the edge #918 by @cike8899 in #1469
• API: start parsing by @cecilia-uu in #1377
• feat: validate the name field of the categorize operator for duplicate names and nulls #918 by @cike8899 in #1471
• feat: support xinference rerank model by @hwzhuhao in #1466
• added SVG for Groq model model providers by @paresh2806 in #1470
• feat: add KeywordExtractForm and BaiduForm and DuckDuckGoForm #918 by @cike8899 in #1477
• fix bugs of rerank model with xinference by @KevinHuSh in #1481
• fix bug about divided by zero by @KevinHuSh in #1482
• add Gemini key by @hangters in #1480
• feat: modify DuckDuckGo's style #918 by @cike8899 in #1485
• feat: translate text from DuckDuckGo #918 by @cike8899 in #1486
• fix: add message_history_window_size to GenerateForm #1472 by @cike8899 in #1487
• Fix baidusearch and duckduckgosearch by @guoyuhao2330 in #1488
• fix ollama max token issue by @KevinHuSh in #1489
• feat: add llm Select to KeywordExtractForm #918 by @cike8899 in #1492
• refine db connection by @KevinHuSh in #1494
• refine db connection by @KevinHuSh in #1495
• refactor name of duckduckgo by @KevinHuSh in #1496
• fix duckduckgo.py by @guoyuhao2330 in #1497
• fix: fixed the issue where the greeting message could not be displayed when opening the debug window #918 by @cike8899 in #1499
• fix: fixed the issue where parameters of DuckDuckGo could not be saved to the backend after being dragged to the canvas #918 by @cike8899 in #1503
• Fix: Implement DOMPurify to sanitize HTML content before rendering by @dservian in #1498
• feat: remove dagre and elkjs #918 by @cike8899 in #1506
• fix: fixed the issue that the llm field in the KeywordExtract form had no default value by @cike8899 in #1510
• fix: Add Model Providers:Azure-OpenAI error #1402 by @cike8899 in #1512
• Add component Wikipedia by @guoyuhao2330 in #1513
• API: show status of parsing by @cecilia-uu in #1504
• Set wikipedia lang by @guoyuhao2330 in #1515
• feat: add Wikipedia operator #918 by @cike8899 in #1516
• Added supported LLMs by @writinwaters in #1517
• fix: fixed the issue of error when opening the canvas #918 by @cike8899 in #1520
• fix wikipedia language by @guoyuhao2330 in #1519
• fix bugs about multi input for generate by @KevinHuSh in #1525
• fix minimax bug by @KevinHuSh in #1528
• fix: Hundreds of chunks list can't choose page #1238 by @cike8899 in #1532
• fix the tokens error that occurred when adding the xinference model by @hwzhuhao in #1527
• 'load llm infomation from a json file and add support for OpenRouter' by @hangters in #1533
• fix: Bulk disable the chunk, the UI still shows they are enabled #1236 by @cike8899 in #1534
• hide referece when disable cite by @KevinHuSh in #1535
• remove dependency by @KevinHuSh in #1536
• fix minimax init error by @KevinHuSh in #1537
• feat: replace open-router.svg #1467 by @cike8899 in #1538
• API Documentation by @cecilia-uu in #1526
• Update requirements.txt with pybind11 2.13.1 by @voulkon in #1548
• Update the Dockerfile.cuda by @cnjack in #1545
• fix: add group id field to ApiKeyModal #1353 by @cike8899 in #1540
• fix emmpty input in graph by @KevinHuSh in #1560
• refine generate by @KevinHuSh in #1562
• Update deploy_local_llm.md by @cnjack in #1557
• Add templates/websearch assistant by @guoyuhao2330 in #1559
• Fix web search and template max tokens by @guoyuhao2330 in #1564
• fix: remove duplicate MessageItem #1289 by @cike8899 in #1566
• fix MiniMax api error by @hangters in #1567
• fix requirements.txt by @guoyuhao2330 in #1570
• API: Stop parsing by @cecilia-uu in #1556
• fix: new message appears in wrong chat window. #1289 by @cike8899 in #1571
• feat: change all file names to lowercase #1574 by @cike8899 in #1575
• [Bug]: IndentationError: unindent does not match any outer indentatio… by @narendra-bluebash in #1579
• Updated deploy a local llm using IPEX-LLM by @writinwaters in #1578
• Fix component debug by @guoyuhao2330 in #1583
• fix: display total items on chunk list page #900 by @cike8899 in #1584
• fix: Minimax API is error! #1353 by @cike8899 in #1585
• Add component pubmed by @guoyuhao2330 in #1586
• Add component arxiv by @guoyuhao2330 in #1587
• fix generate error by @KevinHuSh in #1590
• add docker-compose-gpu.yml to support gpu by @Seaver-Zhu in #1591
• feat: add PubMed operator #918 by @cike8899 in #1589
• fix: after logging out and entering the knowledge base page again, the data before still exists #1306 by @cike8899 in #1597
• fix embeding model for Azure by @KevinHuSh in #1601
• fix function create to slove chunk number wrong by @Seaver-Zhu in #1604
• Fix component exception by @guoyuhao2330 in #1603
• add support for LocalAI by @hangters in #1608
• fix: use @tanstack/react-query to get knowledge base data #1306 by @cike8899 in #1609
• fix delete selected chunks display wrong by @Seaver-Zhu in #1612
• add support for StepFun by @hangters in #1611
• Chat Use CVmodel by @guoyuhao2330 in #1607
• feat: support AWS Bedrock #308 by @cike8899 in #1617
• update quickstart and llm_api_key_setup document by @hangters in #1615
• modify the encryption to first perform base64 encoding and then encrypt by @hwzhuhao in #1621
• add docker-compose-gpu-CN.yml and docker-compose-gpu-CN-oc9.yml to support gpu by @hwzhuhao in #1618
• fix: the content in the chunk card will overflow #1628 by @cike8899 in #1629
• Add sequence2txt model.py by @guoyuhao2330 in #1633
• fix: reference file with 'docx' type can not open #844 by @cike8899 in #1635
• Add information form by @JinHai-CN in #1636
• Add ParsertType Audio by @guoyuhao2330 in #1637
• fix: fetch file list by react-query #1306 by @cike8899 in #1640
• fix bug of ragflowdocxpparser by @KevinHuSh in #1642
• add support for NVIDIA llm by @hangters in #1645
• refine readme, update updates by @KevinHuSh in #1648
• fix redis no such key by @Seaver-Zhu in #1647
• add password reset function by extending the Flask command by @hwzhuhao in #1632
• fix: remove unused libraries #1306 by @cike8899 in #1649
• fix: fetch the file list after uploading the file by @tanstack/react-query #1306 by @cike8899 in #1654
• Feat stepfun by @bdss58 in #1659
• build(deps): bump pymysql from 1.1.0 to 1.1.1 by @dependabot in #1664
• build(deps): bump tqdm from 4.66.2 to 4.66.3 by @dependabot in #1681
• build(deps): bump certifi from 2024.2.2 to 2024.7.4 by @dependabot in #1680
• build(deps): bump pillow from 10.2.0 to 10.3.0 by @dependabot in #1677
• build(deps): bump idna from 3.6 to 3.7 by @dependabot in #1676
• build(deps): bump flask-cors from 4.0.0 to 4.0.1 by @dependabot in #1675
• build(deps): bump jinja2 from 3.1.3 to 3.1.4 by @dependabot in #1674
• build(deps): bump requests from 2.31.0 to 2.32.2 by @dependabot in #1673
• build(deps): bump urllib3 from 2.2.1 to 2.2.2 by @dependabot in #1672
• build(deps): bump scikit-learn from 1.4.1.post1 to 1.5.0 by @dependabot in #1671
• build(deps): bump aiohttp from 3.9.3 to 3.9.4 by @dependabot in #1670
• build(deps): bump werkzeug from 3.0.1 to 3.0.3 by @dependabot in #1669
• build(deps): bump setuptools from 69.5.1 to 70.0.0 by @dependabot in #1666
• fix: Misspelled Variable Name by @Piercinge in #1662
• feat: add arxiv operator #918 by @cike8899 in #1683
• pypdf2 to pypdf by @KevinHuSh in #1684
• add support for LM Studio by @hangters in #1663
• fix graph bug about second retrieval by @KevinHuSh in #1688
• build(deps-dev): bump axios from 0.27.2 to 1.7.2 in /web by @dependabot in #1679
• build(deps): bump follow-redirects from 1.15.4 to 1.15.6 in /web by @dependabot in #1678
• build(deps-dev): bump ws from 8.17.0 to 8.18.0 in /web by @dependabot in #1668
• Fix api reference empty bug by @jiyegege in #1655
• fix: Login with @tanstack/react-query #1306 by @cike8899 in #1691
• fix: Fixed an issue where the project could not be built #1306 by @cike8899 in #1693
• fix: Set the default language to English #1306 by @cike8899 in #1694
• refactor some llm api using openai api format by @hangters in #1692
• fix: Limit the length of the new password input box to no less than 8 #1634 by @cike8899 in #1696
• Fix pdfparser content confusion by @guoyuhao2330 in #1700
• fix embedding_model by @Seaver-Zhu in #1698
• fix: Fix for Empty Reference Array Causing Errors by @iMyth in #1652
• fix bug of regx by @KevinHuSh in #1703
• fix: Set the default value of Self RAG to false #1220 by @cike8899 in #1702
• fix: alter Arxiv to ArXiv #918 by @cike8899 in #1705
• fix: fetch llm list by @tanstack/react-query #1306 by @cike8899 in #1708
• fix: fetch user by @tanstack/react-query #1306 by @cike8899 in #1709
• Fix pdfparser error by @guoyuhao2330 in #1707
• fix: Remove kAModel #1306 by @cike8899 in #1713
• Add Kibana component for Elasticsearch by @Chunshan-Theta in #1710
• Fix docs parser by @guoyuhao2330 in #1714
• fix: test chunk by @tanstack/react-query #1306 by @cike8899 in #1719
• fix OpenAI llm return bug by @hangters in #1728
• Fix multiple generate by @guoyuhao2330 in #1722
• fix: When parsing the bold content in PDF, the result is duplicated. by @leecj in #1729
• Fix docx parser line bug by @guoyuhao2330 in #1715
• fix add local vision llm error when cannot download test pic by @hangters in #1732
• Fix index=true by @guoyuhao2330 in #1734
• fix: Fetch chunk list by @tanstack/react-query #1306 by @cike8899 in #1738
• Add component google,Bing by @guoyuhao2330 in #1737
• fix MiniMax chat bug by @hangters in #1733
• support reset the user email by @hwzhuhao in #1735
• Fix: Embedding err when docx contains unsupported images by @cyhasuka in #1720
• add support for LocalLLM by @hangters in #1744
• feat: Add bing and google operator #918 by @cike8899 in #1745
• fix: ERROR: 'CompletionUsage' object has no attribute 'get' by @chinamerp in #1736
• fix: delete chunk by @tanstack/react-query #1306 by @cike8899 in #1749
• lower openai version in requirements.txt by @hangters in #1747
• add step-1v-8k cv model by @bdss58 in #1686
• refactor stepfun cv model by @hangters in #1751
• feat: Classify nodes based on edge relationships #162 by @cike8899 in #1755
• feat: Increase the distance between nodes #162 by @cike8899 in #1758
• trival by @KevinHuSh in #1760
• API: retrieval api by @Valdanitooooo in #1763
• fix jina module not find bug by @hangters in #1770
• feat: Add KnowledgeGraphModal #162 by @cike8899 in #1766
• feat: Alter style of ForceGraph #162 by @cike8899 in #1774
• fix jina module not find bug by @hangters in #1779
• feat: Add EntityTypesForm #162 by @cike8899 in #1777
• feat: Hide KnowledgeGraphModal #162 by @cike8899 in #1783
• fix: TypeError: Cannot read properties of undefined (reading 'viewport' #1761 by @cike8899 in #1784
• fix: Rename graph to agent #918 by @cike8899 in #1785
• Add component google scholar by @guoyuhao2330 in #1790
• feat: Add IndentedTree #162 by @cike8899 in #1792
• Add graphrag by @KevinHuSh in #1793
• readme update by @KevinHuSh in #1794
• feat: Fixed the issue where the page reports an error when the graph returned by the interface is empty #162 by @cike8899 in #1795
• refine dockerfile by @KevinHuSh in #1801
• refine Dockerfile by @KevinHuSh in #1802
• fix: Fixed the issue that the related form value does not change after selecting the freedom field of the model #1804 by @cike8899 in #1805
• Fix graphrag callback by @guoyuhao2330 in #1806
• refine mindmap prompt by @KevinHuSh in #1808
• remove duplicated key in mind map by @KevinHuSh in #1809
• feat: Translate ForceGraph #162 by @cike8899 in #1810
• be better chunks before graphrag by @KevinHuSh in #1811
• refine docs for 0.9.0 release by @KevinHuSh in #1812
• refine context length by @KevinHuSh in #1813
• refine mindmap by @KevinHuSh in #1817

New Contributors

• @netyum made their first contribution in #1433
• @cyhasuka made their first contribution in #1444
• @paresh2806 made their first contribution in #1470
• @dservian made their first contribution in #1498
• @voulkon made their first contribution in #1548
• @cnjack made their first contribution in #1545
• @narendra-bluebash made their first contribution in #1579
• @Seaver-Zhu made their first contribution in #1591
• @bdss58 made their first contribution in #1659
• @dependabot made their first contribution in #1664
• @Piercinge made their first contribution in #1662
• @jiyegege made their first contribution in #1655
• @iMyth made their first contribution in #1652
• @Chunshan-Theta made their first contribution in #1710
• @leecj made their first contribution in #1729
• @Valdanitooooo made their first contribution in #1763

Full Changelog: v0.8.0...v0.9.0