Skip to content

Latest commit

 

History

History
856 lines (780 loc) · 101 KB

CHANGELOG.md

File metadata and controls

856 lines (780 loc) · 101 KB
Raw

Changelog

0.13.0 (2024-12-06)

Features

  • able to insert without postgres, able to get it to search (54b7771)
  • able to update parameters, not able to see the changes in the (44b1067)
  • add tag_regexes parameter to add more metadata as a (e10b9bd)
  • add additional debugging logs to bulk upload for ingest worker (869166f)
  • add advanced options and usage metrics to each chunk (9c9596d)
  • add api key functionality to demo-ui app (ff00e25)
  • add api key params and expiration (22b84f1)
  • add back user api key management (0c60393)
  • add batch create dataset route (c141890)
  • add CI workflow to push images for pdf2md service (bb88f91)
  • add CLI to test pdf chunking (2543b28)
  • add close button (879aa1f)
  • add docker-compose.yml for local testing (bee90ab)
  • add errors into prometheus /metrics scrape (807ff8c)
  • add example and update adapter to work (5fc58fb)
  • add fall back response if there is no chunks above the score threshold (ba8b5d7)
  • add file chunker microservice (86fec6c)
  • add flag to enable pdf2md (322ec12)
  • add health check + fix redoc (c5e6e18)
  • add initCollections parameter to helm chart (d854b4f)
  • add inputs for OG and alternate chat image (da1922a)
  • add json toggle + copy button to pdf2md (f898cb5)
  • add logging and write to clickhouse (858df03)
  • add match_all to search playground filter options (a82163c)
  • add max req time so we don't stall out workers ever (7fe1a49)
  • add MMR support (784fc9e)
  • add navbar to public page (0fa9748)
  • Add new advanced settings to backend page.html and page handler (66ac2a0)
  • add new parameter for making qdrant collections (dc6f5d1)
  • add open graph to page template (2849204)
  • add open in new window button for public page (a43c57d)
  • add organization_id header when possible to all API reqs (05e4099)
  • add other props to advanced settings (66ac2a0)
  • add page right after its been processed (11623e0)
  • add QDRANT_ONLY to dataset configuration (de775d2)
  • add RERANKER_API_KEY as a dataset setting (6456a0c)
  • add route to get message by id (e7a061c)
  • add skeleton template others can slot into (a97e793)
  • add sort to search over groups (8bfdd76)
  • add styles import (982fb56)
  • add support for subtrace version b134 (71f01e0)
  • add table for tracking tasks (0ee9ebc)
  • add tag_regexes flag into the frontend (6fdd764)
  • add tailwind to public page + roughjs try me preview and arrow (06e6083)
  • add token usage to pdf2md ui (a92b764)
  • add type + groupSearch as options to public page config (213252b)
  • add user routes to ts-sdk and tests (7518c52)
  • add webhooks (93c865b)
  • added api key and file chunking status (1a720a6)
  • adding tags to rag (681b5bb)
  • all done except logic to handle whether it's 0 or 1 indexed (f1c1c35)
  • allow editing a group's tag_set and tracking_id via search playground (79d093d)
  • allow serialization of extra_params (aa7a44b)
  • better chat button placement (45d5385)
  • better error message for average vector strategy (4550234)
  • better tooltip component (2f81c38)
  • build metadata strings (46d1bba)
  • bulk delete chunks (e65889a)
  • completed MVP implementation of the UI (c8f93f3)
  • correct polling + clicking view does something (215451c)
  • create component for role messages (41b247a)
  • custom date picker for analytics overview (591bada)
  • default to useGroupSearch for public page on shopify datasets (e82c65b)
  • different prompt (46d1bba)
  • display all search parameters on the component page (10af639)
  • docker setup for pdf2md service (75d0896)
  • don't let workers run migrations (6f0bad3)
  • dropdown menu for OpenAI/OpenRouter in LLM settings (b5b2a4b)
  • ecomm component grid layout + padding (4a165e1)
  • enable chunk bulk delete as an async operation where chunks matching delete filter can be inserted safely after resp (ba19f02)
  • examples + drag and drop fle functionality (b6a2d1f)
  • export TrieveDocument (5bdcad5)
  • expose custom html based splitter (c7f8b9b)
  • extract chunk_html from payload field, need to extract other (65aca1e)
  • fast load public page (4ba4aea)
  • file-worker to poll and get all chunks (d2d1a83)
  • finish edit groups + display edit button on group search + group metadata filters (fa75860)
  • finish implementing QDRANT_ONLY mode with chunk_count and clear (0eee2f8)
  • finished navbar for pdf2md (6489b08)
  • fix image cropping (46d1bba)
  • fix new dataset modal tooltip sizes (81c4759)
  • fix styling for key command navigation (fc5db95)
  • get signed url for pdf (44e3787)
  • group chat image carousel (c63e09e)
  • group parsing and ui (41f6088)
  • group search view starter for ecommerce, show price range of items (74800eb)
  • handle first load in of table (c9576c8)
  • implement hero background customization (2ebfb92)
  • include advanced options form for pdf2md (29451a5)
  • include cost estimate on pdf view (eec0b1e)
  • include organization url in invitation email (37ef206)
  • include tables (46d1bba)
  • include user resource in ts-sdk docs (2c5a345)
  • initialize logger on clone qdrant script (a13f1de)
  • integrate chunkr as a provider for markdown conversion (afeab01)
  • integrate subtrace b143 (bf66c05)
  • jinja template to render the all component properties (e3aef3e)
  • join similar groups together based on product.title (9651e40)
  • logic to make reranker calls to cohere (5a18771)
  • make BATCH_CHUNK_LIMIT an env to increase it past 120 for (48d772b)
  • make footer look better (1682f3b)
  • make pdf2md ocr optional (1f655cf)
  • make shopify scraping handle null valeus in products.json, fix (f9e0f5b)
  • manual rag with groups (dbe62ad)
  • MMR takes 2x page size such that final results may differ (d4e9e30)
  • mobile view + scrollbar fix for public page (42759c8)
  • move pdla types (46d1bba)
  • multistring input for default questions (e4abc63)
  • new system prompt (46d1bba)
  • open the search component by default (98d1d21)
  • page parameters viewable on the search page, initial styling (7747182)
  • pdf to md :) (89d7020)
  • pdf2md: make LLM provider be the default chunking provider (a2fdbfe)
  • pdf2md: make remove items from processing after message is (4e92c58)
  • pdf2md: update all docker images to have poppler (0e520f4)
  • process 1 page at a time (bbb9d67)
  • public page back button (7dcb165)
  • public page settings tooltips (2dfd91c)
  • queue shopify chunks as you go (247decc)
  • readd recency bias (c1bd516)
  • readd recency bias for groups (962ff9c)
  • rearrange images to display horizontally and make the section collapsable (934e3a6)
  • remove description on groups view in favor of Chunk Count on (e06f169)
  • remove postgresql trigger function for counts (3344662)
  • remove self-hosting.md in favor of mintlify docs (ec2a020)
  • remove subtrace as default option. (e2b7bfb)
  • remove tls on subtrace (c1af4cb)
  • reranker settings added to dashboard (ab99ee0)
  • revert to higher level og options (ce819ce)
  • route to get number of chunks in a group with /api/chunk_group/count (ddb7efc)
  • save chunk boost values to postgres (6dea41c)
  • save orgid in query params on dashboard (04c0ba8)
  • scope api keys to orgs (a2e28fb)
  • separate subpackage for vanilla js (298318b)
  • set defualt values in search UI (3d35fd6)
  • setup fileworker to communicate with pdf2md (2dfe5e5)
  • shortcut to toggle component modes (2c4f378)
  • skeleton for pdf2md demo page (edce843)
  • skeleton upload form for pdf2md (85715a8)
  • skip creating groups in chunk_handler (8151add)
  • smaller search component package (6ff6c72)
  • store hero background name and config (bdaf162)
  • subtrace version bump (a88c148)
  • trieve fumadocs adapter (0c087fa)
  • ui for role management on public page tabs (74ce1bc)
  • uniquify variants (52e3b0b)
  • update chunk boosts (9cf609f)
  • update environment variable in needed locations (ec6b279)
  • update fileworker to receive the correct response type (c30953e)
  • update python SDK (2f9ba2d)
  • update to new route name (5735ef8)
  • update ts-sdk to include dataset routes (c94e26d)
  • updates to frontend propagate correctly to the rendered (5811fd9)
  • upgrade qdrant and use new search over groups query (ad89e4a)
  • upsert_group_tracking_id's in ingestion worker (d4838e0)
  • use api key filters for scroll and add scroll test (a76db38)
  • use carousel for product items in chat (4b01233)
  • use past messages for context (238fae0)
  • use pdla boxes for chunking (46d1bba)
  • use proper struct for public page options (a729c5f)
  • version bump component to 0.0.64 (2f5a255)
  • working API req on file form add (1eb1a52)
  • working CSS for demo-ui on pdf2md (6eb7dba)
  • working tabs on public page (e189b4f)

Enhacements

  • improve organization email template (100b756)
  • move number of retrievals for rag to llm settings (16e4cd3)
  • send an event that scraping has started (e4f8e14)

Cleanup

  • add .env.dist + use env's for model in pdf_chunk.rs (94fcb7f)
  • better public page setting assert (064a84a)
  • clippy (977580d)
  • clippy and lint (d22dcf2)
  • document how to change system prompt and other RAG settings in dashboard (25b65ea)
  • ensure save button for dataset settings is visible (424b1f8)
  • errant print in py sdk chunk_group_api (e090c00)
  • fix rust formatting (ccc06d8)
  • fixed mode switch on mobile such that it's on top-right with (3ac55e3)
  • format and strip html from betterGroupName (8e1685c)
  • formatting (fa633cd)
  • load-in css animation for search component (0aa668a)
  • make support for group_id and group_tracking_id filters more clear (f603016)
  • move test program (46d1bba)
  • remove ChunkDataWithEmbeddingText (bef8cb2)
  • remove creating a file on the file_handler (18930d5)
  • remove credits_used from firecrawl + update env's (eefa854)
  • remove deprecated direction prop for tooltip (9cd45f5)
  • remove old search component syntax highlighted versions (a28e78f)
  • remove red border (6e482bc)
  • remove sentry (156392f)
  • remove test binary (46d1bba)
  • remove tracing and swap back to env_logger (4f041a5)
  • remove unused code (40707fc)
  • return 0 results when there are matching groups for a given set of tracking_id filters instead of erroring (8e57512)
  • slow down clone qdrant worker + add logs (30c4c7e)
  • start carousel to the first page (2b09abc)
  • temporarily remove subtrace for new builds to work (0102754)
  • typo (370e0bd)
  • update to latest Qdrant Rust SDK client (73f5254)
  • use env's for search component example w/ local dev (74da050)

Bug Fixes

  • add back working hero patterns (5d0c96b)
  • add missing fields to PublicPageParameters struct (66ac2a0)
  • change new variable to "nav" (28d5145)
  • change shortcut to ctrl-tab (66bbc6e)
  • change text for brand logo link (a4095d9)
  • chat toggle mode not loading due to misplayzed lazy import of react-markdown (0277ef6)
  • chunk routes on python SDK (2e5e3d4)
  • clear chat when exiting single product chat (9646ffc)
  • correct page count using redis (1bf67dc)
  • default params for search component (2986d64)
  • ecomm group search missing key + query suggestions not always loading on open (da86ea4)
  • fix api key role not being set correctly (21c0d3e)
  • fix broken Dockerfile, update subtrace version (bef55c2)
  • fix peerDependencies (cb1ce2d)
  • fix tests and ensure that expires at functionality works correctly (3d0c662)
  • fixed bug when updating dataset settings (6361201)
  • fumadocs: move trieve-ts-sdk to peerDeps, loosen version range (cfb56a6)
  • group carousel image placement and clear chat (1a1f797)
  • handle empty text better (46d1bba)
  • hide repeated sizes (8dd4b34)
  • hybrid search over groups is not properly returning the first chunk with a cross encoder score (746d04b)
  • image lnik for README not rendering (12805ad)
  • include snippet to handle loading env's at runtime instead of compile (a52308c)
  • infinite refetch on public page settings (41b806c)
  • initial pdf2js UI view tr missing some classes (4783d1b)
  • issues after rebase (46d1bba)
  • keep llm model in env (46d1bba)
  • keep old api key if a new one is not provided (3f1c709)
  • keep same tooltip size in shared ui (7a1f635)
  • log errors into prom metrics (21dc391)
  • make Dockerfile.worker to install poppler (3253c1c)
  • make pdf2md bind to host port (5276259)
  • make timeout error respond with json (2e71d44)
  • mispelled "heading" (ca145a4)
  • modal search position changing on height change + pre tags overflowing (8e14ae2)
  • modify public page types to correct values (9345f17)
  • move cta to far right (b2c0344)
  • pdla server returns floats (46d1bba)
  • prevent duplicate users in orgs (7ddb0d6)
  • properly update chunk full-stop instead of only in Qdrant for (6b05342)
  • public page code indent (beb5e51)
  • python client search reqpayload (ebcd0e8)
  • python SDK search within groups (16e1c59)
  • restore chunk_html to only finding one heading per chunk (7babdaf)
  • revert failed inserts in qdrant_only mode (792def0)
  • safari component height (1dec2ce)
  • search component top position + transparent right arrow instead of hidden (993e835)
  • search/ai queries not showing (c61da81)
  • set 10ms timeout on key input to prevent double key press (23d25c4)
  • set placeholder (8c1a0c0)
  • skip first page of rendered images (da7d930)
  • timeout long standing or failed chat completions (23fc3a2)
  • typo in tooltip (b903637)
  • update bundler (c57520b)
  • update version ranges to allow react 19 (a80291e)
  • user test delete api key using key instead of id in user.test.ts for ts-sdk (1827bf7)
  • working chat with groups (first message only) (2b6ff4d)
  • wrong metatag title name for PDF2MD (9757d04)

Other

  • create seperate docker-compose.yml files for dev and prod (f82f8d2)
  • release search component v0.2.5 and bump in public page (00aa6ef)

Docs

  • add vanilla js method (3c9234e)
  • change style import example on public page (f2c8118)
  • create k8s hosting guide (4a6fa27)
  • include example for problemLink on search-component README (dc6b239)
  • setup README with setup guide and hosting guide (7ba280b)
  • Typo Fix (83f79dd)

Refactoring

  • complete rename from file-chunker to pdf2md (a9eed78)
  • remove task_id and change to id for simplicity of data model (6f899e3)

0.12.1 (2024-10-25)

Features

  • /dataset/visiblity route to the server to set visibility with (ed2c4c0)
  • add /topic/clone to the ts-sdk (8b6fe7e)
  • add debounce for getCounts, search and suggestedQueries (992d8cc)
  • add glasskube argocd app (afc7497)
  • add group products checkmark to crawl settings (a4e23c7)
  • add group variants flag (070a6a5)
  • add ImageConfig for sending images to the llm (1f12579)
  • add kustomization (9480d6d)
  • add noResultsLink + styling: cleanup dark for active tab and search toggle (93a83f2)
  • add noResultsLink + styling: cleanup dark for active tab and search toggle (84479c6)
  • add open and onOpenChange as params for search component (cde5fa2)
  • add packages kustomization (92ce2b0)
  • add prices and images to ecomm chat (9097194)
  • add search component example code (9b9d818)
  • add shopify option to frontend (745d4ea)
  • add shopify scraping (d3a6609)
  • add site path to search results (60ad84a)
  • add the ability to remove strings from titles and content during crawl (8517c98)
  • add trieve hackernews application (9b5620e)
  • add web component (4ca266c)
  • added route to the backend for group_variants (6b2c7d5)
  • added useImages to the frontend as a parameter (c2db61c)
  • allow custom pre and post tags for highlights (c2f3777)
  • allow for specifying allow_external_links and allow_backward_links (aeb9495)
  • allow rag to use group search w/ a size of 1 (95debf6)
  • allow user to specify that chunks' links should be included in generated output (baa9d39)
  • authorization for webhook (e0c060c)
  • auto focus input field (81a0567)
  • bm25 settings when creating new dataset (e1346c7)
  • change snake case to camel case (82035db)
  • checkpoint where scraping of Trieve site + docs works correctly (a84bf93)
  • clone topic and message route (4471cb8)
  • clone topic in chat frontend (71c20d6)
  • consistently link to the id anchor when possible from chunks in the search component, + cleanup: remove happy-dom (0f6ff50)
  • create inital builderio webhook (c4d7ef2)
  • create operator and impl system for content (3f472a4)
  • delete applicationset (0e29e5a)
  • docusarus theme plugin respects color scheme (2082bc3)
  • docusaurus search plugin (6c1c393)
  • event data explorer (a3303c6)
  • fix button element position (23602b4)
  • fixups and auth (2defca7)
  • frontend page to toggle visibility + server dataset config for (78ea4d4)
  • get all event types + bugfix: escape quotes properly for analytics (bc302dc)
  • get page and deny access if no permission is given (8457683)
  • group search in component (ea76b78)
  • headless analytics (a59466d)
  • jinja baseline (990aca5)
  • json parsing (9810fae)
  • lazy load search component (743a068)
  • let embeddings share, gpu, update hash, increase splade-query doc memory (72dfab6)
  • link custom events to actual events (c7724e5)
  • make headings less noticable & extract heading heirarchy from chunks (6d08f4b)
  • move package into folder (596740f)
  • move to HEAD (8c0187d)
  • new org modal in org select page (2250568)
  • page.html renders template arguments passed through and react (646ec78)
  • prevent switching modes (84c7039)
  • properly shrink on page width decrease (f1f63e0)
  • remove @radix-ui/react-dialog and downshift (90abf0a)
  • remove kustomization (22175f0)
  • remove syntax highlighting to reduce bundle size (28e51df)
  • reverted previous code, added the rest needed for end to end (d84d6e1)
  • send json blobs to llm for better parsing (5f5270b)
  • set argocd source to pmig/trieve HEAD (5b0fd13)
  • show first 3 lines of results (1943428)
  • show vote score on dashboard rag table (8c97b2e)
  • special logic to skip sending message to firecrawl (0ca6b5b)
  • switch glasskube argocd deployment to main branch (46ee59b)
  • unpublish (e0c060c)
  • upgrade glasskube to v0.24.0 (3c8661e)
  • upgrade to the latest trieve-gcp package (3fafd77)
  • upload chunks (7cd24e7)
  • use correct color theming for search component (7bbd21b)
  • use same clickhouse and postgres message ids (5fc7799)
  • use tmp deployment branch (2733380)
  • utility functions to get TR-QueryId on all RAG based routes (c1dfa03)
  • validation for bm25 (3213aa1)
  • webhook request type (285e825)

Cleanup

  • add get datset by tracking_id to the openapi + bugfix: delete dataset by tracking_id (f41d397)
  • allow user to opt out of autocomplete search mode (653043f)
  • allow user to use distance_factor or boost_factor for semantic_boost (6ae80ed)
  • avoid panic when ditto api key is not set (5c343d8)
  • cargo fmt (35d4ddd)
  • clippy (e0c060c)
  • clippy again (e0c060c)
  • clippy and server (1f2f2fa)
  • component styling + allow it to create its own TrieveSDK (e8bf984)
  • default get total pages to true in search playground (fa71713)
  • fix floating effect below bottom nav on search-component (25064cb)
  • fixed frontend compile err (7da2bac)
  • ignore clippy too many args (b064266)
  • lower Qdrant conn timeout to 10s (bbb00df)
  • make it optional (d03ebef)
  • more fixes (44f6fa5)
  • more types (fe5aae4)
  • prevent empty chunks in crawl (3aee601)
  • remove errant prints from old PRs (f468101)
  • remove lodash from search-component (7504842)
  • remove lodash-es from search component (7cb65bc)
  • remove max depth as a crawl option field (12121a5)
  • require non star include URL for external URLs (bfb540c)
  • spacing (e0c060c)
  • update crawl settings dashboard display (6950c0d)
  • update firecrawl to v39 (e9ddc36)
  • update mintlify widget (f814543)
  • update to latest firecrawl + better logic for subsequent headings (c43d9c8)
  • upgrade free plan usage (05dd442)

Bug Fixes

  • add path after summary if summary is present (b854b23)
  • autofocus chatInput on open (5510dae)
  • clickhouse: do not force default database (b38d422)
  • crawl options UI not including site_url and allow_external_links in spots (620e905)
  • create embeddings on non receiving thread (1d3bc06)
  • disable keycloak builtin ingress (97bf136)
  • eslint (5cb6c43)
  • fix crawl duplicates from www vs non www (05e0fd6)
  • fix viewing group search results in query details (59e7e09)
  • fixed embedding generation (4c58eb6)
  • handle untagged delta and finish_reason on generate_from_chunks streaming (b153229)
  • handle untagged delta and finish_reason on stream_response (287d679)
  • keep track of message id for voting (5dcf9c1)
  • keyboard nav for search component (a420342)
  • metadata in result card nested too deeply (f816e63)
  • prevent abort signals from printing console.error's (2a952cc)
  • RAG on specified chunks on group page (3503945)
  • remove session_state parameter (feb0aa6)
  • remove trailing slahes from URLs when processing a crawl (b44042c)
  • remove use of error.name instead just use error (653979d)
  • request profile and email scopes (0b4ec80)
  • sanitize organization name to prevent html injections (4559110)
  • stream response false was still sending back a stream (d144bbe)
  • support non-uuid claim subjects (23db60f)
  • update release CI action to push dittofeed and crawl worker (7636b56)
  • use the search events search id in RAG route (4b04a36)

Other

  • add example configuration (8566b5f)
  • add keycloak extra service with gcp backend config POC (383fc7f)
  • add LICENSE for components in package.json and in folder (95f7562)
  • change to single ingress for all services and add gcp managed certificate (08cfa7e)
  • fix keycloak backendconfig (e52d95f)
  • move combined ingress to base (a60aa4d)
  • release v0.12.1 (01d1526)

Docs

  • add additional docstrings and titles (fc80008)
  • add more documentation of group_variants and CrawlShopifyOptions (8aa928e)
  • add TR-QueryID response header to the redoc spec (f60a300)
  • cleanup response types for /chunk/search (f59a834)
  • update example to use responsive: true in docusarus (38e1614)
  • update TR-Dataset and TR-Organization to be uuid::Uuid everywhere (2345823)
  • updated example to have a 200 ms wait time (84fda93)

Refactoring

  • using ScrapingOptions enum instead of is_shopify / (667e9ba)

0.12.0 (2024-10-07)

Features

  • add ability to stop generating message (bc52fe4)
  • add api name to api keys table (a03d8e0)
  • add aws overlay (ca08412)
  • add back button to data explorer (f72f3e7)
  • add chunk counts (b6bdd17)
  • add chunk counts (cff99cb)
  • add css variables for colors (c2483b8)
  • add ctr buttons to search application (00671e1)
  • add ctr data to analytics page (2bc5a30)
  • add dark mode to new chat (2714405)
  • add dataset event types for crawl success/fail (b6dca54)
  • add defaultSearchMode (8a27f95)
  • add editMessageReader and regenerateMessageReader methods (85f1631)
  • add export to more tables; fix obejct export (61d21f9)
  • add final CSV exports (8c0b1bb)
  • add form controls (5575e05)
  • add images and price (3dda75b)
  • add llm response to frontend (aba46a1)
  • add missing resource limits (2369c42)
  • add new alaytics functions to sdk (dc22b6f)
  • add openapi crawl options (6bde73a)
  • add OpenAPI specs to crawling setup (eb7a5db)
  • add popup to rag (69a1096)
  • add price (4831a74)
  • add rate rag route (64657e9)
  • add rate search (54fa779)
  • add refresh chunk count button to dashboard (9aa5f1c)
  • add scraping config as an option to setup when creating a dataset (7a9f36c)
  • add sidebar links (b247c25)
  • add suggested queries prop (b25fa75)
  • add suggested queries to search component (e7c4f80)
  • add tags to search component (73640ee)
  • add tanstack router to have new route for e commerce (c4acaff)
  • add top_score to recommendations-explorer (8170839)
  • add TR-QueryID to ALL llm response routes (62d9317)
  • add track events for users (ee715fd)
  • add trieve aws resource limits (a8ae92c)
  • add TS instructions tab to get started code (43b69d2)
  • add variants change image (c50d12f)
  • added borders to the top and bottom (633d2e6)
  • added new function to trieveSDK to get queryID, and wrapping up (e47ef12)
  • allow for boosting to be prevented when scraping (569d55b)
  • allow stop generate when its loading (da56f5d)
  • allow user to choose the key combination that opens the modal (3d268f2)
  • allow user to pass suggested queries (fdb22d1)
  • change color of words rather than highlight (54ef8b1)
  • clear chat history in search component (e9643f8)
  • context fixes and set tracking id (4f753ac)
  • copy ids in dashboards (fb2bdd9)
  • create new toggle & delete custom item (ea4a7bf)
  • datasetstart checklist component for easier onboarding (68560aa)
  • delete and clear chunks (e41183a)
  • descending order messages and keep it on the same page (2f64815)
  • ditto scraper (4eeb386)
  • docker image + include top search queries and low confidence queries (e910429)
  • don't kill process if dittofeed fails (790a6d4)
  • dont filter duplicates when getting a single query (cb0e85c)
  • first draft at making csv export (a61530b)
  • fix chart alignment (06d8574)
  • fix long org search (6dc14f8)
  • fix single query page (0ed94b1)
  • generate off chunks pass TR QueryID (d65ede3)
  • get crawl settings by dataset id (390e88e)
  • get custom events with a metadata filter and move get CTR data under /events/ (9020e51)
  • getting started links (439a798)
  • inital tw macro and search analyltics (7045621)
  • input validation (10adab6)
  • make tracking_id: "" delete the tracking_id on PUT /api/dataset (5c4a3dc)
  • make button large & only have border (bde22b1)
  • make login screen look better (d13ac3f)
  • make request examples better (cdab105)
  • make signin button highlight when both fields are present (70ba588)
  • merge scraping options on update (1f3a8bf)
  • mobile view to be full screen (d4d74e5)
  • modify behavior to no longer use the title from metadata (0cc2cbb)
  • more convenient api route scoping (eee2e05)
  • more form fields (b4d5d5e)
  • move all files (a239b48)
  • move question to bottom && add suggested questions (5f972e3)
  • move rag analytics (beb3f2a)
  • move search analytics to data explorer (6321399)
  • move to use events endpoints (4470639)
  • pass query id as a header on all rag messages (785927e)
  • prefix TR-QueryID as X-TR-QueryID (c239164)
  • rag data explorer (791d687)
  • recommendations explorer (f0a6fa1)
  • refactor org selector and dataset selector display (37a8f4e)
  • remove h-fit JavaScript (8a279f1)
  • replace all https to http (f92e6f9)
  • send events on dataset crate (9aea394)
  • send to dittofeed on new user (271358e)
  • show org+dataset selector on dataset page and change text to make dataset creation more obvious (928d4b5)
  • show reload button to refetch qiueries (d4c4df6)
  • small fixes (6e3eaf8)
  • spec'd out everything for sending recommendations except for the (f37da18)
  • style openapi req verbs (19e5a2e)
  • update dashboard for rag messages to use search_data.results (f9e03f8)
  • update docker-compose.yml to work with clickhouse (a427090)
  • update ts sdk (aa242ca)

Enhacements

  • add npm install command to trieve-ts-sdk (82d4bf0)

Cleanup

  • abortcontroller's for search and clearing results for empty queries (f178484)
  • add python SDK and remove duplicate detection feature from README (e2c5082)
  • add tooltips and advanced options toggle to create dataset modal for ease of use (58fbe8c)
  • adjust styling for assistant message (7278a3f)
  • allow println's (83e017e)
  • apply light edits to scraper config for better defaults and link tracking plus empty chunk avoidance (51663e6)
  • dedup RAG citations so there are not multiple of the same (7baea76)
  • do not error creating dataset is ditto req fails (1628340)
  • docsearch chat component citations + no-margin paragraph for suggested queries (7752f8a)
  • fix auth issues for /top analytics route (f4cc2ac)
  • fix the names of the analytics request models (604fda3)
  • fixed messed up spacing, only modify height to h-fit on (c263e6a)
  • handle dashboard redirects for 404s for browsers with saved paths (4bfaafa)
  • handle toggle state for search/chat slightly better + tags remove shortcuts (f2f8ed2)
  • make event data and non-destructive RAG easy from dash (53b1298)
  • make playgrounds first in sidebar (217e5df)
  • make tables first in analytics tabs (251f593)
  • moved the graphs above the tables in data explorer (8548938)
  • no longer need to provide org_id when calling routes relating to org (36c7ed7)
  • prevent panic from split_at in search_operator (540a6d9)
  • put scrape config (85f4954)
  • remove analytics stuff (2e30b80)
  • remove eslint_report.json from repo (866da79)
  • remove extra file (6646ed9)
  • remove more analytics stuff (c08640c)
  • remove Union types caused by V2 response type upgrade (9370dff)
  • removed extra console logs (4475e06)
  • rename analytics overview page (6ccaf2c)
  • score has toPrecision(4) (a8ac997)
  • scraper links to heading id or text fragment (c092759)
  • scraping trailing slashes + update: boost titles option in newdataset modal (5510fd9)
  • search component rag scroll bug fixed (6ea7025)
  • simplify tracking_id hashing during scrape to just URL and heading (6a8d4ec)
  • sync dataset page chunk count and datasets (5aeb503)
  • update frontend types (6f086b3)
  • use nullish for schema (30c55f0)
  • use postgres component (ec09919)
  • various dashboard improvements like dataset selector bugfixes and better text in create dataset modal for info (bd4da07)

Bug Fixes

  • /api suffix replace for dashboard's trieve SDK usage (e47cd4d)
  • add handlers to lib.rs (ff655a5)
  • add ingress annotations of aws lb controller (eb74c6e)
  • add keycloak ingress (bb40767)
  • always overwrite openapi options (ffd043f)
  • avoid regen'ing suggestions immediately + refactor: chat even more chat-like (49ded2a)
  • better page count and chunk created count (95eccf5)
  • better placeholder + focus (a07dbb0)
  • break out of loop if lasts more than 60s (0894336)
  • cargo clippy (756aaeb)
  • clear errors on successful submit (9a108c2)
  • concat user messages in search component (ae81f97)
  • correct className conflict of footer (47640e7)
  • datasets in table not reloading when org selected changes (25e2681)
  • default autcomplete function to api version v2 (dd976b8)
  • dont collapse rag queries (cdf7e38)
  • ensure that suggested queries only executes once (3c2187d)
  • ensure that the search component is sending the request id (9356953)
  • ensure that when clicking on llm response, it shows the right one (ed7d79e)
  • eslint (4b8a261)
  • eslint (e64b48e)
  • eslint (e4d25c3)
  • find not a function for dataset memo (a21096e)
  • fix color vars for the outlines (3640949)
  • fix cors error and allow TR-QueryId to be returned through CORS (94052ae)
  • fix crawl worker docker push action (320301d)
  • fix data explorer links (53ebf2c)
  • fix double render (af07178)
  • fix footer on chat (407ff07)
  • fix pr comments; fix input border (581c382)
  • fix question asking on suggested and ask ai button clicks (02a04c8)
  • fix scroll behavior (db2fe12)
  • fix unwrap error with getting ctr search queries (60575ab)
  • fixed right overflow (25eefd1)
  • footer padding fix (8aa4c30)
  • get heading correctly for firecrawl (5079166)
  • get total pages for group search (72d7498)
  • getting wrong crawl options (4f7ba0c)
  • head queries table (4aae83c)
  • include llm response on RAG analytics routes and fix docs (405f7e0)
  • inconsistent page size for hybrid group search (b554c97)
  • inital tracking id loading (fc1b01a)
  • input not showing when id is set (5169184)
  • layout shift from onboarding steps (6219518)
  • log if identity call fails (c75614f)
  • make gpu component work for multiple embeddings (c12de90)
  • make sure to set current_chunk to correct one (31b586d)
  • map not existing in NavbarDatasetSelector (8c6742b)
  • match scrollbar style (9662ef0)
  • missing packages (3615cee)
  • mobile and scroll fixes (93027df)
  • more specific /api replacement on environment variable (378eed1)
  • move props setting to parent (2f6dc2c)
  • navbar not showing up on search playground (6dbd8d2)
  • only put sdk client in debug mode for dev (ae3f8a6)
  • prevent unwrap error on analytics (ec953a6)
  • remove allow_any_header() to fix cors issue (20c3241)
  • remove allowed_header and set cors to allow any header (e6ec903)
  • remove buggy skeleton (021a8b8)
  • remove default namespace (22c3c42)
  • remove duplication (9d38c40)
  • remove rag from sidebar (43a7cc6)
  • remove unused event (82c3f40)
  • scope top datasets to organization (c69bd5a)
  • scroll in next frame (c52da89)
  • search by dataset name instead of id (6371d14)
  • skip highlighting text when only 1 word needs highlights (e6d6918)
  • small tanstacktable type fixes (e6a85d9)
  • styling for analytics, infinite hmr loop, skeletonize all cards (63bad98)
  • terracotta select dropdown submits forms (730979c)
  • update analytics context (e29cd1f)
  • use proper brand colors on chat elements (291d58d)
  • use rag_data to show results instead of search_data for RAG page (02c73e5)
  • use regex to remove and add back chunk_html to properly get (db55ad1)
  • vaccuum lint (5c90990)
  • working add user modal (3d317e5)
  • zod schema (f7992d8)

Other

Docs

  • created quickstart development for search-component in readme (cb46473)

0.11.9-pre (2024-09-20)

Features

Cleanup

  • better CSS scoping for search component (87e847a)
  • change default prompt such that LLM does not provide citations by default (6878f85)
  • fix 2x typos of 'likeclickthrough' (4041788)
  • improve pypi page for python sdk client (cd20b77)
  • move towards html output from firecrawl (56f5c2b)
  • properly handle fallback types for redis for Rust '24 version (6e14d18)
  • put port back (1f21b1e)
  • remove extra file (56da639)
  • remove tooltip and globe icons (5831b53)
  • small fixes (5004a1b)
  • update docker compose .env to include firecrawl (5404c91)
  • update readme (aa46afb)

Bug Fixes

  • add better margins to headings (868bc7c)
  • add delete invitation to openapi spec (7a2b4c0)
  • add get invitations to openapi spec (5f33336)
  • assorted fixes on component from signoz feedback (122801a)
  • auth edge cases (73b888d)
  • concate short chunks to next one (d4dd68d)
  • css fixes for rag page (40bd6ea)
  • dataset warning for chat (82b8f21)
  • ensure that dataset that is scrape to is the new dataset that is created (366380b)
  • eslint (fe47d48)
  • eslint (842cf18)
  • event log (0f5c2f7)
  • extract type (22e0028)
  • fill empty dates in rag usage chart (2537a6d)
  • firecrawl scraping logic next behavior solve (c5103ab)
  • fix copy feature to allow for plain text in chat (f4f2a2c)
  • fix user cache flashing org select (505b5d8)
  • improve start services and add tmuxp to README local Dev guide (2473c0a)
  • incorrect schema type for getting an org (4061254)
  • links to dashboard from search and analytics SPA's post dash rewrite (afcc836)
  • move table code to the top of component (6631ee7)
  • no datasets warning for chat (04a7775)
  • only open modal when there are results (d93b9ef)
  • openapi schema issues w/ dup def for /invitation/{foo} (67af1f8)
  • remove fake delay from testing (ead31c4)
  • remove old-dashboard from monorepo (9f79b46)
  • remove TR-Dataset requirement for create dataset (790c3a9)
  • set topic list to empty if dataset is invalid (b958ed6)
  • show all info in the no results table (36c948e)
  • suspense height tracking (f1d68ed)
  • tsc check (68495dc)
  • use camel case for scrape options (7661e2f)
  • use cn helper && add classname prop to table (8d78f6b)
  • wait for firecrawl if scraping (bf4f298)

Other

Docs

  • fix getting started README docs (6410745)