Releases · truera/trulens

29 Oct 19:40

sfc-gh-jreini

trulens-1.2.1

46f05d0

TruLens v1.2.1 Latest

Latest

Bug Fixes

Don't check for user and account in snowpark sessions because Streamlit apps might hide them. by @sfc-gh-dkurokawa in #1600
catch source code not available in code_line by @sfc-gh-pmardziel in #1592
use float nan in place of numpy for skipped evals by @sfc-gh-chu in #1595
Fix the misspelled trulens-providers-openai package in examples by @SSK-14 in #1601
fix assertion to nan by @sfc-gh-jreini in #1605

New Contributors

@SSK-14 made their first contribution in #1601

Full Changelog: trulens-1.2.0...trulens-1.2.1

Contributors

SSK-14, sfc-gh-dkurokawa, and 3 other contributors

Assets 2

28 Oct 21:31

sfc-gh-jreini

trulens-1.2.0

914b32d

TruLens v1.2.0

What's Changed

Blocking guardrails by @sfc-gh-jreini in #1584
and add dataset preprocessing utils used in benchmarking by @sfc-gh-dhuang in #1559
Use ggshield for local secret scanning by @sfc-gh-jreini in #1585
Clean before uploading docs. by @sfc-gh-dkurokawa in #1594
Update dev guide with git lfs instructions by @sfc-gh-chu in #1597

Bug Fixes

(some) release pipeline fixes by @sfc-gh-pmardziel in #1537
bumping conda package build to 1.1.0 by @sfc-gh-srudenko in #1557
Fix ground truth dataset persistence notebook after the ground truth search metrics update by @sfc-gh-dhuang in #1558
import style by @sfc-gh-pmardziel in #1543
warning and docpage for bad context by @sfc-gh-pmardziel in #1565
test dummy endpoints by @sfc-gh-pmardziel in #1566
fix docs for snowflake connection by @sfc-gh-srudenko in #1576
Use conda channel trulens packages by default. by @sfc-gh-dkurokawa in #1570
Fix 'reason not generated' by @dom7kim in #1561
import rename listings by @sfc-gh-pmardziel in #1568
Use SnowflakeConnector in stored proc. by @sfc-gh-dkurokawa in #1580
Reuse Snowpark session during most tests. by @sfc-gh-dkurokawa in #1536
Have run_leaderboard should fail more clearly if it's unable to authenticate at Snowflake due to being created by a snowpark_session. by @sfc-gh-dkurokawa in #1581
Add tags to schema during snowflake app creation by @sfc-gh-pdharmana in #1577
Use proper golden set format. by @sfc-gh-dkurokawa in #1587
Fix bad merge for snowflake connector. by @sfc-gh-dkurokawa in #1588
Fix poetry.lock boto3 dependency hashes. by @sfc-gh-dkurokawa in #1590
Always ensure endpoint context variable is cleaned up. by @sfc-gh-dkurokawa in #1589
defaults for each contextvar by @sfc-gh-pmardziel in #1586

Examples

Comparison notebook: TruLens groundedness vs RAGAS faithfulness by @sfc-gh-dhuang in #1559
Add quickstarts to docs by @sfc-gh-jreini in #1583

Full Changelog: trulens-1.1.0...trulens-1.2.0

Contributors

dom7kim, sfc-gh-dkurokawa, and 6 other contributors

Assets 2

10 Oct 13:13

sfc-gh-jreini

trulens-1.1.0

208260f

trulens-1.1.0

What's Changed

TruLens 1.1 has a ton of exciting changes - we've grouped the updates into the new features they support so you can jump straight to the updates you're most excited about:

TruLens Dashboard
Feedback Provider Support
Search Metric Support
Adding dataframes to TruLens
OpenTelemetry Support
Async and Streaming Support
More Reliable Feedback Functions
New Examples
Docs Updates
Bug Fixes

TruLens Dashboard

In TruLens 1.1, we re-imagined the dashboard with a focus on making it easy to track large numbers of experiments, make comparisons and improve your apps for production. We also made several improvements performance and usability including dark mode.

Expanded Search Metric Support

TruLens now supports common information retrieval (search) metrics including IR Hit Rate, NDCG, Precision, Recall, Mean Reciprocal Rank and more. These new metrics are accessible as ground truth feedback functions and simply require the addition of expected_chunks to your ground truth data. Try the example

See the change:

Information retrieval (search) metrics computation with ground truth datasets - notebook + metrics implementation by @sfc-gh-dhuang in #1545

Getting started with existing data

It's now easier than ever to get started with TruLens. Starting with a dataframe with query, response and contexts columns, you can load it to TruLens using add_dataframe and easily run feedback functions against your data. Try it yourself

See the change:

add_dataframe method + quickstart by @sfc-gh-jreini in #1474

Experimental support for Open Telemetry

We've added experimental preview support for Open Telemetry, enabled with session.experimental_enable_feature("otel_tracing") . We are collecting feedback and will be continuing to improve the user experience for writing and reading OpenTelemetry traces. If you want to try it out, check it out with custom python or Llama-Index.

See the changes:

OTEL import/export by @sfc-gh-pmardziel in #1485
experimental flags by @sfc-gh-pmardziel in #1427

Restored Async and Streaming Support

memory, threads, and async leakage testing by @sfc-gh-pmardziel in #1470
fix async handling and other release pipeline failures by @sfc-gh-pmardziel in #1441

More reliable feedback functions

Simplify system prompt generation conditions with output space and criteria by @sfc-gh-dhuang in #1554
handle partial functions for feedback functions by @sfc-gh-chu in #1551
More error handling for groundedness internal steps by @sfc-gh-jreini in #1549
RAG triads llm as judges benchmark - adding meta-eval metrics for correlation measurement and experiment notebooks by @sfc-gh-dhuang in #1462
Add option to filter trivial statements for groundedness measure by @sfc-gh-pdharmana in #1556
Fix splitting key_points issue: generalize the solution for splitting key points in _assess_key_point_inclusion() by @dom7kim in #1519'

Feedback Provider Support

Add mistral-large2 to the list of supported models in Cortex feedback provider by @sfc-gh-dhuang in #1496
Claude 3 support for AWS Bedrock by @sfc-gh-chu in #1481
Switch to llama 3.1 8b as default model in cortex by @sfc-gh-dhuang in #1500
Support having a Langchain provider with a BaseLLM and not just BaseChatModel. by @sfc-gh-dkurokawa in #1459

New Examples

Cortex Fine-tuning experiments notebook by @sfc-gh-jreini in #1453
Cortex Chat Quickstart by @sfc-gh-jreini in #1446 and #1460
Server side feedback computation + batch ingestion by @sfc-gh-jreini in #1464
New Custom Streaming example by @sfc-gh-pmardziel in #1441

Docs Updates

az badge update by @sfc-gh-chu in #1436
docs nits by @sfc-gh-jreini in #1434
Docs Changes by @sfc-gh-chu in #1473
website analytics and dark mode fixes by @sfc-gh-chu in #1497
Add blog site and docs grouping by @sfc-gh-chu in #1499
Fix colab links by @sfc-gh-jreini in #1508
Josh/center homepage image text + change app versions compared by @sfc-gh-jreini in #1442
Fix homepage blog link by @sfc-gh-chu in #1535

Bug Fixes

endpoint kwargs by @sfc-gh-chu in #1489
Update threading.py, fix context loss in multi-threading by @glennfeys in #1478
Fix Selector AttributeError by @sfc-gh-chu in #1553
SQLAlchemy joinedload on record.app relationship by @sfc-gh-chu in #1524
release pipeline related fixes by @sfc-gh-pmardziel in #1435
fix trulens_eval migration link by @sfc-gh-pmardziel in #1448
fix typo in Makefile by @sfc-gh-pmardziel in #1463
Add progress bars to data migration scripts by @sfc-gh-chu in #1458
cortex instrumentation fixes by @sfc-gh-chu in #1447
Conda Meta Hash fix by @sfc-gh-srudenko in #1468
fix optionals in core by @sfc-gh-pmardziel in #1471
fix optional import message by @sfc-gh-chu in #1457
Allow for DBs that already have tables (at least if they're not sqlite databases). by @sfc-gh-dkurokawa in #1449
Bumping conda meta to build a conda package 1.0.2 by @sfc-gh-srudenko in #1479
Move requests/Endpoint.post to huggingface provider by @sfc-gh-chu in #1476
relax minor package version constraints by @sfc-gh-chu in #1482
init_server_side=False by default by @sfc-gh-chu in #1483
slight downgrade of minimum dep requirements by @sfc-gh-chu in #1504
Cleanup main pyproject and relax minor versions by @sfc-gh-chu in #1494
Updates to the query planning notebook by @sfc-gh-dhuang in #1512 and @sfc-gh-jreini in #1514
Conda build meta changes by @sfc-gh-srudenko in #1503
Dashboard fixes by @sfc-gh-jreini in #1518
Bump the pip group across 1 directory with 2 updates by @dependabot in #1507
small record ingest formatting fix by @sfc-gh-chu in #1515
Set criteria for feedbacks correctly. by @sfc-gh-dkurokawa in #1526
Allow using Snowflake Connector fo...