TruLens provides a set of tools for developing and monitoring neural nets, including large language models. This includes both tools for evaluation of LLMs and LLM-based applications with TruLens-Eval and deep learning explainability with TruLens-Explain. TruLens-Eval and TruLens-Explain are housed in separate packages and can be used independently.
The best way to support TruLens is to give us a ⭐ on GitHub and join our slack community!
Don't just vibe-check your llm app! Systematically evaluate and track your LLM experiments with TruLens. As you develop your app including prompts, models, retreivers, knowledge sources and more, TruLens-Eval is the tool you need to understand its performance.
Fine-grained, stack-agnostic instrumentation and comprehensive evaluations help you to identify failure modes & systematically iterate to improve your application.
Read more about the core concepts behind TruLens including [Feedback Functions](https://www.trulens.org/trulens_eval/getting_started/core_concepts/ The RAG Triad, and Honest, Harmless and Helpful Evals.
Build your first prototype then connect instrumentation and logging with TruLens. Decide what feedbacks you need, and specify them with TruLens to run alongside your app. Then iterate and compare versions of your app in an easy-to-use user interface 👇
Install the trulens-eval pip package from PyPI.
pip install trulens-evalTo install the latest version from this repository, you can use pip in the following manner:
pip uninstall trulens_eval -y # to remove existing PyPI version
pip install git+https://github.com/truera/trulens#subdirectory=trulens_evalTo install a version from a branch BRANCH, instead use this:
pip uninstall trulens_eval -y # to remove existing PyPI version
pip install git+https://github.com/truera/trulens@BRANCH#subdirectory=trulens_evalWalk through how to instrument and evaluate a RAG built from scratch with TruLens.
Interested in contributing? See our contributing guide for more details.
TruLens-Explain is a cross-framework library for deep learning explainability. It provides a uniform abstraction over a number of different frameworks. It provides a uniform abstraction layer over TensorFlow, Pytorch, and Keras and allows input and internal explanations.
These installation instructions assume that you have conda installed and added to your path.
- Create a virtual environment (or modify an existing one).
conda create -n "<my_name>" python=3 # Skip if using existing environment.
conda activate <my_name>- Install dependencies.
conda install tensorflow-gpu=1 # Or whatever backend you're using.
conda install keras # Or whatever backend you're using.
conda install matplotlib # For visualizations.- [Pip installation] Install the trulens pip package from PyPI.
pip install trulensTo install the latest version from this repository, you can use pip in the following manner:
pip uninstall trulens -y # to remove existing PyPI version
pip install git+https://github.com/truera/trulens#subdirectory=trulens_explainTo install a version from a branch BRANCH, instead use this:
pip uninstall trulens -y # to remove existing PyPI version
pip install git+https://github.com/truera/trulens@BRANCH#subdirectory=trulens_explainTo quickly play around with the TruLens library, check out the following Colab notebooks:
- PyTorch:
- TensorFlow 2 / Keras:
For more information, see TruLens-Explain Documentation.