Skip to content

Latest commit

 

History

History
148 lines (111 loc) · 2.99 KB

README.md

File metadata and controls

148 lines (111 loc) · 2.99 KB

eval-track

Overview

eval-track is a series of LLM-ML observability and tracking services. This series provides APIs for retrieving and storing trace data across multiple programming languages.

Components

  • tracker: data tracking services
  • web: provides API ,and viewer and analyzer of tracked datasets
  • worker: async dataset procedure service that transform non-structured data into structured

Contents

Development Setup

Prerequisites

  • Python 3.8+ with uv
  • Node.js 18+ with pnpm
  • Go 1.20+
  • Rust with Cargo
  • Task for running development commands

Installation

  1. Install Task (if not already installed):

    npm install -g @go-task/cli

  2. Set up Python environment:

    task setup-uv  # if uv not installed
task install-tracker

  3. Install TypeScript dependencies:

    cd tracker-ts
pnpm install

  4. Install Go dependencies:

    cd tracker-go
go mod download

  5. Install Rust dependencies:

    cd tracker-rs
cargo build

Testing

Run tests for each implementation:

# Python tests
task test-tracker
uv run pytest tests/

# TypeScript tests
cd tracker-ts && pnpm test

# Go tests
cd tracker-go && go test ./...

# Rust tests
cd tracker-rs && cargo test

Code Quality

Format and lint the code:

# Python
task check-all-tracker  # Runs ruff format, ruff check, and mypy

# TypeScript
cd tracker-ts && pnpm lint

# Go
cd tracker-go && go fmt ./... && go vet ./...

# Rust
cd tracker-rs && cargo fmt && cargo clippy

Documentation

Local Development

To view and develop the documentation locally:

  1. Install MkDocs and the Material theme:

    pip install mkdocs mkdocs-material

  2. Start the documentation server:

    mkdocs serve

    This will start a local server at http://127.0.0.1:8000

  3. Build the documentation:

    mkdocs build

    This will create a site directory with the built documentation.

Contributing

We welcome contributions to the eval-track project. To get started, follow these steps:

  1. Clone the repository:

    git clone <repository-url>
cd eval-track

# optional
# if task not installed
# other installation is https://taskfile.dev/installation/
npm install -g @go-task/cli
task setup-uv # if uv not installed

  2. Install the necessary dependencies:

    task install-tracker

  3. Run the application:

    task start-tracker-app

  4. Run code quality checks:

    task check-all-tracker  # Python checks