Skip to content

feat: Add comprehensive analytics API with OpenTelemetry observability #1639

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

alvidofaisal
Copy link

@alvidofaisal alvidofaisal commented May 24, 2025

Overview

This PR introduces a comprehensive analytics system with OpenTelemetry instrumentation for enhanced observability and monitoring capabilities.

Features Added

Analytics API Endpoints

  • User Analytics (/api/v1/analytics/user-summary):

    • Total user count
    • Active/inactive user distribution
    • Signup trends (when creation timestamps are available)
  • Item Analytics (/api/v1/analytics/item-trends):

    • Total item count
    • Creation trends (when creation timestamps are available)

OpenTelemetry Integration

  • Distributed Tracing: Full request tracing across the application
  • Database Instrumentation: SQLAlchemy query monitoring
  • FastAPI Auto-instrumentation: Automatic HTTP request/response tracing
  • CI-Friendly Configuration: OTLP exporter only activates when endpoint is explicitly configured
  • Production Ready: No connection timeouts in test/CI environments

Performance Optimizations

  • Polars Integration: Lightning-fast in-memory data processing for analytics
  • Efficient Data Aggregation: Optimized queries with minimal memory footprint
  • Structured Response Models: Type-safe Pydantic models with modern Python 3.9+ annotations

Technical Implementation

New Dependencies

polars              # High-performance DataFrame library
duckdb              # In-memory analytical database
opentelemetry-api   # OpenTelemetry core API
opentelemetry-sdk   # OpenTelemetry SDK
opentelemetry-exporter-otlp           # OTLP protocol exporter
opentelemetry-instrumentation-fastapi # FastAPI auto-instrumentation
opentelemetry-instrumentation-sqlalchemy # SQLAlchemy instrumentation

File Changes

  • backend/app/api/routes/analytics.py - New analytics endpoints (202 lines)
  • backend/app/core/telemetry.py - OpenTelemetry configuration (47 lines)
  • backend/app/tests/api/test_analytics.py - Comprehensive test coverage (56 lines)
  • backend/app/api/main.py - Router registration
  • backend/app/main.py - Telemetry initialization
  • backend/pyproject.toml - New dependencies
  • README.md - Updated documentation

Configuration

Environment Variables

# OpenTelemetry configuration (optional - prevents CI connection issues)
OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4317"  # Your OTLP collector
OTEL_SERVICE_NAME="fastapi-application"              # Service name in traces

Code Quality & CI

  • Type Safety: Full MyPy compliance with proper type annotations
  • Modern Python: Uses list/dict instead of deprecated typing.List/typing.Dict
  • Linting: All Ruff checks pass with modern Python style
  • Pre-commit Hooks: All formatting and style checks pass
  • CI-Ready: OpenTelemetry configuration prevents connection timeouts in test environments
  • Import Optimization: Clean, organized import blocks with no unused imports
  • Return Types: Explicit return type annotations for all functions

Testing

  • ✅ Complete test coverage for all analytics endpoints
  • ✅ Robust error handling for empty datasets and missing timestamps
  • ✅ Compatible with existing test infrastructure
  • ✅ CI-friendly configuration (no external service dependencies)
  • ✅ MyPy type checking compliance

Benefits

  1. Enhanced Monitoring: Deep insights into application performance and user behavior
  2. Scalable Analytics: Efficient data processing suitable for large datasets
  3. Production Ready: Enterprise-grade observability with OpenTelemetry standards
  4. Developer Experience: Type-safe APIs with comprehensive documentation and full type coverage
  5. CI/CD Friendly: No external dependencies, passes all quality checks
  6. Maintainable: Clean code with proper type annotations and formatting

Future Enhancements

  • User signup trends (requires created_at field in User model)
  • Item creation trends (requires created_at field in Item model)
  • Advanced analytics dashboards
  • Real-time metrics and alerting

…ensive analytics functionality with API endpoints, data processing, observability, and tests
…nTelemetry config for CI, remove unused imports, update type annotations
…pe annotations for all functions - Fix Sequence to list conversion for SQLModel results - Add FastAPI import for proper typing - Fix end-of-file and formatting issues
@alvidofaisal
Copy link
Author

Could a maintainer please add the feature label to this PR? It's needed for the CI checks to pass. This PR adds new analytics functionality with OpenTelemetry observability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant