SUMMARY.md

Table of contents

• Overview
  • Features
    • Data pipeline management
  • Concepts
    • Core abstractions
    • Core design principles
    • Blocks
      • Data loading
    • Actions
      • Transformer actions
  • Use cases
• Documentation
  • Quick start
    • Setup and install
    • Load, transform, and export restaurant data
      • Use a completed pipeline
    • Train model on Titanic dataset
      • Use a completed pipeline
  • Tutorials
    • Set up dbt models & orchestrate dbt runs
    • Integrate Mage into an existing Airflow project
    • Clean
    • Data cleaning aka data cleansing
    • Data cleaning workflow
    • Using the tool from end-to-end
  • Guides
    • Blocks
      • R blocks
      • SQL blocks
      • Sensors: blocks depending on external pipelines
    • Pipelines
      • How to build a data integration pipeline
      • Streaming pipeline
    • Data loader
      • Load data from S3
    • Data exporter
      • Export data to S3
    • Triggers
      • Schedule pipelines to run periodically
      • Events
        • Trigger pipeline from AWS Event
    • dbt
      • Add an existing dbt project to Mage
      • dbt connection profiles
      • dbt sources and upstream dependencies
      • Run a single model
      • Run selected models (and optionally exclude others)
      • Running dbt tests
    • Version control
      • Setting up Git on cloud
  • Data integrations
    • Sources
      • Adapt an existing source
      • Add a new source
      • Change Data Capture (CDC) with PostgreSQL
    • Destinations
      • Adapt an existing destination
      • Add a new destination
    • Mage integrations
      • Sources
        • Schema
        • Amplitude
        • BigQuery
        • Google Search Console
        • Google Sheets
        • Intercom
        • MySQL
        • PostgreSQL
        • Redshift
        • Salesforce
          • Client
            • Blacklisted Objects Explanation
            • Changelog
        • Stripe
      • Destinations
        • BigQuery
        • MySQL
        • PostgreSQL
        • Redshift
        • Snowflake
      • Connections
        • Example Amplitude connection
        • Google Analytics
        • PostgreSQL
  • Code block integrations
    • BigQuery
    • Druid
    • MySQL
    • Pinot
    • PostgreSQL
    • Redshift
    • S3
    • Snowflake
  • Kernels
  • Development
    • Multi-development environment in the cloud
  • Production
    • Run pipeline in Airflow
    • Compute Resource
    • Run pipeline in Prefect
    • Runtime Variables
  • Testing
  • Triggers
    • Triggering pipeline via API request
  • Deploy
    • AWS
    • CI/CD
    • Terraform
      • Terraform Amazon Web Services (AWS)
        • Terraform AWS resources
      • Terraform Microsoft Azure
        • Terraform Azure resources
      • Terraform Google Cloud Platform (GCP)
        • Terraform GCP resources
  • Observability
    • Alerting
      • Email
      • Slack
    • Monitoring
  • dbt
  • Spark
  • Frontend
  • Running the server
  • Database migrations
• Contributing
  • Contributing guide
  • Jupyter
  • Testing
  • Adding a chart