Build From Source

Overview

This guide provides step-by-step instructions for building the mssql-python module from source. The process involves setting up a build environment, installing dependencies, compiling the source code, running tests, and publishing the build artifacts. The pipeline configuration provided below is designed to automate these steps using Azure Pipelines.

Pipeline Configuration

The following pipeline configuration is used to build and test the mssql-python module on a Windows environment. The pipeline includes steps for setting up Python, installing dependencies, building the module, running tests, and publishing the results.

Pipeline Definition

name: test-pipeline

jobs:
- job: PytestOnWindows
  pool:
    vmImage: 'windows-latest'
  
  steps:
  - task: UsePythonVersion@0
    inputs:
      versionSpec: '3.13'
      addToPath: true
      # Added a never expire public repo read only Github token in ENV since the new python version is downloaded from GitHub
      githubToken: $(GITHUB_TOKEN)
    displayName: 'Use Python 3.13'

  - script: |
      python -m pip install --upgrade pip
      pip install -r requirements.txt
    displayName: 'Install dependencies'

  - script: |
      cd mssql_python\pybind
      mkdir build
      cd build  
      cmake -DPython3_EXECUTABLE="python3" -DCMAKE_BUILD_TYPE=Debug ..
      cmake --build . --config Debug
      copy Debug\ddbc_bindings.pyd ..\..\ddbc_bindings.pyd
    displayName: 'Build .pyd file'

  - script: |
      python -m pytest -v --junitxml=test-results.xml --cov=. --cov-report=xml --capture=tee-sys --cache-clear
    displayName: 'Run tests with coverage'
    env:
      DB_CONNECTION_STRING: $(DB_CONNECTION_STRING)

  - task: PublishBuildArtifacts@1
    inputs:
      PathtoPublish: 'mssql_python/ddbc_bindings.pyd'
      ArtifactName: 'ddbc_bindings'
      publishLocation: 'Container'
    displayName: 'Publish pyd file as artifact'

  - task: PublishTestResults@2
    condition: succeededOrFailed()
    inputs:
      testResultsFiles: '**/test-results.xml'
      testRunTitle: 'Publish test results'

  - task: PublishCodeCoverageResults@1
    # Have to resort to using PublishCodeCoverageResults@1 because PublishCodeCoverageResults@2 does not use ReportGenerator 
    # https://stackoverflow.com/questions/78200629/azure-publishcodecoverageresults2-issue-with-detailed-report-for-coverage
    # https://devblogs.microsoft.com/devops/new-pccr-task/?commentid=4300#comment-4300
    # There will be warnings in the logs about this, but the coverage report will still be published, and the pipeline will not fail
    # Alternatively, we can use the ReportGenerator tool to generate the coverage report and then publish it using the v2 task.
    inputs:
      codeCoverageTool: 'Cobertura'
      summaryFileLocation: '$(System.DefaultWorkingDirectory)/**/coverage.xml'
    displayName: 'Publish code coverage results'

Steps Explanation

Use Python 3.13: This step sets up Python 3.13 on the build agent. It also adds Python to the system path and uses a GitHub token to download the new Python version.

Install Dependencies: This step upgrades pip and installs the required dependencies listed in the requirements.txt file.

Build .pyd File: This step navigates to the pybind directory, creates a build directory, and uses cmake to configure and build the project. The resulting .pyd file is copied to the appropriate location.

Run Tests with Coverage: This step runs the tests using pytest, generates a test results XML file, and collects code coverage information. The database connection string is provided as an environment variable.

Publish .pyd File as Artifact: This step publishes the built .pyd file as a build artifact, making it available for download.

Publish Test Results: This step publishes the test results XML file, allowing you to view the test results in the Azure Pipelines interface.

Publish Code Coverage Results: This step publishes the code coverage results using the Cobertura format. It uses the PublishCodeCoverageResults@1 task to ensure compatibility with the ReportGenerator tool.