Implement a script to automatically create odbcad configs.

README.md

@@ -16,11 +16,12 @@ Brief explanation of the directory structure under `src`:
   - Download the latest MSSQL Management Studio: https://learn.microsoft.com/en-us/sql/ssms/download-sql-server-management-studio-ssms
     - Note that it can connect to any version of SQL Server, so even if you use 2008, just get the latest Management Studio for better development experience
 - Powershell `sqlserver` module
-  - Open Powershell as Admin and run the following command: `Install-Module -Name sqlserver -Force`
-    - Note that if you're getting errors during the installation, it is likely because because your SQL installation installed its own powershell module which conflicts with the one we intend to use. They're incompatible and behave differently when creating db exports, hence you need to delete it from your System Environment Variable `PSModulePath` and restart powershell to reload without these modules. Basically if you see in your `PSModulePath` environment variable something with `Microsoft SQL Server`, just remove it, since we want to use the module that we intend to use.
+  - Open Powershell as Admin and run the following command: `Install-Module sqlserver -AllowClobber -Force`
+    - Note that if you're getting errors during the installation, it is likely because because your SQL installation installed its own powershell module which conflicts with the one we intend to use. They're incompatible and behave differently when creating db exports, hence you may want to delete it from your System Environment Variable `PSModulePath` and restart powershell to reload without these modules. Basically if you see in your `PSModulePath` environment variable something with `Microsoft SQL Server`, just remove it, since we want to use the module that we intend to use.
 - Python and installing via pip the following packages (if you get errors, run powershell as admin):
   - T-SQL code formatter: `pip install sqlfluff`
   - MSSQL scripter: `pip install mssql-scripter`
+  - Note that if you're using [virtualenv](https://docs.python.org/3/library/venv.html) (recommended), you can simply install the requirements.txt.
 
 
 ### Development
@@ -29,7 +30,7 @@ Note that the development process is inspired from [different db development env
 
 During development we only create migration scripts to alter the current state of the base. The base here refers to the generated scripts in data, procedure and schema.
 
-Every migration script will be prefixed by max 4 leading zeros. For example, `0001_insert_steve_user.sql` will contain an insert statement to `TB_USER` table.
+Every migration script will be prefixed with max 4 leading zeros. For example, `0001_insert_steve_user.sql` will contain an insert statement into `TB_USER` table.
 
 Apart from the benifit of having the database under version control, this also makes it easy to use any SQL version you want. I use both 2008 and 2022 and it works perfectly fine with both.
 
@@ -47,8 +48,14 @@ Maybe in the future this will change if it makes things difficult and we maintai
 
 ### How to use
 
-If you running for the first time the `import.ps1` script, open Powershell as admin and run this command: `Install-Module sqlserver`
+Before running the `import.ps1` script, check that the `$server_name` variable matches with your current server name. If you installed SQL Server using the default instance, then the defaults arguments should work, otherwise you can provide to the script an argument with your custom server name. You can run the following command in powershell to know the names of your current SQL Servers:
+```powershell
+(Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Microsoft SQL Server').InstalledInstances
+```
 
-Before running the `import.ps1` script, check that the `$server_name` variable matches your current server name. If you use the default instance, then defaults should work, otherwise you can provide the script an argument with your custom server name (`.\import.ps1 -server_name "MyServer`).
+Assuming you set a custom name to your SQL Server instance, you may invoke the import command as follows:
+```powershell
+.\import.ps1 -server_name ".\MyCustomInstanceName"
+```
 
-Once you run the script, the database is ready to be used by the server files.
+Once the import script finished importing the db, it will also invoke the `odbcad.ps1` script for you to automatically set odbc configurations so that the server files can connect with your db.

export.ps1

@@ -6,8 +6,6 @@
 param (
     # change server_name if you installed your sql server as a Named Instance.
     # If you installed on the Default Instance, then you can leave this as-is.
-    # If you're still not sure what is your sql server names, you can run the following powershell command:
-    # (Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Microsoft SQL Server').InstalledInstances
     [string][Parameter(Mandatory = $false)]
     $server_name = "localhost",
 
@@ -22,7 +20,13 @@ param (
     $quiet
 )
 
-. "$PSScriptRoot\logger.ps1"
+. "$PSScriptRoot\utils.ps1"
+
+function ValidateArgs {
+  if (-not (ValidateServerNameInput $server_name)) {
+    exit_script 1 $quiet
+  }
+}
 
 
 function GetFileEncoding($Path) {
@@ -46,9 +50,11 @@ function Main {
     MessageError "Error: 'mssql-scripter' command is not available."
     MessageError "Please make sure you have Python installed and then run:"
     MessageError "pip install mssql-scripter"
-    exit 1
+    exit_script 1 $quiet
   }
 
+  ValidateArgs
+
   mssql-scripter `
     -S $server_name `
     -d $db_name `
@@ -106,10 +112,8 @@ function Main {
       "$PSScriptRoot\src\procedure\"
   }
 
-  MessageSuccess "Successfully exported [$db_name] database from [$server_name] SQL server."
+  MessageSuccess "Successfully exported [$db_name] database from [$server_name] SQL server!"
 }
 
 Main
-if (-not $quiet) {
-  cmd /c 'pause'
-}
+exit_script 0 $quiet

import.ps1

@@ -7,8 +7,6 @@
 param (
     # change server_name if you installed your sql server as a Named Instance.
     # If you installed on the Default Instance, then you can leave this as-is.
-    # If you're still not sure what is your sql server names, you can run the following powershell command:
-    # (Get-ItemProperty 'HKLM:\SOFTWARE\Microsoft\Microsoft SQL Server').InstalledInstances
     [string][Parameter(Mandatory = $false)]
     $server_name = "localhost",
 
@@ -18,6 +16,9 @@ param (
     [switch][Parameter(Mandatory = $false)]
     $skip_migration_scripts,
 
+    [switch][Parameter(Mandatory = $false)]
+    $skip_odbc_creation,
+
     # Generate diffs for each migration script that is not archived.
     # Warning: make sure to commit your changes before running the script with this enabled, or you may lose work
     [switch][Parameter(Mandatory = $false)]
@@ -27,10 +28,19 @@ param (
     $quiet
 )
 
-. "$PSScriptRoot\logger.ps1"
+. "$PSScriptRoot\utils.ps1"
+
+function ValidateArgs {
+  if (-not (ValidateServerNameInput $server_name)) {
+    exit_script 1 $quiet
+  }
+
+  if ($skip_migration_scripts -and $generate_diffs) {
+    MessageError "Error: skip_migration_scripts and generate_diffs args are mutually exclusive."
+    exit_script 1 $quiet
+  }
+}
 
-# Note that the script will fail if you don't have powershell sqlserver module installed.
-# To install it, run powershell as admin and execute the following command: `Install-Module sqlserver`
 function ConnectToSqlServer {
   [System.Reflection.Assembly]::LoadWithPartialName("Microsoft.SqlServer.SMO") > $null
   $server = $null
@@ -40,7 +50,7 @@ function ConnectToSqlServer {
     $server.ConnectionContext.Connect()
   } catch {
     Write-Error $_
-    exit 1
+    exit_script 1 $quiet
   }
 
   return $server
@@ -131,19 +141,24 @@ function RunMigrationScriptsAndGenerateDiffs {
 
 function CreateDbCredentials {
   MessageInfo "`n`n### Creating login and user for $db_name... ###"
-  Message "src/misc/create_login.sql"
-  InvokeSqlScript -script_path "src/misc/create_login.sql"
+  $login_script_file = "src/misc/create_login.sql"
+  Message $login_script_file
+  $output_script_file = Join-Path $env:TEMP "create_login_$(New-Guid).sql"
+  (Get-Content $login_script_file) -replace "###DB_NAME###", $db_name | Out-File $output_script_file
+  InvokeSqlScript -script_path "$output_script_file"
 }
 
 function Main {
   # Check if the sqlserver module is installed
   if (-not (Get-Module -Name sqlserver -ListAvailable)) {
     MessageError "Error: The 'sqlserver' powershell module is not installed."
     MessageError "Please open PowerShell as Administrator and execute the following command to install it:"
-    MessageError "Install-Module -Name sqlserver -Force"
-    exit 1
+    MessageError "Install-Module sqlserver -AllowClobber -Force"
+    exit_script 1 $quiet
   }
 
+  ValidateArgs
+
   $server = ConnectToSqlServer
   RecreateDb -server_instance $server
   RunInitialDbScripts
@@ -159,9 +174,14 @@ function Main {
   }
 
   CreateDbCredentials
+
+  if (-not $skip_odbc_creation) {
+    MessageInfo "Creating odbc configurations..."
+    .\odbcad.ps1 -server_name $server_name -quiet
+  }
+
+  MessageSuccess "Successfully imported [$db_name] database into [$server_name] SQL server!"
 }
 
 Main
-if (-not $quiet) {
-  cmd /c 'pause'
-}
+exit_script 0 $quiet

odbcad.ps1

@@ -0,0 +1,151 @@
+# If you invoke the script from the terminal, you can specify the server and db name in case you changed it.
+# Otherwise just stick to the default values.
+param (
+    # change server_name if you installed your sql server as a Named Instance.
+    # If you installed on the Default Instance, then you can leave this as-is.
+    [string][Parameter(Mandatory = $false)]
+    $server_name = "localhost",
+
+    [string][Parameter(Mandatory = $false)]
+    $db_name = "kodb",
+
+    # 64 or 32
+    [string][Parameter(Mandatory = $false)]
+    $platform = "32",
+
+    # When testing connection, we can decide whether to use the Odbc or SqlClient module
+    [switch][Parameter(Mandatory = $false)]
+    $test_odbc_module,
+
+    [switch][Parameter(Mandatory = $false)]
+    $quiet
+)
+
+. "$PSScriptRoot\utils.ps1"
+
+function ValidateArgs {
+  if ($platform -ne "32" -and $platform -ne "64") {
+    MessageError "Wrong platform argument [$platform]. Choose either 64 or 32."
+    exit_script 1 $quiet
+  }
+
+  if (-not (ValidateServerNameInput $server_name)) {
+    exit_script 1 $quiet
+  }
+}
+
+
+
+function SelectSqlDriver {
+  $drivers = Get-OdbcDriver | Where-Object { $_.Name -like "*SQL Server*" -and $_.Platform -eq "$platform-bit" }
+  if (-not $drivers) {
+    MessageWarn "Are you sure SQL Server is installed? I couldn't find any drivers."
+    exit_script 1 $quiet
+  }
+
+  $selected_driver = $null
+
+  # In quiet mode we'll just return the last driver, assuming it is the most up to date one
+  if ($quiet) {
+    $selected_driver = $drivers[-1]
+    MessageInfo "Selected SQL Driver: {$($selected_driver.Name)}"
+    return $selected_driver
+  }
+
+  if ($drivers.Count -eq 1) {
+    # Select the first one if we only have one driver
+    $selected_driver = $drivers[0]
+  } else {
+    while (-not $selected_driver) {
+      MessageInfo "Enter a number to select your SQL Driver:"
+      for ($i = 0; $i -lt $drivers.Count; $i++) {
+        Message "$($i+1). $($drivers[$i].Name)"
+      }
+
+      $user_input = -1
+      $input_valid = [int]::TryParse((Read-Host "Input"), [ref]$user_input)
+      if (-not $input_valid -or $user_input -lt 1 -or $user_input -gt $drivers.Count) {
+        MessageWarn "Invalid selection."
+      } else {
+        $selected_driver = $drivers[$user_input - 1]
+        MessageInfo "Selected SQL Driver: {$($selected_driver.Name)}"
+        break
+      }
+    }
+  }
+
+  return $selected_driver
+}
+
+function CreateOdbcConnection {
+  param ([string][Parameter(Mandatory)] $driver_name)
+  Remove-OdbcDsn -Name $db_name -DsnType "User" -ErrorAction Ignore
+  Add-OdbcDsn `
+    -Name $db_name `
+    -DriverName $driver_name `
+    -DsnType "User" `
+    -SetPropertyValue @(
+      "Server=$server_name",
+      "Database=$db_name",
+      "AutoTranslate=No",
+      "Trusted_Connection=Yes"
+    )
+}
+
+# Refs:
+# https://learn.microsoft.com/en-us/dotnet/api/system.data.odbc.odbcconnection.connectionstring
+# https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlconnection.connectionstring
+# https://github.com/microsoft/referencesource/blob/master/System.Data/System/Data/Odbc/OdbcConnection.cs
+# https://github.com/microsoft/referencesource/blob/master/System.Data/System/Data/SqlClient/SqlConnection.cs
+function TestOdbcConnection {
+  param ([string][Parameter(Mandatory)] $driver_name)
+  $result = $false
+
+  $con_timeout_seconds = 15
+  # Note that in later sql servers `Trusted_Connection` replaced `Integrated Security` and changed it to a synonym. Therefore for backwards
+  # compatibility we're using `Integrated Security` instead.
+  $con_str = "Server={0};UID={1};PWD={1};Timeout={2};Integrated Security=True" -f $server_name, ($db_name + "_user"), $con_timeout_seconds
+  $con = $null
+  $module = ""
+  if ($test_odbc_module) {
+    $module = "Odbc"
+    $con_str += ";DSN=$db_name;Driver={$driver_name}"
+    $con = New-Object System.Data.Odbc.OdbcConnection($con_str)
+  } else {
+    $module = "SqlClient"
+    $con_str += ";Database=$db_name;Encrypt=False;TrustServerCertificate=True"
+    $con = New-Object System.Data.SqlClient.SqlConnection($con_str)
+  }
+
+  MessageInfo "Testing connection string with $module module: [$($con.ConnectionString)]"
+  try {
+    $con.Open()
+    $result = $con.State -eq [System.Data.ConnectionState]::Open
+  } catch {
+    MessageError "An error occurred while testing the connection: $_"
+  } finally {
+    if ($con.State -ne [System.Data.ConnectionState]::Closed) {
+      $con.Close()
+    }
+  }
+
+  return $result
+}
+
+function Main {
+  ValidateArgs
+
+  $selected_driver = SelectSqlDriver
+  CreateOdbcConnection -driver_name $selected_driver.Name
+  $is_successful = TestOdbcConnection -driver_name $selected_driver.Name
+  if ($is_successful) {
+    MessageSuccess "Successfully created odbc connection driver and tested connection!"
+  } else {
+    MessageError "Failed to test connection. Check that you first imported the database."
+    MessageError "If that didn't work, depending on how you installed MSSQL (Default or Named Instance), you may need to change the server above from localhost to yours."
+    exit_script 1 $quiet
+  }
+}
+
+Main
+exit_script 0 $quiet

src/migration/README.md

@@ -12,7 +12,7 @@ Couple of rules and notes when writing migration scripts:
 
 Below are instructions for the release engineer in order to create a new db release:
 - Create a new release branch following the db new release version according version semantics (`git checkout -b release/1.0.1`)
-- Run the import script skipping the migration scripts (`.\import.ps1 -run_migration_scripts $false`)
+- Run the import script skipping the migration scripts (`.\import.ps1 -skip_migration_scripts`)
 - Run the export script, to be sure that no diff is produced (`.\export.ps1` and then `git status`)
   - If there are local changes, something is probably off. Repeat the steps above
   - If you're sure all in order, best is if you create a new separate PR with the changes, in case empty spaces and such were added

src/misc/create_login.sql

@@ -1,12 +1,12 @@
 USE [master]
-IF EXISTS (SELECT * FROM sys.server_principals WHERE name = N'kodb_user')
-  DROP LOGIN [kodb_user];
-CREATE LOGIN [kodb_user] WITH PASSWORD=N'kodb_user', DEFAULT_DATABASE=[kodb], DEFAULT_LANGUAGE=[us_english], CHECK_EXPIRATION=OFF, CHECK_POLICY=ON;
+IF EXISTS (SELECT * FROM sys.server_principals WHERE name = N'###DB_NAME###_user')
+  DROP LOGIN [###DB_NAME###_user];
+CREATE LOGIN [###DB_NAME###_user] WITH PASSWORD=N'###DB_NAME###_user', DEFAULT_DATABASE=[###DB_NAME###], DEFAULT_LANGUAGE=[us_english], CHECK_EXPIRATION=OFF, CHECK_POLICY=ON;
 GO
 
-USE [kodb]
-IF EXISTS (SELECT * FROM sys.database_principals WHERE name = N'kodb_user')
-  DROP USER [kodb_user];
-CREATE USER [kodb_user] FOR LOGIN [kodb_user];
-EXEC sp_addrolemember N'db_owner', N'kodb_user';
+USE [###DB_NAME###]
+IF EXISTS (SELECT * FROM sys.database_principals WHERE name = N'###DB_NAME###_user')
+  DROP USER [###DB_NAME###_user];
+CREATE USER [###DB_NAME###_user] FOR LOGIN [###DB_NAME###_user];
+EXEC sp_addrolemember N'db_owner', N'###DB_NAME###_user';
 GO