diff --git a/airbyte-integrations/connectors/destination-mssql-v2/README.md b/airbyte-integrations/connectors/destination-mssql-v2/README.md new file mode 100644 index 000000000000..4f2fe1572dfa --- /dev/null +++ b/airbyte-integrations/connectors/destination-mssql-v2/README.md @@ -0,0 +1,91 @@ +# Microsoft SQL Server V2 (Bulk CDK) Destination + +## Build + +### airbyte-ci + +To build the connector via the [Airbyte CI CLI tool](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md), navigate to the root of the [Airbyte repository](https://github.com/airbytehq/airbyte) and execute the following command: + +```shell +> airbyte-ci connectors --name=destination-mssql-v2 build +``` + +### Gradle + +To build the connector via [Gradle](https://gradle.org/), navigate to the root of the [Airbyte repository](https://github.com/airbytehq/airbyte) and execute the following command: + +```shell +> ./gradlew :airbyte-integrations:connectors:destination-mssql-v2:build +``` +## Execute + +### Local Execution via Docker + +In order to run the connector image locally, first either build the connector's [Docker](https://www.docker.com/) image using the commands found +in this section of this document OR build the image using the following command: + +```shell +> ./gradlew :airbyte-integrations:connectors:destination-mssql-v2:buildConnectorImage +``` + +The built image will automatically be tagged with the `dev` label. To run the connector image, use the following commands: + +```shell +docker run --rm airbyte/destination-mssql-v2:dev spec +docker run --rm -v $(pwd)/secrets:/secrets airbyte/destination-mssql-v2:dev check --config /secrets/config.json +docker run --rm -v $(pwd)/secrets:/secrets airbyte/destination-mssql-v2:dev discover --config /secrets/config.json +docker run --rm -v $(pwd)/secrets:/secrets -v $(pwd)/integration_tests:/integration_tests airbyte/destination-mssql-v2:dev read --config /secrets/config.json --catalog /integration_tests/configured_catalog.json +``` + +## Test + +The connector contains both unit and acceptance tests which can each be executed from the local environment. + +### Unit Tests + +The connector uses a combination of [Kotlin](https://kotlinlang.org/), [JUnit 5](https://junit.org/junit5/) and [MockK](https://mockk.io/) +to implement unit tests. Existing tests can be found within the destination-mssql-v2 module in the conventional `src/test/kotlin` source folder. New tests should also be added to this location. + +The unit tests can be executed either via the [Airbyte CI CLI tool](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) or [Gradle](https://gradle.org/): + +###### Airbyte CI CLI +```shell +> airbyte-ci connectors --name=destination-mssql-v2 test +``` + +###### Gradle +```shell +> ./gradlew :airbyte-integrations:connectors:destination-mssql-v2:test +``` + +### Acceptance Tests + +The [Airbyte project](https://github.com/airbytehq/airbyte) a standard test suite that all destination connectors must pass. The tests require specific implementations of a few components in order to connect the acceptance test suite with the connector's specific logic. The existing acceptance test scaffolding can be found in the conventional `src/test-integration/kotlin` source folder. + +The acceptance tests can be executed either via the [Airbyte CI CLI tool](https://github.com/airbytehq/airbyte/blob/master/airbyte-ci/connectors/pipelines/README.md) or [Gradle](https://gradle.org/): + +###### Airbyte CI CLI +```shell +> airbyte-ci connectors --name=destination-mssql-v2 test +``` + +###### Gradle +```shell +> ./gradlew :airbyte-integrations:connectors:destination-mssql-v2:integrationTest +``` + +## Release + +### Publishing a new version of the connector + +You've checked out the repo, implemented a million dollar feature, and you're ready to share your changes with the world. Now what? + +1. Make sure your changes are passing our test suite: `airbyte-ci connectors --name=destination-mssql-v2 test` +2. Bump the connector version in `metadata.yaml`: increment the `dockerImageTag` value. Please follow [semantic versioning for connectors](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#semantic-versioning-for-connectors). +3. Make sure the `metadata.yaml` content is up to date. +4. Make the connector documentation and its changelog is up to date (`docs/integrations/destinations/mssql-v2.md`). +5. Create a Pull Request: use [our PR naming conventions](https://docs.airbyte.com/contributing-to-airbyte/resources/pull-requests-handbook/#pull-request-title-convention). +6. Pat yourself on the back for being an awesome contributor. +7. Someone from Airbyte will take a look at your PR and iterate with you to merge it into master. + + diff --git a/airbyte-integrations/connectors/destination-mssql-v2/build.gradle.kts b/airbyte-integrations/connectors/destination-mssql-v2/build.gradle.kts new file mode 100644 index 000000000000..4cc0467ff7bd --- /dev/null +++ b/airbyte-integrations/connectors/destination-mssql-v2/build.gradle.kts @@ -0,0 +1,52 @@ +/* + * Copyright (c) 2024 Airbyte, Inc., all rights reserved. + */ + +plugins { + id("application") + id("airbyte-bulk-connector") +} + +airbyteBulkConnector { + core = "load" + toolkits = listOf() + cdk = "local" +} + +application { + mainClass = "io.airbyte.integrations.destination.mssql.v2.MSSQLDestination" + + applicationDefaultJvmArgs = listOf("-XX:+ExitOnOutOfMemoryError", "-XX:MaxRAMPercentage=75.0") + + // Uncomment and replace to run locally + //applicationDefaultJvmArgs = listOf("-XX:+ExitOnOutOfMemoryError", "-XX:MaxRAMPercentage=75.0", "--add-opens", "java.base/sun.nio.ch=ALL-UNNAMED", "--add-opens", "java.base/sun.security.action=ALL-UNNAMED", "--add-opens", "java.base/java.lang=ALL-UNNAMED") +} + +val junitVersion = "5.11.3" + +configurations.configureEach { + // Exclude additional SLF4J providers from all classpaths + exclude(mapOf("group" to "org.slf4j", "module" to "slf4j-reload4j")) +} + +// Uncomment to run locally +//tasks.run.configure { +// standardInput = System.`in` +//} + +dependencies { + implementation("com.microsoft.sqlserver:mssql-jdbc:12.8.1.jre11") + implementation("io.github.oshai:kotlin-logging-jvm:7.0.0") + implementation("jakarta.inject:jakarta.inject-api:2.0.1") + implementation("com.github.spotbugs:spotbugs-annotations:4.8.6") + implementation("io.micronaut:micronaut-inject:4.6.1") + + testImplementation("io.mockk:mockk:1.13.13") + testImplementation("org.junit.jupiter:junit-jupiter-api:$junitVersion") + testImplementation("org.junit.jupiter:junit-jupiter-params:$junitVersion") + testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine:$junitVersion") +} + +tasks.named("test") { + systemProperties(mapOf("mockk.junit.extension.keepmocks" to "true", "mockk.junit.extension.requireParallelTesting" to "true")) +} diff --git a/airbyte-integrations/connectors/destination-mssql-v2/gradle.properties b/airbyte-integrations/connectors/destination-mssql-v2/gradle.properties new file mode 100644 index 000000000000..4dbe8b8729df --- /dev/null +++ b/airbyte-integrations/connectors/destination-mssql-v2/gradle.properties @@ -0,0 +1 @@ +testExecutionConcurrency=-1 diff --git a/airbyte-integrations/connectors/destination-mssql-v2/icon.svg b/airbyte-integrations/connectors/destination-mssql-v2/icon.svg new file mode 100644 index 000000000000..edcaeb77c8f2 --- /dev/null +++ b/airbyte-integrations/connectors/destination-mssql-v2/icon.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/airbyte-integrations/connectors/destination-mssql-v2/metadata.yaml b/airbyte-integrations/connectors/destination-mssql-v2/metadata.yaml new file mode 100644 index 000000000000..1b09be9a1ad8 --- /dev/null +++ b/airbyte-integrations/connectors/destination-mssql-v2/metadata.yaml @@ -0,0 +1,34 @@ +data: + connectorSubtype: database + connectorType: destination + definitionId: 37a928c1-2d5c-431a-a97d-ae236bd1ea0c + dockerImageTag: 0.1.0 + dockerRepository: airbyte/destination-mssql-v2 + githubIssueLabel: destination-mssql-v2 + icon: icon.svg + license: ELv2 + name: MSSQL V2 Destination + registryOverrides: + cloud: + enabled: false + oss: + enabled: false + releaseStage: alpha + documentationUrl: https://docs.airbyte.com/integrations/destinations/mssql-v2 + tags: + - language:java + ab_internal: + sl: 100 + ql: 100 + supportLevel: community + supportsRefreshes: true + connectorTestSuitesOptions: + - suite: unitTests + - suite: integrationTests + testSecrets: + - name: SECRET_DESTINATION-S3-V2-MINIMAL-REQUIRED-CONFIG + fileName: s3_dest_v2_minimal_required_config.json + secretStore: + type: GSM + alias: airbyte-connector-testing-secret-store +metadataSpecVersion: "1.0" diff --git a/airbyte-integrations/connectors/destination-mssql-v2/src/main/kotlin/io/airbyte/integrations/destination/mssql/v2/MSSQLDestination.kt b/airbyte-integrations/connectors/destination-mssql-v2/src/main/kotlin/io/airbyte/integrations/destination/mssql/v2/MSSQLDestination.kt new file mode 100644 index 000000000000..cb1d288390aa --- /dev/null +++ b/airbyte-integrations/connectors/destination-mssql-v2/src/main/kotlin/io/airbyte/integrations/destination/mssql/v2/MSSQLDestination.kt @@ -0,0 +1,14 @@ +/* + * Copyright (c) 2024 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.integrations.destination.mssql.v2 + +import io.airbyte.cdk.AirbyteDestinationRunner + +object MSSQLDestination { + @JvmStatic + fun main(args: Array) { + AirbyteDestinationRunner.run(*args) + } +} diff --git a/airbyte-integrations/connectors/destination-mssql-v2/src/main/kotlin/io/airbyte/integrations/destination/mssql/v2/config/MSSQLConfiguration.kt b/airbyte-integrations/connectors/destination-mssql-v2/src/main/kotlin/io/airbyte/integrations/destination/mssql/v2/config/MSSQLConfiguration.kt new file mode 100644 index 000000000000..ca468c08caa7 --- /dev/null +++ b/airbyte-integrations/connectors/destination-mssql-v2/src/main/kotlin/io/airbyte/integrations/destination/mssql/v2/config/MSSQLConfiguration.kt @@ -0,0 +1,28 @@ +/* + * Copyright (c) 2024 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.integrations.destination.mssql.v2.config + +import dagger.Component.Factory +import io.airbyte.cdk.load.command.DestinationConfiguration +import io.airbyte.cdk.load.command.DestinationConfigurationFactory +import jakarta.inject.Singleton + +data class MSSQLConfiguration(val placeholder: String) : DestinationConfiguration() + +@Singleton +class MSSQLConfigurationFactory : + DestinationConfigurationFactory { + override fun makeWithoutExceptionHandling(pojo: MSSQLSpecification): MSSQLConfiguration { + TODO("Not yet implemented") + } +} + +@Factory +class MSSQLConfigurationProvider(private val config: DestinationConfiguration) { + @Singleton + fun get(): MSSQLConfiguration { + return config as MSSQLConfiguration + } +} diff --git a/airbyte-integrations/connectors/destination-mssql-v2/src/main/kotlin/io/airbyte/integrations/destination/mssql/v2/config/MSSQLSpecification.kt b/airbyte-integrations/connectors/destination-mssql-v2/src/main/kotlin/io/airbyte/integrations/destination/mssql/v2/config/MSSQLSpecification.kt new file mode 100644 index 000000000000..30a7f9769c63 --- /dev/null +++ b/airbyte-integrations/connectors/destination-mssql-v2/src/main/kotlin/io/airbyte/integrations/destination/mssql/v2/config/MSSQLSpecification.kt @@ -0,0 +1,26 @@ +/* + * Copyright (c) 2024 Airbyte, Inc., all rights reserved. + */ + +package io.airbyte.integrations.destination.mssql.v2.config + +import com.kjetland.jackson.jsonSchema.annotations.JsonSchemaTitle +import io.airbyte.cdk.command.ConfigurationSpecification +import io.airbyte.cdk.load.spec.DestinationSpecificationExtension +import io.airbyte.protocol.models.v0.DestinationSyncMode +import jakarta.inject.Singleton + +@Singleton +@JsonSchemaTitle("MSSQL V2 Destination Specification") +class MSSQLSpecification : ConfigurationSpecification() {} + +@Singleton +class MSSQLSpecificationExtension : DestinationSpecificationExtension { + override val supportedSyncModes = + listOf( + DestinationSyncMode.OVERWRITE, + DestinationSyncMode.APPEND, + DestinationSyncMode.APPEND_DEDUP + ) + override val supportsIncremental = true +} diff --git a/docs/integrations/destinations/mssql-v2.md b/docs/integrations/destinations/mssql-v2.md new file mode 100644 index 000000000000..0e168a4ea5b7 --- /dev/null +++ b/docs/integrations/destinations/mssql-v2.md @@ -0,0 +1,12 @@ +# MSSQL (V2) + +## Changelog + +
+ Expand to review + +| Version | Date | Pull Request | Subject | +|:--------|:-----------| :--------------------------------------------------------- |:---------------| +| 0.1.0 | 2024-12-16 | [\#49460](https://github.com/airbytehq/airbyte/pull/49460) | Initial commit | + +
\ No newline at end of file