Skip to content

Commit

Permalink
go.mod: bump github.com/conduitio/conduit-connector-sdk from 0.9.1 to…
Browse files Browse the repository at this point in the history
… 0.10.0 (#113)

* go.mod: bump github.com/conduitio/conduit-connector-sdk

Bumps [github.com/conduitio/conduit-connector-sdk](https://github.com/conduitio/conduit-connector-sdk) from 0.9.1 to 0.10.0.
- [Commits](ConduitIO/conduit-connector-sdk@v0.9.1...v0.10.0)

---
updated-dependencies:
- dependency-name: github.com/conduitio/conduit-connector-sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* fix upgrade

* fix

* updates

* update

* remove test

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Raúl Barroso <[email protected]>
  • Loading branch information
dependabot[bot] and raulb authored Aug 13, 2024
1 parent 92d1f30 commit 572ad26
Show file tree
Hide file tree
Showing 21 changed files with 345 additions and 727 deletions.
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,10 @@ When interrupted, after restarted, it iterates all over container again.
After Snapshot reading is finished, connector switches to **CDC mode**.
In this mode, connector monitors the container each `pollingPeriod` period and notifies about changes detected.
The iterator stores the last modification timestamp of the files iterated in the previous cycle. When detecting new changes, this timestamp is used to discard files created before it.
When interrupted, after restarted, it iterates using the timestamp stored in sdk.Position, passed to source's Open method.
When interrupted, after restarted, it iterates using the timestamp stored in opencdc.Position, passed to source's Open method.

Both iterators paginate over the container via [List Blobs](https://docs.microsoft.com/rest/api/storageservices/list-blobs) query, with up to `maxResults` items per page, to read the list of available items and their metadata (`Last-Modified` and `Content-Type`).
When creating the sdk.Record, the contents of the file is additionally requested via [Get Blob](https://docs.microsoft.com/rest/api/storageservices/get-blob) query.
When creating the opencdc.Record, the contents of the file is additionally requested via [Get Blob](https://docs.microsoft.com/rest/api/storageservices/get-blob) query.

### Supported storage changes

Expand Down
7 changes: 4 additions & 3 deletions acceptance_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ import (
"time"

"github.com/Azure/azure-sdk-for-go/sdk/storage/azblob"
"github.com/conduitio/conduit-commons/opencdc"
sdk "github.com/conduitio/conduit-connector-sdk"
"github.com/miquido/conduit-connector-azure-storage/source"
helper "github.com/miquido/conduit-connector-azure-storage/test"
Expand All @@ -32,18 +33,18 @@ type CustomConfigurableAcceptanceTestDriver struct {
containerName string
}

func (d *CustomConfigurableAcceptanceTestDriver) GenerateRecord(t *testing.T, op sdk.Operation) sdk.Record {
func (d *CustomConfigurableAcceptanceTestDriver) GenerateRecord(t *testing.T, op opencdc.Operation) opencdc.Record {
t.Helper()

record := d.ConfigurableAcceptanceTestDriver.GenerateRecord(t, op)

// Override Key
record.Key = sdk.RawData(fmt.Sprintf("file-%d.txt", time.Now().UnixMicro()))
record.Key = opencdc.RawData(fmt.Sprintf("file-%d.txt", time.Now().UnixMicro()))

return record
}

func (d *CustomConfigurableAcceptanceTestDriver) WriteToSource(_ *testing.T, records []sdk.Record) (results []sdk.Record) {
func (d *CustomConfigurableAcceptanceTestDriver) WriteToSource(_ *testing.T, records []opencdc.Record) (results []opencdc.Record) {
for _, record := range records {
_ = helper.CreateBlob(
d.blobClient,
Expand Down
105 changes: 56 additions & 49 deletions go.mod
Original file line number Diff line number Diff line change
@@ -1,11 +1,12 @@
module github.com/miquido/conduit-connector-azure-storage

go 1.22.3
go 1.22.6

require (
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.14.0
github.com/Azure/azure-sdk-for-go/sdk/storage/azblob v1.4.0
github.com/conduitio/conduit-connector-sdk v0.9.1
github.com/conduitio/conduit-commons v0.3.0
github.com/conduitio/conduit-connector-sdk v0.10.0
github.com/golangci/golangci-lint v1.59.1
github.com/jaswdr/faker v1.19.1
github.com/matryer/moq v0.3.4
Expand All @@ -21,12 +22,12 @@ require (
github.com/Abirdcfly/dupword v0.0.14 // indirect
github.com/Antonboom/errname v0.1.13 // indirect
github.com/Antonboom/nilnil v0.1.9 // indirect
github.com/Antonboom/testifylint v1.3.1 // indirect
github.com/Antonboom/testifylint v1.4.3 // indirect
github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 // indirect
github.com/BurntSushi/toml v1.4.0 // indirect
github.com/Crocmagnon/fatcontext v0.2.2 // indirect
github.com/Crocmagnon/fatcontext v0.3.0 // indirect
github.com/Djarvur/go-err113 v0.0.0-20210108212216-aea10b59be24 // indirect
github.com/GaijinEntertainment/go-exhaustruct/v3 v3.2.0 // indirect
github.com/GaijinEntertainment/go-exhaustruct/v3 v3.3.0 // indirect
github.com/Masterminds/goutils v1.1.1 // indirect
github.com/Masterminds/semver/v3 v3.2.1 // indirect
github.com/Masterminds/sprig/v3 v3.2.3 // indirect
Expand All @@ -40,28 +41,27 @@ require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/bkielbasa/cyclop v1.2.1 // indirect
github.com/blizzy78/varnamelen v0.8.0 // indirect
github.com/bombsimon/wsl/v4 v4.2.1 // indirect
github.com/bombsimon/wsl/v4 v4.4.1 // indirect
github.com/breml/bidichk v0.2.7 // indirect
github.com/breml/errchkjson v0.3.6 // indirect
github.com/butuzov/ireturn v0.3.0 // indirect
github.com/butuzov/mirror v1.2.0 // indirect
github.com/catenacyber/perfsprint v0.7.1 // indirect
github.com/ccojocar/zxcvbn-go v1.0.2 // indirect
github.com/cespare/xxhash/v2 v2.2.0 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/charithe/durationcheck v0.0.10 // indirect
github.com/chavacava/garif v0.1.0 // indirect
github.com/ckaznocha/intrange v0.1.2 // indirect
github.com/conduitio/conduit-commons v0.2.0 // indirect
github.com/conduitio/conduit-connector-protocol v0.6.0 // indirect
github.com/conduitio/conduit-connector-protocol v0.7.0 // indirect
github.com/curioswitch/go-reassign v0.2.0 // indirect
github.com/daixiang0/gci v0.13.4 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
github.com/denis-tingaikin/go-header v0.5.0 // indirect
github.com/ettle/strcase v0.2.0 // indirect
github.com/fatih/color v1.17.0 // indirect
github.com/fatih/structtag v1.2.0 // indirect
github.com/firefart/nonamedreturns v1.0.5 // indirect
github.com/fsnotify/fsnotify v1.5.4 // indirect
github.com/fsnotify/fsnotify v1.7.0 // indirect
github.com/fzipp/gocyclo v0.6.0 // indirect
github.com/ghostiam/protogetter v0.3.6 // indirect
github.com/go-critic/go-critic v0.11.4 // indirect
Expand All @@ -75,11 +75,11 @@ require (
github.com/go-viper/mapstructure/v2 v2.0.0 // indirect
github.com/go-xmlfmt/xmlfmt v1.1.2 // indirect
github.com/gobwas/glob v0.2.3 // indirect
github.com/goccy/go-json v0.10.2 // indirect
github.com/goccy/go-json v0.10.3 // indirect
github.com/gofrs/flock v0.8.1 // indirect
github.com/golang/protobuf v1.5.4 // indirect
github.com/golangci/dupl v0.0.0-20180902072040-3e9179ac440a // indirect
github.com/golangci/gofmt v0.0.0-20231018234816-f50ced29576e // indirect
github.com/golangci/gofmt v0.0.0-20231019111953-be8c47862aaa // indirect
github.com/golangci/misspell v0.6.0 // indirect
github.com/golangci/modinfo v0.3.4 // indirect
github.com/golangci/plugin-module-register v0.1.1 // indirect
Expand All @@ -92,8 +92,9 @@ require (
github.com/gostaticanalysis/comment v1.4.2 // indirect
github.com/gostaticanalysis/forcetypeassert v0.1.0 // indirect
github.com/gostaticanalysis/nilerr v0.1.1 // indirect
github.com/hashicorp/go-hclog v1.5.0 // indirect
github.com/hashicorp/go-plugin v1.6.0 // indirect
github.com/hamba/avro/v2 v2.24.0 // indirect
github.com/hashicorp/go-hclog v1.6.3 // indirect
github.com/hashicorp/go-plugin v1.6.1 // indirect
github.com/hashicorp/go-version v1.7.0 // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/hashicorp/yamux v0.1.1 // indirect
Expand All @@ -104,8 +105,9 @@ require (
github.com/jgautheron/goconst v1.7.1 // indirect
github.com/jingyugao/rowserrcheck v1.1.1 // indirect
github.com/jirfag/go-printf-func-name v0.0.0-20200119135958-7558a9eaa5af // indirect
github.com/jjti/go-spancheck v0.6.1 // indirect
github.com/jjti/go-spancheck v0.6.2 // indirect
github.com/jpillora/backoff v1.0.0 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/julz/importas v0.1.0 // indirect
github.com/karamaru-alpha/copyloopvar v1.1.0 // indirect
github.com/kisielk/errcheck v1.7.0 // indirect
Expand All @@ -119,73 +121,77 @@ require (
github.com/leonklingele/grouper v1.1.2 // indirect
github.com/lufeee/execinquery v1.2.1 // indirect
github.com/macabu/inamedparam v0.1.3 // indirect
github.com/magiconair/properties v1.8.6 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/maratori/testableexamples v1.0.0 // indirect
github.com/maratori/testpackage v1.1.1 // indirect
github.com/matoous/godox v0.0.0-20230222163458-006bad1f9d26 // indirect
github.com/matoous/godox v0.0.0-20240105082147-c5b5e0e7c0c0 // indirect
github.com/matryer/is v1.4.1 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mattn/go-runewidth v0.0.9 // indirect
github.com/matttproud/golang_protobuf_extensions v1.0.1 // indirect
github.com/mattn/go-runewidth v0.0.15 // indirect
github.com/mgechev/revive v1.3.7 // indirect
github.com/mitchellh/copystructure v1.0.0 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-testing-interface v1.14.1 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/mitchellh/reflectwalk v1.0.0 // indirect
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/moricho/tparallel v0.3.1 // indirect
github.com/nakabonne/nestif v0.3.1 // indirect
github.com/nishanths/exhaustive v0.12.0 // indirect
github.com/nishanths/predeclared v0.2.2 // indirect
github.com/nunnatsa/ginkgolinter v0.16.2 // indirect
github.com/oklog/run v1.1.0 // indirect
github.com/olekukonko/tablewriter v0.0.5 // indirect
github.com/pelletier/go-toml v1.9.5 // indirect
github.com/pelletier/go-toml/v2 v2.2.2 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/polyfloyd/go-errorlint v1.5.2 // indirect
github.com/prometheus/client_golang v1.12.1 // indirect
github.com/prometheus/client_model v0.2.0 // indirect
github.com/prometheus/common v0.32.1 // indirect
github.com/prometheus/procfs v0.7.3 // indirect
github.com/prometheus/client_golang v1.19.1 // indirect
github.com/prometheus/client_model v0.6.1 // indirect
github.com/prometheus/common v0.54.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/quasilyte/go-ruleguard v0.4.2 // indirect
github.com/quasilyte/go-ruleguard/dsl v0.3.22 // indirect
github.com/quasilyte/gogrep v0.5.0 // indirect
github.com/quasilyte/regex/syntax v0.0.0-20210819130434-b3f0c404a727 // indirect
github.com/quasilyte/stdinfo v0.0.0-20220114132959-f7386bf02567 // indirect
github.com/rs/zerolog v1.32.0 // indirect
github.com/rivo/uniseg v0.4.7 // indirect
github.com/rs/zerolog v1.33.0 // indirect
github.com/ryancurrah/gomodguard v1.3.2 // indirect
github.com/ryanrolds/sqlclosecheck v0.5.1 // indirect
github.com/sagikazarmark/locafero v0.6.0 // indirect
github.com/sagikazarmark/slog-shim v0.1.0 // indirect
github.com/sanposhiho/wastedassign/v2 v2.0.7 // indirect
github.com/santhosh-tekuri/jsonschema/v5 v5.3.1 // indirect
github.com/sashamelentyev/interfacebloat v1.1.0 // indirect
github.com/sashamelentyev/usestdlibvars v1.26.0 // indirect
github.com/sashamelentyev/usestdlibvars v1.27.0 // indirect
github.com/securego/gosec/v2 v2.20.1-0.20240525090044-5f0084eb01a9 // indirect
github.com/shazow/go-diff v0.0.0-20160112020656-b6b7b6733b8c // indirect
github.com/shopspring/decimal v1.2.0 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/sivchari/containedctx v1.0.3 // indirect
github.com/sivchari/tenv v1.7.1 // indirect
github.com/sivchari/tenv v1.10.0 // indirect
github.com/sonatard/noctx v0.0.2 // indirect
github.com/sourcegraph/conc v0.3.0 // indirect
github.com/sourcegraph/go-diff v0.7.0 // indirect
github.com/spf13/afero v1.11.0 // indirect
github.com/spf13/cast v1.5.0 // indirect
github.com/spf13/cobra v1.8.0 // indirect
github.com/spf13/jwalterweatherman v1.1.0 // indirect
github.com/spf13/cast v1.6.0 // indirect
github.com/spf13/cobra v1.8.1 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/spf13/viper v1.12.0 // indirect
github.com/spf13/viper v1.19.0 // indirect
github.com/ssgreg/nlreturn/v2 v2.2.1 // indirect
github.com/stbenjam/no-sprintf-host-port v0.1.1 // indirect
github.com/stretchr/objx v0.5.2 // indirect
github.com/subosito/gotenv v1.4.1 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/t-yuki/gocover-cobertura v0.0.0-20180217150009-aaee18c8195c // indirect
github.com/tdakkota/asciicheck v0.2.0 // indirect
github.com/tetafro/godot v1.4.16 // indirect
github.com/timakin/bodyclose v0.0.0-20230421092635-574207250966 // indirect
github.com/timakin/bodyclose v0.0.0-20240125160201-f835fa56326a // indirect
github.com/timonwong/loggercheck v0.9.4 // indirect
github.com/tomarrell/wrapcheck/v2 v2.8.3 // indirect
github.com/tommy-muehle/go-mnd/v2 v2.5.1 // indirect
github.com/twmb/go-cache v1.2.1 // indirect
github.com/ultraware/funlen v0.1.0 // indirect
github.com/ultraware/whitespace v0.1.1 // indirect
github.com/uudashr/gocognit v1.1.2 // indirect
Expand All @@ -195,23 +201,24 @@ require (
github.com/ykadowak/zerologlint v0.1.5 // indirect
gitlab.com/bosi/decorder v0.4.2 // indirect
go-simpler.org/musttag v0.12.2 // indirect
go-simpler.org/sloglint v0.7.1 // indirect
go-simpler.org/sloglint v0.7.2 // indirect
go.uber.org/automaxprocs v1.5.3 // indirect
go.uber.org/mock v0.4.0 // indirect
go.uber.org/multierr v1.11.0 // indirect
go.uber.org/zap v1.27.0 // indirect
golang.org/x/crypto v0.25.0 // indirect
golang.org/x/exp v0.0.0-20240416160154-fe59bbe5cc7f // indirect
golang.org/x/exp/typeparams v0.0.0-20240314144324-c7f7c6466f7f // indirect
golang.org/x/mod v0.18.0 // indirect
golang.org/x/net v0.27.0 // indirect
golang.org/x/sync v0.7.0 // indirect
golang.org/x/sys v0.22.0 // indirect
golang.org/x/text v0.16.0 // indirect
golang.org/x/time v0.5.0 // indirect
golang.org/x/tools v0.22.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240415180920-8c6c420018be // indirect
google.golang.org/grpc v1.63.2 // indirect
google.golang.org/protobuf v1.33.0 // indirect
golang.org/x/crypto v0.26.0 // indirect
golang.org/x/exp v0.0.0-20240613232115-7f521ea00fb8 // indirect
golang.org/x/exp/typeparams v0.0.0-20240613232115-7f521ea00fb8 // indirect
golang.org/x/mod v0.20.0 // indirect
golang.org/x/net v0.28.0 // indirect
golang.org/x/sync v0.8.0 // indirect
golang.org/x/sys v0.23.0 // indirect
golang.org/x/text v0.17.0 // indirect
golang.org/x/time v0.6.0 // indirect
golang.org/x/tools v0.24.0 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20240624140628-dc46fd24d27d // indirect
google.golang.org/grpc v1.65.0 // indirect
google.golang.org/protobuf v1.34.2 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
Expand Down
Loading

0 comments on commit 572ad26

Please sign in to comment.