Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CLC-311, CLC-314, CLC-326] DMT Updates #398

Merged
merged 57 commits into from
Oct 20, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
57 commits
Select commit Hold shift + click to select a range
2f281d6
* add initial tests for DMT
kutluhanmetin Sep 6, 2023
f79befd
add errors and logs
kutluhanmetin Sep 6, 2023
8b68b51
[CLC-311]: Add status command to DMT
kutluhanmetin Sep 7, 2023
3f76b0f
refactor
kutluhanmetin Sep 7, 2023
98ea33f
add overall completion percentage to status output
kutluhanmetin Sep 7, 2023
ad4bc17
refactor
kutluhanmetin Sep 7, 2023
d0dfafa
fix PR comments
kutluhanmetin Sep 7, 2023
318fff5
fix dependency
kutluhanmetin Sep 7, 2023
b529b9a
fix PR comment: change error msg
kutluhanmetin Sep 7, 2023
d5591a0
fix PR comments
kutluhanmetin Sep 7, 2023
48a42ce
subscribe to topic if status is not terminal
kutluhanmetin Sep 7, 2023
999a3bf
write migration report to file
kutluhanmetin Sep 7, 2023
aed1d01
refactor remove listener
kutluhanmetin Sep 8, 2023
75654cd
add debug log files
kutluhanmetin Sep 8, 2023
dc45c7a
add progress
kutluhanmetin Sep 18, 2023
243f7b1
add flag for output dir of report file
kutluhanmetin Sep 18, 2023
db210e3
add flag for output dir of report file
kutluhanmetin Sep 18, 2023
e42e56c
save debug logs to clc log file
kutluhanmetin Sep 18, 2023
45c4933
fix tests
kutluhanmetin Sep 18, 2023
dfaddaf
fix PR comment
kutluhanmetin Sep 18, 2023
f874293
delete unnecessary struct
kutluhanmetin Sep 18, 2023
b5554a9
fix PR comments
kutluhanmetin Sep 18, 2023
e35e7b5
fix PR comments
kutluhanmetin Sep 19, 2023
b4fc17e
Merge branch 'dmt' into CLC-306
kutluhanmetin Sep 19, 2023
44260f5
merge
kutluhanmetin Sep 19, 2023
ae6c008
set status
kutluhanmetin Sep 19, 2023
476be4e
query with sql
kutluhanmetin Sep 21, 2023
f486385
fix status command
kutluhanmetin Sep 21, 2023
547cf14
fix status command
kutluhanmetin Sep 21, 2023
9c493b9
refactor
kutluhanmetin Sep 21, 2023
1b95eb8
add timeout
kutluhanmetin Sep 22, 2023
752972f
refactor start test code
kutluhanmetin Sep 22, 2023
2f224bc
refactor status test code
kutluhanmetin Sep 22, 2023
7b2d93e
remove member logs in test
kutluhanmetin Sep 22, 2023
6b7119b
comment log tests
kutluhanmetin Sep 22, 2023
eb122a1
yuce's fix for default log path in tests
kutluhanmetin Sep 22, 2023
01cd2f3
add sleep
kutluhanmetin Sep 22, 2023
7dbbcba
Merge branch 'main' into DMT-Updates
kutluhanmetin Sep 28, 2023
e0f9627
Merge branch 'dmt' into DMT-Updates
kutluhanmetin Sep 28, 2023
d6f2e4b
fix tests
kutluhanmetin Sep 28, 2023
df939b2
fix tests
kutluhanmetin Sep 28, 2023
cd61ae5
set text for individual migration
kutluhanmetin Sep 28, 2023
c2f15d2
add deleted build-dmt to Makefile again
kutluhanmetin Sep 28, 2023
9425bc7
fix wait logic
kutluhanmetin Oct 11, 2023
57e1df2
add migration id info to the logs
kutluhanmetin Oct 11, 2023
f57aa14
fix tests
kutluhanmetin Oct 11, 2023
677d455
fix tests
kutluhanmetin Oct 11, 2023
5ac0b8d
Merge remote-tracking branch 'upstream/dmt' into DMT-Updates
kutluhanmetin Oct 12, 2023
8cde4fd
fix PR comment
kutluhanmetin Oct 12, 2023
ccfb425
fix Serkan's comments
kutluhanmetin Oct 16, 2023
2d916cf
fix Serkan's comment
kutluhanmetin Oct 18, 2023
0c7028c
fix Yuce's comments 1
kutluhanmetin Oct 19, 2023
f50a1de
fix Yuce's comments 2
kutluhanmetin Oct 19, 2023
db6348f
try fixing tests
kutluhanmetin Oct 19, 2023
d8a8917
try fixing tests
kutluhanmetin Oct 19, 2023
4a9371d
fix comments
kutluhanmetin Oct 20, 2023
1037acb
fix comments
kutluhanmetin Oct 20, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/test-all-386.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -57,4 +57,4 @@ jobs:

- name: "Run All Tests"
run: |
GOARCH=386 make test TEST_FLAGS="-v -count 1 -timeout 30m"
GOARCH=386 make test-dmt TEST_FLAGS="-v -count 1 -timeout 30m"
2 changes: 1 addition & 1 deletion .github/workflows/test-all.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -70,4 +70,4 @@ jobs:

- name: "Run All Tests"
run: |
make test
make test-dmt
6 changes: 6 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,15 @@ TARGZ ?= true
build:
CGO_ENABLED=0 go build -tags base,std,hazelcastinternal,hazelcastinternaltest -ldflags "$(LDFLAGS)" -o build/$(BINARY_NAME) ./cmd/clc

build-dmt:
CGO_ENABLED=0 go build -tags base,migration,config,home,version,hazelcastinternal,hazelcastinternaltest -ldflags "$(LDFLAGS)" -o build/$(BINARY_NAME) ./cmd/clc

test:
go test -tags base,std,hazelcastinternal,hazelcastinternaltest -p 1 $(TEST_FLAGS) ./...

test-dmt:
go test -tags base,std,migration,config,home,version,hazelcastinternal,hazelcastinternaltest -p 1 $(TEST_FLAGS) ./...

test-cover:
go test -tags base,std,hazelcastinternal,hazelcastinternaltest -p 1 $(TEST_FLAGS) -coverprofile=coverage.out -coverpkg $(PACKAGES) -coverprofile=$(COVERAGE_OUT) ./...

Expand Down
33 changes: 29 additions & 4 deletions base/commands/migration/const.go
Original file line number Diff line number Diff line change
@@ -1,8 +1,33 @@
//go:build std || migration

package migration

const (
startQueueName = "__datamigration_start_queue"
statusMapEntryName = "status"
argDMTConfig = "dmtConfig"
argTitleDMTConfig = "DMT configuration"
StartQueueName = "__datamigration_start_queue"
StatusMapEntryName = "status"
StatusMapName = "__datamigration_migrations"
UpdateTopicPrefix = "__datamigration_updates_"
DebugLogsListPrefix = "__datamigration_debug_logs_"
MigrationsInProgressList = "__datamigrations_in_progress"
startQueueName = "__datamigration_start_queue"
statusMapEntryName = "status"
argDMTConfig = "dmtConfig"
argTitleDMTConfig = "DMT configuration"
)

type Status string

const (
StatusStarted Status = "STARTED"
StatusCanceling Status = "CANCELING"
StatusComplete Status = "COMPLETED"
StatusCanceled Status = "CANCELED"
StatusFailed Status = "FAILED"
StatusInProgress Status = "IN_PROGRESS"
)

const flagOutputDir = "output-dir"

const banner = `Hazelcast Data Migration Tool v5.3.0
(c) 2023 Hazelcast, Inc.
`
2 changes: 2 additions & 0 deletions base/commands/migration/dummy.go
Original file line number Diff line number Diff line change
@@ -1 +1,3 @@
package migration

// This file exists only for compilation
2 changes: 1 addition & 1 deletion base/commands/migration/migration.go
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
//go:build migration
//go:build std || migration

package migration

Expand Down
Loading
Loading