Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix conn data race #123

Closed
wants to merge 125 commits into from
Closed

Conversation

liangjunmo
Copy link

@liangjunmo liangjunmo commented Aug 17, 2023

  • Do only one thing
  • Non breaking API changes
  • Tested

What did this pull request do?

Fix conn DARA RACE issue. There are some related issues like #96.

User Case Description

I use this package for sharding bill table. It runs normally on test environment, but occur DATA RACE issue on online environment.

I write unit test for this issue:

  • sharding_test.go
func TestDataRace(t *testing.T) {
	ctx, cancel := context.WithCancel(context.Background())
	ch := make(chan error)

	for i := 0; i < 2; i++ {
		go func() {
			for {
				select {
				case <-ctx.Done():
					return
				default:
					err := db.Model(&Order{}).Where("user_id", 100).Find(&[]Order{}).Error
					if err != nil {
						ch <- err
						return
					}
				}
			}
		}()
	}

	select {
	case <-time.After(time.Millisecond * 50):
		cancel()
	case err := <-ch:
		cancel()
		t.Fatal(err)
	}
}

Run this unit test on main branch with go test --race -count=1 -v --run=TestDataRace, it will output:

=== RUN   TestDataRace
==================
WARNING: DATA RACE
Write at 0x00c0000b3048 by goroutine 15:
  command-line-arguments.(*Sharding).switchConn()
      /Users/liangjunmo/workspace/gorm-sharding/sharding.go:268 +0x168
  command-line-arguments.(*Sharding).switchConn-fm()
      <autogenerated>:1 +0x44
  gorm.io/gorm.(*processor).Execute()
      /Users/liangjunmo/.gvm/pkgsets/go1.19.9/global/pkg/mod/gorm.io/[email protected]/callbacks.go:130 +0xbe1
  gorm.io/gorm.(*DB).Find()
      /Users/liangjunmo/.gvm/pkgsets/go1.19.9/global/pkg/mod/gorm.io/[email protected]/finisher_api.go:172 +0x238
  command-line-arguments.TestDataRace.func1()
      /Users/liangjunmo/workspace/gorm-sharding/sharding_test.go:413 +0x20d

Previous write at 0x00c0000b3048 by goroutine 14:
  command-line-arguments.(*Sharding).switchConn()
      /Users/liangjunmo/workspace/gorm-sharding/sharding.go:268 +0x168
  command-line-arguments.(*Sharding).switchConn-fm()
      <autogenerated>:1 +0x44
  gorm.io/gorm.(*processor).Execute()
      /Users/liangjunmo/.gvm/pkgsets/go1.19.9/global/pkg/mod/gorm.io/[email protected]/callbacks.go:130 +0xbe1
  gorm.io/gorm.(*DB).Find()
      /Users/liangjunmo/.gvm/pkgsets/go1.19.9/global/pkg/mod/gorm.io/[email protected]/finisher_api.go:172 +0x238
  command-line-arguments.TestDataRace.func1()
      /Users/liangjunmo/workspace/gorm-sharding/sharding_test.go:413 +0x20d

Goroutine 15 (running) created at:
  command-line-arguments.TestDataRace()
      /Users/liangjunmo/workspace/gorm-sharding/sharding_test.go:407 +0x87
  testing.tRunner()
      /Users/liangjunmo/.gvm/gos/go1.19.9/src/testing/testing.go:1446 +0x216
  testing.(*T).Run.func1()
      /Users/liangjunmo/.gvm/gos/go1.19.9/src/testing/testing.go:1493 +0x47

Goroutine 14 (running) created at:
  command-line-arguments.TestDataRace()
      /Users/liangjunmo/workspace/gorm-sharding/sharding_test.go:407 +0x87
  testing.tRunner()
      /Users/liangjunmo/.gvm/gos/go1.19.9/src/testing/testing.go:1446 +0x216
  testing.(*T).Run.func1()
      /Users/liangjunmo/.gvm/gos/go1.19.9/src/testing/testing.go:1493 +0x47
==================
    testing.go:1319: race detected during execution of test
--- FAIL: TestDataRace (0.05s)
=== CONT
    testing.go:1319: race detected during execution of test
FAIL
FAIL	command-line-arguments	0.867s
FAIL

hyperphoton and others added 22 commits October 14, 2022 11:01
….io/driver/mysql-1.4.1

Bump gorm.io/driver/mysql from 1.3.4 to 1.4.1
Bumps [gorm.io/gorm](https://github.com/go-gorm/gorm) from 1.23.5 to 1.24.0.
- [Release notes](https://github.com/go-gorm/gorm/releases)
- [Commits](go-gorm/gorm@v1.23.5...v1.24.0)

---
updated-dependencies:
- dependency-name: gorm.io/gorm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [gorm.io/plugin/dbresolver](https://github.com/go-gorm/dbresolver) from 1.2.2 to 1.3.0.
- [Release notes](https://github.com/go-gorm/dbresolver/releases)
- [Commits](go-gorm/dbresolver@v1.2.2...v1.3.0)

---
updated-dependencies:
- dependency-name: gorm.io/plugin/dbresolver
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
….io/gorm-1.24.0

Bump gorm.io/gorm from 1.23.5 to 1.24.0
….io/plugin/dbresolver-1.3.0

Bump gorm.io/plugin/dbresolver from 1.2.2 to 1.3.0
Bumps [gorm.io/driver/postgres](https://github.com/go-gorm/postgres) from 1.3.7 to 1.4.4.
- [Release notes](https://github.com/go-gorm/postgres/releases)
- [Commits](go-gorm/postgres@v1.3.7...v1.4.4)

---
updated-dependencies:
- dependency-name: gorm.io/driver/postgres
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
….io/driver/postgres-1.4.4

Bump gorm.io/driver/postgres from 1.3.7 to 1.4.4
Bumps [gorm.io/driver/postgres](https://github.com/go-gorm/postgres) from 1.4.8 to 1.5.0.
- [Release notes](https://github.com/go-gorm/postgres/releases)
- [Commits](go-gorm/postgres@v1.4.8...v1.5.0)

---
updated-dependencies:
- dependency-name: gorm.io/driver/postgres
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Bumps [actions/setup-go](https://github.com/actions/setup-go) from 3 to 4.
- [Release notes](https://github.com/actions/setup-go/releases)
- [Commits](actions/setup-go@v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-go
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [gorm.io/gorm](https://github.com/go-gorm/gorm) from 1.24.7-0.20230306060331-85eaf9eeda11 to 1.25.1.
- [Release notes](https://github.com/go-gorm/gorm/releases)
- [Commits](https://github.com/go-gorm/gorm/commits/v1.25.1)

---
updated-dependencies:
- dependency-name: gorm.io/gorm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
… an error was reported in the primary key generation
…m.io/gorm-1.25.1

Bump gorm.io/gorm from 1.24.7-0.20230306060331-85eaf9eeda11 to 1.25.1
Bumps [gorm.io/hints](https://github.com/go-gorm/hints) from 1.1.1 to 1.1.2.
- [Commits](go-gorm/hints@v1.1.1...v1.1.2)

---
updated-dependencies:
- dependency-name: gorm.io/hints
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
When the table suffix is non numeric, the insertion operation will report an error when generating the primary key ID
…m.io/hints-1.1.2

Bump gorm.io/hints from 1.1.1 to 1.1.2
@liangjunmo
Copy link
Author

Please correct me if I am wrong or miss something.

@liangjunmo
Copy link
Author

liangjunmo commented Aug 17, 2023

Seems like the MariaDB job failed, because of container initializing failure.

@liangjunmo liangjunmo closed this Aug 25, 2023
@liangjunmo liangjunmo deleted the fix-conn-data-race branch August 25, 2023 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants