Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

data races in runUpload, runDownload and runDelete functions #14

Open
krisiasty opened this issue Aug 23, 2019 · 1 comment
Open

data races in runUpload, runDownload and runDelete functions #14

krisiasty opened this issue Aug 23, 2019 · 1 comment

Comments

@krisiasty
Copy link

The uploadFinish variable in runUpload function, downloadFinish variable in runDownload function as well as downloadFinish variable in runDownload functions are updated in unsafe way causing data race when number of threads is greater than one.

Built-in go race detector confirms that (see below). I suggest using WaitGroup to wait for all go routines to finish and calculate time outside of go routines (just after wg.Wait() unblocks).

I reported this originally here: https://github.com/minio/s3-benchmark/issues/4 with proposed fix here: https://github.com/minio/s3-benchmark/pull/5, but https://github.com/minio/s3-benchmark maintainers asked to fix it at original repo before they merge the fix.

C:\code\go\s3-benchmark>go run -race s3-benchmark.go -b racetest -t 4
S3 benchmark program v3.0
Parameters: url=https://play.min.io, bucket=racetest, duration=10, threads=4, loops=1, size=1M
==================
WARNING: DATA RACE
Write at 0x000000d3f870 by goroutine 17:
  main.runUpload()
      C:/code/go/s3-benchmark/s3-benchmark.go:271 +0x76b

Previous write at 0x000000d3f870 by goroutine 15:
  main.runUpload()
      C:/code/go/s3-benchmark/s3-benchmark.go:271 +0x76b

Goroutine 17 (running) created at:
  main.main()
      C:/code/go/s3-benchmark/s3-benchmark.go:393 +0x954

Goroutine 15 (finished) created at:
  main.main()
      C:/code/go/s3-benchmark/s3-benchmark.go:393 +0x954
==================
Thu, 22 Aug 2019 23:22:27 GMT Loop 1: PUT time 14.4 secs, objects = 8, speed = 570.4KB/sec, 0.6 operations/sec.
==================
WARNING: DATA RACE
Write at 0x000000d3f810 by goroutine 41:
  main.runDownload()
      C:/code/go/s3-benchmark/s3-benchmark.go:290 +0x4b0

Previous write at 0x000000d3f810 by goroutine 42:
  main.runDownload()
      C:/code/go/s3-benchmark/s3-benchmark.go:290 +0x4b0

Goroutine 41 (running) created at:
  main.main()
      C:/code/go/s3-benchmark/s3-benchmark.go:419 +0xdf6

Goroutine 42 (finished) created at:
  main.main()
      C:/code/go/s3-benchmark/s3-benchmark.go:419 +0xdf6
==================
Thu, 22 Aug 2019 23:22:39 GMT Loop 1: GET time 11.6 secs, objects = 12, speed = 1MB/sec, 1.0 operations/sec.
==================
WARNING: DATA RACE
Write at 0x000000d3f7f0 by goroutine 49:
  main.runDelete()
      C:/code/go/s3-benchmark/s3-benchmark.go:309 +0x345

Previous write at 0x000000d3f7f0 by goroutine 50:
  main.runDelete()
      C:/code/go/s3-benchmark/s3-benchmark.go:309 +0x345

Goroutine 49 (running) created at:
  main.main()
      C:/code/go/s3-benchmark/s3-benchmark.go:445 +0x12a4

Goroutine 50 (finished) created at:
  main.main()
      C:/code/go/s3-benchmark/s3-benchmark.go:445 +0x12a4
==================
==================
WARNING: DATA RACE
Write at 0x000000d3f7f0 by goroutine 48:
  main.runDelete()
      C:/code/go/s3-benchmark/s3-benchmark.go:309 +0x345

Previous write at 0x000000d3f7f0 by goroutine 50:
  main.runDelete()
      C:/code/go/s3-benchmark/s3-benchmark.go:309 +0x345

Goroutine 48 (running) created at:
  main.main()
      C:/code/go/s3-benchmark/s3-benchmark.go:445 +0x12a4

Goroutine 50 (finished) created at:
  main.main()
      C:/code/go/s3-benchmark/s3-benchmark.go:445 +0x12a4
==================
==================
WARNING: DATA RACE
Write at 0x000000d3f7f0 by goroutine 51:
  main.runDelete()
      C:/code/go/s3-benchmark/s3-benchmark.go:309 +0x345

Previous write at 0x000000d3f7f0 by goroutine 50:
  main.runDelete()
      C:/code/go/s3-benchmark/s3-benchmark.go:309 +0x345

Goroutine 51 (running) created at:
  main.main()
      C:/code/go/s3-benchmark/s3-benchmark.go:445 +0x12a4

Goroutine 50 (finished) created at:
  main.main()
      C:/code/go/s3-benchmark/s3-benchmark.go:445 +0x12a4
==================
Thu, 22 Aug 2019 23:22:39 GMT Loop 1: DELETE time 0.4 secs, 20.2 operations/sec.
Benchmark completed.
Found 5 data race(s)
exit status 66

@metadaddy
Copy link

Not only is there a data race, but the results will be inaccurate for more than one thread. The time of the slowest thread will be used in calculating performance, rather than the time for each thread.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants