Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use syscall.Gettimeofday to get current time #961

Merged
merged 7 commits into from
Jan 1, 2025

Conversation

hjweddie
Copy link
Contributor

@hjweddie hjweddie commented Dec 27, 2024

If we use syscall.Gettimeofday instead of time.Now, we could get a bit performance increasement.

Here are benchmark code and some result:

package main_test

import (
	"syscall"
	"testing"
	"time"
)

func Now() time.Time {
	var tv syscall.Timeval
	if err := syscall.Gettimeofday(&tv); nil != err {
		return time.Now()
	}

	return time.Unix(0, syscall.TimevalToNsec(tv))
}

func BenchmarkOld(t *testing.B) {
	t.ResetTimer()
	for n := 0; n < t.N; n++ {
		_ = time.Now()
	}
	t.StopTimer()
}

func BenchmarkNew(t *testing.B) {
	t.ResetTimer()
	for n := 0; n < t.N; n++ {
		_ = Now()
	}
	t.StopTimer()
}
goos: linux
goarch: amd64
pkg: test
cpu: Intel(R) Xeon(R) Platinum 8575C
BenchmarkOld-4   	22126742	        54.39 ns/op
BenchmarkNew-4   	37930492	        31.78 ns/op
PASS
ok  	test	2.500s


goos: darwin
goarch: arm64
pkg: test
cpu: Apple M2
BenchmarkOld-8   	20453368	        54.23 ns/op
BenchmarkNew-8   	34531131	        34.67 ns/op
PASS
ok  	test	5.448s

Copy link
Collaborator

@dveeden dveeden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This might be useful background info:

I think utils.Now() should probably have some tests.

@dveeden
Copy link
Collaborator

dveeden commented Dec 27, 2024

This doesn't seem to be an improvement on FreeBSD on Intel Atom.

goos: freebsd
goarch: amd64
cpu: Intel(R) Atom(TM) CPU  C2550  @ 2.40GHz
BenchmarkOld
BenchmarkOld-4   	10124980	       119.3 ns/op
BenchmarkNew
BenchmarkNew-4   	 1965103	       610.6 ns/op
PASS
ok  	command-line-arguments	3.154s

But I can confirm the improvements on Linux on Intel Core Ultra 7

goos: linux
goarch: amd64
cpu: Intel(R) Core(TM) Ultra 7 165U
BenchmarkOld
BenchmarkOld-14    	52909362	        22.65 ns/op
BenchmarkNew
BenchmarkNew-14    	65706342	        15.82 ns/op
PASS
ok  	command-line-arguments	2.285s

@dveeden
Copy link
Collaborator

dveeden commented Dec 27, 2024

What I don't really know is if time.Now() and syscall.Gettimeofday() behave the same for:

  • Timezones (With system clock set to UTC or not, etc)
  • Precision
  • Monotonic increases
  • etc

@hjweddie
Copy link
Contributor Author

According to https://tpaschalis.me/golang-time-now/, for linux amd64, when we call time.Now(), it uses runtime.vdsoClockgettimeSym, which makes a system call with context-switch overhead.
When we call syscall.Gettimeofday, it is called directly by user application, acceing the memory without context-switch overhead. This is why syscall.Gettimeofday may be faster than time.Now in linux.

But for freebsd, https://github.com/golang/go/blob/master/src/runtime/vdso_freebsd.go#L108 , time.Now prefers to use vdsoClockgettimeSym, and the same as linux.
When we call syscall.Gettimeofday, because theris no vdsoGettimeofdaySym in freebsd vdso. I think it just make a system call Gettimeofday without vdso, which has more interrupt handing overhead than vdsoGettimeofdaySym. This is why now.Now is faster than syscall.Gettimeofday for freebsd.

@hjweddie
Copy link
Contributor Author

What I don't really know is if time.Now() and syscall.Gettimeofday() behave the same for:

  • Timezones (With system clock set to UTC or not, etc)
  • Precision
  • Monotonic increases
  • etc

I agree with you that we should probably have some tests.
As what we had found out, we could gain performance increasement in linux but not all os, we may focus on linux os first and just use syscall.Gettimeofday on linux os.

@dveeden
Copy link
Collaborator

dveeden commented Dec 28, 2024

This also fixes support for Windows:

dvaneeden@dve-carbon:~/dev/go-mysql$ git status
On branch hjweddie/master
nothing to commit, working tree clean
dvaneeden@dve-carbon:~/dev/go-mysql$ make GOOS=windows build
go build -o bin/go-mysqlbinlog cmd/go-mysqlbinlog/main.go
go build -o bin/go-mysqldump cmd/go-mysqldump/main.go
go build -o bin/go-canal cmd/go-canal/main.go
go build -o bin/go-binlogparser cmd/go-binlogparser/main.go
go build -o bin/go-mysqlserver cmd/go-mysqlserver/main.go
dvaneeden@dve-carbon:~/dev/go-mysql$ git checkout HEAD^1
HEAD is now at 45f2ba0 use syscall.Gettimeofday to get current time
dvaneeden@dve-carbon:~/dev/go-mysql$ git status
HEAD detached at 45f2ba0
nothing to commit, working tree clean
dvaneeden@dve-carbon:~/dev/go-mysql$ make GOOS=windows build
go build -o bin/go-mysqlbinlog cmd/go-mysqlbinlog/main.go
# github.com/go-mysql-org/go-mysql/utils
utils/time.go:16:30: undefined: syscall.TimevalToNsec
make: *** [Makefile:8: build] Error 1
dvaneeden@dve-carbon:~/dev/go-mysql$ 

utils/now.go Outdated
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would probably call this now_unix.go and the other one now.go or now_generic.go or so.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea and I rename files name to now_unix.go and now.go

@hjweddie
Copy link
Contributor Author

hjweddie commented Dec 28, 2024

What I don't really know is if time.Now() and syscall.Gettimeofday() behave the same for:

  • Timezones (With system clock set to UTC or not, etc)
  • Precision
  • Monotonic increases
  • etc

What I don't really know is if time.Now() and syscall.Gettimeofday() behave the same for:

  • Timezones (With system clock set to UTC or not, etc)
  • Precision
  • Monotonic increases
  • etc

According to man doc: https://man7.org/linux/man-pages/man2/gettimeofday.2.html
Gettimeofday percision is one millisecond and not monotonic. I can confirm it in unittest code.
I think we should use time.Now with seeding to get a more random int.
For the most use cases in go-mysql project, syscall.Gettimeofday may be enough.

Copy link
Collaborator

@lance6716 lance6716 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest lgtm

// Now is a faster method to get current time
func Now() time.Time {
var tv syscall.Timeval
if err := syscall.Gettimeofday(&tv); nil != err {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if err := syscall.Gettimeofday(&tv); nil != err {
if err := syscall.Gettimeofday(&tv); err != nil {

nit. just a style preference. golang will not have if err = nil {} problems

customTimestamp := Now().UnixNano()

// two timestamp should within 1 percistion
assert.Equal(t, timestamp+int64(precision) >= customTimestamp && timestamp-int64(precision) <= customTimestamp, true, fmt.Sprintf("Loop: %d: customTimestamp should within %s. timestamp: %d, customTimestamp: %d", i, precision.String(), timestamp, customTimestamp))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you can use assert.WithinRange or assert.InDelta

// two timestamp should within 1 percistion
assert.Equal(t, timestamp+int64(precision) >= customTimestamp && timestamp-int64(precision) <= customTimestamp, true, fmt.Sprintf("Loop: %d: customTimestamp should within %s. timestamp: %d, customTimestamp: %d", i, precision.String(), timestamp, customTimestamp))

os.Setenv("TZ", fmt.Sprintf("UTC%d", 14-i%27))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please add a comment for its purpose. I don't clearly understand

@lance6716 lance6716 merged commit dc262fe into go-mysql-org:master Jan 1, 2025
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants