Upstream changes #1
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Reduce allocs when doing query host selection by pooling rands. updates #1401
Allow specifying MaxInterval for exponential reconnection policy.
Added authentication example
We've seen increased per-query latency after upgrading to latest gocql. This commit fixes the latency issue for us. This partially reverts 7b17705. There were multiple unrelated changes in that commit. The relevant part of that commit's description is: Fix finding the correct token in the token ring for host selection. Unfortunately that description does not contain the details of the issue it was aimed to fix. The change in 7b17705 also made the two binary search implementations (tokenRingReplicas.replicasFor and tokenRing.GetHostForToken) inconsistent, reverting the change fixed this discrepancy as well. The token values returned in system.peers and system.local tables are ends of ranges that the nodes are responsible for[1], so we never need to rollunder. When searching for a node responsible for token X (computed from partition key), we need to find hostTokens with end token Y with the lowest value such that Y >= X. In case X is larger than the largest end token value in tokenRingReplicas, we wrap the ring and the node with the lowest token is responsible (i.e. the hostTokens at index 0). [1] https://docs.datastax.com/en/dse/6.7/dse-arch/datastax_enterprise/dbArch/archAboutDataDistribute.html
This function is not used anymore.
Control queries only ever do reads on system tables but AWS MCS treats a prepare as write which it blocks on system tables. We can avoid doing preapres on selects as long as we get the result metadata.
Keyspace could define replication that doesn't cover all datacenters. More over: such case certainly happens when new data is added to cluster. This commit fixes handling of this situation in networkTopology replicaMap by skipping tokens from excess datacenters. fixes #1349
ObservedQuery and ObservedBatch expose a pointer to hostMetrics struct to the respective observers. However, the fields of the returned *hostMetrics are not safe to read, since the observer that might access hostMetrics' fields is called without the lock held. In order to return a safe value to clients, we need to make a copy of the hostMetrics struct while the lock is still locked. This commit also optimizes the count of lock/unlock calls, previously we did 3 pairs (for updating attempts, for updating latency and for getting hostMetrics), now we only do one.
* tuples: correct unmarshalling of unset tuple values In case the tuple value has never been written the unmarshalling used to panic. This is now handled by inspecting the data from the database and if it is missing (nil) it will set the tuple value to the corresponding default value. Fixes: #1343 * tuples: scan target vars reset of unset tuple values
* Parse smallint data type in metadata * Parse duration and date data type in metadata
There were missing checks for length which could lead to panic when the server sends malformed data. This commits adds the missing checks.
Co-authored-by: Bala <[email protected]>
* keep legacy metadata views, add materialized views
I executed
$ gofmt -w *.go
locally. This came out of #1456 where
my editor proposed making certain format changes.
...for full sentence comment. This makes the comment consistent with the rest of the documentation.
* fix tuple unmarshaling
There are two problems.
First, tuples with a null value are not handled properly which results in a
panic:
panic: runtime error: slice bounds out of range [:-1]
goroutine 1 [running]:
github.com/gocql/gocql.unmarshalTuple(0x769880, 0xc0000e8cc0, 0xc0000ae83a, 0x10, 0x4a, 0xc000186540, 0xc000190480, 0xc000190480, 0x16)
/home/vincent/dev/go/src/github.com/gocql/gocql/marshal.go:1950 +0x1276
github.com/gocql/gocql.Unmarshal(0x769880, 0xc0000e8cc0, 0xc0000ae836, 0x10, 0x4a, 0xc000186540, 0xc000190480, 0xc0000e8cc0, 0xc0000ae832)
/home/vincent/dev/go/src/github.com/gocql/gocql/marshal.go:167 +0xd56
github.com/gocql/gocql.unmarshalMap(0x769800, 0xc0000e8d00, 0xc0000ae836, 0x10, 0x4e, 0x698740, 0xc000190450, 0x0, 0x677f11)
/home/vincent/dev/go/src/github.com/gocql/gocql/marshal.go:1669 +0x8fb
github.com/gocql/gocql.Unmarshal(0x769800, 0xc0000e8d00, 0xc0000ae828, 0x1e, 0x58, 0x698740, 0xc000190450, 0x0, 0x0)
/home/vincent/dev/go/src/github.com/gocql/gocql/marshal.go:159 +0xcbf
github.com/gocql/gocql.scanColumn(0xc0000ae828, 0x1e, 0x58, 0xc0000b0988, 0x6, 0xc0000b09a0, 0xc, 0xc0000b09b0, 0x4, 0x769800, ...)
/home/vincent/dev/go/src/github.com/gocql/gocql/session.go:1314 +0x274
github.com/gocql/gocql.(*Iter).Scan(0xc0001963f0, 0xc000190460, 0x2, 0x2, 0x1)
/home/vincent/dev/go/src/github.com/gocql/gocql/session.go:1414 +0x2e5
main.main()
/home/vincent/tmp/gocql-tuple-map/main.go:90 +0x9cc
It panics when unmarshaling into a struct or into an array/slice.
Second, the unmarshaling code does not work with pointer fields in a
struct or pointer elements in an array/slice which results in a panic:
panic: reflect.Set: value of type time.Time is not assignable to type *time.Time
goroutine 1 [running]:
reflect.Value.assignTo(0x6fa700, 0xc000282280, 0x199, 0x7026c8, 0xb, 0x6fc0e0, 0x0, 0x0, 0x6fc0e0, 0xc000282280)
/usr/local/go/src/reflect/value.go:2403 +0x426
reflect.Value.Set(0x6fc0e0, 0xc0002821b8, 0x196, 0x6fa700, 0xc000282280, 0x199)
/usr/local/go/src/reflect/value.go:1532 +0xbd
github.com/gocql/gocql.unmarshalTuple(0x7698c0, 0xc000384100, 0xc0003aa0b6, 0x10, 0x4a, 0xc00028c1c0, 0xc0002821a0, 0xc0002821a0, 0x16)
/home/vincent/dev/go/src/github.com/gocql/gocql/marshal.go:1952 +0xf06
github.com/gocql/gocql.Unmarshal(0x7698c0, 0xc000384100, 0xc0003aa0b6, 0x10, 0x4a, 0xc00028c1c0, 0xc0002821a0, 0xc000384100, 0xc0003aa0b2)
/home/vincent/dev/go/src/github.com/gocql/gocql/marshal.go:166 +0xd56
github.com/gocql/gocql.unmarshalMap(0x769840, 0xc000384140, 0xc0003aa0b6, 0x10, 0x4e, 0x698720, 0xc000282170, 0x0, 0x677e51)
/home/vincent/dev/go/src/github.com/gocql/gocql/marshal.go:1668 +0x8fb
github.com/gocql/gocql.Unmarshal(0x769840, 0xc000384140, 0xc0003aa0a8, 0x1e, 0x58, 0x698720, 0xc000282170, 0x0, 0x0)
/home/vincent/dev/go/src/github.com/gocql/gocql/marshal.go:158 +0xcbf
github.com/gocql/gocql.scanColumn(0xc0003aa0a8, 0x1e, 0x58, 0xc000382018, 0x6, 0xc000382030, 0xc, 0xc000382040, 0x4, 0x769840, ...)
/home/vincent/dev/go/src/github.com/gocql/gocql/session.go:1314 +0x274
github.com/gocql/gocql.(*Iter).Scan(0xc00029a120, 0xc000282180, 0x2, 0x2, 0x1)
/home/vincent/dev/go/src/github.com/gocql/gocql/session.go:1414 +0x2e5
main.main()
/home/vincent/tmp/gocql-tuple-map/main.go:90 +0x9cc
This commit fixes both problems
* add tests for unmarshalTuple
* fix marshalTuple for null tuple elements Per the CQL spec a null [bytes] value should be encoded as the length -1 only, the previous behavior of marshalTuple would instead write the length 0. This broke both cqlsh and the Datastax Java driver when attempting to unmarshal the value. * improve marshalTuple tests
We support two latest versions of Go.
Advertising driver's name in the system.clients table can be helpful when debugging issues, e.g. when a connection imbalance occurs and allows to narrow down the culprit application/driver better.
If all nodes in the cluster change their IPs at one time, driver used to no longer be able to ever contact the cluster; the only solution was to restart the driver. A fallback is added to the control connection `reconnect()` logic so that when no known host is reachable, all hostnames provided in ClusterConfig (initial contact points) are reresolved and control connection is attempted to be opened to any of them. If this succeeds, a metadata fetch is issued normally and the whole cluster is discovered with its new IPs. For the cluster to correctly learn new IPs in case that nodes are accessible indirectly (e.g. through a proxy), that is, by translated address and not `rpc_address` or `broadcast_address`, the code introduced in #1682 was extended to remove and re-add a host also when its translated address changed (even when its internal address stays the same). As a bonus, a misnamed variable `hostport` is renamed to a suitable `hostaddr`.
…or} authenticators These are already allowed in github.com/scylladb/gocql. Closes #1703
This information is needed for testing shard awareness. Also it is more in line with cqlsh tracing.
Previously TokenAwarePolicy always used Keyspace explicitly set in cluster.Keyspace regardless of the keyspace in the Query. Now after preparing statement Keyspace and Table names are transferred to the Query and it can make use of that. Fixes: #1621
Reassignment of copyright, where approved, and relicensing. Change phrases of "gocql team" to "the Cassandra project". New repository will be github.com/apache/cassandra-gocql-driver
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Automatically opened by http://forkbomb.io