Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate the table name in NewTable #52

Merged
merged 1 commit into from
Sep 6, 2024

Conversation

joamaki
Copy link
Contributor

@joamaki joamaki commented Sep 6, 2024

To avoid constructing table names which are not usable for metrics (e.g. as Prometheus metrics label), restrict the allowed name to "^[a-z][a-z0-9_\-]{0,30}$", e.g. must be lower-case, start with letter and be between 1-31 characters long (is this long enough limit?)

@joamaki joamaki requested a review from ovidiutirla September 6, 2024 08:06
@joamaki joamaki requested a review from a team as a code owner September 6, 2024 08:06
@joamaki joamaki requested review from pippolo84 and removed request for a team September 6, 2024 08:06
To avoid constructing table names which are not usable for metrics
(e.g. as Prometheus metrics label), restrict the allowed name to
"^[a-z][a-z0-9_\\-]{1,30}$".

Signed-off-by: Jussi Maki <[email protected]>
@joamaki joamaki force-pushed the pr/joamaki/validate-table-names branch from 4c65b74 to 84a92b9 Compare September 6, 2024 08:08
Copy link

github-actions bot commented Sep 6, 2024

$ make test
go: downloading github.com/mitchellh/mapstructure v1.5.0
go: downloading golang.org/x/term v0.16.0
go: downloading github.com/spf13/cast v1.6.0
go: downloading github.com/fsnotify/fsnotify v1.7.0
go: downloading github.com/spf13/afero v1.11.0
go: downloading github.com/sagikazarmark/slog-shim v0.1.0
go: downloading golang.org/x/sys v0.17.0
go: downloading github.com/subosito/gotenv v1.6.0
go: downloading github.com/hashicorp/hcl v1.0.0
go: downloading gopkg.in/ini.v1 v1.67.0
go: downloading github.com/magiconair/properties v1.8.7
go: downloading github.com/pelletier/go-toml/v2 v2.1.0
go: downloading golang.org/x/text v0.14.0
	github.com/cilium/statedb/reconciler/benchmark		coverage: 0.0% of statements
	github.com/cilium/statedb/reconciler/example		coverage: 0.0% of statements
ok  	github.com/cilium/statedb	5.657s	coverage: 88.4% of statements
ok  	github.com/cilium/statedb/index	0.005s	coverage: 44.6% of statements
ok  	github.com/cilium/statedb/internal	0.010s	coverage: 93.3% of statements
ok  	github.com/cilium/statedb/part	2.568s	coverage: 82.8% of statements
ok  	github.com/cilium/statedb/reconciler	2.514s	coverage: 92.2% of statements
-----
$ make bench
go test ./... -bench . -benchmem -test.run xxx
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb
cpu: AMD EPYC 7763 64-Core Processor                
BenchmarkDB_WriteTxn_1-4                    	  457713	      2609 ns/op	    383252 objects/sec	    2800 B/op	      32 allocs/op
BenchmarkDB_WriteTxn_10-4                   	 1238235	       972.8 ns/op	   1028001 objects/sec	     742 B/op	      10 allocs/op
BenchmarkDB_WriteTxn_100-4                  	 1570862	       762.2 ns/op	   1311993 objects/sec	     597 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_1000-4                 	 1506288	       832.3 ns/op	   1201475 objects/sec	     549 B/op	       7 allocs/op
BenchmarkDB_WriteTxn_100_SecondaryIndex-4   	  505645	      2294 ns/op	    435888 objects/sec	    1488 B/op	      37 allocs/op
BenchmarkDB_Modify-4                        	    1335	    925372 ns/op	   1080648 objects/sec	  777948 B/op	    8467 allocs/op
BenchmarkDB_GetInsert-4                     	    1237	    991687 ns/op	   1008383 objects/sec	  756084 B/op	    8461 allocs/op
BenchmarkDB_RandomInsert-4                  	    2344	    500502 ns/op	   1997993 objects/sec	  402379 B/op	    7099 allocs/op
BenchmarkDB_RandomReplace-4                 	     321	   3702982 ns/op	    270053 objects/sec	 2348883 B/op	   48574 allocs/op
BenchmarkDB_SequentialInsert-4              	    1520	    803481 ns/op	   1244585 objects/sec	  548798 B/op	    7284 allocs/op
BenchmarkDB_Changes_Baseline-4              	    1274	    939822 ns/op	   1064032 objects/sec	  553115 B/op	   10253 allocs/op
BenchmarkDB_Changes-4                       	     705	   1688827 ns/op	    592127 objects/sec	  993294 B/op	   14493 allocs/op
BenchmarkDB_RandomLookup-4                  	   21930	     52852 ns/op	  18920693 objects/sec	     144 B/op	       1 allocs/op
BenchmarkDB_SequentialLookup-4              	   27206	     44555 ns/op	  22444345 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_FullIteration_All-4             	     802	   1477651 ns/op	  67675076 objects/sec	     464 B/op	      12 allocs/op
BenchmarkDB_FullIteration_Get-4             	     214	   5461222 ns/op	  18310951 objects/sec	       0 B/op	       0 allocs/op
BenchmarkDB_PropagationDelay-4              	  482569	      2292 ns/op	        20.00 50th_µs	        23.00 90th_µs	        75.00 99th_µs	    1587 B/op	      24 allocs/op
PASS
ok  	github.com/cilium/statedb	26.592s
PASS
ok  	github.com/cilium/statedb/index	0.004s
PASS
ok  	github.com/cilium/statedb/internal	0.003s
goos: linux
goarch: amd64
pkg: github.com/cilium/statedb/part
cpu: AMD EPYC 7763 64-Core Processor                
Benchmark_Insert_RootOnlyWatch-4    	    8686	    131932 ns/op	   7579642 objects/sec	  104163 B/op	    2041 allocs/op
Benchmark_Insert-4                  	    5994	    187310 ns/op	   5338735 objects/sec	  219074 B/op	    3065 allocs/op
Benchmark_Modify-4                  	    8362	    143200 ns/op	   6983246 objects/sec	  212817 B/op	    1205 allocs/op
Benchmark_GetInsert-4               	    6681	    177247 ns/op	   5641835 objects/sec	  212364 B/op	    1204 allocs/op
Benchmark_Replace-4                 	26997331	        44.78 ns/op	  22333720 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Replace_RootOnlyWatch-4   	26981832	        44.16 ns/op	  22645852 objects/sec	       0 B/op	       0 allocs/op
Benchmark_txn_1-4                   	 3020631	       399.7 ns/op	   2501914 objects/sec	     448 B/op	       7 allocs/op
Benchmark_txn_10-4                  	 7304109	       168.3 ns/op	   5942914 objects/sec	     154 B/op	       2 allocs/op
Benchmark_txn_100-4                 	 7947171	       150.7 ns/op	   6637907 objects/sec	     224 B/op	       2 allocs/op
Benchmark_txn_1000-4                	 7039929	       169.1 ns/op	   5914203 objects/sec	     216 B/op	       2 allocs/op
Benchmark_txn_delete_1-4            	 3108022	       383.2 ns/op	   2609291 objects/sec	     856 B/op	       6 allocs/op
Benchmark_txn_delete_10-4           	 8589652	       139.5 ns/op	   7170754 objects/sec	     132 B/op	       1 allocs/op
Benchmark_txn_delete_100-4          	10722538	       110.8 ns/op	   9023906 objects/sec	      60 B/op	       1 allocs/op
Benchmark_txn_delete_1000-4         	11421139	       105.0 ns/op	   9522204 objects/sec	      26 B/op	       1 allocs/op
Benchmark_Get-4                     	   38995	     31075 ns/op	  32180286 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Iterate-4                 	  179643	      6822 ns/op	 146588044 objects/sec	      80 B/op	       3 allocs/op
Benchmark_Hashmap_Insert-4          	   15645	     76523 ns/op	  13068064 objects/sec	   86547 B/op	      64 allocs/op
Benchmark_Hashmap_Get_Uint64-4      	  163923	      7441 ns/op	 134391126 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Hashmap_Get_Bytes-4       	  156024	      7699 ns/op	 129882104 objects/sec	       0 B/op	       0 allocs/op
Benchmark_Uint64Map_Random-4        	    1393	    856168 ns/op	   1167996 items/sec	 2702476 B/op	    9026 allocs/op
Benchmark_Uint64Map_Sequential-4    	    1485	    793483 ns/op	   1260267 items/sec	 2492408 B/op	    9749 allocs/op
PASS
ok  	github.com/cilium/statedb/part	28.414s
PASS
ok  	github.com/cilium/statedb/reconciler	0.004s
?   	github.com/cilium/statedb/reconciler/benchmark	[no test files]
?   	github.com/cilium/statedb/reconciler/example	[no test files]
go run ./reconciler/benchmark -quiet
1000000 objects reconciled in 2.94 seconds (batch size 1000)
Throughput 339967.80 objects per second
Allocated 6011261 objects, 424773kB bytes, 541792kB bytes still in use

Copy link

@ovidiutirla ovidiutirla left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

TODO, I think we need to put a validation on the jobgroups too, they also emit metrics.

malformed, for example if `IDIndex` is not unique (primary index has to be), or if
the indexers have overlapping names.
more secondary indexes. The table name must match the regular expression
"^[a-z][a-z0-9_\\-]{0,30}$".

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we allow _ or should we enforce - to be used always?

Is there a particular reason for selecting 30? 30 chars -> ~4-8 words.
Considering a metric name needs to be usually below 200 chars, but the table name could be used in situation like jobgroup prefix -> table name -> suffix, I think it's more than enough.

The largest table name we currently have is 16 chars, bandwidth-qdiscs which IMO is fine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's two parts to the names in the job library. One is the name used in the module health and that is concat of the full module id (agent.control-plane.foo) and the job name (job-control-loop). The module ids are already validated, but indeed we don't validate the job name (e.g. the name argument passed to job.OneShot for example).

Would you be interested in doing a PR for cilium/hive to add this validation? Probably same or similar constraint as used here makes sense.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we allow _ or should we enforce - to be used always?

I think allowing both is fine and using a similar constraint as we use in e.g. cilium/hive for module ids makes it easier to guess what is allowed.

@joamaki joamaki merged commit b6fa4df into main Sep 6, 2024
1 check passed
@joamaki joamaki deleted the pr/joamaki/validate-table-names branch September 6, 2024 08:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants