Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validation required for avro schema before creating it in schema registry #215

Open
sandeepk6 opened this issue May 11, 2023 · 1 comment
Labels
✨ Feature Request New feature or request

Comments

@sandeepk6
Copy link

schemaRegistry.createSchema proceeds and adds schema into schema registry without schema validation.
Creation is successful, but the writing of record to Kafka fails with an exception which is not useful to debug the issue.

Exception Trace:
ERRO[0001] panic: runtime error: invalid memory address or nil pointer dereference
goroutine 53 [running]:
runtime/debug.Stack()
scenarioruntime/debug/stack.go:24 +0x65
go.k6.io/k6/js/common.RunWithPanicCatching.func1()
go.k6.io/[email protected]/js/common/util.go:82 +0x191
panic({0x18bef80, 0x28d87b0})
runtime/panic.go:884 +0x212
github.com/dop251/goja.(*Runtime).runWrapped.func1()
github.com/dop251/[email protected]/runtime.go:2409 +0xdc
panic({0x18bef80, 0x28d87b0})
runtime/panic.go:884 +0x212
github.com/dop251/goja.(*vm).try.func1()
github.com/dop251/[email protected]/vm.go:626 +0x70f
panic({0x18bef80, 0x28d87b0})
runtime/panic.go:884 +0x212
github.com/dop251/goja.(*Runtime).runWrapped.func1()
github.com/dop251/[email protected]/runtime.go:2409 +0xdc
panic({0x18bef80, 0x28d87b0})
runtime/panic.go:884 +0x212
github.com/dop251/goja.(*vm).try.func1()
github.com/dop251/[email protected]/vm.go:626 +0x70f
panic({0x18bef80, 0x28d87b0})
runtime/panic.go:884 +0x212
github.com/linkedin/goavro/v2.(*Codec).NativeFromTextual(0xc00177ef50?, {0xc0036ea000, 0x186, 0x1a0})
github.com/linkedin/goavro/[email protected]/codec.go:500 +0x23
github.com/mostafa/xk6-kafka.(*AvroSerde).Serialize(0x18b3880, {0x18b4c60?, 0xc001f2de90?}, 0x4?)
github.com/mostafa/[email protected]/avro.go:14 +0x79
github.com/mostafa/xk6-kafka.(*Kafka).serialize(0xc002558dc0, 0xc001f2de30)
github.com/mostafa/[email protected]/serdes.go:46 +0x119
github.com/mostafa/xk6-kafka.(*Kafka).schemaRegistryClientClass.func4({{0x1dd64d8, 0xc0021546f0}, {0xc0025fe8a0, 0x1, 0x1a}})
github.com/mostafa/[email protected]/schema_registry.go:210 +0x12d
github.com/dop251/goja.(*vm)._nativeCall(0xc002537300, 0xc0000bb900, 0x1)
github.com/dop251/[email protected]/vm.go:3233 +0x1e3
github.com/dop251/goja.call.exec(0xa?, 0xc002537300)
github.com/dop251/[email protected]/vm.go:3203 +0x33b
github.com/dop251/goja.(*vm).run(0xc002537300)
github.com/dop251/[email protected]/vm.go:495 +0x8b
github.com/dop251/goja.(*baseJsFuncObject)._call(0xc0029a3d40, {0x0?, 0x0, 0x1e61b060598?}, {0x0?, 0x0}, {0x1dd6dc0?, 0x2950a20?})
github.com/dop251/[email protected]/func.go:318 +0x3f8
github.com/dop251/goja.(*baseJsFuncObject).call(...)
github.com/dop251/[email protected]/func.go:328
github.com/dop251/goja.(*baseJsFuncObject).Call(0xaa0853?, {{0x1dd6dc0, 0x2950a20}, {0x0, 0x0, 0x0}})
github.com/dop251/[email protected]/func.go:280 +0xa5
github.com/dop251/goja.AssertFunction.func1.1()
github.com/dop251/[email protected]/runtime.go:2369 +0x74
github.com/dop251/goja.(*vm).try(0xc002537300, 0x110091b?)
github.com/dop251/[email protected]/vm.go:634 +0x18a
github.com/dop251/goja.(*Runtime).runWrapped(0xc0027a3880, 0xc000499000?)
github.com/dop251/[email protected]/runtime.go:2413 +0x7c
github.com/dop251/goja.AssertFunction.func1({0x1dd6dc0?, 0x2950a20?}, {0x0?, 0x4?, 0xc001522c30?})
github.com/dop251/[email protected]/runtime.go:2368 +0x92
go.k6.io/k6/js/modules/k6.(*K6).Group(0xc0002fd9b0, {0xc00191c2e0, 0x1b}, 0xc0026ac978)
go.k6.io/[email protected]/js/modules/k6/k6.go:122 +0x203
reflect.Value.call({0x18aa100?, 0xc0002fd9e0?, 0x1e61b060598?}, {0x1a653ba, 0x4}, {0xc001d549c0, 0x2, 0xc0006af060?})
reflect/value.go:584 +0x8c5
reflect.Value.Call({0x18aa100?, 0xc0002fd9e0?, 0xc001d54930?}, {0xc001d549c0?, 0xc002622d10?, 0xc000088000?})
reflect/value.go:368 +0xbc
github.com/dop251/goja.(*Runtime).wrapReflectFunc.func1({{0x1dd6dc0, 0x2950a20}, {0xc0025fe7d0, 0x2, 0x27}})
github.com/dop251/[email protected]/runtime.go:1952 +0x1da
github.com/dop251/goja.(*vm)._nativeCall(0xc002537300, 0xc0029a3c80, 0x2)
github.com/dop251/[email protected]/vm.go:3233 +0x1e3
github.com/dop251/goja.call.exec(0x5?, 0xc002537300)
github.com/dop251/[email protected]/vm.go:3205 +0x14d
github.com/dop251/goja.(*vm).run(0xc002537300)
github.com/dop251/[email protected]/vm.go:495 +0x8b
github.com/dop251/goja.(*baseJsFuncObject)._call(0xc002204300, {0xc000617c80?, 0x1, 0x8?}, {0x0?, 0x0}, {0x1dd6dc0?, 0x2950a20?})
github.com/dop251/[email protected]/func.go:318 +0x3f8
github.com/dop251/goja.(*baseJsFuncObject).call(...)
github.com/dop251/[email protected]/func.go:328
github.com/dop251/goja.(*baseJsFuncObject).Call(0x2030007?, {{0x1dd6dc0, 0x2950a20}, {0xc000617c80, 0x1, 0x1}})
github.com/dop251/[email protected]/func.go:280 +0xa5
github.com/dop251/goja.AssertFunction.func1.1()
github.com/dop251/[email protected]/runtime.go:2369 +0x74
github.com/dop251/goja.(*vm).try(0xc002537300, 0x0?)
github.com/dop251/[email protected]/vm.go:634 +0x18a
github.com/dop251/goja.(*Runtime).runWrapped(0xc0027a3880, 0x8?)
github.com/dop251/[email protected]/runtime.go:2413 +0x7c
github.com/dop251/goja.AssertFunction.func1({0x1dd6dc0?, 0x2950a20?}, {0xc000617c80?, 0x1?, 0x1?})
github.com/dop251/[email protected]/runtime.go:2368 +0x92
go.k6.io/k6/js.(*VU).runFn.func2.1()
go.k6.io/[email protected]/js/runner.go:774 +0x43
go.k6.io/k6/js/eventloop.(*EventLoop).Start(0xc0007610e0, 0xc001d54780)
go.k6.io/[email protected]/js/eventloop/eventloop.go:171 +0x1b7
go.k6.io/k6/js.(*VU).runFn.func2()
go.k6.io/[email protected]/js/runner.go:773 +0xeb
go.k6.io/k6/js/common.RunWithPanicCatching({0x1dd9a60?, 0xc0007d7200?}, 0xa2f986?, 0x18d6de0?)
go.k6.io/[email protected]/js/common/util.go:86 +0x87
go.k6.io/k6/js.(*VU).runFn(0xc002491bc0, {0x1dc9be0, 0xc0025bed80}, 0xc0?, 0xc0001f3d58, 0xc000617c70, {0xc000617c80, 0x1, 0x1})
go.k6.io/[email protected]/js/runner.go:772 +0x28a
go.k6.io/k6/js.(*ActiveVU).RunOnce(0xc0025bed40)
go.k6.io/[email protected]/js/runner.go:717 +0x412
go.k6.io/k6/lib/executor.getIterationRunner.func1({0x1dc9c88, 0xc001d54510}, {0x1dbf760?, 0xc0025bed40?})
go.k6.io/[email protected]/lib/executor/helpers.go:124 +0x64
go.k6.io/k6/lib/executor.PerVUIterations.Run.func5({0x1dc51c0, 0xc002491bc0})
go.k6.io/[email protected]/lib/executor/per_vu_iterations.go:227 +0x40a
created by go.k6.io/k6/lib/executor.PerVUIterations.Run
go.k6.io/[email protected]/lib/executor/per_vu_iterations.go:240 +0xc29

Avro Schema that was created and was allowed:
"namespace": "test.1",
"type": "record",
"name": "myschema",

As per Avro official documentation, it looks like namespace cannot start with a number like 1.test etc. Avro schema validation is highly desired that should not allow an invalid schema to be added to schema registry.

@mostafa
Copy link
Owner

mostafa commented May 12, 2023

Hey @sandeepk6,

It is expected of the developer/tester to provide valid schema in the code, which is why there is no validation. But I'd be happy to review contributions for schema validation.

@mostafa mostafa added the ✨ Feature Request New feature or request label May 12, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
✨ Feature Request New feature or request
Projects
Status: Todo
Development

No branches or pull requests

2 participants