Custom serde plugin example

[chris-nolan]

Hi,

I've been playing with the latest master docker image to which I've added a fat jar containing an implementation of com.provectus.kafka.ui.serde.api.Serde and configured environment variables to pick it up as a custom plugin but kafka-ui fails to start up:

2022-11-21 16:38:58,516 INFO  [main] c.p.k.u.s.ClusterSerdes: Loading custom serde ExampleCodec
2022-11-21 16:38:58,519 WARN  [main] o.s.b.w.r.c.AnnotationConfigReactiveWebServerApplicationContext: Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'messagesService' defined in URL [jar:file:/kafka-ui-api.jar!/BOOT-INF/classes!/com/provectus/kafka/ui/service/MessagesService.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'deserializationService' defined in URL [jar:file:/kafka-ui-api.jar!/BOOT-INF/classes!/com/provectus/kafka/ui/service/DeserializationService.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.provectus.kafka.ui.service.DeserializationService]: Constructor threw exception; nested exception is java.lang.NullPointerException
2022-11-21 16:38:58,535 INFO  [main] o.s.b.a.l.ConditionEvaluationReportLoggingListener: 

Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2022-11-21 16:38:58,556 ERROR [main] o.s.b.SpringApplication: Application run failed
org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'messagesService' defined in URL [jar:file:/kafka-ui-api.jar!/BOOT-INF/classes!/com/provectus/kafka/ui/service/MessagesService.class]: Unsatisfied dependency expressed through constructor parameter 1; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'deserializationService' defined in URL [jar:file:/kafka-ui-api.jar!/BOOT-INF/classes!/com/provectus/kafka/ui/service/DeserializationService.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.provectus.kafka.ui.service.DeserializationService]: Constructor threw exception; nested exception is java.lang.NullPointerException
        at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800)
        at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:955)
        at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
        at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
        at org.springframework.boot.web.reactive.context.ReactiveWebServerApplicationContext.refresh(ReactiveWebServerApplicationContext.java:66)
        at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:734)
        at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:408)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:308)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1306)
        at org.springframework.boot.SpringApplication.run(SpringApplication.java:1295)
        at com.provectus.kafka.ui.KafkaUiApplication.main(KafkaUiApplication.java:15)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:568)
        at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
        at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
        at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:65)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'deserializationService' defined in URL [jar:file:/kafka-ui-api.jar!/BOOT-INF/classes!/com/provectus/kafka/ui/service/DeserializationService.class]: Bean instantiation via constructor failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.provectus.kafka.ui.service.DeserializationService]: Constructor threw exception; nested exception is java.lang.NullPointerException
        at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:315)
        at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:296)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582)
        at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542)
        at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
        at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
        at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
        at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
        at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1391)
        at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1311)
        at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887)
        at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791)
        ... 27 common frames omitted
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.provectus.kafka.ui.service.DeserializationService]: Constructor threw exception; nested exception is java.lang.NullPointerException
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:224)
        at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:117)
        at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:311)
        ... 41 common frames omitted
Caused by: java.lang.NullPointerException: null
        at java.base/java.util.Objects.requireNonNull(Objects.java:208)
        at java.base/sun.nio.fs.UnixFileSystem.getPath(UnixFileSystem.java:263)
        at java.base/java.nio.file.Path.of(Path.java:147)
        at com.provectus.kafka.ui.serdes.CustomSerdeLoader.loadAndConfigure(CustomSerdeLoader.java:42)
        at com.provectus.kafka.ui.serdes.ClusterSerdes.loadCustom(ClusterSerdes.java:157)
        at com.provectus.kafka.ui.serdes.ClusterSerdes.initSerdeFromConfig(ClusterSerdes.java:141)
        at com.provectus.kafka.ui.serdes.ClusterSerdes.<init>(ClusterSerdes.java:70)
        at com.provectus.kafka.ui.service.DeserializationService.<init>(DeserializationService.java:34)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
        at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
        at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
        at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
        ... 43 common frames omitted

I can create a repo with the code in, but I was wondering if there are any examples anywhere, or even if master is currently working for custom serde plugins?

[Haarolean]

Hi, currently there are two custom serdes:
https://github.com/provectus/kafkaui-smile-serde
https://github.com/provectus/kafkaui-glue-sr-serde

The exception you encounter means there's a SerdeConfig with null filePath.

And here's documentation for custom serdes (currently unmerged):
https://github.com/provectus/kafka-ui/pull/2702/files

Let me know if you need any further help

[chris-nolan]

Thanks for the links, that has helped. My biggest issue is having com.provectus.kafka-ui-serde-api to link against, I guess you'll publish this to maven at some point, but I'd rather not build the whole project to get it, is it possible to get it thrown up somewhere?

[chris-nolan]

Not to worry....built it all locally and now have it working!

[Haarolean]

@chris-nolan oh yeah, we're working on publishing it on maven central rn! Sorry, forgot to mention that.