Conductor Integration with MySQL Database

[shyamsundarsekhar]

Describe the bug
Integration with MySQL Database throws an error due to missing attribute setting for Cache (cacheRefreshTime) fetched from properties

Details
Conductor version:3.9.0-SNAPSHOT
Persistence implementation: MySQL
Queue implementation: Dynoqueues
Lock: Redis

To Reproduce
Steps to reproduce the behavior:

1. Netflix Conductor setup with db type as mysql
2. Startup of server results in exception for missing dependency on MySQLMetadataDAO
3. Exception is caused because the cacheRefreshTime is not fetched correctly from Properties

Expected behavior
Server startup results in successful handshake with MySQL via Flyway

Additional context
The code works fine when setup with In-memory database. The issue is only seen with MySQL integration.

[Prebiusta]

I have had a similar issue with Postgres persistence. Problem is that the default conductor repository does not contain a module for MySQL persistence, as you can see here. To solve the issue, you can use a community-made persistence package for MySQL persistence.
Add runtimeOnly group: 'com.netflix.conductor', name: 'conductor-mysql-persistence', version: '3.9.0' into server/build.gradle file. This package can also be found at Maven Central

[shyamsundarsekhar]

I have had a similar issue with Postgres persistence. Problem is that the default conductor repository does not contain a module for MySQL persistence, as you can see here. To solve the issue, you can use a community-made persistence package for MySQL persistence. Add runtimeOnly group: 'com.netflix.conductor', name: 'conductor-mysql-persistence', version: '3.9.0' into server/build.gradle file. This package can also be found at Maven Central

Hi @Prebiusta, Thank you for the response. I have added the dependency. Prior to 3.7.3, this folder was within Conductor-Main. But after the refactoring to Conductor-Community, we did make the necessary changes to reflect the dependency.

The same setup works when run via "../gradlew bootRun" command and the fix that was performed is here:

conductor-community/persistence/mysql-persistence/src/main/java/com/netflix/conductor/mysql/dao/MySQLMetadataDAO.java

Line 53 in 77f72e2

long cacheRefreshTime = properties.getTaskDefCacheRefreshInterval().getSeconds();

This value if hard-coded to 60, rather than fetch from properties will boot the server fine in bootRun mode. But if they are added as gradle dependencies and run via docker it fails.

This can be easily reproduced by setting the db mode to MySQL and leaving everything else vanilla.

[Prebiusta]

@shyamsundarsekhar Unfortunately, I am not aware of any fix for that. But if you find the solution, please do post it here

[BhuhanJ]

Hi @Prebiusta

Can you please let me know the steps to integrate with mysql without docker. I am currently facing issue.

Error creating bean with name 'workflowResource' defined in URL [jar:file:/Users/a13400917/Airtel/POC's/conductor-community-poc/conductor-community/community-server/build/libs/conductor-community-server-3.14.0-SNAPSHOT-boot.jar!/BOOT-INF/lib/conductor-rest-3.13.7.jar!/com/netflix/conductor/rest/controllers/WorkflowResource.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'workflowServiceImpl' defined in URL [jar:file:/Users/a13400917/Airtel/POC's/conductor-community-poc/conductor-community/community-server/build/libs/conductor-community-server-3.14.0-SNAPSHOT-boot.jar!/BOOT-INF/lib/conductor-core-3.13.7.jar!/com/netflix/conductor/service/WorkflowServiceImpl.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'workflowExecutor' defined in URL [jar:file:/Users/a13400917/Airtel/POC's/conductor-community-poc/conductor-community/community-server/build/libs/conductor-community-server-3.14.0-SNAPSHOT-boot.jar!/BOOT-INF/lib/conductor-core-3.13.7.jar!/com/netflix/conductor/core/execution/WorkflowExecutor.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'deciderService' defined in URL [jar:file:/Users/a13400917/Airtel/POC's/conductor-community-poc/conductor-community/community-server/build/libs/conductor-community-server-3.14.0-SNAPSHOT-boot.jar!/BOOT-INF/lib/conductor-core-3.13.7.jar!/com/netflix/conductor/core/execution/DeciderService.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mySqlMetadataDAO' defined in class path resource [com/netflix/conductor/mysql/config/MySQLConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.netflix.conductor.mysql.dao.MySQLMetadataDAO]: Factory method 'mySqlMetadataDAO' threw exception; nested exception is java.lang.IllegalArgumentException
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800) ~[conductor-es7-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[conductor-es7-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[conductor-es7-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]

Any help will be appreciated.

[BhuhanJ]

@shyamsundarsekhar I am also facing the issue while integrating with mysql (without docker), can you please help.

Below issue i am facing:

Error creating bean with name 'workflowResource' defined in URL [jar:file:/Users/a13400917/Airtel/POC's/conductor-community-poc/conductor-community/community-server/build/libs/conductor-community-server-3.14.0-SNAPSHOT-boot.jar!/BOOT-INF/lib/conductor-rest-3.13.7.jar!/com/netflix/conductor/rest/controllers/WorkflowResource.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'workflowServiceImpl' defined in URL [jar:file:/Users/a13400917/Airtel/POC's/conductor-community-poc/conductor-community/community-server/build/libs/conductor-community-server-3.14.0-SNAPSHOT-boot.jar!/BOOT-INF/lib/conductor-core-3.13.7.jar!/com/netflix/conductor/service/WorkflowServiceImpl.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'workflowExecutor' defined in URL [jar:file:/Users/a13400917/Airtel/POC's/conductor-community-poc/conductor-community/community-server/build/libs/conductor-community-server-3.14.0-SNAPSHOT-boot.jar!/BOOT-INF/lib/conductor-core-3.13.7.jar!/com/netflix/conductor/core/execution/WorkflowExecutor.class]: Unsatisfied dependency expressed through constructor parameter 0; nested exception is org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'deciderService' defined in URL [jar:file:/Users/a13400917/Airtel/POC's/conductor-community-poc/conductor-community/community-server/build/libs/conductor-community-server-3.14.0-SNAPSHOT-boot.jar!/BOOT-INF/lib/conductor-core-3.13.7.jar!/com/netflix/conductor/core/execution/DeciderService.class]: Unsatisfied dependency expressed through constructor parameter 2; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mySqlMetadataDAO' defined in class path resource [com/netflix/conductor/mysql/config/MySQLConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [com.netflix.conductor.mysql.dao.MySQLMetadataDAO]: Factory method 'mySqlMetadataDAO' threw exception; nested exception is java.lang.IllegalArgumentException
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:800) ~[conductor-es7-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229) ~[conductor-es7-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1372) ~[conductor-es7-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1222) ~[conductor-es7-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:582) ~[conductor-es7-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:542) ~[conductor-es7-persistence-3.14.0-SNAPSHOT.jar!/:3.14.0-SNAPSHOT]