Using Conductor without Redis

[vfouqueron]

Hello !

I just upgraded from Netflix conductor 3.15.0 to Conductor OSS 3.16.0. We were using conductor with just a mysql database and no Redis, RabbitMQ nor Elasticsearch. Worklfow executions were broken in the UI, but it was alright at the time. However I have the feeling that a Redis or a RabbitMQ is required now pour queuing. I cannot find any documentation about that. Is it a new requirement for using Conductor ?

[brianhks]

I just got 3.16.0 working with postgres only. The trick is that when you build the server you have to remove the other persistence projects from server/build.gradle. I commented out redis, cassandra, mysql and es7 and got it working.

It looks like the control of what modules are loaded isn't very good. Just make sure the only modules in the build are the ones you are using.

[JCHacking]

In my opinion, while it is a way to make it work, a customer should not have to modify the source code to execute conductor (since this could not be done with the official docker image for example).

Should a memory queue type be added for this?

[brianhks]

I'm not really familiar with spring boot, my preference is to use configuration to selectively load modules. The configuration kinda supports this but not really. It doesn't prevent some modules from loading. I'll look at the code and see if I can come up with a pull request to change that.

[antz-H]

When will version 3.16.0 of the image be released?

[JCHacking]

Hi @vfouqueron

I think if you add the following configuration it should work:

conductor.queue.type=mysql

It seems that it is now necessary to specify which queuing system to use.

[rajanarahul93]

Hello,
I don't have much knowledge but my basic knowledge. It looks like the upgrade to Conductor OSS 3.16.0 brought in some changes, especially regarding how the system handles persistence modules. While Redis or RabbitMQ weren't mandatory before, the newer version might assume their presence for queuing purposes.

A user @brianhks shared a workaround by tweaking the build in the server/build. gradle file, removing unnecessary persistence projects. They managed to make it work with PostgreSQL only.

But, there's a valid concern from JCHacking that users shouldn't ideally mess with the source code, especially when using the official Docker image. JCHacking suggests adding a configuration like a conductor.queue.type=mysql to specify the queuing system, which seems like a solution.

Keep an eye out for updates and potential releases, as mentioned by antz-H regarding version 3.16.0 of the Docker image.

[JCHacking]

Hello, try applying the above.
On my systems I have tested and it works with that configuration in version 3.17.0 (I guess in 3.16.0 too).

It seems that when you add persistence you also create a queues table, so that db serves as queue system as well.

I think the biggest problem is that this now mandatory configuration is not documented.

[ksgnextuple]

One doubt here, then we cannot have redis for queues and postgres for persistence?

[JCHacking]

@ksgnextuple you can, now the persistence and queues have independent configuration

[ksgnextuple]

Thanks, working now. Also this config parameter
queues.dynomite.threads -> Is it still valid with latest code?