Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set cluster mode config #97

Merged
merged 3 commits into from
Nov 10, 2023
Merged

Set cluster mode config #97

merged 3 commits into from
Nov 10, 2023

Conversation

manan164
Copy link
Contributor

@manan164 manan164 commented Nov 9, 2023

Logs,

mananbhatt@Manans-MacBook-Pro charts % kubectl logs orkes-conductor-58fd9fbcb6-tlgtp -n orkes-ns                       
nginx: [warn] the "user" directive makes sense only if the master process runs with super-user privileges, ignored in /etc/nginx/nginx.conf:3
Starting Conductor Server and UI
Running Nginx in background
Using config properties
Starting Conductor with -Xms1500M -Xmx2500M memory settings
12:16:58.230 [main] INFO io.orkes.conductor.OrkesConductorApplication - Starting conductor enterprise edition ... v2
12:16:58.259 [main] INFO io.orkes.conductor.OrkesConductorApplication - Loaded 31 properties from /app/config/config.properties
Cloud environment is AWS

  ______   .______       __  ___  _______     _______.
 /  __  \  |   _  \     |  |/  / |   ____|   /       |
|  |  |  | |  |_)  |    |  '  /  |  |__     |   (----`
|  |  |  | |      /     |    <   |   __|     \   \
|  `--'  | |  |\  \----.|  .  \  |  |____.----)   |
 \______/  | _| `._____||__|\__\ |_______|_______/

  ______   ______   .__   __.  _______   __    __    ______ .___________.  ______   .______
 /      | /  __  \  |  \ |  | |       \ |  |  |  |  /      ||           | /  __  \  |   _  \
|  ,----'|  |  |  | |   \|  | |  .--.  ||  |  |  | |  ,----'`---|  |----`|  |  |  | |  |_)  |
|  |     |  |  |  | |  . `  | |  |  |  ||  |  |  | |  |         |  |     |  |  |  | |      /
|  `----.|  `--'  | |  |\   | |  '--'  ||  `--'  | |  `----.    |  |     |  `--'  | |  |\  \----.
 \______| \______/  |__| \__| |_______/  \______/   \______|    |__|      \______/  | _| `._____|

                        (v2.6.32) :::Orkes Conductor Server
                       Copyright © 2022 Orkes Inc. All Rights Reserved.
2023-11-09 12:17:01,010 INFO  [main] org.springframework.boot.StartupInfoLogger: Starting OrkesConductorApplication v2.6.32 using Java 11.0.19 on orkes-conductor-58fd9fbcb6-tlgtp with PID 10 (/app/libs/server.jar started by conductor in /app/libs)
2023-11-09 12:17:01,011 INFO  [main] org.springframework.boot.SpringApplication: The following 2 profiles are active: "logrotate", "postgres"
2023-11-09 12:17:03,792 INFO  [main] org.springframework.data.repository.config.RepositoryConfigurationDelegate: Multiple Spring Data modules found, entering strict repository configuration mode
2023-11-09 12:17:03,796 INFO  [main] org.springframework.data.repository.config.RepositoryConfigurationDelegate: Bootstrapping Spring Data JPA repositories in DEFAULT mode.
2023-11-09 12:17:03,832 INFO  [main] org.springframework.data.repository.config.RepositoryConfigurationDelegate: Finished Spring Data repository scanning in 21 ms. Found 0 JPA repository interfaces.
2023-11-09 12:17:04,969 INFO  [main] org.springframework.boot.web.embedded.tomcat.TomcatWebServer: Tomcat initialized with port(s): 8080 (http)
2023-11-09 12:17:04,979 INFO  [main] org.apache.juli.logging.DirectJDKLog: Initializing ProtocolHandler ["http-nio-8080"]
2023-11-09 12:17:04,983 INFO  [main] org.apache.juli.logging.DirectJDKLog: Starting service [Tomcat]
2023-11-09 12:17:04,988 INFO  [main] org.apache.juli.logging.DirectJDKLog: Starting Servlet engine: [Apache Tomcat/9.0.81]
2023-11-09 12:17:05,079 INFO  [main] org.apache.juli.logging.DirectJDKLog: Initializing Spring embedded WebApplicationContext
2023-11-09 12:17:05,080 INFO  [main] org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext: Root WebApplicationContext: initialization completed in 3985 ms
2023-11-09 12:17:05,431 INFO  [main] io.orkes.conductor.metrics.MetricsCollector: skipLabels: false
2023-11-09 12:17:05,621 INFO  [main] com.netflix.conductor.redis.jedis.OrkesJedisProxy: Initialized OrkesJedisProxy with metrics io.orkes.conductor.metrics.MetricsCollector@6cd56321
2023-11-09 12:17:05,625 INFO  [main] com.netflix.conductor.redis.dao.RedisMonitorDAO: Redis critical memory threshold is set as 70
2023-11-09 12:17:05,627 INFO  [main] io.orkes.conductor.dao.server.ServerMonitorDAO: Server cpu usage threshold is set as 1.0, heap as 1.0
2023-11-09 12:17:05,996 INFO  [main] io.orkes.conductor.mq.dao.ClusteredRedisQueueDAO: Queues initialized using io.orkes.conductor.mq.dao.ClusteredRedisQueueDAO
2023-11-09 12:17:06,101 INFO  [main] com.netflix.conductor.redis.dao.OrkesMetadataDAO: taskDefCacheTTL set to 60000
2023-11-09 12:17:06,101 INFO  [main] com.netflix.conductor.redis.dao.OrkesMetadataDAO: maxTaskInWorkflow set to 100
2023-11-09 12:17:06,105 INFO  [main] io.orkes.conductor.id.TimeBasedUUIDGenerator: Using TimeBasedUUIDGenerator to generate Ids
2023-11-09 12:17:06,110 INFO  [main] com.netflix.conductor.redis.dao.OrkesRedisExecutionDAO: Conductor will wait for 5000 ms to replicate workflows to 1 replicas
2023-11-09 12:17:06,125 INFO  [main] com.netflix.conductor.redis.dao.OrkesRedisExecutionDAO: maxWorkflowSize: 10485760 bytes
2023-11-09 12:17:06,126 INFO  [main] com.netflix.conductor.redis.dao.OrkesRedisExecutionDAO: maxTaskSize: 102400 bytes
2023-11-09 12:17:06,126 INFO  [main] com.netflix.conductor.redis.dao.OrkesRedisExecutionDAO: maxTaskInWorkflow: 1000
2023-11-09 12:17:06,134 INFO  [main] io.orkes.conductor.events.conductor.ConductorEventQueueProvider: Ready...
2023-11-09 12:17:06,328 INFO  [main] org.flywaydb.core.internal.logging.slf4j.Slf4jLog: Flyway Community Edition 8.5.13 by Redgate
2023-11-09 12:17:06,328 INFO  [main] org.flywaydb.core.internal.logging.slf4j.Slf4jLog: See what's new here: https://flywaydb.org/documentation/learnmore/releaseNotes#8.5.13
2023-11-09 12:17:06,328 INFO  [main] org.flywaydb.core.internal.logging.slf4j.Slf4jLog: 
2023-11-09 12:17:06,387 INFO  [main] com.zaxxer.hikari.HikariDataSource: HikariPool-1 - Starting...
2023-11-09 12:17:06,506 INFO  [main] com.zaxxer.hikari.HikariDataSource: HikariPool-1 - Start completed.
2023-11-09 12:17:06,520 INFO  [main] org.flywaydb.core.internal.logging.slf4j.Slf4jLog: Database: jdbc:postgresql://postgresql:5432/pgdb (PostgreSQL 10.1)
2023-11-09 12:17:06,601 INFO  [main] org.flywaydb.core.internal.logging.slf4j.Slf4jLog: Successfully validated 26 migrations (execution time 00:00.066s)
2023-11-09 12:17:06,610 INFO  [main] org.flywaydb.core.internal.logging.slf4j.Slf4jLog: Current version of schema "public": 303
2023-11-09 12:17:06,610 WARN  [main] org.flywaydb.core.internal.logging.slf4j.Slf4jLog: outOfOrder mode is active. Migration of schema "public" may not be reproducible.
2023-11-09 12:17:06,610 INFO  [main] org.flywaydb.core.internal.logging.slf4j.Slf4jLog: Schema "public" is up to date. No migration necessary.
2023-11-09 12:17:06,617 INFO  [main] io.orkes.conductor.server.service.lock.RedisLock: 

Using fail-fast simple redis based locks


2023-11-09 12:17:06,635 INFO  [main] io.orkes.conductor.dao.postgres.archive.partition.PartitionManager: io.orkes.conductor.dao.postgres.archive.partition.PartitionManagerInitialized with PartitionManagerProperties(rowsThreshold=500000, partitionSplitTimeMin=15, retentionDays=366)
2023-11-09 12:17:06,686 INFO  [main] io.orkes.conductor.dao.postgres.archive.PostgresDocumentStoreDAO: ExecutionDAO is set as com.netflix.conductor.redis.dao.OrkesRedisExecutionDAO@313f8301 with ttl as 30
2023-11-09 12:17:06,689 INFO  [main] io.orkes.conductor.id.TimeBasedUUIDGenerator: Using TimeBasedUUIDGenerator to generate Ids
2023-11-09 12:17:06,689 INFO  [main] io.orkes.conductor.dao.postgres.archive.PostgresDocumentStoreDAO: ExecutionDAO is set as null with ttl as 30
2023-11-09 12:17:06,689 INFO  [main] io.orkes.conductor.dao.postgres.archive.PostgresArchiveDAO: Using HikariDataSource (HikariPool-1) as search datasource
2023-11-09 12:17:06,689 INFO  [main] io.orkes.conductor.dao.postgres.archive.PostgresArchiveDAO: Using jdbc:postgresql://postgresql:5432/pgdb as search datasource
2023-11-09 12:17:06,691 INFO  [main] io.orkes.conductor.dao.archive.ArchivedExecutionDAO: Initialized ArchivedExecutionDAO as Execution DAO with OrkesRedisExecutionDAO as primary DAO
2023-11-09 12:17:06,695 INFO  [main] com.netflix.conductor.redis.dao.RedisPollDataDAO: Using OrkesPollDataDAO
2023-11-09 12:17:06,697 INFO  [main] com.netflix.conductor.core.config.ConductorCoreConfiguration: Initialized dummy payload storage!
2023-11-09 12:17:06,710 INFO  [main] com.netflix.conductor.core.execution.OrkesExecutionDAOFacade: sendAllTaskUpdatesValue: false
2023-11-09 12:17:06,711 INFO  [main] com.netflix.conductor.core.execution.OrkesExecutionDAOFacade: statusChangePayloadLimitInBytes: 1000
2023-11-09 12:17:06,711 INFO  [main] com.netflix.conductor.core.execution.OrkesExecutionDAOFacade: Status Publisher sendAllTasks? false
2023-11-09 12:17:06,711 INFO  [main] com.netflix.conductor.core.execution.OrkesExecutionDAOFacade: OrkesExecutionDAOFacade initialized with com.netflix.conductor.redis.dao.RedisRateLimitingDAO@2676dc05, com.netflix.conductor.redis.dao.WorkflowRateLimiterDAO@5f631ca0
2023-11-09 12:17:06,908 INFO  [main] com.netflix.conductor.core.execution.tasks.OrkesJoin: Using EfficientJoin v2 to do joins with null evaluator
2023-11-09 12:17:07,495 INFO  [main] com.netflix.conductor.core.execution.mapper.OrkesForkJoinDynamicTaskMapper: OrkesForkJoinDynamicTaskMapper initialized
2023-11-09 12:17:07,495 INFO  [main] com.netflix.conductor.core.execution.mapper.OrkesHTTPPollTakMapper: Using OrkesHTTPPollTakMapper
2023-11-09 12:17:07,499 INFO  [main] com.netflix.conductor.core.execution.mapper.OrkesBusinessRuleWorkerTaskMapper: Using OrkesBusinessRuleWorkerTaskMapper
2023-11-09 12:17:07,506 INFO  [main] com.netflix.conductor.core.execution.mapper.OrkesOpsGenieTakMapper: Using OrkesOpsGenieTakMapper
2023-11-09 12:17:07,506 INFO  [main] com.netflix.conductor.core.execution.mapper.OrkesAWSLambdaTaskMapper: Using OrkesAWSLambdaTaskMapper
2023-11-09 12:17:07,507 INFO  [main] com.netflix.conductor.core.execution.mapper.OrkesQueryProcessorTaskMapper: Using OrkesQueryProcessorTaskMapper
2023-11-09 12:17:07,508 INFO  [main] com.netflix.conductor.core.execution.mapper.OrkesSimpleTaskMapper: Using OrkesSimpleTaskMapper
2023-11-09 12:17:07,508 INFO  [main] com.netflix.conductor.core.execution.mapper.OrkesSubWorkflowTaskMapper: Using OrkesSubWorkflowTaskMapper
2023-11-09 12:17:07,509 INFO  [main] com.netflix.conductor.core.execution.mapper.OrkesJDBCTaskMapper: Using OrkesJDBCTaskMapper
2023-11-09 12:17:07,550 INFO  [main] com.netflix.conductor.core.execution.OrkesWorkflowExecutor: OrkesWorkflowExecutor initialized with io.orkes.conductor.server.service.ParameterUtilsModified@5fbe155
2023-11-09 12:17:07,552 INFO  [main] com.netflix.conductor.service.OrkesExecutionService: Initialized OrkesExecutionService with com.netflix.conductor.redis.dao.OrkesMetadataDAO@7f37b6d9
2023-11-09 12:17:07,600 INFO  [main] com.netflix.conductor.redis.dao.RedisSchedulerDAO: maxScheduleConfigs set to 200 
2023-11-09 12:17:07,615 INFO  [main] com.netflix.conductor.core.reconciliation.WorkflowRepairService: WorkflowRepairService Initialized
2023-11-09 12:17:07,627 INFO  [main] io.orkes.conductor.events.OrkesEventProcessor: Event Processing is ENABLED
2023-11-09 12:17:07,634 INFO  [main] io.orkes.conductor.events.OrkesEventQueueManager: Ready to serve event queues io.orkes.conductor.events.OrkesEventQueues@58a2b917
2023-11-09 12:17:07,648 INFO  [main] io.orkes.conductor.server.service.OrkesWorkflowServiceImpl: OrkesWorkflowServiceImpl initialized with workflowSearchCacheEnabled = false
2023-11-09 12:17:07,694 INFO  [main] com.netflix.conductor.core.events.queue.DefaultEventQueueProcessor: DefaultEventQueueProcessor initialized with 0 queues
2023-11-09 12:17:07,714 INFO  [main] com.netflix.conductor.core.execution.tasks.SystemTaskWorker: SystemTaskWorker initialized with 12 threads
2023-11-09 12:17:07,715 INFO  [main] com.netflix.conductor.core.execution.tasks.IsolatedTaskQueueProducer: Isolated System Task Worker DISABLED
2023-11-09 12:17:07,719 INFO  [main] com.netflix.conductor.core.reconciliation.WorkflowSweeper: WorkflowSweeper initialized.
2023-11-09 12:17:07,742 INFO  [main] io.orkes.conductor.server.grpc.AuthInterceptor: AuthInterceptor::INIT
2023-11-09 12:17:07,753 INFO  [main] io.orkes.conductor.server.rest.WorkflowResource: localHostIp: 10.244.0.77
2023-11-09 12:17:07,804 INFO  [main] io.orkes.conductor.scheduler.service.SchedulerService: Started listening for task: conductor_system_scheduler in queue: conductor_system_scheduler
2023-11-09 12:17:07,810 INFO  [main] io.orkes.conductor.scheduler.service.SchedulerService: Started listening for archival records : conductor_system_scheduler_archival in queue: conductor_system_scheduler_archival
2023-11-09 12:17:07,847 INFO  [main] io.orkes.conductor.server.service.monitor.WorkflowMonitor: WorkflowMonitor initialized reporting every 30 seconds
2023-11-09 12:17:07,866 INFO  [main] io.orkes.conductor.server.service.grpc.WorkflowExecutionServiceImpl: WorkflowExecutionServiceImpl STARTED
2023-11-09 12:17:07,868 INFO  [main] io.orkes.conductor.server.service.sweeper.OrkesWorkflowSweeper: Initializing sweeper with 20 threads
2023-11-09 12:17:11,116 INFO  [main] io.orkes.conductor.WebhookWorker: WebhookWorker::INIT with threadCount = 1, pollingInterval = 1000 and pollBatchSize = 10
2023-11-09 12:17:11,495 INFO  [webhookWorker-thread-0] io.orkes.conductor.mq.redis.cluster.ConductorRedisClusterQueue: ConductorRedisClusterQueue started serving conductor_queues.test.QUEUE._webhook_queue.c
2023-11-09 12:17:11,586 INFO  [main] io.orkes.conductor.dao.indexer.IndexWorker: IndexWorker::INIT with primaryExecDAO = com.netflix.conductor.redis.dao.OrkesRedisExecutionDAO@313f8301, threadCount = 5, pollingInterval = 1 and pollBatchSize = 30
2023-11-09 12:17:11,640 INFO  [indexer-thread-0] io.orkes.conductor.mq.redis.cluster.ConductorRedisClusterQueue: ConductorRedisClusterQueue started serving conductor_queues.test.QUEUE._index_queue.c
2023-11-09 12:17:11,804 INFO  [main] io.orkes.conductor.server.grpc.GrpcConfiguration: GRPCServer::interceptors: [io.orkes.conductor.server.grpc.AuthInterceptor@3caafa67]
2023-11-09 12:17:18,103 INFO  [main] io.orkes.conductor.server.grpc.GRPCServer: grpc: Server started, listening on 8090
2023-11-09 12:17:22,632 INFO  [pool-13-thread-1] io.orkes.conductor.events.OrkesEventQueueManager: Refreshing event queues []
mananbhatt@Manans-MacBook-Pro charts % kubectl get endpoints -n orkes-ns                    
NAME                        ENDPOINTS                                                        AGE
my-redis-cluster            10.244.0.44:6379,10.244.0.45:6379,10.244.0.46:6379 + 3 more...   6h23m
my-redis-cluster-headless   10.244.0.44:6379,10.244.0.45:6379,10.244.0.46:6379 + 9 more...   6h23m
orkes-conductor             10.244.0.77:5000                                                 2m5s
orkes-conductor-app         10.244.0.77:8080                                                 2m5s
postgresql                  10.244.0.27:5432                                                 6h40m

@manan164 manan164 requested a review from boney9 November 9, 2023 11:49
@v1r3n v1r3n merged commit 4f09596 into main Nov 10, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants