* Issue #KN-427 feat: Knowledge service cloud-agnostic
* added artifact upload for oci oss storage
Signed-off-by: Deepak Devadathan <[email protected]>
* owerwrite oss object if already exists
Signed-off-by: Deepak Devadathan <[email protected]>
* testing with artifact download for neo4j from oci oss
Signed-off-by: Deepak Devadathan <[email protected]>
* changed the oss object download command
Signed-off-by: Deepak Devadathan <[email protected]>
* added checkpoint
Signed-off-by: Deepak Devadathan <[email protected]>
* removed checkpoint for debuging
Signed-off-by: Deepak Devadathan <[email protected]>
* added a cloud storage type selector for oci
Signed-off-by: Deepak Devadathan <[email protected]>
* jinja indentation
Signed-off-by: Deepak Devadathan <[email protected]>
* jinja indentation correction
Signed-off-by: Deepak Devadathan <[email protected]>
* removed the extra line for cloud_storage_type
Signed-off-by: Deepak Devadathan <[email protected]>
* added s3 credential configuration for flink
Signed-off-by: Deepak Devadathan <[email protected]>
* disabled s3 storage temporarily
Signed-off-by: Deepak Devadathan <[email protected]>
* s3 config for flink jobmanager only
Signed-off-by: Deepak Devadathan <[email protected]>
* corrected if else syntax
Signed-off-by: Deepak Devadathan <[email protected]>
* added s3 config for taskmanager
Signed-off-by: Deepak Devadathan <[email protected]>
* added base url configuration for s3
Signed-off-by: Deepak Devadathan <[email protected]>
* jinja templating syntax error correction
Signed-off-by: Deepak Devadathan <[email protected]>
* corrected base url for s3
Signed-off-by: Deepak Devadathan <[email protected]>
* updated flink-conf for questionset-publish
Signed-off-by: Deepak Devadathan <[email protected]>
* testing with s3 configuration
Signed-off-by: Deepak Devadathan <[email protected]>
* endpoint url changed home region
Signed-off-by: Deepak Devadathan <[email protected]>
* updated values.j2
cloud_service_provider
cloud_storage_endpoint
Signed-off-by: Deepak Devadathan <[email protected]>
* added closing (") for variable cloud_service_provider
Signed-off-by: Deepak Devadathan <[email protected]>
* added hadoop-fs parameters to flink-conf for questionset
Signed-off-by: Deepak Devadathan <[email protected]>
* corrected the flink-conf for questionset
Signed-off-by: Deepak Devadathan <[email protected]>
* updated presto specific variables in flink-conf.yaml
Signed-off-by: Deepak Devadathan <[email protected]>
* added s3-fs-hadoop values in flink-conf for question set
Signed-off-by: Deepak Devadathan <[email protected]>
* updated the correct values for flink-conf
Signed-off-by: Deepak Devadathan <[email protected]>
* removed values form flink-conf.yaml
Signed-off-by: Deepak Devadathan <[email protected]>
* updated the deployment template
Signed-off-by: Deepak Devadathan <[email protected]>
* added hive.s3.path-style-access
Signed-off-by: Deepak Devadathan <[email protected]>
* removed hive access path
Signed-off-by: Deepak Devadathan <[email protected]>
* template value for path.style
Signed-off-by: Deepak Devadathan <[email protected]>
* added s3.aws.cred provider
Signed-off-by: Deepak Devadathan <[email protected]>
* removed s3.aws.credentials.provider
Signed-off-by: Deepak Devadathan <[email protected]>
* testing with flink-conf.yaml
Signed-off-by: Deepak Devadathan <[email protected]>
* added s3.aws.credentials.provider
Signed-off-by: Deepak Devadathan <[email protected]>
* updated value of s3.aws.credentials.provider
Signed-off-by: Deepak Devadathan <[email protected]>
* removed s3.aws.credentials.provider
Signed-off-by: Deepak Devadathan <[email protected]>
* added s3.aws.credentials.provider for asset-enrichment
Signed-off-by: Deepak Devadathan <[email protected]>
* added acces key for asset enrichment
Signed-off-by: Deepak Devadathan <[email protected]>
* added provider in the jobmanager deployment
Signed-off-by: Deepak Devadathan <[email protected]>
* removed aws provider from deployment and asset-enrichment
Signed-off-by: Deepak Devadathan <[email protected]>
* added aws cred provider in job mgr deployment
Signed-off-by: Deepak Devadathan <[email protected]>
* testing with signing algo
Signed-off-by: Deepak Devadathan <[email protected]>
* testing with S3SignerType
Signed-off-by: Deepak Devadathan <[email protected]>
* update AWS3SignerType
Signed-off-by: Deepak Devadathan <[email protected]>
* removed signing algorithm
Signed-off-by: Deepak Devadathan <[email protected]>
* trying with s3a
Signed-off-by: Deepak Devadathan <[email protected]>
* reverted to s3
Signed-off-by: Deepak Devadathan <[email protected]>
* AWS4SignerType algo
Signed-off-by: Deepak Devadathan <[email protected]>
* AWS4UnsignedPayloadSignerType
Signed-off-by: Deepak Devadathan <[email protected]>
* NoOpSignerType algo
Signed-off-by: Deepak Devadathan <[email protected]>
* S3SignerType
Signed-off-by: Deepak Devadathan <[email protected]>
* removed signer algo
Signed-off-by: Deepak Devadathan <[email protected]>
* added aws logging
Signed-off-by: Deepak Devadathan <[email protected]>
* added http logging
Signed-off-by: Deepak Devadathan <[email protected]>
* added flink debug logs
Signed-off-by: Deepak Devadathan <[email protected]>
* added http logging
Signed-off-by: Deepak Devadathan <[email protected]>
* removed http logging
Signed-off-by: Deepak Devadathan <[email protected]>
* testing bucket probing
Signed-off-by: Deepak Devadathan <[email protected]>
* bucketprobe=2
Signed-off-by: Deepak Devadathan <[email protected]>
* bucketprobe=1
Signed-off-by: Deepak Devadathan <[email protected]>
* disable bucket probe
Signed-off-by: Deepak Devadathan <[email protected]>
* removed bucket probe
Signed-off-by: Deepak Devadathan <[email protected]>
* testing with disabing bucket probe
Signed-off-by: Deepak Devadathan <[email protected]>
* passing the region to hadoop configuration
Signed-off-by: Deepak Devadathan <[email protected]>
* presto config
Signed-off-by: Deepak Devadathan <[email protected]>
* added the region to deployment
Signed-off-by: Deepak Devadathan <[email protected]>
* updated taskmanager with presto variables
Signed-off-by: Deepak Devadathan <[email protected]>
* removed oss connection details
removed from flink-conf.yaml for questionset-publish as it no longer required
Signed-off-by: Deepak Devadathan <[email protected]>
* Added oci cloud-store-sdk
updated artifact id to cloud-store-sdk_2.11
version: 1.45
Signed-off-by: Deepak Devadathan <[email protected]>
* cloud-store-sdk with exclusion
Signed-off-by: Deepak Devadathan <[email protected]>
* removed exlusion from cloud-store-sdk dependency
Signed-off-by: Deepak Devadathan <[email protected]>
* rearranged modules for service
Signed-off-by: Deepak Devadathan <[email protected]>
* reverted modules ordering
Signed-off-by: Deepak Devadathan <[email protected]>
* added exclusion for guava for cloud-store-sdk
Signed-off-by: Deepak Devadathan <[email protected]>
* removed exlusion for guava
Signed-off-by: Deepak Devadathan <[email protected]>
* updated cassandra driver version to 3.2.0
This is test guava 20
Signed-off-by: Deepak Devadathan <[email protected]>
* update cloud-store-sdk version to 1.4.6
Signed-off-by: Deepak Devadathan <[email protected]>
* passing an empty region
in cloud-store-sdk 1.4.6 region is defined as an Option<String> , so need to pass it as a optional string
Signed-off-by: Deepak Devadathan <[email protected]>
* updated cassandra driver to 3.1.2
cloud-store-sdk 1.4.6 uses guava 19, so reverted the change from 3.2.0
Signed-off-by: Deepak Devadathan <[email protected]>
* Updated the oss_bucket_name
* Updated the instance principal
* Added env var for OCI
* Corrected the oss bucket name key
* added oci specs for video-stream deployment for kp flinkjobs
Signed-off-by: Deepak Devadathan <[email protected]>
* updated the zookeeper ip from local host to the ingestion and processing
Signed-off-by: Deepak Devadathan <[email protected]>
* disabled service_fact gathering
Signed-off-by: Deepak Devadathan <[email protected]>
* disabled service_facts
Signed-off-by: Deepak Devadathan <[email protected]>
* Update values.j2
To add the media workflow id
* Issue #KN-0000 feat: Config update
* Issue #KN-0000 feat: Config update
* Issue #KN-0000 feat: Config update
* Issue #KN-0000 feat: Config update
* added endpoint to flinkjob configuration
Signed-off-by: Deepak Devadathan <[email protected]>
* added the alter replication and partition
Signed-off-by: Deepak Devadathan <[email protected]>
* testing list length condition
Signed-off-by: Deepak Devadathan <[email protected]>
* added condition to create topic
Signed-off-by: Deepak Devadathan <[email protected]>
* removed replication factor alter
Signed-off-by: Deepak Devadathan <[email protected]>
* oci media streaming changes
* added cloud_storage_type key
Signed-off-by: Deepak Devadathan <[email protected]>
* udpated cloud-store-sdk id
Signed-off-by: Deepak Devadathan <[email protected]>
* resolved conflicts
Signed-off-by: Deepak Devadathan <[email protected]>
* added cloud_storage_proxy_host for qrcode job
Signed-off-by: Deepak Devadathan <[email protected]>
* added value for proxy host
Signed-off-by: Deepak Devadathan <[email protected]>
* Issue #KN-195 feat: CSP migrator parallelism increase
* To troubleshoot download content issue
* Adding the missing property
* Reverting the changes made in this deploy branch
* missing property
* for testing publish unit test
Signed-off-by: Deepak Devadathan <[email protected]>
* updated the continer name value for unit test case
Signed-off-by: Deepak Devadathan <[email protected]>
* testing unit tests
Signed-off-by: Deepak Devadathan <[email protected]>
* updated endpoint for unit testing
Signed-off-by: Deepak Devadathan <[email protected]>
* youtube chcks
Signed-off-by: Deepak Devadathan <[email protected]>
* for testing unit case with azure
Signed-off-by: Deepak Devadathan <[email protected]>
* testing unit cse
Signed-off-by: Deepak Devadathan <[email protected]>
* added cloud ep for unit test
Signed-off-by: Deepak Devadathan <[email protected]>
* adding the missing config
* updated generic variables
Signed-off-by: Deepak Devadathan <[email protected]>
* for test runs in circleci
Signed-off-by: Deepak Devadathan <[email protected]>
* updated test data
Signed-off-by: Deepak Devadathan <[email protected]>
* updated youtube link
Signed-off-by: Deepak Devadathan <[email protected]>
* updated invalid youtube links
Signed-off-by: Deepak Devadathan <[email protected]>
* updated youtube link for test cases
Signed-off-by: Deepak Devadathan <[email protected]>
* testing creative commons lic
Signed-off-by: Deepak Devadathan <[email protected]>
* passing mime type field for test case
Signed-off-by: Deepak Devadathan <[email protected]>
* reverted the mimetype value
Signed-off-by: Deepak Devadathan <[email protected]>
* KN-CSP Changes
* KN-CSP Changes
* KN-Fix For CSP Changes
* KN-Fix For CSP Changes
* KN-Fix For CSP Changes
* KN-Fix For CSP Changes
* KN-Fix For CSP Changes
* KN-Fix For CSP Changes
* KN-Fix For CSP Changes
* KN-Fix For CSP Changes
* KN-Fix For CSP Changes
* KN-Fix For CSP Changes
* Issue #KN-920 fix: Update kubernetes/ansible/roles/flink-jobs-deploy/defaults/main.yml
Co-authored-by: Mahesh Kumar Gangula <[email protected]>
* Revert "KN-CSP Changes"
* Issue KN-921 fix: config issue fix
* Issue KN-921 fix: config issue fix
* Issue KN-921 fix: config issue fix
* Issue KN-921 fix: AWS support for Flink jobs
* Issue KN-921 fix: Syntax issue fix
---------
Signed-off-by: Deepak Devadathan <[email protected]>
Co-authored-by: Jayaprakash8887 <[email protected]>
Co-authored-by: vinukumar-vs <[email protected]>
Co-authored-by: Deepak Devadathan <[email protected]>
Co-authored-by: Kenneth Heung <[email protected]>
Co-authored-by: subhash_chandra_budde <[email protected]>
Co-authored-by: rjanart <[email protected]>
Co-authored-by: Anil Gupta <[email protected]>
Co-authored-by: Ramya <[email protected]>
Co-authored-by: Aiman Sharief <[email protected]>
Co-authored-by: Mahesh Kumar Gangula <[email protected]>
