Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migeng 371 avoid messages duplication issues #70

Open
wants to merge 21 commits into
base: master
Choose a base branch
from

Commits on Jan 9, 2020

  1. MIGENG-365 - End2end integration test (project-xavier#20)

    * WINDUP-2398 changed mimetype and created sample available json
    
    * changed kafka topic
    
    * changed kafka topic and category
    
    * refactored code
    
    * replaced seda by direct as it seemed not to work
    
    * Converted calculator into Bean
    Added Sonar + Jacoco execution
    
    * Created first test for a simple route in order to know how to properly test Camel
    
    * Created unit test for AnalysticsCalculator
    Added JUnit 5 and AssertJ to pom
    Aplied some clean-code suggestions
    
    * Removed sonar token from pom
    
    * WINDUP-2398
    Added support for JUnit 5 on old maven versions
    Added constructors on model
    Fixed AnalyticsCalculatorTest test
    Added Sonar info to README
    
    * WINDUP-2398
    Added lombok file to add Generated annotation that will be avoided by JaCoCo
    Added sample file to test multipart
    Added test for the multipart
    Removed accessors/modifiers for cloudforms classes
    Cleaned CustomizedMultipartDataFormat
    
    * WINDUP-2398
    Renamed test methods
    
    * WINDUP-2398
    Renamed test methods
    
    * WINDUP-2398
    Moved back to JUnit 4 as Camel Test support doesnt support JUnit 5 yet
    
    * WINDUP-2398
    Added very simple test for direct:store
    
    * WINDUP-2398
    Added test for direct-upload with individual files and zip file
    
    * WINDUP-2398
    Added test for direct:insights route
    
    * WINDUP-2398
    minor change on xmlroutestest
    
    * WINDUP-2398 Applied changes from travis PR
    
    * WNDUP-2398 added test for kafka listener route
    
    * WNDUP-2398 added a test for the direct-download route ( not finished )
    
    * WNDUP-2398 fixed test for direct:download route
    
    * WNDUP-2398 added test for direct:calculate
    added cloudforms export file
    
    * WNDUP-2398 testing the rest endpoint
    
    * WNDUP-2398 testing the rest-upload endpoint
    
    * WNDUP-2398 testing the rest report endpoint
    
    * WNDUP-2398 switched to regular endpoints for rest, as rest DSL has some "issues"
    
    * WNDUP-2398 renamed kie helper methods with no Random, and removed unused code
    
    * WNDUP-2404 testing call-kie-extract-reports route
    removed testing route
    
    * WNDUP-2398 testing the  direct:decisionserver endpoint
    
    * WNDUP-2404 changed kie endpoint from http4 to http as a ClientProtocolException appears
    
    * WINDUP-2404 adjusted the tests to consider the new context-path
    
    * windup-2398 considering non file params as headers for all messages
    
    * windup-2398 testing the param from multi part form
    
    * WINDUP-2398 removed customerid from the url on the rest endpoint
    
    * WINDUP-2398 removed customerid from the test of rest endpoint
    
    * WINDUP-2398 modified tests to work with dynamic metadata properties and filtering for messages having all those properties
    
    * MIGENG-23 added new version of Cloudforms file, refactored Calculator
    
    * MIGENG-23 replaced the object model to be sent to JMS with the one in xavier-analytics
    
    * MIGENG-23 adapted tests to new report model
    
    * MIGENG-23 changed cloudforms model types Integer to Long
    
    * MIGENG-23 changed the way we obtain RequestId as before it was always the same
    
    * MIGENG-23 fixed tests according to the new model
    
    * MIGENG-23 replaced cloudforms test files
    
    * MIGENG-23 made host.type filter configurable with a property
    
    * MIGENG-23 added tar and gz support to the route
    
    * MIGENG-23 added TAR.GZ support and added test for that
    
    * MIGENG-23 added exception handling when the json crashes on the unmarshal
    
    * MIGENG-23 fixed mocking log endpoint in test
    
    * MIGENG-23 fixed attributes and filter names
    
    * MIGENG-23 fixed tests with the new "MA_metadata" header
    now we use the RHIdentity object passed as x-rh-identity header if it exists and we populate our params
    
    * MIGENG-106 Modified the routes in order to do the unzip after receiving the message from kafka and not in the upload
    
    * MIGENG-106 Fixed tests in order to work with the unzipping after the kafka message
    
    * MIGENG-106 Removed all structure for CloudForms
    Modified test and added paths to properties
    
    * MIGENG-106 Small refactors
    
    * MIGENG-106 Renamed test
    
    * end2end added test to allow an End To End test
    using TestContainers and real layers
    using WireMock to simulate Rest InsightsUpload service
    
    * end2end working on issue with Ryuk and test containers
    
    * MIGENG-106 code prepared to use the version on the manifest
    
    * MIGENG-106 added test to check versioning fallback
    
    * MIGENG-106 fixed tests according the new payload taxonomy
    
    * MIGENG-106 added tar.gz file for only 1 json inside
    
    * MIGENG-106 modified getRHIdentity to modify only the identity.internal element
    
    * MIGENG-106 fixes to issues during deployment and test
    
    * MIGENG-106 fixed the calculation method for the number of hypervisors
    
    * MIGENG-106 fixing and adding tests to changes on getRHIdentity and extractMAmetadata
    
    * MIGENG-106 replaced artemis with activemq
    Improvements in the E2E test with the containers
    
    * end2end added hoverfly to simulate insights upload service, firsts attempts
    
    * end2end typo fix
    
    * end2end added logging for testing, and changed to mock-server to mock rest API
    
    * end2end Fixed the test with correct multipart sending, correct jpa retrieving, and kie sample files as response
    
    * end2end Fix the End2End test with correct version of TestContainers
    Fixed test for Kie Service with proper expectation
    Fixed test for Direct Download for considering the proper endpoint having query params
    
    * end2end Removed analytics dependency
    
    * end2end Removed Debug LOG lines
    
    * end2end Removed hoverfly logging
    
    * MIGENG-150 Creation of initial classes and routes for VMWorkloadInventory
    
    * end2end renamed the EndToEnd to a regular Test
    
    * end2end fixing the filtering to camel headers , and switching back to previous false state
    
    * end2end making the identity of the ..with-targz.json file the full one
    
    * end2end making the identity of the platform.upload.xavier.json.gz and .json file the full one
    
    * end2end fixing base64 identity to have the same
    fix the type of the file loaded to be consistent with the one in the base64 identity
    
    * end2end fixed test dependencies
    
    * end2end removed Spring Boot ActiveMQ dependency
    
    * MIGENG-150 Added Test for the VMWorkloadInventoryCalculator
    Changed Ids for second host in sample file (.json)
    Added multicast route to send message to both calculators
    Renamed cost savings route
    Added paths to obtain workload inventory properties
    
    * MIGENG-150 Fixed test
    
    * MIGENG-150 removed parallel processing
    
    * MIGENG-150 added test for direct:calculate-vmworkloadinventory
    
    * MIGENG-150 adapted route using decisionServer to use the right methods
    
    * MIGENG-150 set right values for kie command and session for WKI
    
    * MIGENG-150 fixed value for test
    
    * MIGENG-150 fixed multicast and split issue as seen on CI deployment
    
    * MIGENG-150 changed the method call from setWorkload ... to AddWorkLoad
    
    * MIGENG-150 changed the test for calculate vmworkload to consider individual elements return instead of a collection
    
    * MIGENG-150 added Serializable to the WorkloadInventory Model object
    
    * MIGENG-150 moved class VMWorkloadinventory to another package
    
    * MIGENG-150 fixed depending classes after moved class VMWorkloadinventory to another package
    
    * end2end added info about Ryuk and Testcontainers property
    
    * MIGENG-151 adding methods to get the collection fields
    
    * MIGENG-151 added methods to set values for the collection fields
    added info to the test payload
    
    * MIGENG-150 working on the Files map retrieval
    
    * MIGENG-150 fixed the path for the files map
    fixed the lambda to get the files map value pairs
    
    * MIGENG-151 working on the Files map retrieval
    
    * MIGENG-151 fixed the path for the files map
    fixed the lambda to get the files map value pairs
    
    * MIGENG-151 changed expected values considering also the collection fields for the workloadinventory
    
    * MIGENG-155 saving and persisting the entity
    
    * MIGENG-155 adding methods on the analysisService to get and update the analysismodel
    
    * MIGENG-155 modified WLI route to use the new method in analysisService
    modified test on DirectStore
    
    * MIGENG-155 renamed resources for inticial cost savings report
    
    * end2end Added S3 client bean to customise its endpoint url
    Added PostgreSQL testcontainers container
    Added Localstack testcontainers container
    Fixed analysisid in json file as response from kafka
    
    * end2end Added reports files for demolab payload
    added tests for the 3 reports
    added test for S3
    added performance test
    
    * end2end Added demolab payloads files
    added report file for demolab
    
    * end2end Added Drools and KIE containers
    Added Ingress image build and container creation
    
    * end2end reordered code
    
    * end2end Modified KAFKA estrategy to earliest to take all pending messages
    Added properties for kieserver to connect to local container
    Added properties for KIE container to connect to DROOLS container
    Added docker-compose containers for Ingress
    Removed mocks
    Added commands to import projecto into drools
    
    * end2end added more env vars
    added bash script file with docker and curl commands to simulate the infrastructure deployment
    
    * end2end add env vars
    
    * end2end adjusted testcontainers env vars
    added sleeps in the bash script
    
    * end2end using shell commands to execute docker images for drools and kie
    
    * end2end added travis stuff for docker
    changed the way we get test files from FS
    
    * end2end fixed ocd analytics-template version
    
    * end2end fixed ScanRunModel accessors of Type
    
    * end2end fixed date value for summary report scan run model test file
    
    * end2end fixed Spring Boot error for Port 8080 already in use
    modified folder on ignored test for S3
    added live replacement on ingress docker-compose to make it work
    increased performance test timeout to 1 minute
    
    * end2end addded changes to run Testcontainers in Travis
    
    * end2end added again H2 as test database
    configured the env properties for End2End Test with postgresql
    
    * end2end changes to travis to fix the issue with Testcontainers
    
    * end2end setting log level to debug on tests to check on travis
    
    * end2end log level back to info
    
    * end2end increasing time to wait until initialcostsavings report is generated
    
    * end2end increased time for CSR to 3 minutes, and used awaitillity to continue earlier if its finished
    
    * end2end added log level to hibernate to check connections to DB
    
    * end2end added commands to upgrade docker version at Travis
    removed testcontainers kafka dependency as it's not used
    
    * end2end making kafka to fail in the connection
    
    * end2end added print out of docker logs
    added explicit naming of kafka container
    
    * end2end using master branch of ingress at 20191115
    
    * end2end renamed folder of downloaded insights as it causes issues with docker-compose
    
    * end2end added docker logs in travis
    
    * end2end added logger output to critical containers
    
    * end2end added time before the test starts
    
    * end2end added logger to activemq
    switched back to old version of ingress
    
    * end2end added more logs to containers and increased times
    
    * end2end added advertising for kafka
    
    * end2end enabled again ryuk
    
    * end2end removed -DtestLogRootLevel=warn -Pcoverage from the test script
    
    * end2end adding log and more time to start the test
    
    * end2end added zookeeper logs
    
    * end2end moved docker compose containers to Testcontainers
    added temporary hacks in the download-file route to test
    
    * end2end enabled the download of ingress-go project from github
    
    * end2end printing ScanRunModels and using sameDay as comparison between dates in ScanRunModel
    
    * end2end not considering scanrunmodel.date in the test
    
    * end2end removing tracing because Travis kills the job if we exceed
    
    * end2end removed log of docker containers in travis yaml
    
    * end2end fixed unit tests
    set all tests to be run in travis
    refactored comparisson of sets
    
    * end2end reduced log level to reduce logs in Travis
    
    * end2end changed hardcoded values
    set the logging level in the travis pipeline
    removed unused files
    
    * end2end re-added Profile Coverage to mvn execution
    
    * end2end some PR clean tasks
    - fixed Testcontainers name in Readme
    - removed unused files
    - refactored ...DirectCalculateVMWorkloadInventoryModelTest assert on scanRunModel
    - fixed typo on test properties
    - removed comments on ScanRunModel
    
    * end2end added minio.start
    
    * end2end replaced timeouts with something way shorter (in  milliseconds)
    
    * end2end added Github action to build project and red hat repositories for dependencies
    
    * Update maven.yml
    
    * MIGENG-365 removed github pipeline
    
    * MIGENG-365 cleanup downloaded files
    
    * MIGENG-365 added timouts to the test properties and increased the value
    fixed not available minio.host prop on runtime
    
    * MIGENG-365 as we are not specifying profile, moved the regular SpringConfiguration to any profile except test
    
    * MIGENG-365 added delay between Minio and MINIOMC containers
    
    * MIGENG-365 removed test code from the production route, and moved to the test
    
    * MIGENG-365 cleaned the route on the download-file side, and moved parts to the test
    
    * change `type` by `smartStateEnabled`
    
    * restore `type` by `smartStateEnabled` since it will modify the database
    
    * removed unused import
    
    * fix test
    
    * MIGENG-365 [PR] removed unused jmsTemplate
    
    * MIGENG-365 [PR] moved Docker only test files to subfolder
    
    * MIGENG-365 [PR] renamed minio shell script
    
    * end2end removed minio-mc commands setting policies for the buckets as they are wrong and apparently not needed
    
    Co-authored-by: Carlos E. Feria Vila <[email protected]>
    Jonathan Vila and carlosthe19916 committed Jan 9, 2020
    Configuration menu
    Copy the full SHA
    cdf6246 View commit details
    Browse the repository at this point in the history

Commits on Jan 17, 2020

  1. Configuration menu
    Copy the full SHA
    dba6ad4 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    e8ac231 View commit details
    Browse the repository at this point in the history

Commits on Jan 20, 2020

  1. MIGENG-371 upgraded testcontainers version

    added test after stoping camel context
    commented out the idempotentConsumer
    added debug info on localstack
    jonathan committed Jan 20, 2020
    Configuration menu
    Copy the full SHA
    06d2f8b View commit details
    Browse the repository at this point in the history
  2. MIGENG-371 upgraded testcontainers version

    added test after stoping camel context
    commented out the idempotentConsumer
    added debug info on localstack
    jonathan committed Jan 20, 2020
    Configuration menu
    Copy the full SHA
    1f0dfc6 View commit details
    Browse the repository at this point in the history

Commits on Jan 21, 2020

  1. MIGENG-371 commented out to test on CI

    jonathan committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    745eca0 View commit details
    Browse the repository at this point in the history
  2. MIGENG-371 commented out to test on CI

    jonathan committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    b5b5e18 View commit details
    Browse the repository at this point in the history
  3. MIGENG-371 added tests to check duplication of rows with the Kafka co…

    …nsumer
    jonathan committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    0975cb5 View commit details
    Browse the repository at this point in the history
  4. MIGENG-371 added tests to check duplication of rows with the Kafka co…

    …nsumer
    jonathan committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    f99745b View commit details
    Browse the repository at this point in the history
  5. MIGENG-371 removed the comment on the idempotentConsumer, for the mom…

    …ent no need to use it
    jonathan committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    29c1139 View commit details
    Browse the repository at this point in the history
  6. MIGENG-371 removed the comment on the idempotentConsumer, for the mom…

    …ent no need to use it
    jonathan committed Jan 21, 2020
    Configuration menu
    Copy the full SHA
    30095a5 View commit details
    Browse the repository at this point in the history

Commits on Jan 22, 2020

  1. MIGENG-371 refactoring and giving more time as we have discrepancies …

    …between local and Travis
    jonathan committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    2130709 View commit details
    Browse the repository at this point in the history
  2. MIGENG-371 refactoring and giving more time as we have discrepancies …

    …between local and Travis
    jonathan committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    e6a73d6 View commit details
    Browse the repository at this point in the history
  3. MIGENG-371 reintroduced autoOffsetReset=latest

    refactored duplication tests
    jonathan committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    e889ada View commit details
    Browse the repository at this point in the history
  4. MIGENG-371 reintroduced autoOffsetReset=latest

    refactored duplication tests
    jonathan committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    98d9870 View commit details
    Browse the repository at this point in the history
  5. MIGENG-371 parametrised sleep between duplication check and start

    fix typo
    jonathan committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    b556339 View commit details
    Browse the repository at this point in the history
  6. MIGENG-371 parametrised sleep between duplication check and start

    fix typo
    jonathan committed Jan 22, 2020
    Configuration menu
    Copy the full SHA
    2adbdf5 View commit details
    Browse the repository at this point in the history

Commits on Jan 23, 2020

  1. MIGENG-371 fixed typo on value injection

    jonathan committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    01c8eb4 View commit details
    Browse the repository at this point in the history
  2. MIGENG-371 fixed typo on value injection

    jonathan committed Jan 23, 2020
    Configuration menu
    Copy the full SHA
    fad6692 View commit details
    Browse the repository at this point in the history

Commits on Feb 25, 2020

  1. Configuration menu
    Copy the full SHA
    544d971 View commit details
    Browse the repository at this point in the history
  2. Merging master

    jonathan committed Feb 25, 2020
    Configuration menu
    Copy the full SHA
    04b0c0d View commit details
    Browse the repository at this point in the history