v5.5.0

Version Summary

Version Number : v5.5.0 [tag: v5.5.0]
Date: 03-April-2024

Projects / Repos Included in this release: IUDX Data Ingestion Server, IUDX Deployment and installation, IUDX's Documentation

Release Summary :

India Urban Data Exchange (IUDX) | Data Ingestion Server v5.5.0 is released with enhanced features.

Highlights:

• Updated Software Client libraries
• Updated Software Testing
  • Updated Postman Collection
  • Included REST Assured based integration testing
  • Improved test coverage
• Software License:
  • With this release, all IUDX Softwares will be distributed under Apache 2.0 License

Test Reports:

Please find the release test details and reports here

API Docs

The api docs can be found here.

How to use the Release:

External Dependencies Installation

The Data ingestion pipeline connects to external dependencies namely

• RabbitMQ

Make configuration

Make a config file based on the template in ./configs/config-example.json

• Generate a certificate using Lets Encrypt or other methods
• Make a Java Keystore File and mention its path and password in the appropriate sections
• Modify the database url and associated credentials in the appropriate sections

Docker based

1. Install docker and docker-compose
2. Clone this repo
3. Build the images
   ./docker/build.sh
4. Modify the docker-compose.yml file to map the config file you just created
5. Start the server in production (prod) or development (dev) mode using docker-compose
   docker-compose up prod

Maven based

1. Install java 11 and maven
2. Use the maven exec plugin based starter to start the server
   mvn clean compile exec:java@data-ingestion-server

JAR based

1. Install java 11 and maven
2. Set Environment variables

export DI_URL=https://<rs-domain-name>
export LOG_LEVEL=INFO

3. Use maven to package the application as a JAR
   mvn clean package -Dmaven.test.skip=true
4. 2 JAR files would be generated in the target/ directory
   • iudx.data.ingestion.server-cluster-0.0.1-SNAPSHOT-fat.jar - clustered vert.x containing micrometer metrics
   • iudx.data.ingestion.server-dev-0.0.1-SNAPSHOT-fat.jar - non-clustered vert.x and does not contain micrometer metrics

Running the clustered JAR

Note: The clustered JAR requires Zookeeper to be installed. Refer here to learn more about how to set up Zookeeper. Additionally, the zookeepers key in the config being used needs to be updated with the IP address/domain of the system running Zookeeper.
The JAR requires 3 runtime arguments when running:

• --config/-c : path to the config file
• --hostname/-i : the hostname for clustering
• --modules/-m : comma separated list of module names to deploy
  e.g. java -jar ./fatjar.jar --host $(hostname) -c configs/config.json -m iudx.data.ingestion.server.ApiServerVerticle
  Use the --help/-h argument for more information. You may additionally append an DI_JAVA_OPTS environment variable containing any Java options to pass to the application.
  e.g.

$ export RS_JAVA_OPTS="-Xmx4096m"
$ java $RS_JAVA_OPTS -jar target/iudx.data.ingestion.server-cluster-0.0.1-SNAPSHOT-fat.jar ...

Running the non-clustered JAR

The JAR requires 1 runtime argument when running:

• --config/-c : path to the config file
  e.g. java -Dvertx.logger-delegate-factory-class-name=io.vertx.core.logging.Log4j2LogDelegateFactory -jar target/iudx.data.ingestion.server-dev-0.0.1-SNAPSHOT-fat.jar -c configs/config.json
  Use the --help/-h argument for more information. You may additionally append an RS_JAVA_OPTS environment variable containing any Java options to pass to the application.
  e.g.

$ export RS_JAVA_OPTS="-Xmx1024m"
$ java $RS_JAVA_OPTS -jar target/iudx.data.ingestion.server-dev-0.0.1-SNAPSHOT-fat.jar ...

License

With this release, all IUDX Softwares will be distributed under Apache 2.0 License

What's Changed

• Jenkins CI: Updated image tag to 5.5.0-alpha by @karun-singh in #87
• Software Updates by @Kranthi-Guribilli in #88
• License information update by @sushanthakumar in #89

New Contributors

• @Kranthi-Guribilli made their first contribution in #88
• @sushanthakumar made their first contribution in #89

Full Changelog: v5.0.0...v5.5.0