Self Signed Certificates Refactor and Unit Test (#404)

* Fix iterating over properties using incorrect source map.
Add debug logging.

* Fix the spelling on the class name for Base[r]ServerSetup

* Support for multiple certificate keystores accessed by identifier.

* Support passing the self signed trust store into the HTTP context

* Add new properties for trust store and update test configs.

* Fix case when user does not choose a response.

* Add support for importing certs within the certificate factory.
Make the PKCS12 factory only required for scheduled polling of the file
system keystore.

* Add identifiers for certificate factories.

* Support multiple certificate factories

* Refactor self signed certificate handling so that trust store is managed
as a component.

* Fix deprecation warnings

* Standardise on config.dir property key

* Add new properties

* Fix tests failing after certificate factory changes

* Upgrade packages to fix warnings

* Remove unused imports

* Support an "enabled" attribute on the component nodes

* Make a standalone class for ENV replacement so it is reusable for
property loading

* Upgrade to 4.1.0

* Release notes for 4.1.0

* Update dependencies to latest

* Prevent using milestone releases

* Allow Healthcheck module to be fully configured by properties

* Removed all XML based config examples as part of move to properties.
Reformatted and restructured to make logical sense.
Updated numerous sections

* Make methods public to failitate testing.

* Enhancements to support extended testing

* Unit testing Self Signed Certificates for SSL

* Add properties to make modules fully configurable by props.
Rearrange property order to be more logical.

* Refactoring the AS2Message name

* Ensure self signed KeyManagers are in SSL Context

* Support PKCS12 certificate store for SSL certificates.

* Support certificate generation

* Update release notes

* Cater for Windows mess with backslashes in file path when rendering as
string

RELEASE-NOTES.md

@@ -1,32 +1,29 @@
 #              OpenAS2 Server
-#              Version 4.0.0
+#              Version 4.1.0
 #              RELEASE NOTES
 -----
-The OpenAS2 project is pleased to announce the release of OpenAS2 4.0.0
+The OpenAS2 project is pleased to announce the release of OpenAS2 4.1.0
 
-The release download file is: OpenAS2Server-4.0.0.zip
+The release download file is: OpenAS2Server-4.1.0.zip
 
 The zip file contains a PDF document (OpenAS2HowTo.pdf) providing information on installing and using the application.
 ## NOTE: Testing covers Java 11 to 21.
 ##       Java 8 is NO LONGER SUPPORTED.
 
-Version 4.0.0 - 2024-10-15
+Version 4.1.0 - 2024-12-04
 
-This is an major change and enhancement release.
+This is an enhancement release.
        **IMPORTANT NOTE**: Please review upgrade notes below if you are upgrading
 
-1. Java 8 is NO LONGER supported. Support is for Java 11 and newer releases only.
-2. The logging facade was switched from Jakarta Commons Logging to SLF4J and the default implementation uses the Logback framework.
-3. The DB state logging for external databases now uses an Hikari connection pool and persists to the database using a separate thread to improve performance.
-4. Switch to Jakarta package for mail and REST API implementations
-5. Remove the insecure "remote" socket command processor and associated support class in the OpenAS2 server.
-6. Remove redundant packages from maven build.
-7. Fix MDN cleanup occurring when resend mode is entered.
+1. Support for Elliptic curve certificates.
+2. Enhanced support for using SSL with self signed certificates
+3. Support PKCS12 certificate keystore for SSL certificates.
+4. Significantly updated the OpenAS2HowTo documentation.
 
 ##Upgrade Notes
  See the openAS2HowTo appendix for the general process on upgrading OpenAS2.
 
-### Upgrading to 4.0 or newer any older version:
+### Upgrading to 4.0 or newer from any older version:
       1. Ensure you implement all logging that you had configured for ealrier versions using the logback configuration or replace with another framework that works with SLF4J facade. See the OpenAS2HowTo.pdf logging section for more details.
       2. The property for email configuration in the config.xml changed:
           Change ALL occurrences of javax.mail.properties to jakarta.mail.properties in config.xml and the .properties file if you implemented it.

Server/pom.xml

@@ -7,7 +7,7 @@
         <!-- DO NOT CHANGE THIS "groupId" WITHOUT CHANGING XMLSession.getManifestAttributes.MANIFEST_VENDOR_ID_ATTRIB -->
         <groupId>net.sf.openas2</groupId>
         <artifactId>OpenAS2</artifactId>
-        <version>4.0.0</version>
+        <version>4.1.0</version>
         <relativePath>../pom.xml</relativePath>
     </parent>
 
@@ -24,6 +24,7 @@
         <project.dist.package.name>${project.parent.artifactId}Server-${project.version}.zip</project.dist.package.name>
         <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
         <help.file.src>${project.basedir}/../docs/OpenAS2HowTo.pdf</help.file.src>
+        <project.license>${project.basedir}/../LICENSE.txt</project.license>
         <release.notes>${project.basedir}/../RELEASE-NOTES.md</release.notes>
         <release.history>${project.basedir}/../changes.txt</release.history>
         <package.assembly.dir>${project.build.directory}/dist</package.assembly.dir>
@@ -95,6 +96,8 @@
                                       todir="${package.assembly.dir}" verbose="true"/>
                                 <copy file="${release.history}"
                                       todir="${package.assembly.dir}" verbose="true"/>
+                                <copy file="${project.license}"
+                                      todir="${package.assembly.dir}" verbose="true"/>
                                 <!-- create distribution package
                                     basedir="${package.assembly.dir}"
                                 -->

Server/src/bin/gen_p12_key_par.sh

@@ -73,7 +73,7 @@ if [ "1" != "$IS_AUTOMATED_EXEC" ]; then
   echo "Generate a certificate to a PKCS12 key store."
   echo "Generating certificate:  using alias $certAlias to ${tgtStore}.p12 $PRE_GEN_MSG_ADDITIONAL"
   read -p "Do you wish to execute this request? [Y/N]" Response
-  if [  $Response != "Y" -a $Response != "y"  ] ; then
+  if [  "$Response" != "Y" -a "$Response" != "y"  ] ; then
     exit 1
   fi
   read -p "Enter password for keystore:" ksPwd

Server/src/config/config.xml

@@ -1,10 +1,22 @@
 <?xml version="1.0" encoding="UTF-8"?>
 <openas2>
-   <properties storageBaseDir="%home%/../data"
+   <properties config.dir="%home%"
+               storageBaseDir="%home%/../data"
+               partnership_file="%home%/partnerships.xml"
                log_date_format="yyyy-MM-dd HH:mm:ss.SSS"
                sql_timestamp_format="yyyy-MM-dd HH:mm:ss.SSS"
                as2_message_id_format="$date.yyyyMMddHHmmssZ$-$rand.123$-$msg.sender.as2_id$_$msg.receiver.as2_id$_$msg.attributes.filename$"
                as2_receive_message_filename_fallback="$rand.shortUUID$"
+               as2_async_mdn_url="http://localhost:$properties.async_mdn_receiver_port$"
+               as2_keystore="%home%/as2_certs.p12"
+               as2_keystore_password="testas2"
+               as2_keystore.refresh_interval="300"
+               ssl_trust_keystore.enabled="false"
+               ssl_trust_keystore="%home%/ssl_trust_certs.p12"
+               ssl_trust_keystore_password="testas2"
+               ssl_trust_keystore.refresh_interval="300"
+               ssl_keystore="%home%/ssl_certs.jks"
+               ssl_keystore_password="testas2"
                console.command.processor.enabled="false"
                restapi.command.processor.enabled="false"
                restapi.command.processor.baseuri="http://localhost:8080"
@@ -33,14 +45,12 @@
                module.AS2MDNReceiverModule.https.enabled="false"
                module.AS2MDNReceiverModule.https.port="10444"
                module.HealthCheckModule.enabled="false"
+               module.HealthCheckModule.protocol="http"
+               module.HealthCheckModule.address="localhost"
                module.HealthCheckModule.port="10099"
+               module.HealthCheckModule.keystore="$properties.ssl_keystore$"
+               module.HealthCheckModule.keystore_password="$properties.ssl_keystore_password$"
                async_mdn_receiver_port="$properties.module.AS2MDNReceiverModule.http.port$"
-               as2_async_mdn_url="http://localhost:$properties.async_mdn_receiver_port$"
-               as2_keystore="%home%/as2_certs.p12"
-               as2_keystore_password="testas2"
-               ssl_keystore="%home%/ssl_certs.jks"
-               ssl_keystore_password="testas2"
-               partnership_file="%home%/partnerships.xml"
                jakarta.mail.properties.file="%home%/java.mail.properties"
                email.from="Open AS2 Server&lt;[email protected]&gt;"
                email.to="your email address"
@@ -74,12 +84,20 @@
                pollerConfigBase.process_files_in_paralllel="false"
                pollerConfigBase.max_parallel_files="20"
                partnerships.polling.interval="120"
+               messages.enabled="false"
                messages.polling.interval="120"
              />
    <certificates classname="org.openas2.cert.PKCS12CertificateFactory"
+                 identifier="as2_certs"
                  filename="$properties.as2_keystore$"
                  password="$properties.as2_keystore_password$"
-                 interval="300"/>
+                 interval="$properties.as2_keystore.refresh_interval$"/>
+   <certificates enabled="$properties.ssl_trust_keystore.enabled$"
+                 classname="org.openas2.cert.PKCS12CertificateFactory"
+                 identifier="ssl_trust_certs"
+                 filename="$properties.ssl_trust_keystore$"
+                 password="$properties.ssl_trust_keystore_password$"
+                 interval="$properties.ssl_trust_keystore.refresh_interval$"/>
    <commands classname="org.openas2.cmd.XMLCommandRegistry"
              filename="%home%/commands.xml"/>
    <commandProcessors>
@@ -117,20 +135,6 @@
               format="sender.as2_id, receiver.as2_id, attributes.filename"
               mimetype="application/EDI-X12"/>
       -->
-      <!-- This directory polling module is dedicated to sending to partner PartnerA_OID -->
-      <!-- NOTE: This mechanism is no longer the preferred way to add pollers. Use the partnership config as of 3.0.0
-                 See "Partnerships XML Defined Directory Poller" in the OpenAS2HowTo for details.
-      <module classname="org.openas2.processor.receiver.AS2DirectoryPollingModule"
-              outboxdir="$properties.storageBaseDir$/toPartnerA/"
-              errordir="$properties.storageBaseDir$/toPartnerA/error/$date.YYYY$/$date.MM$"
-              stored_error_filename="$msg.attributes.filename$-$date.YYYY$-$date.MM$-$msg.headers.message-id$"
-              sentdir="$properties.storageBaseDir$/$msg.sender.as2_id$-$msg.receiver.as2_id$/sent/$date.YYYY$/$date.MM$"
-              stored_sent_filename="$msg.attributes.filename$-$msg.headers.message-id$"
-              interval="5"
-              defaults="sender.as2_id=MyCompany_OID, receiver.as2_id=PartnerA_OID"
-              sendfilename="true"
-              mimetype="application/EDI-X12"/>
-      -->
       <module enabled="$properties.module.DbTrackingModule.enabled$"
               classname="org.openas2.processor.msgtracking.DbTrackingModule"
               use_embedded_db="$properties.msg_tracking.use_embedded_db$"
@@ -188,7 +192,11 @@
               resenddelay="$properties.module.DirectoryResenderModule.resenddelay$"/>
       <module enabled="$properties.module.HealthCheckModule.enabled$"
               classname="org.openas2.processor.receiver.HealthCheckModule"
-              port="$properties.module.HealthCheckModule.port$"/>
+              protocol="$properties.module.HealthCheckModule.protocol"
+              address="$properties.module.HealthCheckModule.address$"
+              port="$properties.module.HealthCheckModule.port$"
+              ssl_keystore="$properties.module.HealthCheckModule.ssl_keystore$"
+              ssl_keystore_password="$properties.module.HealthCheckModule.ssl_keystore_password$"/>
    </processor>
    <!-- The pollerConfigBase provides the base config for the partnership directory pollers. It must be placed at the top of the file -->
    <pollerConfigBase classname="org.openas2.processor.receiver.AS2DirectoryPollingModule"
@@ -204,7 +212,8 @@
                  filename="$properties.partnership_file$"
                  interval="$properties.partnerships.polling.interval$"/>
 
-   <messages classname="org.openas2.message.XMLMessageFactory"
-                 filename="%home%/messages.xml"
-                 interval="$properties.messages.polling.interval$"/>
+   <messages enabled="$properties.messages.enabled$"
+             classname="org.openas2.message.XMLMessageFactory"
+             filename="%home%/messages.xml"
+             interval="$properties.messages.polling.interval$"/>
 </openas2>

Server/src/config/openas2.properties.sample

@@ -1,10 +1,31 @@
 storageBaseDir=/opt/MyCompanyAS2/data
-as2_keystore=$properties.storageBaseDir$/as2_certs.p12
+# Where to find the partnerships XML file
 partnership_file=$properties.storageBaseDir$/partnerships.xml
+# the path to and name ofthe keystore fiule containing the AS2 certificates
+as2_keystore=$properties.storageBaseDir$/as2_certs.p12
+# The password for the AS2 certificates keystore
 as2_keystore_password=032scali
+# The time between checks for a changed AS2 keystore file
+as2_keystore.refesh_interval=300
+# The keystore for SSL certificates if SSL is enabled
+ssl_keystore=$properties.storageBaseDir$/ssl_certs.jks
+# The SSL certificates keystore password
+ssl_keystore_password=testas2
+# The SSL certificates keystore interval in seconds checking for changed file
+ssl_trust_keystore.refresh_interval=300
+# The trusted Self Signed SSL certificates keystore
+ssl_trust_keystore=%home%/ssl_trust_certs.p12
+# The trusted Self Signed SSL certificates keystore password
+ssl_trust_keystore_password=testas2
+# The trusted Self Signed SSL certificates keystore interval in seconds checking for changed file
+ssl_trust_keystore.refresh_interval=300
+# The format of the logging entry timestamp
 log_date_format=yyyy-MM-dd HH:mm:ss.SSS
+# The SQL servers timestamp format - default is SQL92 so should work for most DB's
 sql_timestamp_format=yyyy-MM-dd HH:mm:ss.SSS
+# The AS2 message ID is generated using this property
 as2_message_id_format=$date.ddMMyyyyHHmmssZ$-$rand.UUID$
+# The AS2 message payload file name fallback option if it is not passed in the AS2 message headers
 as2_receive_message_filename_fallback=$rand.shortUUID$
 log_invalid_http_request=false
 # The command processor that works within the command line console after startup. NOT to be used for production deployments.
@@ -46,26 +67,47 @@ module.AS2MDNReceiverModule.https.enabled=false
 module.AS2MDNReceiverModule.https.port=10444
 # Supports a healthcheck API to monitor the OpenAS2 server
 module.HealthCheckModule.enabled=false
+# The host address that the healthcheck module binds to
+module.HealthCheckModule.address=localhost
 # What port can the healthcheck module run on
-module.HealthCheckModule.port="10099"
+module.HealthCheckModule.port=10099
 # The ASYNC URL that will be used if you tunr on ASYN mode MDN for a partner. Can be overridden explicitly in the partnership definition
 #as2_async_mdn_url=https://myas2.mycomany.com:$properties.module.AS2MDNReceiverModule.https.port$
-# The keystore for SSL certificates if SSL is enabled
-ssl_keystore=$properties.storageBaseDir$/ssl_certs.jks
-# The SSL certificates keystore password
-ssl_keystore_password=Fantini0101
 # The location of the DB tracking database if using the H@ database which is the default in OpenAS2
 msg_tracking.db_directory=$properties.storageBaseDir$/DB
+# Use the H2 embedded database that is the default for OpenAS2. Change to false for other DB's
 msg_tracking.use_embedded_db=true
+# Only necessary for older JDBC drivers. Mostly just ignored.
 msg_tracking.force_load_jdbc_driver=false
+# Database user name
 msg_tracking.db_user=sa
+# Database password
 msg_tracking.db_pwd=OpenAS2
+# Database instance name
 msg_tracking.db_name=openas2
+# Database tracking table name
 msg_tracking.table_name=msg_metadata
+# Database JDBC driver
 msg_tracking.jdbc_driver=org.h2.Driver
+# The JDBC connect string. The below string is specific to H2 database
 msg_tracking.jdbc_connect_string=jdbc:h2:$component.db_directory$/$component.db_name$
 msg_tracking.sql_escape_character='
+# Specific to the embedded database that comes with OpenAS2. Ignored otherwise.
 msg_tracking.tcp_server_start=true
+# Specific to the embedded database that comes with OpenAS2. Ignored otherwise.
 msg_tracking.tcp_server_port=10092
+# Specific to the embedded database that comes with OpenAS2. Ignored otherwise.
 msg_tracking.tcp_server_password=openas2
+# Requires all received AS2 messages to be signed. If you have a partner who requires sending unsigned messages then change this to false
 reject_unsigned_messages=true
+# The following are all related to the partnership based directory poller.
+pollerConfigBase.outboxdir=$properties.storageBaseDir$/outbox/$partnership.receiver.as2_id$
+pollerConfigBase.errordir=$properties.storageBaseDir$/outbox/error/$date.YYYY$-$date.MM$-$date.dd$/$partnership.receiver.as2_id$
+pollerConfigBase.interval=5
+pollerConfigBase.defaults=sender.as2_id=$partnership.sender.as2_id$, receiver.as2_id=$partnership.receiver.as2_id$
+pollerConfigBase.sendfilename=true
+pollerConfigBase.mimetype=application/EDI-X12
+pollerConfigBase.process_files_in_paralllel=false
+pollerConfigBase.max_parallel_files=20
+# The time between checks for a changed partnerships.xml file for auto reload.
+partnerships.polling.interval=120

Server/src/main/java/CheckCertificate.java

@@ -221,7 +221,7 @@ public int CheckCertStore(String host, int port, String uri, String targetKeySto
 
     private void checkUsingApacheHttp(String host, int port, String uri, String targetKeyStore, String keyStorePwd) throws Exception {
         System.out.println("Trying using Apache HTTP Client...");
-        Map<String, String> httpOptions = new HashMap<String, String>();
+        Map<String, Object> httpOptions = new HashMap<String, Object>();
         if (auth_user != null) {
             httpOptions.put(HTTPUtil.PARAM_HTTP_USER, auth_user);
             httpOptions.put(HTTPUtil.PARAM_HTTP_PWD, auth_pwd);

Server/src/main/java/org/openas2/BaseSession.java

@@ -54,8 +54,8 @@ public void stop() throws Exception {
         }
     }
 
-    public CertificateFactory getCertificateFactory() throws ComponentNotFoundException {
-        return (CertificateFactory) getComponent(CertificateFactory.COMPID_CERTIFICATE_FACTORY);
+    public CertificateFactory getCertificateFactory(String componentID) throws ComponentNotFoundException {
+        return (CertificateFactory) getComponent(componentID);
     }
 
     public Map<String, Map<String, Object>> getPolledDirectories() {
@@ -71,10 +71,14 @@ public void setPolledDirectories(Map<String, Map<String, Object>> polledDirector
      *
      * @param componentID registers the component to this ID
      * @param comp        component to register
+     * @throws OpenAS2Exception 
      * @see Component
      */
-    public void setComponent(String componentID, Component comp) {
+    public void setComponent(String componentID, Component comp) throws OpenAS2Exception {
         Map<String, Component> objects = getComponents();
+        if (objects.containsKey(componentID)) {
+            throw new OpenAS2Exception("A component with this ID has already been regiostered: " + componentID);
+        }
         objects.put(componentID, comp);
     }
 
@@ -207,6 +211,10 @@ public DirectoryPollingModule getPartnershipPoller(String senderAs2Id, String re
 
     public void loadPartnershipPoller(Node moduleNode, String partnershipName, String configSource) throws OpenAS2Exception {
         DirectoryPollingModule procmod = (DirectoryPollingModule) XMLUtil.getComponent(moduleNode, this);
+        if (procmod == null) {
+            // Must be disable so do nothing
+            return;
+        }
         String pollerDir = procmod.getParameters().get(DirectoryPollingModule.PARAM_OUTBOX_DIRECTORY);
         try {
             checkPollerModuleConfig(pollerDir);

Server/src/main/java/org/openas2/Session.java

@@ -51,7 +51,7 @@ public interface Session {
      * @see CertificateFactory
      * @see Component
      */
-    CertificateFactory getCertificateFactory() throws ComponentNotFoundException;
+    CertificateFactory getCertificateFactory(String componentID) throws ComponentNotFoundException;
 
     /**
      * Gets the <code>Component</code> currently registered with an ID