Mail client auto-configuration updated

Mail server testing additions

server/mail/autoconfig.rst

@@ -1,14 +1,233 @@
 Mail Client Auto-Configuration
 ==============================
 
-Some mail clients can retrieve their configuration for a mail address
-automatically, trough a combination of DNS and HTTP queries leading to
-configuration data provided as XML on a webserver.
+With mail-client auto-configuration, users who setup a new mail-client or
+smartphone only need to provide their mail-address and password. All the
+complicated configurations for incoming and outgoing mailservers, port-number,
+encryption and login methods, address-books and calendars will then be setup
+automatically.
+
+These mail clients retrieve configuration data for a mail-address
+trough a combination of DNS and HTTP queries from the domain-part of the address.
+
+As there is no univeersal standard for this, different mail-clients use
+their own proprietary method, which service providers need to support on there
+servers for this to work. Other clients don't even support such functionality.
+
+Three different methods are known today to support automatic mail client
+configuration:
+
+- Apple Mac, iPhone and iPad:
+
+  - iPhone and iPod touch with iOS 4 or later
+  - iPad with iOS 4.3 or later or iPadOS 13.1 or later
+  - Mac computers with OS X 10.7 or later
+  - Apple TV with tvOS 9 or later
+
+- Microsoft Outlook
+- Moziila Thunderbird, also supported by:
+
+  - Evolution (for Linux GNOME desktops)
+  - FairEmail (for Android phones)
+  - K9 Mail (for Android phones)
+  - KMail aka Kontact (for Lunux KDE desktops)
+  - NextCloud Mail App
+
+Manually providing and mainting all these different formart is tedious.
+
+automx2 App
+-----------
+
+`automx2 <https://automx.org/en/>`_ is a auto-configuration web-service who
+can provide configuration data in all three formats to requesting clients.
+
+
+Prerequesites
+^^^^^^^^^^^^^
+
+The following needs to be available beforehand:
+
+* :doc:`/server/mariadb/index`
+* :doc:`/server/nginx/index`
+
+Create a system user who will run the service::
+
+    $ sudo adduser --system --home /var/www/example.net/automx2 automx2
+
+
+Create a database access password::
+
+    $ pwgen -s 32 1
+    jyHZdNnB3Fe3sTotihMTiuf51BH6EEq9YkCd0zTWU6GekkkO
+
+Create a database and a user in MariaDB server to hold configuration data::
+
+    mysql -p
+
+::
+
+    mysql> CREATE DATABASE `automx2` COLLATE 'utf8mb4_general_ci';
+    mysql> GRANT SELECT ON automx2.* TO 'automx2'@'127.0.0.1' \
+    mysql>     IDENTIFIED BY 'jyHZdNnB3Fe3sTotihMTiuf51BH6EEq9YkCd0zTWU6GekkkO';
+    mysql> FLUSH PRIVILEGES;
+    mysql> exit
+
+
+Create a Python virtual environment for the software to be installed under::
+
+    $ sudo -u automx2 -Hs
+    $ cd /var/www/example.net/automx2
+    $ wget https://github.com/rseichter/automx2/raw/master/contrib/setupvenv.sh
+    $ chmod u+x setupvenv.sh
+    $ ./setupvenv.sh
+
+
+Software Installation
+^^^^^^^^^^^^^^^^^^^^^
+
+::
+
+    $ sudo -u automx2 -Hs
+    $ source .venv/bin/activate
+    $ pip install automx2
+
+
+Software Configuration
+^^^^^^^^^^^^^^^^^^^^^^
+
+Create the file :file:`/etc/automx2/automx2.conf`.
+
+.. code:: ini
+
+    [automx2]
+
+    # A typical production setup would use loglevel WARNING.
+    loglevel = DEBUG
+
+    # Echo SQL commands into log? Used for debugging.
+    db_echo = no
+
+    # MySQL database on a remote server. This example does not use an encrypted
+    # connection and is therefore *not* recommended for production use.
+    #db_uri = mysql://username:[email protected]/db
+
+    # Database server connection
+    db_uri = mysql+pymysql://automx2:jyHZdNnB3Fe3sTotihMTiuf51BH6EEq9YkCd0zTWU6GekkkO@localhost/automx2?charset=utf8mb4
+
+    # Number of proxy servers between automx2 and the client (default: 0).
+    # If your logs only show 127.0.0.1 or ::1 as the source IP for incoming
+    # connections, proxy_count probably needs to be changed.
+    proxy_count = 1
+
+
+Initialize
+^^^^^^^^^^
+
+Initialize the database::
+
+    $ curl http://127.0.0.1:4243/initdb/
+
+
+Copy and edit the file :file:`/var/www/example.net/automx2/contrib/seed-example.json`
+
+.. code-block:: json
+
+    {
+        "provider": "Example Net.",
+        "domains": ["example.net", "example.org", "example.com"],
+        "servers": [
+            {"name": "mail.example.net", "type": "imap"},
+            {"name": "mail.example.net", "type": "smtps"}
+        ]
+    }
+
+
+SystemD Service
+---------------
+
+Copy the provided service file
+:file:`/var/www/example.net/automx2/contrib/automx2.service` to the
+:file:`/etc/systemd/system/` directory.
+
+Ajust the file path of the ExecStart and WorkingDirectory lines to our
+Installation.
+
+.. code-block:: ini
+
+    [Unit]
+    After=network.target
+    Description=MUA configuration service
+    Documentation=https://rseichter.github.io/automx2/
+
+    [Service]
+    Environment=FLASK_APP=automx2.server:app
+    Environment=FLASK_CONFIG=production
+    ExecStart=/var/www/example.net/automx2/bin/flask run --host=127.0.0.1 --port=4243
+    Restart=always
+    User=automx2
+    WorkingDirectory=/var/lib/automx2
+
+    [Install]
+    WantedBy=multi-user.target
+
+Reload SystemD and enable the service::
+
+    $ sudo systemctl daemon-reload
+    $ sudo systemctl enable automx2
+
+
+Updating
+--------
+
+Updating the Software
+^^^^^^^^^^^^^^^^^^^^^
+
+::
+
+    $ sudo -u automx2 -Hs
+    $ cd /srv/web/automx2
+    $ source .venv/bin/activate
+    $ pip install --upgrade automx2
+
+
+Updating the Database
+^^^^^^^^^^^^^^^^^^^^^
+
+::
+
+    $ sudo -u automx2 -Hs
+    $ cd /srv/web/automx2
+    $ export RELEASE="2021.6"
+    $ wget https://github.com/rseichter/automx2/archive/refs/tags/$RELEASE.zip
+    $ unzip $RELEASE.zip
+    $ cd automx2-$RELEASE/alembic
+
+
+Edit the file :file:`/var/www/example.net/automx2/alembic/alembic.ini`
+
+.. code-block:: ini
+
+    # Database server connection
+    sqlalchemy.url = mysql://automx2:jyHZdNnB3Fe3sTotihMTiuf51BH6EEq9YkCd0zTWU6GekkkO@localhost/automx2?charset=utf8mb4
+
+
+Do the upgrade::
+
+    $ source .venv/bin/activate
+    make upgrade
 
 
 Mozilla Thunderbird
 -------------------
 
+Thunderbird looks for configuration data in XML-format at predefined
+(well-known) URLs.
+
+This method of autonconfiguration
+
+This also works for ...
+
+
 Evolution and KMail have adopted this format too.
 
 The process is desribed at the `Autoconfiguration in Thunderbird
@@ -255,10 +474,81 @@ this should will as follows:
 The example above is for three domains only. For every addiotional domain, the
 number of hostnames who need to be certfied by your CA increases exponentially.
 
+Testing
+-------
+
+Microsoft
+^^^^^^^^^
+
+* `Microsoft Remote Connectivity Analyzer <https://testconnectivity.microsoft.com/>`_
+* `Outlook Connectivity <https://testconnectivity.microsoft.com/tests/O365Ola/input>`_
+
+
+Other Projects
+--------------
+
+* `The automx2 Web Application <https://rseichter.github.io/automx2/>`_
+* `<https://github.com/smartlyway/email-autoconfig-php>`_
+* `<https://github.com/olkitu/Autoconfig-PHP>`_
+* `Milkys Homepage: Mail autoconfiguration for MS Outlook, Thunderbird and Apple devices <https://mcmilk.de/projects/autoconfig/>`_
+
+
+References
+----------
+
+RFCs
+^^^^
+
+* :rfc:`6186` - "Use of SRV Records for Locating Email Client Services"
+* :rfc:`6764` - "Locating Services for CalDAV and CardDAV"
+
+
+Mozilla Thunderbird
+^^^^^^^^^^^^^^^^^^^
+
+Mozilla Wiki:
+
+* `Thunderbird:Autoconfiguration <https://wiki.mozilla.org/Thunderbird:Autoconfiguration>`_ (2021)
+* `Thunderbird:Autoconfiguration:DNSBasedLookup <https://wiki.mozilla.org/Thunderbird:Autoconfiguration:DNSBasedLookup>`_ (2009)
+* `Thunderbird:Autoconfiguration:ConfigFileFormat <https://wiki.mozilla.org/Thunderbird:Autoconfiguration:ConfigFileFormat>`_ (2022)
+
+Ben Bucksch (Moziila Dev):
+
+* `Thunderbird Autoconfiguration <https://www.bucksch.org/1/projects/thunderbird/autoconfiguration/>`_ (2022)
+
+
+Microsoft Outlook
+^^^^^^^^^^^^^^^^^
+
+Microsoft Support:
+
+* `Outlook 2016 implementation of Autodiscover <https://support.microsoft.com/en-us/topic/outlook-2016-implementation-of-autodiscover-0d7b2709-958a-7249-1c87-434d257b9087>`_
+
+Microsoft Build:
+
+* `Autodiscover for Exchange <https://docs.microsoft.com/en-us/exchange/client-developer/exchange-web-services/autodiscover-for-exchange>`_
+* `Microsoft Build: Autodiscover web service reference for Exchange <https://docs.microsoft.com/en-us/exchange/client-developer/web-service-reference/autodiscover-web-service-reference-for-exchange>`_
+* `Microsoft Build: Autodiscover service in Exchange Server <https://docs.microsoft.com/en-us/Exchange/architecture/client-access/autodiscover?view=exchserver-2019>`_
+
+Third-Party:
+
+* `MSXFAQ: Autodiscover V2 <https://www.msxfaq.de/exchange/autodiscover/autodiscover_v2.htm>`_
+
+
+Apple
+^^^^^
+
+Apple Support:
+
+* `Intro to mobile device management <https://support.apple.com/de-de/guide/deployment/depc0aadd3fe/web>`_
+* `Mail MDM payload settings for Apple devices <https://support.apple.com/de-de/guide/deployment/dep9c14bfc5/1/web/1.0>`_
+* `Subscribed Calendars MDM payload settings for Apple devices <https://support.apple.com/de-de/guide/deployment/dep950bfdb6/1/web/1.0>`_
+* `Distribute profiles manually with Profile Manager <https://support.apple.com/guide/profile-manager/distribute-profiles-manually-pmdbd71ebc9/mac>`_
+
+Apple Developers:
+
+* `Configuration Profile Reference (PDF) <https://developer.apple.com/business/documentation/Configuration-Profile-Reference.pdf>`_
 
-References:
+Third-Party:
 
- * https://automx.org/en/
- * https://testconnectivity.microsoft.com/
- * https://github.com/smartlyway/email-autoconfig-php
- * :rfc:`6186` - "Use of SRV Records for Locating Email Submission/Access Services"
+* `Over-the-air IPhone Setup Using a Signed .mobileconfig File <http://www.rootmanager.com/iphone-ota-configuration/iphone-ota-setup-with-signed-mobileconfig.html>`_

server/mail/testing.rst

@@ -117,12 +117,18 @@ Mail Message
     > 250 2.1.5 Ok
     DATA
     > 354 End data with <CR><LF>.<CR><LF>
-    From: [email protected]
-    Subject: Test message!
+    Message-ID: <[email protected]>
+    Date: Sun, 3 Jul 2022 20:58:50 +0200
+    MIME-Version: 1.0
+    Subject: Test Message
+    To: John Doe <[email protected]>
+    Content-Language: en-US
+    From: John Doe <[email protected]>
+    Subject: Test Message
 
     Hi,
 
-    This is a test message!
+    This is a test message.
 
     Best,
     Widmore