interTwin-eu · vrbanecd · Oct 16, 2024 · Sep 11, 2024 · Sep 12, 2024 · Sep 12, 2024
diff --git a/CONFIGURATION.md b/CONFIGURATION.md
@@ -1,20 +1,19 @@
 # Configuration
 
+All configuration information for both Teapot and StoRM WebDAV is contained in
+the `config.ini` file. Please update this file as needed to meet your requirements.
+
 ## Certificates
 
 Both Teapot and StoRM WebDAV servers require `SSL` certificates. Teapot requires
-a certificate/key pair for the machine's DNS name to be added to
+an appropriate server certificate/key pair for the machine to be added to
 `/var/lib/teapot/webdav`. The certificate should also be added to the system's
 trust store. To generate self-signed certificates for StoRM WebDAV and add them
 to the trust store, run `sudo /usr/share/teapot/self-signed-cert-gen.sh`.
 
-In `teapot.py` line 945, uvicorn hostname needs to be replaced with the machine
-DNS name, and paths to the certificate/key pair for that DNS name should be
-added here.
-
-## Storm-webdav configuration
+## Additional Storm-webdav configuration
 
-Two main pieces of information are required to configure StoRM-WebDAV:
+Two main pieces of additional information are required to configure StoRM-WebDAV:
 
 - Storage Areas (SA), which are folders assigned to each user
 - OIDC provider and identity mapping information
@@ -47,9 +46,9 @@ For information on how to configure storage-areas, please refer to the
 ### OIDC provider and identity mapping information
 
 OIDC provider information used to authenticate users must be provided. See below
-for information on where to change the OIDC provider information. For more
-details on authentication to the storage areas, refer to the
-[StoRM WebDAV Guidelines](https://github.com/italiangrid/storm-webdav/blob/master/doc/storage-area-configuration.md).
+for information on where to change the OIDC provider information, besides the
+changes already made in `config.ini` file. For more details on authentication to
+the storage areas, refer to the [StoRM WebDAV Guidelines](https://github.com/italiangrid/storm-webdav/blob/master/doc/storage-area-configuration.md).
 
 To configure the OIDC provider information, make the following changes:
 
@@ -59,8 +58,6 @@ To configure the OIDC provider information, make the following changes:
    modifying the `iss` information which stands for issuer.
 3. Modify the OIDC providers that have access to the storage area by modifying
    the `org` information in `/usr/share/teapot/storage_element.properties`.
-4. Modify the OIDC provider list in `teapot.py` under
-   `flaat.set_trusted_OP_list`.
 
 If no other way for mapping user's local and global identities is provided, the
 rudementary way is defined as explained next. The `user-mapping.csv` file is to

diff --git a/README.md b/README.md
@@ -16,7 +16,7 @@ will start another StoRM-WebDAV server for that user in the same way.
 
 Teapot needs `Python3` to run, the specific distribution depends on the operating system.
 The required Python packages for installing Teapot are listed in the [requirements.txt](https://github.com/interTwin-eu/teapot/blob/main/requirements.txt)
-file. Python packages that aren't provided in the rpm format are already included in Teapot's rpm file.
+file. Python packages that aren't available in the rpm format are already included in Teapot's rpm file.
 To automatically generate self-signed certificate `openssl>3` is needed. Storm-webdav needs `java-11`.
 
 ### Installation & Configuration
@@ -27,8 +27,8 @@ feel free to contact us and we’ll gladly provide you with one for your specifi
 
 To install teapot, first a user `teapot` must be created. To do that run `sudo adduser teapot`.
 
-Please refer to [CONFIGURATION.md](https://github.com/interTwin-eu/teapot/blob/main/CONFIGURATION.md) for information on
-how to configure Teapot.
+Please refer to [CONFIGURATION.md](https://github.com/interTwin-eu/teapot/blob/main/CONFIGURATION.md)
+for information on how to configure Teapot and Storm-Webdav.
 
 ### Starting
 

diff --git a/Service_Reference_Card.md b/Service_Reference_Card.md
@@ -12,12 +12,15 @@ start another StoRM-WebDAV server for that user in the same way.
 
 ## List of configuration files
 
-All templates for configuration files are located in `/templates`. Teapot
-configuration file can be found in `/etc/teapot/`. All configuration files for
-StoRM-WebdDAV for each user are automatically generated in
-`/var/lib/teapot/user-$USER/` when user sends its first request. To provide the
-necessary information for configuration files and for more details on
-configuration files, please refer to [CONFIGURATION.md](https://github.com/interTwin-eu/teapot/blob/main/CONFIGURATION.md).
+The primary configuration for Teapot and Storm WebDAV can be found at
+`/etc/teapot/config.ini`. All templates for configuration files for individual users
+are located in `usr/share/teapot/templates`. All configuration files for
+StoRM-WebdDAV's individual users are automatically generated in their user directories
+as defined in `config.ini`, when a particular user sends its first request. To provide
+the necessary information for configuration files and for more details on configuration
+files, please refer to [CONFIGURATION.md](https://github.com/interTwin-eu/teapot/blob/main/CONFIGURATION.md).
+
+## Run setting
 
 `teapot.py` must be run as the `teapot` user, with the Python modules listed in
 `requirements.txt` and the sudoers permissions defined in the `templates/teapot` file.

diff --git a/config.ini b/config.ini
@@ -0,0 +1,75 @@
+[Teapot]
+# Here is the configuration for the Teapot server
+APP_NAME=teapot
+# One less than the first port that is going to be used by any storm webdav
+STARTING_PORT=32399
+# instance timeout, Storm-webdav instances are automatically deleted after a period of inactivity specified here
+# default: 10 minutes
+INSTANCE_TIMEOUT_SEC=600
+# interval between instance timeout checks in stop_expired_instances
+# default: 3 minutes
+CHECK_INTERVAL_SEC=180
+# Hostname for the Teapot's uvicorn server (without the https:// part)
+hostname = teapot
+# Port number Teapot is listening on
+port = 8081
+# List of trusted identity providers
+trusted_OP = https://keycloak:8443/realms/test-realm, https://aai-demo.egi.eu/auth/realms/egi
+# Path to the CA for the DNS certificate
+Teapot_CA = /etc/pki/ca-trust/source/anchors/Teapot-testing.crt
+# Path to the certificate for the machine's DNS
+Teapot_ssl_certificate = /var/lib/teapot/webdav/teapot.crt
+# Path to the key for the machine's DNS
+Teapot_ssl_key = /var/lib/teapot/webdav/teapot.key
+# Path to the Teapot log file location
+log_location = /var/log/teapot/teapot.log
+
+[Storm-webdav]
+# Here is the configuration for the Storm-Webdav server for each user
+# General guide to storm-webdav configuration is here: https://github.com/italiangrid/storm-webdav/blob/master/doc/storm-webdav-guide.md
+# CA for Storm-webdav - by default Storm-webdav uses self-signed certificate
+Storm-webdav_CA = /etc/pki/ca-trust/source/anchors/localhost.crt
+# Memory allocation for Storm-webdav JVM
+JVM_OPTS = -Xms2048m -Xmx2048m -Djava.security.egd=file:/dev/./urandom
+# Hostname for Storm-webdav server
+SERVER_ADDRESS = localhost
+# Port number for the https port for Storm-webdav server
+HTTPS_PORT = ${Current-user:port}
+# Port number for the http port for Storm-webdav server
+HTTP_PORT = ${Current-user:port1}
+# Path to Storm-webdav server certificate
+CERTIFICATE_PATH = /var/lib/teapot/webdav/localhost.crt
+# Path to Storm-webdav server key
+PRIVATE_KEY_PATH = /var/lib/teapot/webdav/localhost.key
+# Path to the trust anchor directory
+TRUST_ANCHORS_DIR = /etc/ssl/certs
+# Trust anchors refresh interval
+TRUST_ANCHORS_REFRESH_INTERVAL = 86400
+# The size of the thread pool used to serve incoming requests
+MAX_CONNECTIONS = 300
+MAX_QUEUE_SIZE = 900
+CONNECTOR_MAX_IDLE_TIME = 30000
+# Path to storage areas configuration directories
+SA_CONFIG_DIR = /var/lib/teapot/user-${Current-user:username}/sa.d
+# Path to Storm-webdav jar file
+JAR = /usr/share/java/storm-webdav/storm-webdav-server.jar
+# Path to Storm-webdav log file
+LOG = /var/log/teapot/storm-webdav-server-user-${Current-user:username}.log
+# Path to Storm webdav output file
+OUT = /var/log/teapot/storm-webdav-server-user-${Current-user:username}.out
+# Path to Storm-webdav error file
+ERR = /var/log/teapot/storm-webdav-server-user-${Current-user:username}.err
+# Path to Storm-webdav log configuration file
+LOG_CONFIGURATION = /etc/teapot/logback.xml
+# Path to Storm-webdav access log configuration file
+ACCESS_LOG_CONFIGURATION = /etc/teapot/logback-access.xml
+# Storm-webdav VO files (disabled)
+VO_MAP_FILES_ENABLE = false
+# Refresh interval for the VO files (disabled)
+VO_MAP_FILES_REFRESH_INTERVAL = 21600
+# The size of the thread pool used to serve third-party copy requests
+TPC_MAX_CONNECTIONS = 50
+TPC_VERIFY_CHECKSUM = false
+# Require client certificate is disabled as we use tokens instead
+REQUIRE_CLIENT_CERT = false
+TPC_USE_CONSCRYPT = true
diff --git a/requirements.txt b/requirements.txt
@@ -7,4 +7,6 @@ uvicorn
 anyio
 asyncio
 psutil
+configparser
+dotenv
 
diff --git a/robot/teapot-requirements.dockerfile b/robot/teapot-requirements.dockerfile
@@ -7,7 +7,7 @@ RUN \
     dnf -y group install "Development Tools" && \
     dnf -y install libffi libffi-devel cronie openssl python3-pip && \
     dnf -y install python3-fastapi python3-httpx python3-pydantic && \
-    dnf -y install python3-uvicorn python3-anyio python3-psutil && \
+    dnf -y install python3-uvicorn python3-anyio python3-psutil python3-dotenv && \
     dnf clean all
 
 WORKDIR /usr/local/ssl

diff --git a/rpm/teapot-alma.spec b/rpm/teapot-alma.spec
@@ -43,6 +43,8 @@ mkdir -p %{buildroot}/%{_sysconfdir}/%name
 cp %{_builddir}/%name-%version/templates/issuers %{buildroot}/%{_sysconfdir}/%name/
 cp %{_builddir}/%name-%version/templates/logback.xml %{buildroot}/%{_sysconfdir}/%name/
 cp %{_builddir}/%name-%version/templates/logback-access.xml %{buildroot}/%{_sysconfdir}/%name/
+cp %{_builddir}/%name-%version/config.ini %{buildroot}/%{_sysconfdir}/%name/
+cp %{_builddir}/%name-%version/user_config.ini %{buildroot}/%{_sysconfdir}/%name/
 mkdir -p %{buildroot}/%{_sysconfdir}/sudoers.d/
 cp %{_builddir}/%name-%version/templates/teapot %{buildroot}/%{_sysconfdir}/sudoers.d/
 mkdir -p %{buildroot}/%{_datadir}/%name
@@ -85,6 +87,8 @@ fi
 %attr(644, root, root) %{_sysconfdir}/%name/issuers
 %attr(744, teapot, teapot) %{_sysconfdir}/%name/logback.xml
 %attr(744, teapot, teapot) %{_sysconfdir}/%name/logback-access.xml
+%attr(744, teapot, teapot) %{_sysconfdir}/%name/config.ini
+%attr(744, teapot, teapot) %{_sysconfdir}/%name/user_config.ini
 %attr(644, root, root) %{_datadir}/%name/storage_authorizations
 %attr(644, root, root) %{_datadir}/%name/storage_element.properties
 %attr(755, teapot, teapot) %{_localstatedir}/log/%name/
@@ -101,5 +105,5 @@ fi
 %attr(755, root, root) %{_exec_prefix}/local/lib/python3.9/site-packages/*
 
 %changelog
-* Fri Sep 04 2024 Dijana Vrbanec <[email protected]>
+* Thu Sep 19 2024 Dijana Vrbanec <[email protected]>
 - %{version}
diff --git a/rpm/teapot-fedora.spec b/rpm/teapot-fedora.spec
@@ -43,6 +43,8 @@ mkdir -p %{buildroot}/%{_sysconfdir}/%name
 cp %{_builddir}/%name-%version/templates/issuers %{buildroot}/%{_sysconfdir}/%name/
 cp %{_builddir}/%name-%version/templates/logback.xml %{buildroot}/%{_sysconfdir}/%name/
 cp %{_builddir}/%name-%version/templates/logback-access.xml %{buildroot}/%{_sysconfdir}/%name/
+cp %{_builddir}/%name-%version/config.ini %{buildroot}/%{_sysconfdir}/%name/
+cp %{_builddir}/%name-%version/user_config.ini %{buildroot}/%{_sysconfdir}/%name/
 mkdir -p %{buildroot}/%{_sysconfdir}/sudoers.d/
 cp %{_builddir}/%name-%version/templates/teapot %{buildroot}/%{_sysconfdir}/sudoers.d/
 mkdir -p %{buildroot}/%{_datadir}/%name
@@ -80,11 +82,13 @@ fi
 
 %files
 %attr(644, root, root) %{_datadir}/java/storm-webdav/storm-webdav-server.jar
-%attr(774, teapot, teapot) %{_datadir}/%name/teapot.py
+%attr(744, teapot, teapot) %{_datadir}/%name/teapot.py
 %attr(774, root, root) %{_datadir}/%name/self-signed-cert-gen.sh
 %attr(644, root, root) %{_sysconfdir}/%name/issuers
 %attr(744, teapot, teapot) %{_sysconfdir}/%name/logback.xml
 %attr(744, teapot, teapot) %{_sysconfdir}/%name/logback-access.xml
+%attr(744, teapot, teapot) %{_sysconfdir}/%name/config.ini
+%attr(744, teapot, teapot) %{_sysconfdir}/%name/user_config.ini
 %attr(644, root, root) %{_datadir}/%name/storage_authorizations
 %attr(644, root, root) %{_datadir}/%name/storage_element.properties
 %attr(755, teapot, teapot) %{_localstatedir}/log/%name/
@@ -101,5 +105,5 @@ fi
 %attr(755, root, root) %{_exec_prefix}/local/lib/python3.12/site-packages/*
 
 %changelog
-* Wed Sep 04 2024 Dijana Vrbanec <[email protected]>
+* Fri Sep 20 2024 Dijana Vrbanec <[email protected]>
 - %{version}
diff --git a/rpm/teapot-rocky.spec b/rpm/teapot-rocky.spec
@@ -43,6 +43,8 @@ mkdir -p %{buildroot}/%{_sysconfdir}/%name
 cp %{_builddir}/%name-%version/templates/issuers %{buildroot}/%{_sysconfdir}/%name/
 cp %{_builddir}/%name-%version/templates/logback.xml %{buildroot}/%{_sysconfdir}/%name/
 cp %{_builddir}/%name-%version/templates/logback-access.xml %{buildroot}/%{_sysconfdir}/%name/
+cp %{_builddir}/%name-%version/config.ini %{buildroot}/%{_sysconfdir}/%name/
+cp %{_builddir}/%name-%version/user_config.ini %{buildroot}/%{_sysconfdir}/%name/
 mkdir -p %{buildroot}/%{_sysconfdir}/sudoers.d/
 cp %{_builddir}/%name-%version/templates/teapot %{buildroot}/%{_sysconfdir}/sudoers.d/
 mkdir -p %{buildroot}/%{_datadir}/%name
@@ -85,6 +87,8 @@ fi
 %attr(644, root, root) %{_sysconfdir}/%name/issuers
 %attr(744, teapot, teapot) %{_sysconfdir}/%name/logback.xml
 %attr(744, teapot, teapot) %{_sysconfdir}/%name/logback-access.xml
+%attr(744, teapot, teapot) %{_sysconfdir}/%name/config.ini
+%attr(744, teapot, teapot) %{_sysconfdir}/%name/user_config.ini
 %attr(644, root, root) %{_datadir}/%name/storage_authorizations
 %attr(644, root, root) %{_datadir}/%name/storage_element.properties
 %attr(755, teapot, teapot) %{_localstatedir}/log/%name/
@@ -101,5 +105,5 @@ fi
 %attr(755, root, root) %{_exec_prefix}/local/lib/python3.9/site-packages/*
 
 %changelog
-* Fri Sep 04 2024 Dijana Vrbanec <[email protected]>
+* Thu Sep 19 2024 Dijana Vrbanec <[email protected]>
 - %{version}
diff --git a/rpm/teapot-rpm-alma.sh b/rpm/teapot-rpm-alma.sh
@@ -14,7 +14,8 @@ rm -r teapot-"$version_rpm"
 mv teapot-"$version_rpm".tar.gz "$HOME"/rpmbuild/SOURCES/
 
 #getting the storm-webdav.jar file
-curl -O https://repo.cloud.cnaf.infn.it/repository/storm-rpm-stable/centos7/storm-webdav-1.4.2-1.el7.noarch.rpm
+#curl -O https://repo.cloud.cnaf.infn.it/repository/storm-rpm-stable/centos7/storm-webdav-1.4.2-1.el7.noarch.rpm
+curl -O https://syncandshare.desy.de/index.php/s/GwSKbqF8DQZ4KzG/download/storm-webdav-1.4.2-1.el7.noarch.rpm
 rpm2cpio storm-webdav-1.4.2-1.el7.noarch.rpm | cpio -id ./usr/share/java/storm-webdav/storm-webdav-server.jar
 rm storm-webdav-1.4.2-1.el7.noarch.rpm
 mv usr/share/java/storm-webdav/storm-webdav-server.jar ./

diff --git a/rpm/teapot-rpm-fedora.sh b/rpm/teapot-rpm-fedora.sh
@@ -14,7 +14,8 @@ rm -r teapot-"$version_rpm"
 mv teapot-"$version_rpm".tar.gz "$HOME"/rpmbuild/SOURCES/
 
 #getting the storm-webdav.jar file
-curl -O https://repo.cloud.cnaf.infn.it/repository/storm-rpm-stable/centos7/storm-webdav-1.4.2-1.el7.noarch.rpm
+#curl -O https://repo.cloud.cnaf.infn.it/repository/storm-rpm-stable/centos7/storm-webdav-1.4.2-1.el7.noarch.rpm
+curl -O https://syncandshare.desy.de/index.php/s/GwSKbqF8DQZ4KzG/download/storm-webdav-1.4.2-1.el7.noarch.rpm
 rpm2cpio storm-webdav-1.4.2-1.el7.noarch.rpm | cpio -id ./usr/share/java/storm-webdav/storm-webdav-server.jar
 rm storm-webdav-1.4.2-1.el7.noarch.rpm
 mv usr/share/java/storm-webdav/storm-webdav-server.jar ./

diff --git a/rpm/teapot-rpm-rocky.sh b/rpm/teapot-rpm-rocky.sh
@@ -14,7 +14,8 @@ rm -r teapot-"$version_rpm"
 mv teapot-"$version_rpm".tar.gz "$HOME"/rpmbuild/SOURCES/
 
 #getting the storm-webdav.jar file
-curl -O https://repo.cloud.cnaf.infn.it/repository/storm-rpm-stable/centos7/storm-webdav-1.4.2-1.el7.noarch.rpm
+#curl -O https://repo.cloud.cnaf.infn.it/repository/storm-rpm-stable/centos7/storm-webdav-1.4.2-1.el7.noarch.rpm
+curl -O https://syncandshare.desy.de/index.php/s/GwSKbqF8DQZ4KzG/download/storm-webdav-1.4.2-1.el7.noarch.rpm
 rpm2cpio storm-webdav-1.4.2-1.el7.noarch.rpm | cpio -id ./usr/share/java/storm-webdav/storm-webdav-server.jar
 rm storm-webdav-1.4.2-1.el7.noarch.rpm
 mv usr/share/java/storm-webdav/storm-webdav-server.jar ./
-Original file line number
+Diff line change
@@ Expand Up / @@ -7,4 +7,6 @@ uvicorn @@
     anyio
     asyncio
     psutil
+    configparser
+    dotenv