From 778b700d8057d28f53fc950e6535b07540216109 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 13 Dec 2024 15:59:01 -0600 Subject: [PATCH 01/11] WIP: Install OSDF Cache via Container doc (SOFTWARE-6013) --- docs/data/osdf/install-cache-container.md | 267 ++++++++++++++++++++++ 1 file changed, 267 insertions(+) create mode 100644 docs/data/osdf/install-cache-container.md diff --git a/docs/data/osdf/install-cache-container.md b/docs/data/osdf/install-cache-container.md new file mode 100644 index 000000000..00486d279 --- /dev/null +++ b/docs/data/osdf/install-cache-container.md @@ -0,0 +1,267 @@ +title: Installing the OSDF Cache by Container + +Installing the OSDF Cache by Container +====================================== + +This document describes how to run an Open Science Data Federation (OSDF) Cache service via containers. +This service allows a site or regional network to cache data frequently used in Open Science Pool jobs, +reducing data transfer over the wide-area network and increasing throughput to jobs. + +Before Starting +--------------- + +Before starting the installation process, consider the following requirements: + +* __Docker:__ For the purpose of this guide, the host must have a running docker service + and you must have the ability to start containers (i.e., belong to the `docker` Unix group). +* __File Systems:__ The cache should have a partition of its own for storing data and metadata. +* __Network ports:__ The cache service requires the following open ports: + * Inbound TCP port 8443 for authenticated file access via the HTTP(S) and XRoot protocols. + * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; + if enabled, access to this port should be restricted to the LAN. + * Outbound UDP port 9930 for reporting to `xrd-report.osgstorage.org` and `xrd-mon.osgstorage.org` for monitoring +* __Host certificate:__ Required for authentication. See note below. +* __Service requirements:__ + * A cache serving the OSDF federation as a regional cache should have at least: + * 8 cores + * 40 Gbps connectivity + * 50-200 TB of NVMe disk for the cache partition; you may distribute the disk, e.g., by using an NVMe-backed Ceph pool, + if you cannot fit that much disk into a single chassis + * 24 GB of RAM + * A cache being used to serve data from the OSDF to a single site should have at least: + * 8 cores + * 40 Gbps connectivity + * 2 TB of NVMe disk for the cache partition + * 24 GB of RAM + + +!!! note "Host certificates" + Caches are accessed by users through browsers, meaning caches need a certificate from a CA acceptable to a standard browser. + Examples include [Let's Encrypt](../../security/host-certs/lets-encrypt.md) or the InCommon RSA CA. + Caches without a valid certificate for the browser cannot be added to the OSDF. + Note that, unlike legacy grid software, the public certificate file will need to contain the "full chain", including any + intermediate CAs (if you're unsure about your setup, try accessing your cache from your browser). + + The certificate chain and key should be mounted into the following locations: + + * **Host Certificate Chain**: `/etc/pelican/certificates/tls.crt` + * **Host Key**: `/etc/pelican/certificates/tls.key` + + +!!! note "osdf-cache 7.12" + This document covers versions 7.12 and later of the `osdf-cache` container image. + + +Configuring the Cache Server +---------------------------- + +In addition to the required configuration above (ports and file systems), +you may also configure the behavior of your cache with the following variables using an environment variable file: + +Where the environment file on the docker host, `/opt/osdf-cache/.env`, has (at least) the following contents, +replacing `` with the name of your resource as +[registered in Topology](install-cache.md#registering-the-cache) +and `` with the public DNS name that should be used to contact your cache: + +```file +PELICAN_XROOTD_SITENAME= +PELICAN_CACHE_PORT=8443 +``` + + + + + + +Migrating from XCache-based OSDF cache (OSG 23 or earlier) +---------------------------------------------------------- + + + + + +### Optional configuration ### + +Further behavior of the cache can be configured by setting the following in the environment variable file: + +- `XC_SPACE_HIGH_WM`, `XC_SPACE_LOW_WM`: High-water and low-water marks for disk usage, + as numbers between 0.00 (0%) and 1.00 (100%); + when usage goes above the high-water mark, the cache will delete files until it hits the low-water mark. +- `XC_RAMSIZE`: Amount of memory to use for storing blocks before writting them to disk. (Use higher for slower disks). +- `XC_BLOCKSIZE`: Size of the blocks in the cache. +- `XC_PREFETCH`: Number of blocks to prefetch from a file at once. + This controls how aggressive the cache is to request portions of a file. + + +Preparing for Initial Startup +----------------------------- + +1. The cache identifies itself to the federation via public key authentication; +before starting the cache for the first time, it is recommended to generate a keypair. +Download the Pelican client from and run: + + :::console + user@host$ pelican generate keygen + + + The newly created files, `issuer.jwk` and `issuer-pub.jwks` are the private and public keys, respectively. + +1. **Save these files**; if you lose the `issuer.jwk`, your cache will need to be re-approved. + + +Running a Cache +--------------- + +Cache containers may be run with either multiple mounted host partitions (recommended) or a single host +partition. + +It is recommended to use a container orchestration service such as [docker-compose](https://docs.docker.com/compose/) +or [kubernetes](https://kubernetes.io/) whose details are beyond the scope of this document. +The following sections provide examples for starting cache containers from the command-line as well as a more +production-appropriate method using systemd. + +### Multiple host partitions (recommended) ### + +For improved performance and storage, +especially if your cache is serving over 10 TB of data, +we recommend multiple partitions for handling namespaces (HDD, SSD, or NVMe), data (HDDs), and metadata (SSDs or NVMe). + +!!! note + Under this configuration the `` is not used to store the files. + Instead, the partition stores symlinks to the files in the metadata and data partitions. + +```console +user@host $ docker run --rm \ + --publish :8443 \ + --publish :8444 \ + --volume :/etc/pelican/issuer.jwk \ + --volume :/etc/pelican/certificates/tls.crt \ + --volume :/etc/pelican/certificates/tls.key \ + --volume :/run/pelican/cache/namespace \ + --volume :/run/pelican/cache/meta \ + --volume :/run/pelican/cache/data \ + --name osdf-cache \ + --env-file=/opt/osdf-cache/.env \ + hub.opensciencegrid.org/pelican_platform/osdf-cache:latest +``` + +### Single host partition ### + +For a simpler installation, you may use a single host partition mounted to `/run/pelican/cache/`: + +```console +user@host $ docker run --rm \ + --publish :8443 \ + --publish :8443 \ + --volume :/run/pelican/cache \ + --volume :/etc/pelican/certificates/tls.crt \ + --volume :/etc/pelican/certificates/tls.key \ + --name osdf-cache \ + --env-file=/opt/xcache/.env \ + hub.opensciencegrid.org/pelican_platform/osdf-cache:latest +``` + +### Running a cache on container with systemd (TODO) + +An example systemd service file for the OSDF cache. +This will require creating the environment file in the directory `/opt/xcache/.env`. + +!!! note + This example systemd file assumes `` is `8000`, `` is `8443`, + `` is `/srv/cache`, and the cert and key to use are in `/etc/ssl/host.crt` and `/etc/ssl/host.key`, + respectively. + +Create the systemd service file `/etc/systemd/system/docker.stash-cache.service` as follows: + +```file +[Unit] +Description=Cache Container +After=docker.service +Requires=docker.service + +[Service] +TimeoutStartSec=0 +Restart=always +ExecStartPre=-/usr/bin/docker stop %n +ExecStartPre=-/usr/bin/docker rm %n +ExecStartPre=/usr/bin/docker pull opensciencegrid/stash-cache:23-release +ExecStart=/usr/bin/docker run --rm --name %n \ + --publish 8000:8000 \ + --publish 8443:8443 \ + --volume /srv/cache:/xcache \ + --volume /etc/ssl/host.crt:/etc/grid-security/hostcert.pem \ + --volume /etc/ssl/host.key:/etc/grid-security/hostkey.pem \ + --env-file /opt/xcache/.env \ + opensciencegrid/stash-cache:23-release + +[Install] +WantedBy=multi-user.target +``` + +Enable and start the service with: + +```console +root@host $ systemctl enable docker.stash-cache +root@host $ systemctl start docker.stash-cache +``` + +!!! warning + You must [register](install-cache.md#registering-the-cache) the cache before starting it up. + + + +### Network optimization ### + +For caches that are connected to NICs over 40 Gbps we recommend that you disable the virtualized network and "bind" the +container to the host network: + +```console +user@host $ docker run --rm \ + --network="host" \ + --volume :/cache \ + --volume :/etc/grid-security/hostcert.pem \ + --volume :/etc/grid-security/hostkey.pem \ + --name osdf-cache \ + --env-file=/opt/xcache/.env \ + hub.opensciencegrid.org/pelican_platform/osdf-cache:latest +``` + +### Memory optimization ### + +The cache uses the host's memory for two purposes: + +1. Caching files recently read from disk (via the kernel page cache). +1. Buffering files recently received from the network before writing them to disk (to compensate for slow disks). + +An easy way to increase the performance of the cache is to assign it more memory. +If you set a limit on the container's memory usage via the docker option `--memory` or Kubernetes resource limits, +make sure it is at least twice the value of `XC_RAMSIZE`. + +Validating the Cache +--------------------- + +You can use the Pelican client (from ) to verify that the cache is functional. + +Download a test file from the OSDF through your cache (replacing `CACHE_HOSTNAME` with the host name of your cache) + + +```console +user@host $ pelican object get -c CACHE_HOSTNAME:8443 osdf://ospool/uc-shared/public/OSG-Staff/validation/test.txt /tmp/test.txt +user@host $ cat /tmp/test.txt + +Hello, World! +``` +If the download fails, rerun the above `pelican object get` command with the `-d` flag added. +Additional debugging information is located in the logs of your cache container. + +```console +root@host $ docker logs osdf-cache +``` + +See [this page](../../common/help.md) for requesting assistance; please include the logs and the `pelican object get -d` output in your request. + + +Getting Help +------------ + +To get assistance, please use the [this page](../../common/help.md). From 71b52588b4b0f38c60922c8da9b97863ee4afdeb Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 13 Dec 2024 17:58:08 -0600 Subject: [PATCH 02/11] Fill out most of the OSDF cache installation via container doc (SOFTWARE-6013) --- docs/data/osdf/install-cache-container.md | 338 +++++++++++++--------- 1 file changed, 209 insertions(+), 129 deletions(-) diff --git a/docs/data/osdf/install-cache-container.md b/docs/data/osdf/install-cache-container.md index 00486d279..877077499 100644 --- a/docs/data/osdf/install-cache-container.md +++ b/docs/data/osdf/install-cache-container.md @@ -12,15 +12,16 @@ Before Starting Before starting the installation process, consider the following requirements: -* __Docker:__ For the purpose of this guide, the host must have a running docker service +* __Container runtime:__ The examples in this guide assume using Docker as the container runtime, + for which the host must have a running `docker` service, and you must have the ability to start containers (i.e., belong to the `docker` Unix group). -* __File Systems:__ The cache should have a partition of its own for storing data and metadata. +* __File systems:__ The cache should have a partition of its own for storing data and metadata. +* __Host certificate:__ Required for authentication. See note below. * __Network ports:__ The cache service requires the following open ports: * Inbound TCP port 8443 for authenticated file access via the HTTP(S) and XRoot protocols. * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; if enabled, access to this port should be restricted to the LAN. * Outbound UDP port 9930 for reporting to `xrd-report.osgstorage.org` and `xrd-mon.osgstorage.org` for monitoring -* __Host certificate:__ Required for authentication. See note below. * __Service requirements:__ * A cache serving the OSDF federation as a regional cache should have at least: * 8 cores @@ -51,46 +52,26 @@ Before starting the installation process, consider the following requirements: !!! note "osdf-cache 7.12" This document covers versions 7.12 and later of the `osdf-cache` container image. +!!! note "root" + The paths used as examples on this page (e.g., `/etc/pelican`) require root to edit; + if you do not have root on the host, modify the directories to a path you do have access to. + Configuring the Cache Server ---------------------------- -In addition to the required configuration above (ports and file systems), -you may also configure the behavior of your cache with the following variables using an environment variable file: - -Where the environment file on the docker host, `/opt/osdf-cache/.env`, has (at least) the following contents, -replacing `` with the name of your resource as -[registered in Topology](install-cache.md#registering-the-cache) -and `` with the public DNS name that should be used to contact your cache: +The preferred method of configuring a Pelican-based OSDF cache is via a YAML file that will be +mounted into the container into the `/etc/pelican/config.d` directory. +This document will use `/etc/pelican/cache-config.yaml` as the name of the configuration file outside of the container. +Create `/etc/pelican/cache-config.yaml` with the following contents: ```file -PELICAN_XROOTD_SITENAME= -PELICAN_CACHE_PORT=8443 +Cache: + Port: 8443 +# XRootD: +# Sitename: ``` - - - - - - -Migrating from XCache-based OSDF cache (OSG 23 or earlier) ----------------------------------------------------------- - - - - - -### Optional configuration ### - -Further behavior of the cache can be configured by setting the following in the environment variable file: - -- `XC_SPACE_HIGH_WM`, `XC_SPACE_LOW_WM`: High-water and low-water marks for disk usage, - as numbers between 0.00 (0%) and 1.00 (100%); - when usage goes above the high-water mark, the cache will delete files until it hits the low-water mark. -- `XC_RAMSIZE`: Amount of memory to use for storing blocks before writting them to disk. (Use higher for slower disks). -- `XC_BLOCKSIZE`: Size of the blocks in the cache. -- `XC_PREFETCH`: Number of blocks to prefetch from a file at once. - This controls how aggressive the cache is to request portions of a file. +You will uncomment and fill in the value for `XRootD.Sitename` in a later step, after registration is complete. Preparing for Initial Startup @@ -101,7 +82,8 @@ before starting the cache for the first time, it is recommended to generate a ke Download the Pelican client from and run: :::console - user@host$ pelican generate keygen + root@host$ cd /etc/pelican + root@host$ pelican generate keygen The newly created files, `issuer.jwk` and `issuer-pub.jwks` are the private and public keys, respectively. @@ -112,87 +94,220 @@ Download the Pelican client from ` is not used to store the files. - Instead, the partition stores symlinks to the files in the metadata and data partitions. ```console user@host $ docker run --rm \ - --publish :8443 \ - --publish :8444 \ - --volume :/etc/pelican/issuer.jwk \ - --volume :/etc/pelican/certificates/tls.crt \ - --volume :/etc/pelican/certificates/tls.key \ - --volume :/run/pelican/cache/namespace \ - --volume :/run/pelican/cache/meta \ - --volume :/run/pelican/cache/data \ + --publish :8443 \ + --publish :8444 \ + --volume :/etc/pelican/issuer.jwk \ + --volume :/etc/pelican/certificates/tls.crt \ + --volume :/etc/pelican/certificates/tls.key \ + --volume :/etc/pelican/config.d/99-local.yaml \ + --volume :/run/pelican/cache \ --name osdf-cache \ - --env-file=/opt/osdf-cache/.env \ hub.opensciencegrid.org/pelican_platform/osdf-cache:latest ``` -### Single host partition ### +Using the example values from the table, this is -For a simpler installation, you may use a single host partition mounted to `/run/pelican/cache/`: +```console +user@host $ docker run --rm \ + --publish 8443:8443 \ + --publish 8444:8444 \ + --volume /etc/pelican/issuer.jwk:/etc/pelican/issuer.jwk \ + --volume /etc/pelican/hostcert.pem:/etc/pelican/certificates/tls.crt \ + --volume /etc/pelican/hostkey.pem:/etc/pelican/certificates/tls.key \ + --volume /etc/pelican/cache-config.yaml:/etc/pelican/config.d/99-local.yaml \ + --volume /mnt/cache:/run/pelican/cache \ + --name osdf-cache \ + hub.opensciencegrid.org/pelican_platform/osdf-cache:latest +``` + + +### Multiple host partitions + +A cache may be configured to store namespace information, data, and metadata on separate partitions. +This improves performance and is recommended for caches with more than 10 TB of capacity. + + +```console +user@host $ docker run --rm \ + --publish :8443 \ + --publish :8444 \ + --volume :/etc/pelican/issuer.jwk \ + --volume :/etc/pelican/certificates/tls.crt \ + --volume :/etc/pelican/certificates/tls.key \ + --volume :/etc/pelican/config.d/99-local.yaml \ + --volume :/run/pelican/cache/namespace \ + --volume :/run/pelican/cache/meta \ + --volume :/run/pelican/cache/data \ + --name osdf-cache \ + hub.opensciencegrid.org/pelican_platform/osdf-cache:latest +``` + +Using the example values from the table, this is ```console user@host $ docker run --rm \ - --publish :8443 \ - --publish :8443 \ - --volume :/run/pelican/cache \ - --volume :/etc/pelican/certificates/tls.crt \ - --volume :/etc/pelican/certificates/tls.key \ + --publish 8443:8443 \ + --publish 8444:8444 \ + --volume /etc/pelican/issuer.jwk:/etc/pelican/issuer.jwk \ + --volume /etc/pelican/hostcert.pem:/etc/pelican/certificates/tls.crt \ + --volume /etc/pelican/hostkey.pem:/etc/pelican/certificates/tls.key \ + --volume /etc/pelican/cache-config.yaml:/etc/pelican/config.d/99-local.yaml \ + --volume /mnt/cache-namespace:/run/pelican/cache/namespace \ + --volume /mnt/cache-meta:/run/pelican/cache/meta \ + --volume /mnt/cache-data:/run/pelican/cache/data \ --name osdf-cache \ - --env-file=/opt/xcache/.env \ hub.opensciencegrid.org/pelican_platform/osdf-cache:latest ``` -### Running a cache on container with systemd (TODO) -An example systemd service file for the OSDF cache. -This will require creating the environment file in the directory `/opt/xcache/.env`. +Validating the Cache +--------------------- + +You can use the Pelican client (from ) to verify that the cache is functional. + +Download a test file from the OSDF through your cache (replacing `CACHE_HOSTNAME` with the host name of your cache) + + +```console +user@host $ pelican object get -c CACHE_HOSTNAME:8443 osdf://ospool/uc-shared/public/OSG-Staff/validation/test.txt /tmp/test.txt +user@host $ cat /tmp/test.txt + +Hello, World! +``` + +If the download fails, rerun the above `pelican object get` command with the `-d` flag added. +Additional debugging information is located in the logs of your cache container. + +```console +user@host $ docker logs osdf-cache +``` + +To increase the debugging information in the cache, edit your cache configuration file and set: +``` +Debug: true +``` + +See [this page](../../common/help.md) for requesting assistance; please include the logs and the `pelican object get -d` output in your request. + + +Joining the Cache to the Federation +----------------------------------- + +The cache must be registered with the OSG prior to joining the data federation. +Send mail to requesting registration; provide the following information: + +* Cache hostname +* Administrative and security contact(s) +* Institution that the cache belongs to + +OSG Staff will register the cache and respond with the Resource Name that the cache was registered as. +Edit your cache configuration file (`/etc/pelican/cache-config.yaml` in the example above): +uncomment `XRootD.Sitename` and change the value to the Resource Name that OSG Staff responded with. +Then, restart your cache. + + + + + + + +Advanced topics +--------------- -!!! note - This example systemd file assumes `` is `8000`, `` is `8443`, - `` is `/srv/cache`, and the cert and key to use are in `/etc/ssl/host.crt` and `/etc/ssl/host.key`, - respectively. +### Persisting logs -Create the systemd service file `/etc/systemd/system/docker.stash-cache.service` as follows: +By default, the cache sends logs to the console, where it is captured by the container runtime's logging mechanism. +To save logs to a file instead, set the log location in your cache config file (e.g., `/etc/pelican/cache-config.yaml`) +as follows: + +```file +Logging: + LogLocation: +``` + +Note: the osdf-cache image does not come with log rotation; if you save logs to a file, you must +set up logration yourself to avoid running the cache out of disk space. + + + + +### Managing a cache with systemd + +If you do not have a container orchestration service but still want to manage a container-based cache, +you may run the container via a systemd service. +The following example uses the 'single host partition' setup from [above](#single-host-partition). + +Create the systemd service file `/etc/systemd/system/docker-osdf-cache.service` as follows: ```file [Unit] -Description=Cache Container +Description=OSDF Cache Container After=docker.service Requires=docker.service [Service] TimeoutStartSec=0 Restart=always -ExecStartPre=-/usr/bin/docker stop %n -ExecStartPre=-/usr/bin/docker rm %n +ExecStartPre=-/usr/bin/docker stop osdf-cache +ExecStartPre=-/usr/bin/docker rm osdf-cache ExecStartPre=/usr/bin/docker pull opensciencegrid/stash-cache:23-release -ExecStart=/usr/bin/docker run --rm --name %n \ - --publish 8000:8000 \ +ExecStart=/usr/bin/docker run --rm --name osdf-cache \ --publish 8443:8443 \ - --volume /srv/cache:/xcache \ - --volume /etc/ssl/host.crt:/etc/grid-security/hostcert.pem \ - --volume /etc/ssl/host.key:/etc/grid-security/hostkey.pem \ - --env-file /opt/xcache/.env \ - opensciencegrid/stash-cache:23-release + --publish 8444:8444 \ + --volume /etc/pelican/issuer.jwk:/etc/pelican/issuer.jwk \ + --volume /etc/pelican/hostcert.pem:/etc/pelican/certificates/tls.crt \ + --volume /etc/pelican/hostkey.pem:/etc/pelican/certificates/tls.key \ + --volume /etc/pelican/cache-config.yaml:/etc/pelican/config.d/99-local.yaml \ + --volume /mnt/cache:/run/pelican/cache \ + --name osdf-cache \ + hub.opensciencegrid.org/pelican_platform/osdf-cache:latest [Install] WantedBy=multi-user.target @@ -201,64 +316,29 @@ WantedBy=multi-user.target Enable and start the service with: ```console -root@host $ systemctl enable docker.stash-cache -root@host $ systemctl start docker.stash-cache +root@host $ systemctl enable docker-osdf-cache +root@host $ systemctl start docker-osdf-cache ``` -!!! warning - You must [register](install-cache.md#registering-the-cache) the cache before starting it up. - - ### Network optimization ### For caches that are connected to NICs over 40 Gbps we recommend that you disable the virtualized network and "bind" the -container to the host network: +container to the host network. +The following example uses the 'single host partition' setup from [above](#single-host-partition): ```console user@host $ docker run --rm \ --network="host" \ - --volume :/cache \ - --volume :/etc/grid-security/hostcert.pem \ - --volume :/etc/grid-security/hostkey.pem \ + --volume /etc/pelican/issuer.jwk:/etc/pelican/issuer.jwk \ + --volume /etc/pelican/hostcert.pem:/etc/pelican/certificates/tls.crt \ + --volume /etc/pelican/hostkey.pem:/etc/pelican/certificates/tls.key \ + --volume /etc/pelican/cache-config.yaml:/etc/pelican/config.d/99-local.yaml \ + --volume /mnt/cache:/run/pelican/cache \ --name osdf-cache \ - --env-file=/opt/xcache/.env \ hub.opensciencegrid.org/pelican_platform/osdf-cache:latest ``` -### Memory optimization ### - -The cache uses the host's memory for two purposes: - -1. Caching files recently read from disk (via the kernel page cache). -1. Buffering files recently received from the network before writing them to disk (to compensate for slow disks). - -An easy way to increase the performance of the cache is to assign it more memory. -If you set a limit on the container's memory usage via the docker option `--memory` or Kubernetes resource limits, -make sure it is at least twice the value of `XC_RAMSIZE`. - -Validating the Cache ---------------------- - -You can use the Pelican client (from ) to verify that the cache is functional. - -Download a test file from the OSDF through your cache (replacing `CACHE_HOSTNAME` with the host name of your cache) - - -```console -user@host $ pelican object get -c CACHE_HOSTNAME:8443 osdf://ospool/uc-shared/public/OSG-Staff/validation/test.txt /tmp/test.txt -user@host $ cat /tmp/test.txt - -Hello, World! -``` -If the download fails, rerun the above `pelican object get` command with the `-d` flag added. -Additional debugging information is located in the logs of your cache container. - -```console -root@host $ docker logs osdf-cache -``` - -See [this page](../../common/help.md) for requesting assistance; please include the logs and the `pelican object get -d` output in your request. Getting Help From b6fec957fff9adbe2ed4122869c499049f9c5b69 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 13 Dec 2024 18:06:42 -0600 Subject: [PATCH 03/11] Add OSDF-cache-via-container doc to TOC (SOFTWARE-6013) --- mkdocs.yml | 1 + 1 file changed, 1 insertion(+) diff --git a/mkdocs.yml b/mkdocs.yml index fce96b24d..062b98bcf 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -69,6 +69,7 @@ nav: - 'Data Federation (OSDF)': - 'Overview': 'data/stashcache/overview.md' - Data Cache: + - 'Install from container': 'data/osdf/install-cache-container.md' - 'Install from RPM': 'data/osdf/install-cache-rpm.md' - Data Origin: - 'Install from RPM': 'data/osdf/install-origin-rpm.md' From 2593a3c7fb20f28af10771c1c939e7b0643e71d2 Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 13 Dec 2024 18:17:00 -0600 Subject: [PATCH 04/11] whack-a-stash --- docs/data/osdf/install-cache-container.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/data/osdf/install-cache-container.md b/docs/data/osdf/install-cache-container.md index 877077499..67cba6865 100644 --- a/docs/data/osdf/install-cache-container.md +++ b/docs/data/osdf/install-cache-container.md @@ -297,7 +297,7 @@ TimeoutStartSec=0 Restart=always ExecStartPre=-/usr/bin/docker stop osdf-cache ExecStartPre=-/usr/bin/docker rm osdf-cache -ExecStartPre=/usr/bin/docker pull opensciencegrid/stash-cache:23-release +ExecStartPre=/usr/bin/docker pull hub.opensciencegrid.org/pelican_platform/osdf-cache:latest ExecStart=/usr/bin/docker run --rm --name osdf-cache \ --publish 8443:8443 \ --publish 8444:8444 \ From 7e4f5d16b5d49b0801fdd50e5a98e7d057d7d8f9 Mon Sep 17 00:00:00 2001 From: Brian Lin Date: Fri, 20 Dec 2024 12:37:00 -0600 Subject: [PATCH 05/11] Fix list indentation --- docs/data/osdf/install-cache-container.md | 30 +++++++++++------------ 1 file changed, 15 insertions(+), 15 deletions(-) diff --git a/docs/data/osdf/install-cache-container.md b/docs/data/osdf/install-cache-container.md index 67cba6865..9684e7334 100644 --- a/docs/data/osdf/install-cache-container.md +++ b/docs/data/osdf/install-cache-container.md @@ -18,22 +18,22 @@ Before starting the installation process, consider the following requirements: * __File systems:__ The cache should have a partition of its own for storing data and metadata. * __Host certificate:__ Required for authentication. See note below. * __Network ports:__ The cache service requires the following open ports: - * Inbound TCP port 8443 for authenticated file access via the HTTP(S) and XRoot protocols. - * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; - if enabled, access to this port should be restricted to the LAN. - * Outbound UDP port 9930 for reporting to `xrd-report.osgstorage.org` and `xrd-mon.osgstorage.org` for monitoring + * Inbound TCP port 8443 for authenticated file access via the HTTP(S) and XRoot protocols. + * (Optional) Inbound TCP port 8444 for access to the web interface for monitoring and configuration; + if enabled, access to this port should be restricted to the LAN. + * Outbound UDP port 9930 for reporting to `xrd-report.osgstorage.org` and `xrd-mon.osgstorage.org` for monitoring * __Service requirements:__ - * A cache serving the OSDF federation as a regional cache should have at least: - * 8 cores - * 40 Gbps connectivity - * 50-200 TB of NVMe disk for the cache partition; you may distribute the disk, e.g., by using an NVMe-backed Ceph pool, - if you cannot fit that much disk into a single chassis - * 24 GB of RAM - * A cache being used to serve data from the OSDF to a single site should have at least: - * 8 cores - * 40 Gbps connectivity - * 2 TB of NVMe disk for the cache partition - * 24 GB of RAM + * A cache serving the OSDF federation as a regional cache should have at least: + * 8 cores + * 40 Gbps connectivity + * 50-200 TB of NVMe disk for the cache partition; you may distribute the disk, e.g., by using an NVMe-backed Ceph pool, + if you cannot fit that much disk into a single chassis + * 24 GB of RAM + * A cache being used to serve data from the OSDF to a single site should have at least: + * 8 cores + * 40 Gbps connectivity + * 2 TB of NVMe disk for the cache partition + * 24 GB of RAM !!! note "Host certificates" From 9e6fb30c1af0ca3154307e0a66243d9f3e413daa Mon Sep 17 00:00:00 2001 From: Brian Lin Date: Fri, 20 Dec 2024 13:58:57 -0600 Subject: [PATCH 06/11] Miscellaneous nitpicks --- docs/data/osdf/install-cache-container.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/data/osdf/install-cache-container.md b/docs/data/osdf/install-cache-container.md index 9684e7334..e49500d1a 100644 --- a/docs/data/osdf/install-cache-container.md +++ b/docs/data/osdf/install-cache-container.md @@ -54,7 +54,7 @@ Before starting the installation process, consider the following requirements: !!! note "root" The paths used as examples on this page (e.g., `/etc/pelican`) require root to edit; - if you do not have root on the host, modify the directories to a path you do have access to. + if you do not have root on the host, choose new locations where you have modification permissions. Configuring the Cache Server @@ -323,9 +323,9 @@ root@host $ systemctl start docker-osdf-cache ### Network optimization ### -For caches that are connected to NICs over 40 Gbps we recommend that you disable the virtualized network and "bind" the +For caches that are connected to NICs over 40 Gbps we recommend that you disable the virtualized network and bind the container to the host network. -The following example uses the 'single host partition' setup from [above](#single-host-partition): +The following example uses the [single host partition](#single-host-partition) documented above. ```console user@host $ docker run --rm \ From 5bcf642e743d87f00afe8ee91da40f6d2a7d68bc Mon Sep 17 00:00:00 2001 From: Matyas Selmeci Date: Fri, 27 Dec 2024 16:56:08 -0600 Subject: [PATCH 07/11] danger! --- docs/data/osdf/install-cache-container.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docs/data/osdf/install-cache-container.md b/docs/data/osdf/install-cache-container.md index 67cba6865..bd1efb272 100644 --- a/docs/data/osdf/install-cache-container.md +++ b/docs/data/osdf/install-cache-container.md @@ -258,8 +258,9 @@ Logging: LogLocation: ``` -Note: the osdf-cache image does not come with log rotation; if you save logs to a file, you must -set up logration yourself to avoid running the cache out of disk space. +!!! danger + Note: the osdf-cache image does not come with log rotation; if you save logs to a file, you must + set up logration yourself to avoid running the cache out of disk space.