Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multiple SERVER Releases #1141

Merged
merged 4 commits into from
Mar 11, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -675,6 +675,40 @@ MinIO supports the following condition keys for all supported

For additional keys supported by a specific S3 action, see the reference documentation for that action.

MinIO Extended Condition Keys
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

MinIO extends the S3 standard condition keys with the following extended key:

``sts:DurationSeconds``

.. versionadded:: MinIO SERVER RELEASE.2024-02-06T21-36-22Z

Specify a time in seconds to limit the duration of *all* Security Token Service credentials generated by :ref:`minio-sts-assumerolewithwebidentity`.

This value overrides the ``DurationSeconds`` field specified to the client.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Point of clarity. If the client specifies a shorter duration, which duration wins? This one? Or the client one?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good question - @vadmeste , I'm assuming the shorter value always wins regardless of where it's set.

That is, if I set the duration field here to be higher, then setting Duration to be lower in the STS API endpoint should work.


For example:

.. code-block:: json

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRoleWithWebIdentity"
],
"Condition": {
"NumericLessThanEquals": {
"sts:DurationSeconds": "300"
}
}
}
]
}

.. _minio-policy-mc-admin-actions:

``mc admin`` Policy Action Keys
Expand Down
4 changes: 4 additions & 0 deletions source/operations/checklists/hardware.rst
Original file line number Diff line number Diff line change
Expand Up @@ -189,6 +189,10 @@ The following table provides general guidelines for allocating memory for use by
* - More than 1 Pebibyte (Pi)
- 128GiB

.. important::

Starting with :minio-release:`RELEASE.2024-01-28T22-35-53Z`, MinIO preallocates 2GiB of memory per node in distributed setups and 1GiB of memory for a single-node setup.

.. _minio-hardware-checklist-storage:

Storage
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ This procedure assumes a *total loss* of one or more peer sites versus replicati
All healthy peers in the site replication configuration update for the new peer automatically.
You can use the :mc-cmd:`mc admin replicate info` command to verify the new site replication configuration.

#. Resynchronize the new peer with :mc:`mc admin replicate resync`.
#. Resynchronize the new peer with :mc-cmd:`mc admin replicate resync`.

.. code-block:: shell
:class: copyable
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -104,6 +104,16 @@ Storage Requirements
:start-after: start-storage-requirements-desc
:end-before: end-storage-requirements-desc

Memory Requirements
~~~~~~~~~~~~~~~~~~~

.. versionchanged:: RELEASE.2024-01-28T22-35-53Z

MinIO pre-allocates 2GiB of system memory at startup.

MinIO recommends a *minimum* of 32GiB of memory per host.
See :ref:`minio-hardware-checklist-memory` for more guidance on memory allocation in MinIO.

Time Synchronization
~~~~~~~~~~~~~~~~~~~~

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,16 @@ Storage Requirements
:start-after: start-storage-requirements-desc
:end-before: end-storage-requirements-desc

Memory Requirements
~~~~~~~~~~~~~~~~~~~

.. versionchanged:: RELEASE.2024-01-28T22-35-53Z

MinIO pre-allocates 2GiB of system memory at startup.

MinIO recommends a *minimum* of 32GiB of memory per host.
See :ref:`minio-hardware-checklist-memory` for more guidance on memory allocation in MinIO.

.. _deploy-minio-standalone-multidrive:

Deploy Single-Node Multi-Drive MinIO
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,14 +11,8 @@ Deploy MinIO: Single-Node Single-Drive
:depth: 2

The procedures on this page cover deploying MinIO in a Single-Node Single-Drive (SNSD) configuration for early development and evaluation.
|SNSD| deployments provide no added reliability or availability beyond what the underlying storage volume implements (RAID, LVM, ZFS, etc.).

Starting with :minio-release:`RELEASE.2022-06-02T02-11-04Z`, MinIO implements a zero-parity erasure coded backend for single-node single-drive deployments.
This feature allows access to :ref:`erasure coding dependent features <minio-erasure-coding>` without the requirement of multiple drives.
This mode **requires** accessing stored objects through the S3 API, and does **not** support direct access to objects through the filesystem/POSIX interface.

MinIO only starts in |SNSD| mode if the storage volume or path is empty *or* only contain files generated by a previous |SNSD| deployment.
See the documentation on :ref:`SNSD behavior with pre-existing data <minio-snsd-pre-existing-data>` for more information.
|SNSD| deployments use a zero-parity erasure coded backend that provides no added reliability or availability beyond what the underlying storage volume implements.
These deployments are best suited for local testing and evaluation, or for small-scale data workloads that do not have availability or performance requirements.

.. cond:: container

Expand Down Expand Up @@ -67,6 +61,37 @@ The following table lists the possible storage volume states and MinIO behavior:

.. versionchanged:: RELEASE.2022-10-29T06-21-33Z

Prerequisites
-------------

Storage Requirements
~~~~~~~~~~~~~~~~~~~~

The following requirements summarize the :ref:`minio-hardware-checklist-storage` section of MinIO's hardware recommendations:

Use Local Storage
Direct-Attached Storage (DAS) has significant performance and consistency advantages over networked storage (:abbr:`NAS (Network Attached Storage)`, :abbr:`SAN (Storage Area Network)`, :abbr:`NFS (Network File Storage)`).
MinIO strongly recommends flash storage (NVMe, SSD) for primary or "hot" data.

Use XFS-Formatting for Drives
MinIO strongly recommends provisioning XFS formatted drives for storage.
MinIO uses XFS as part of internal testing and validation suites, providing additional confidence in performance and behavior at all scales.

Persist Drive Mounting and Mapping Across Reboots
Use ``/etc/fstab`` to ensure consistent drive-to-mount mapping across node reboots.

Non-Linux Operating Systems should use the equivalent drive mount management tool.

Memory Requirements
~~~~~~~~~~~~~~~~~~~

.. versionchanged:: RELEASE.2024-01-28T22-35-53Z

MinIO pre-allocates 2GiB of system memory at startup.

MinIO recommends a *minimum* of 32GiB of memory per host.
See :ref:`minio-hardware-checklist-memory` for more guidance on memory allocation in MinIO.

.. _deploy-minio-standalone:

Deploy Single-Node Single-Drive MinIO
Expand All @@ -83,9 +108,6 @@ The following procedure deploys MinIO consisting of a single MinIO server and a
MinIO cannot provide consistency guarantees if the underlying storage
volumes are NFS or a similar network-attached storage volume.

For deployments that *require* using network-attached storage, use
NFSv4 for best results.

.. cond:: linux

.. include:: /includes/linux/steps-deploy-minio-single-node-single-drive.rst
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -362,6 +362,7 @@ Settings marked with an asterisk :guilabel:`*` are *required*:
* - :guilabel:`Memory per Node [Gi]`
- Specify the total amount of memory (RAM) to allocate per MinIO server pod.
See :ref:`minio-hardware-checklist-memory` for guidance on setting this value.
MinIO **requires** a minimum of 2GiB of memory per worker.

The Kubernetes cluster *must* have worker nodes with sufficient free RAM to match the pod request.

Expand Down
20 changes: 20 additions & 0 deletions source/operations/monitoring/metrics-and-alerts.rst
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,20 @@ MinIO provides scraping endpoints for the following metric groups:
Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.

.. tab-item:: Resource Metrics

.. versionadded:: RELEASE.2023-10-07T15-07-38Z

You can scrape :ref:`resource metrics <minio-available-resource-metrics>` using the following URL endpoint:

.. code-block:: shell
:class: copyable

http://HOSTNAME:PORT/minio/v2/metrics/resource

Replace ``HOSTNAME:PORT`` with the :abbr:`FQDN (Fully Qualified Domain Name)` and port of the MinIO deployment.
For deployments with a load balancer managing connections between MinIO nodes, specify the address of the load balancer.


MinIO by default requires authentication for scraping the metrics endpoints.
Use the :mc-cmd:`mc admin prometheus generate` command to generate the necessary bearer tokens.
Expand Down Expand Up @@ -110,6 +124,7 @@ Each metric includes a label for the MinIO server which generated that metric.

- :ref:`Cluster Metrics <minio-available-cluster-metrics>`
- :ref:`Bucket Metrics <minio-available-bucket-metrics>`
- :ref:`Resource Metrics <minio-available-resource-metrics>`

.. _minio-available-cluster-metrics:

Expand All @@ -121,6 +136,11 @@ Each metric includes a label for the MinIO server which generated that metric.
.. include:: /includes/common-metrics-bucket.md
:parser: myst_parser.sphinx_

.. _minio-available-resource-metrics:

.. include:: /includes/common-metrics-resource.md
:parser: myst_parser.sphinx_

.. toctree::
:titlesonly:
:hidden:
Expand Down
77 changes: 46 additions & 31 deletions source/operations/monitoring/minio-logging.rst
Original file line number Diff line number Diff line change
Expand Up @@ -320,51 +320,66 @@ MinIO audit logs resemble the following JSON document:

{
"version": "1",
"deploymentid": "bc0e4d1e-bacc-42eb-91ad-2d7f3eacfa8d",
"time": "2019-08-12T21:34:37.187817748Z",
"deploymentid": "8ca2b7ad-20cf-4d07-9efb-28b2f519f4a5",
"time": "2024-02-29T19:39:25.744431903Z",
"event": "",
"trigger": "incoming",
"api": {
"name": "PutObject",
"bucket": "testbucket",
"object": "hosts",
"name": "CompleteMultipartUpload",
"bucket": "data",
"object": "test-data.csv",
"status": "OK",
"statusCode": 200,
"timeToFirstByte": "366333ns",
"timeToResponse": "16438202ns"
"rx": 267,
"tx": 358,
"txHeaders": 387,
"timeToFirstByte": "2096989ns",
"timeToFirstByteInNS": "2096989",
"timeToResponse": "2111986ns",
"timeToResponseInNS": "2111986"
},
"remotehost": "127.0.0.1",
"requestID": "15BA4A72C0C70AFC",
"userAgent": "MinIO (linux; amd64) minio-go/v6.0.32 mc/2019-08-12T18:27:13Z",
"requestID": "17B86CB0ED88EBE9",
"userAgent": "MinIO (linux; amd64) minio-go/v7.0.67 mc/RELEASE.2024-02-24T01-33-20Z",
"requestPath": "/data/test-data.csv",
"requestHost": "minio.example.net:9000",
"requestQuery": {
"uploadId": "OGNhMmI3YWQtMjBjZi00ZDA3LTllZmItMjhiMmY1MTlmNGE1LmU3MjNlNWI4LTNiYWYtNDYyNy1hNzI3LWMyNDE3NTVjMmMzNw"
},
"requestHeader": {
"Authorization": "AWS4-HMAC-SHA256 Credential=minio/20190812/us-east-1/s3/aws4_request,SignedHeaders=host;x-amz-content-sha256;x-amz-date;x-amz-decoded-content-length,Signature=d3f02a6aeddeb29b06e1773b6a8422112890981269f2463a26f307b60423177c",
"Content-Length": "686",
"Accept-Encoding": "zstd,gzip",
"Authorization": "AWS4-HMAC-SHA256 Credential=minioadmin/20240229/us-east-1/s3/aws4_request, SignedHeaders=content-type;host;x-amz-content-sha256;x-amz-date, Signature=ccb3acdc1763509a88a7e4a3d7fe431ef0ee5ca3f66ccb430d5a09326e87e893",
"Content-Length": "267",
"Content-Type": "application/octet-stream",
"User-Agent": "MinIO (linux; amd64) minio-go/v6.0.32 mc/2019-08-12T18:27:13Z",
"X-Amz-Content-Sha256": "STREAMING-AWS4-HMAC-SHA256-PAYLOAD",
"X-Amz-Date": "20190812T213437Z",
"X-Amz-Decoded-Content-Length": "512"
"User-Agent": "MinIO (linux; amd64) minio-go/v7.0.67 mc/RELEASE.2024-02-24T01-33-20Z",
"X-Amz-Content-Sha256": "d61969719ee94f43c4e87044229b7a13b54cab320131e9a77259ad0c9344f6d3",
"X-Amz-Date": "20240229T193925Z"
},
"responseHeader": {
"Accept-Ranges": "bytes",
"Content-Length": "0",
"Content-Security-Policy": "block-all-mixed-content",
"ETag": "a414c889dc276457bd7175f974332cb0-1",
"Server": "MinIO/DEVELOPMENT.2019-08-12T21-28-07Z",
"Vary": "Origin",
"X-Amz-Request-Id": "15BA4A72C0C70AFC",
"Content-Length": "358",
"Content-Type": "application/xml",
"ETag": "1d9fdc88af5e74f5eac0a3dd750ce58e-2",
"Server": "MinIO",
"Strict-Transport-Security": "max-age=31536000; includeSubDomains",
"Vary": "Origin,Accept-Encoding",
"X-Amz-Id-2": "dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8",
"X-Amz-Request-Id": "17B86CB0ED88EBE9",
"X-Content-Type-Options": "nosniff",
"X-Xss-Protection": "1; mode=block"
},
"tags": {
"objectErasureMap": {
"object": {
"objectLocation": {
"name": "Mousepad Template-v03final.jpg",
"poolId": 1,
"setId": 10,
"setId": 1,
"disks": [
"http://server01/mnt/pool1/disk01",
"http://server02/mnt/pool1/disk02",
"http://server03/mnt/pool1/disk03",
"http://server04/mnt/pool1/disk04"
"/mnt/drive-1",
"/mnt/drive-2",
"/mnt/drive-3",
"/mnt/drive-4"
]
}
}
}
}
},
"accessKey": "minioadmin"
}
12 changes: 12 additions & 0 deletions source/reference/minio-mc-admin/mc-admin-update.rst
Original file line number Diff line number Diff line change
Expand Up @@ -88,3 +88,15 @@ Syntax
:optional:

Pass this flag to confirm the update and bypass the confirmation prompt.

Behavior
--------

Binary Compression
~~~~~~~~~~~~~~~~~~

.. versionchanged:: RELEASE.2024-01-28T22-35-53Z

:mc-cmd:`mc admin update` compresses the binary before sending to all nodes in the deployment.

This feature does not apply to :ref:`systemctl managed deployments <minio-upgrade-systemctl>`.
9 changes: 8 additions & 1 deletion sync-minio-server-docs.sh
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,14 @@ set -x
function main() {
curl --retry 10 -Ls https://raw.githubusercontent.com/minio/minio/master/docs/metrics/prometheus/list.md | csplit - /"# Bucket Metrics"/
mv xx00 source/includes/common-metrics-cluster.md
mv xx01 source/includes/common-metrics-bucket.md

# Kludgy. Does csplit again on the Bucket Metrics file fragment
# Tried to get smart using `grep '^# [A-Za-z]` to get line numbers but got stuck

cat xx01 | csplit - /"# Resource Metrics"/

mv xx00 source/includes/common-metrics-bucket.md
mv xx01 source/includes/common-metrics-resource.md
}

main "$@"