Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds information about tiering rules and exclusive access #1220

Merged
merged 5 commits into from
May 22, 2024
Merged
Changes from 4 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions source/administration/object-management.rst
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,48 @@ Conversion to or from XML may be required for importing rules created on S3 or s

See :ref:`minio-lifecycle-management` for more complete documentation.

Target Bucket Considerations
----------------------------

MinIO does *not* require that the target bucket match object management or versioning configurations with the source bucket.
The target bucket *may* have its own set of object management rules, if defined with care.

Target buckets should *not* have their own rules for expiration or additional tiering.
Expiration rules can result in removal of tiered data still in use by the source bucket.
Tiering to an additional remote creates an additional network hop between the hot tier and it's data while also increasing operational complexity.

You *may* configure object locking or versioning on the remote bucket.
Use caution, as mismatched rules between the source and the target buckets run a risk of data corruption and loss.
djwfyi marked this conversation as resolved.
Show resolved Hide resolved

Creating separate rules on the target bucket may have effects such as the following:
djwfyi marked this conversation as resolved.
Show resolved Hide resolved

- Object locking set on the target bucket may prevent desired ``delete`` operations from the source bucket from completing.

- MinIO tiers objects with their own ``UUID``, so versioning on the target bucket is not required.

If you enable versioning on the target bucket, you may experience the following unexpected or undesired results:

- Excess data usage on the target remote after a delete operation on the source
- Reduced storage efficiency on the target, as ``delete`` operations result in ``DeleteMarker`` rather than freeing space
- Duplicate delete markers on source and target buckets
djwfyi marked this conversation as resolved.
Show resolved Hide resolved

Exclusive Access to Remote Data
-------------------------------

MinIO **must** have *exclusive* access to the target bucket.
No other user, process, application, or resource should have any access to or perform any actions against the target bucket.

All access to the transitioned objects *must* occur through MinIO via S3 API operations only.
Manually modifying a transitioned object - whether the metadata on the “hot” MinIO tier or the object data on the remote “warm/cold” tier - may result in loss of that object data.

MinIO ignores any objects in the remote bucket or bucket prefix not explicitly managed by the MinIO deployment. Automatic transition and transparent object retrieval depend on the following assumptions:

- No external mutation, migration, or deletion of objects on the remote storage.
- No lifecycle management rules (such as transition or expiration) on the remote storage bucket.

To facilitate this exclusive access, grant the lifecycle management user ``read``, ``write``, and ``delete`` access to the target bucket in its :ref:`policy <minio-policy>`.
All other policies should ``deny`` access to the target bucket.

.. toctree::
:titlesonly:
:hidden:
Expand Down