This change makes object-store backed metadata more ergonomic and consistent with our snapshots configuration.

cc: @igalshilman if you'd like to chime in!

Changes:

Switch object-store metadata to use utils crate

The config struct for metadata client has switched from an explicit bucket name
to a URL and renamed to "path", consistent with snapshots configuration.

Additionally we now use the shared object store options struct which allows
users to directly set a variety of credentials config keys.

Finally, we switch to using our object-store-utils crate which handles the
client building and AWS credentials caching. This allows us to reuse the AWS
SDK conventions for more ergonomic and consistent configuration.

With this change, we now support simplified server configuration like:

[metadata-client]
type = "object-store"
# bucket = "bucket" # removed in favor of path:
path = "s3://bucket"
aws-profile = "aws-profile"
# aws_region = "eu-central-1"
# aws_access_key_id "..."
# ... and more, from ObjectStoreOptions - same as [worker.snapshots] options

# no longer used; creds can be set within the [metadata-client] section
# [metadata-client.credentials]
# type = "aws-env"

Note thought that thanks to the AWS SDK config resolution mechanism, creds are entirely optional. We will use AWS_PROFILE, AWS_REGION, etc. or server environment STS credentials without any additional configuration.

Note too, that this breaks compatibility. Since we've never documented or advertised this feature, I feel justified in changing the config keys without maintaining backcompat - let me know if you disagree.

Add support for bucket prefixes

This allows users to use a single bucket for multiple use cases, e.g. metadata and snapshots for the same cluster.

We can now do this - note the metadata-0 prefix supplied as part of the object store path:

[metadata-client]
type = "object-store"
path = "s3://restate-snapshots-afs1/metadata-0"

Example:

❯ aws s3 ls s3://restate-snapshots-afs1/
                           PRE metadata-0/
                           PRE metadata-1/

❯ aws s3 ls s3://restate-snapshots-afs1/metadata-0/
2025-02-27 08:45:48       1619 bifrost_config
2025-02-27 08:45:46        478 nodes_config
2025-02-27 08:45:48       2169 partition_table
2025-02-27 08:45:48        122 pp_epoch_0
2025-02-27 08:45:48        122 pp_epoch_1

Closes: #2795