Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reworked storage configuration section - added storage URI #43

Merged
merged 1 commit into from
Mar 29, 2024

Conversation

percona-ysorokin
Copy link
Collaborator

Changed the way how storage backend is configured: instead of 'type' and 'path' parameters we now have single 'uri', which incorporates both and is open for future extensions
Currently, two schemas are supported:

  • 'file://' for local filesystem;
  • 's3://<access_key_id>:<secret_access_key>@<bucket_name>/' for AWS S3. Please note that for 's3' credentials can be omitted, in other words 's3://<bucket_name>/' is also a valid URI provided that this <bucket_name> bucket is publicly accessible.

's3_storage_backend' class (still a stub) extended with missing AWS S3 configuration parameters:

  • 'access_key_id';
  • 'secret_access_key';
  • 'bucket'.

Reworked constructors of both 's3_storage_backend' and 'filesystem_storage_backend': they now accept single 'boost::urls::url_view_base' parameter. These constructors extended with basic URI validation (for schema, userinfo, host, port, path, query, and fragment).

'do_get_description()' method of both 's3_storage_backend' and 'filesystem_storage_backend' extended with including more configuration parameters:

  • 'path' for local filesystem;
  • 'bucket' and 'path' for AWS S3 (credentials are omitted deliberately).

Reworked 'storage_backend_factory::create()' method: it now determines which concrete implementation of the 'basic_storage_backend' interface it should construct based on the schema part of the 'uri' parameter from the 'storage' configuration section.

Update "help" message for the main application to reflect new 'storage.uri' parameter.

Updated sample configuration JSON file.

Changed the way how storage backend is configured: instead of 'type' and 'path'
parameters we now have single 'uri', which incorporates both and is open for
future extensions
Currently, two schemas are supported:
* 'file://<path>' for local filesystem;
* 's3://<access_key_id>:<secret_access_key>@<bucket_name>/<path>' for AWS S3.
Please note that for 's3' credentials can be omitted, in other words
's3://<bucket_name>/<path>' is also a valid URI provided that this <bucket_name>
bucket is publicly accessible.

's3_storage_backend' class (still a stub) extended with missing AWS S3
configuration parameters:
* 'access_key_id';
* 'secret_access_key';
* 'bucket'.

Reworked constructors of both 's3_storage_backend' and
'filesystem_storage_backend': they now accept single
'boost::urls::url_view_base' parameter. These constructors extended with basic
URI validation (for schema, userinfo, host, port, path, query, and fragment).

'do_get_description()' method of both 's3_storage_backend' and
'filesystem_storage_backend' extended with including more configuration
parameters:
* 'path' for local filesystem;
* 'bucket' and 'path' for AWS S3 (credentials are omitted deliberately).

Reworked 'storage_backend_factory::create()' method: it now determines which
concrete implementation of the 'basic_storage_backend' interface it should
construct based on the schema part of the 'uri' parameter from the 'storage'
configuration section.

Update "help" message for the main application to reflect new 'storage.uri'
parameter.

Updated sample configuration JSON file.
@percona-ysorokin percona-ysorokin merged commit 1d19abd into Percona-Lab:main Mar 29, 2024
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant