Skip to content

Commit

Permalink
chore: adding plugin definition
Browse files Browse the repository at this point in the history
  • Loading branch information
Nicolas ESTRADA committed Jun 20, 2024
1 parent 660546b commit e667621
Showing 1 changed file with 198 additions and 0 deletions.
198 changes: 198 additions & 0 deletions 42school.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,198 @@
capabilities:
- about
- schema-flattening
- stream-maps
description: AWS S3 (Multi Format)
domain_url: https://aws.amazon.com/s3/
executable: target-s3
keywords:
- meltano_sdk
- aws
- file
label: S3 (Multi Format)
logo_url: /assets/logos/loaders/s3.png
maintenance_status: active
name: target-s3
namespace: target_s3
next_steps: ''
pip_url: git+https://github.com/42School/target-s3.git
quality: platinum
repo: https://github.com/42School/target-s3
settings:
- description: A flag to append the date to the key filename.
kind: boolean
label: Append Date To Filename
name: append_date_to_filename
value: true
- description: The grain of the date to append to the filename.
kind: options
label: Append Date To Filename Grain
name: append_date_to_filename_grain
options:
- label: Year
value: year
- label: Month
value: month
- label: Day
value: day
- label: Hour
value: hour
- label: Minute
value: minute
- label: Second
value: second
- label: Microsecond
value: microsecond
value: day
- description: A flag to append the date to the key prefix.
kind: boolean
label: Append Date To Prefix
name: append_date_to_prefix
value: true
- description: The grain of the date to append to the prefix.
kind: options
label: Append Date To Prefix Grain
name: append_date_to_prefix_grain
options:
- label: Year
value: year
- label: Month
value: month
- label: Day
value: day
- label: Hour
value: hour
- label: Minute
value: minute
- label: Second
value: second
- label: Microsecond
value: microsecond
value: day
- description: Your AWS access key ID.
kind: password
label: Cloud Provider AWS AWS Access Key ID
name: cloud_provider.aws.aws_access_key_id
sensitive: true
- description: Your AWS access key ID.
kind: password
label: Cloud Provider AWS AWS Bucket
name: cloud_provider.aws.aws_bucket
sensitive: true
- description: The AWS endpoint url to use for testing purposes.
kind: password
label: Cloud Provider AWS AWS Endpoint Override
name: cloud_provider.aws.aws_endpoint_override
sensitive: true
- description: Your AWS profile name to optionally use for auth.
kind: password
label: Cloud Provider AWS AWS Profile Name
name: cloud_provider.aws.aws_profile_name
sensitive: true
- description: Your AWS region.
kind: password
label: Cloud Provider AWS AWS Region
name: cloud_provider.aws.aws_region
sensitive: true
- description: Your AWS secret access key.
kind: password
label: Cloud Provider AWS AWS Secret Access Key
name: cloud_provider.aws.aws_secret_access_key
sensitive: true
- description: ''
kind: password
label: Cloud Provider AWS AWS Session Token
name: cloud_provider.aws.aws_session_token
sensitive: true
- description: The cloud provider type to use.
kind: password
label: Cloud Provider Cloud Provider Type
name: cloud_provider.cloud_provider_type
options:
- label: AWS
value: aws
sensitive: true
value: aws
- description: A flag indictating to flatten records.
kind: boolean
label: Flatten Records
name: flatten_records
- description: "'True' to enable schema flattening and automatically expand nested
properties."
kind: boolean
label: Flattening Enabled
name: flattening_enabled
- description: The max depth to flatten schemas.
kind: integer
label: Flattening Max Depth
name: flattening_max_depth
- description: This flag determines whether the data types of incoming data elements
should be validated. When set `True`, a schema is created from the first record
and all subsequent records that don't match that data type are cast. Default False.
kind: boolean
label: Format Format Parquet Validate
name: format.format_parquet.validate
value: false
- description: The format type of the output files.
kind: options
label: Format Format Type
name: format.format_type
options:
- label: Parquet
value: parquet
- label: Json
value: json
- label: JSON Lines
value: jsonl
- description: A flag indicating whether to append _process_date to record.
kind: boolean
label: Include Process Date
name: include_process_date
value: false
- description: A flag (only works if append_date_to_prefix is enabled) to have partitioning
name formatted e.g. 'year=2023/month=01/day=01'.
kind: boolean
label: Partition Name Enabled
name: partition_name_enabled
value: false
- description: The prefix for the key.
kind: string
label: Prefix
name: prefix
- description: User-defined config values to be used within map expressions.
kind: object
label: Stream Map Config
name: stream_map_config
- description: Config object for stream maps capability. For more information check
out [Stream Maps](https://sdk.meltano.com/en/latest/stream_maps.html).
kind: object
label: Stream Maps
name: stream_maps
- description: The S3 key stream name override.
kind: password
label: Stream Name Path Override
name: stream_name_path_override
sensitive: true
- description: A flag to force the filename to be identical to the stream name
kind: boolean
label: Use Raw Stream Name
name: use_raw_stream_name
value: false
- description: Maximum time in minutes between state messages when records are streamed in
kind: integer
label: Maximum Batch Age
name: max_batch_age
value: 5
- description: Maximum size of batches when records are streamed in
kind: integer
label: Maximum Batch Size
name: max_batch_size
value: 10000
settings_group_validation:
- - cloud_provider.aws.aws_bucket
- cloud_provider.aws.aws_region
- cloud_provider.cloud_provider_type
- format.format_type
settings_preamble: ''
usage: ''
variant: 42School

0 comments on commit e667621

Please sign in to comment.