Skip to content

Commit

Permalink
docs: Release notes v0.2.8 (#268)
Browse files Browse the repository at this point in the history
  • Loading branch information
wwoytenko authored Feb 25, 2025
1 parent 7ed2529 commit 4145920
Show file tree
Hide file tree
Showing 32 changed files with 432 additions and 226 deletions.
30 changes: 15 additions & 15 deletions docs/built_in_transformers/standard_transformers/masking.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,29 +2,29 @@ Mask a value using one of the masking rules depending on your domain. `NULL` val

## Parameters

| Name | Description | Default | Required | Supported DB types |
|--------|-----------------------------------------------------------------------------------------------------------------|-----------|----------|--------------------|
| column | The name of the column to be affected | | Yes | text, varchar |
| type | Data type of attribute (`default`, `password`, `name`, `addr`, `email`, `mobile`, `tel`, `id`, `credit`, `url`) | `default` | No | - |
| Name | Description | Default | Required | Supported DB types |
|--------|-----------------------------------------------------------------------------------------------------------------|-----------|----------|-------------------------------------|
| column | The name of the column to be affected | | Yes | text, varchar, char, bpchar, citext |
| type | Data type of attribute (`default`, `password`, `name`, `addr`, `email`, `mobile`, `tel`, `id`, `credit`, `url`) | `default` | No | - |

## Description

The `Masking` transformer replaces characters with asterisk `*` symbols depending on the provided data type. If the
value is `NULL`, it is kept unchanged. It is based on [ggwhite/go-masker](https://github.com/ggwhite/go-masker) and
supports the following masking rules:

| Type | Description |
|:-----------:|:-------------------------------------------------------------------------------------------------------------------------------------------------------------|
| default | Returns `*` symbols with the same length, e.g. input: `test1234` output: `********` |
| name | Masks the second letter the third letter in a word, e. g. input: `ABCD` output: `A**D` |
| password | Always returns `************` |
| address | Keeps first 6 letters, masks the rest, e. g. input: `Larnaca, makarios st` output: `Larnac*************` |
| email | Keeps a domain and the first 3 letters, masks the rest, e. g. input: `[email protected]` output: `ggw****@gmail.com` |
| mobile | Masks 3 digits starting from the 4th digit, e. g. input: `0987654321` output: `0987***321` |
| Type | Description |
|:-----------:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| default | Returns `*` symbols with the same length, e.g. input: `test1234` output: `********` |
| name | Masks the second letter the third letter in a word, e. g. input: `ABCD` output: `A**D` |
| password | Always returns `************` |
| address | Keeps first 6 letters, masks the rest, e. g. input: `Larnaca, makarios st` output: `Larnac*************` |
| email | Keeps a domain and the first 3 letters, masks the rest, e. g. input: `[email protected]` output: `ggw****@gmail.com` |
| mobile | Masks 3 digits starting from the 4th digit, e. g. input: `0987654321` output: `0987***321` |
| telephone | Removes `(`, `)`, ` `, `-` chart, and masks last 4 digits of telephone number, then formats it to `(??)????-????`, e. g. input: `0227993078` output: `(02)2799-****` |
| id | Masks last 4 digits of ID number, e. g. input: `A123456789` output: `A12345****` |
| credit_cart | Masks 6 digits starting from the 7th digit, e. g. input `1234567890123456` output `123456******3456` |
| url | Masks the password part of the URL, if applicable, e. g. `http://admin:mysecretpassword@localhost:1234/uri` output: `http://admin:xxxxx@localhost:1234/uri` |
| id | Masks last 4 digits of ID number, e. g. input: `A123456789` output: `A12345****` |
| credit_cart | Masks 6 digits starting from the 7th digit, e. g. input `1234567890123456` output `123456******3456` |
| url | Masks the password part of the URL, if applicable, e. g. `http://admin:mysecretpassword@localhost:1234/uri` output: `http://admin:xxxxx@localhost:1234/uri` |

## Example: Masking employee national ID number

Expand Down
Original file line number Diff line number Diff line change
@@ -1,19 +1,27 @@
The `RandomAmountWithCurrency` transformer is specifically designed to populate specified database columns with random financial amounts accompanied by currency codes. Ideal for applications requiring the simulation of financial transactions, this utility enhances the realism of financial datasets by introducing variability in amounts and currencies.
The `RandomAmountWithCurrency` transformer is specifically designed to populate specified database columns with random
financial amounts accompanied by currency codes. Ideal for applications requiring the simulation of financial
transactions, this utility enhances the realism of financial datasets by introducing variability in amounts and
currencies.

## Parameters

| Name | Description | Default | Required | Supported DB types |
|------------|----------------------------------------------------|---------|----------|--------------------|
| column | The name of the column to be affected | | Yes | text, varchar |
| keep_null | Indicates whether NULL values should be preserved | `false` | No | - |
| Name | Description | Default | Required | Supported DB types |
|-----------|---------------------------------------------------|---------|----------|-------------------------------------|
| column | The name of the column to be affected | | Yes | text, varchar, char, bpchar, citext |
| keep_null | Indicates whether NULL values should be preserved | `false` | No | - |

## Description

This transformer automatically generates random financial amounts along with corresponding global currency codes (e. g., `250.00 USD`, `300.00 EUR`), injecting them into the designated database column. It provides a straightforward solution for populating financial records with varied and realistic data, suitable for testing payment systems, data anonymization, and simulation of economic models.
This transformer automatically generates random financial amounts along with corresponding global currency codes (e. g.,
`250.00 USD`, `300.00 EUR`), injecting them into the designated database column. It provides a straightforward solution
for populating financial records with varied and realistic data, suitable for testing payment systems, data
anonymization, and simulation of economic models.

## Example: Populate the `payments` table with random amounts and currencies

This example shows how to configure the `RandomAmountWithCurrency` transformer to populate the `payment_details` column in the `payments` table with random amounts and currencies. It is an effective approach to simulating a diverse range of payment transactions.
This example shows how to configure the `RandomAmountWithCurrency` transformer to populate the `payment_details` column
in the `payments` table with random amounts and currencies. It is an effective approach to simulating a diverse range of
payment transactions.

```yaml title="RandomAmountWithCurrency transformer example"
- schema: "public"
Expand All @@ -25,4 +33,6 @@ This example shows how to configure the `RandomAmountWithCurrency` transformer t
keep_null: false
```
In this setup, the `payment_details` column will be updated with random financial amounts and currency codes for each entry, replacing any existing non-NULL values. The `keep_null` parameter, when set to `true`, ensures that existing NULL values in the column remain unchanged, preserving the integrity of records without specified payment details.
In this setup, the `payment_details` column will be updated with random financial amounts and currency codes for each
entry, replacing any existing non-NULL values. The `keep_null` parameter, when set to `true`, ensures that existing NULL
values in the column remain unchanged, preserving the integrity of records without specified payment details.
Original file line number Diff line number Diff line change
@@ -1,19 +1,26 @@
The `RandomCCNumber` transformer is specifically designed to populate specified database columns with random credit card numbers. This utility is crucial for applications that involve simulating financial data, testing payment systems, or anonymizing real credit card numbers in datasets.
The `RandomCCNumber` transformer is specifically designed to populate specified database columns with random credit card
numbers. This utility is crucial for applications that involve simulating financial data, testing payment systems, or
anonymizing real credit card numbers in datasets.

## Parameters

| Name | Description | Default | Required | Supported DB types |
|------------|-------------------------------------------------------|---------|----------|--------------------|
| column | The name of the column to be affected | | Yes | text, varchar |
| keep_null | Indicates whether NULL values should be preserved | `false` | No | - |
| Name | Description | Default | Required | Supported DB types |
|-----------|---------------------------------------------------|---------|----------|-------------------------------------|
| column | The name of the column to be affected | | Yes | text, varchar, char, bpchar, citext |
| keep_null | Indicates whether NULL values should be preserved | `false` | No | - |

## Description

By leveraging algorithms capable of generating plausible credit card numbers that adhere to standard credit card validation rules (such as the Luhn algorithm), the `RandomCCNumber` transformer injects random credit card numbers into the designated database column. This approach ensures the generation of credit card numbers that are realistic for testing and development purposes, without compromising real-world applicability and security.
By leveraging algorithms capable of generating plausible credit card numbers that adhere to standard credit card
validation rules (such as the Luhn algorithm), the `RandomCCNumber` transformer injects random credit card numbers into
the designated database column. This approach ensures the generation of credit card numbers that are realistic for
testing and development purposes, without compromising real-world applicability and security.

## Example: Populate random credit card numbers for the `payment_information` table

This example demonstrates configuring the `RandomCCNumber` transformer to populate the `cc_number` column in the `payment_information` table with random credit card numbers. It is an effective strategy for creating a realistic set of payment data for application testing or data anonymization.
This example demonstrates configuring the `RandomCCNumber` transformer to populate the `cc_number` column in the
`payment_information` table with random credit card numbers. It is an effective strategy for creating a realistic set of
payment data for application testing or data anonymization.

```yaml title="RandomCCNumber transformer example"
- schema: "public"
Expand All @@ -25,4 +32,6 @@ This example demonstrates configuring the `RandomCCNumber` transformer to popula
keep_null: false
```
With this setup, the `cc_number` column will be updated with random credit card numbers for each entry, replacing any existing non-NULL values. If the `keep_null` parameter is set to `true`, it will ensure that existing NULL values in the column are preserved, maintaining the integrity of records where credit card information is not applicable or available.
With this setup, the `cc_number` column will be updated with random credit card numbers for each entry, replacing any
existing non-NULL values. If the `keep_null` parameter is set to `true`, it will ensure that existing NULL values in the
column are preserved, maintaining the integrity of records where credit card information is not applicable or available.
25 changes: 17 additions & 8 deletions docs/built_in_transformers/standard_transformers/random_cc_type.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,26 @@
The `RandomCCType` transformer is designed to populate specified database columns with random credit card types. This tool is essential for applications that require the simulation of financial transaction data, testing payment processing systems, or anonymizing credit card type information in datasets.
The `RandomCCType` transformer is designed to populate specified database columns with random credit card types. This
tool is essential for applications that require the simulation of financial transaction data, testing payment processing
systems, or anonymizing credit card type information in datasets.

## Parameters

| Name | Description | Default | Required | Supported DB types |
|------------|-------------------------------------------------------|---------|----------|--------------------|
| column | The name of the column to be affected | | Yes | text, varchar |
| keep_null | Indicates whether NULL values should be preserved | `false` | No | - |
| Name | Description | Default | Required | Supported DB types |
|-----------|---------------------------------------------------|---------|----------|-------------------------------------|
| column | The name of the column to be affected | | Yes | text, varchar, char, bpchar, citext |
| keep_null | Indicates whether NULL values should be preserved | `false` | No | - |

## Description

Utilizing a predefined list of credit card types (e.g., VISA, MasterCard, American Express, Discover), the `RandomCCType` transformer injects random credit card type names into the designated database column. This feature allows for the creation of realistic and varied financial transaction datasets by simulating a range of credit card types without using real card data.
Utilizing a predefined list of credit card types (e.g., VISA, MasterCard, American Express, Discover), the
`RandomCCType` transformer injects random credit card type names into the designated database column. This feature
allows for the creation of realistic and varied financial transaction datasets by simulating a range of credit card
types without using real card data.

## Example: Populate random credit card types for the `transactions` table

This example shows how to configure the `RandomCCType` transformer to populate the `card_type` column in the `transactions` table with random credit card types. It is a straightforward method for simulating diverse payment methods across transactions.
This example shows how to configure the `RandomCCType` transformer to populate the `card_type` column in the
`transactions` table with random credit card types. It is a straightforward method for simulating diverse payment
methods across transactions.

```yaml title="RandomCCType transformer example"
- schema: "public"
Expand All @@ -25,4 +32,6 @@ This example shows how to configure the `RandomCCType` transformer to populate t
keep_null: false
```
In this configuration, the `card_type` column will be updated with random credit card types for each entry, replacing any existing non-NULL values. If the `keep_null` parameter is set to `true`, existing NULL values in the column will be preserved, maintaining the integrity of records where card type information is not applicable.
In this configuration, the `card_type` column will be updated with random credit card types for each entry, replacing
any existing non-NULL values. If the `keep_null` parameter is set to `true`, existing NULL values in the column will be
preserved, maintaining the integrity of records where card type information is not applicable.
Loading

0 comments on commit 4145920

Please sign in to comment.