Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add anonymizer feature #27

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

mazsudo
Copy link
Contributor

@mazsudo mazsudo commented Feb 25, 2020

Hi there,
here is a first implementation to answer to this #5 feature request.

Concept is:

  • provides 2 annotation (at entity & property levels): AnonymizedEntity & AnonymizedProperty

  • AnonymizedEntity has 2 possible actions (anonymize & truncate) and a where sql clause to be able to keep some data as they are (admin account or something like that)

  • AnonymizedProperty has 3 types (static, composed, expression) and a value field:
    => static : a static value that will be applied on all raw of the table
    => composed : a composed value based on another field of the entity which allow you to update all lines into a format like: email="test+<id>@test.com where id is the identifier of the table which allow to answer to unique index issue (at sql level)
    => expression : a valid sql expression like CONCAT(FLOOR(1 + (RAND() * 1000)), id) to answer to unique constraints issue (at validator level)

This MR mainly contains a command that :

  • gather & build AnonymizedMetadata by entity
  • validate this AnonymizedMetadata object (check some nullable/unique constraint to give best chances to generated queries to succeed)
  • build a query for each AnonymizedMetadata object
  • execute those queries

Would love some feedback before going further and writing test/doc ;)

@mazsudo mazsudo force-pushed the add-anonymize-feature branch 8 times, most recently from 6fa3604 to 26da8ec Compare March 3, 2020 11:50
@qsomazzi
Copy link
Member

qsomazzi commented Mar 3, 2020

I'm not sure it's possible (or the goal), but it would be awesome to run this command in a production env, and instead of executing thoose queries it will create an anonymized dump of the DB in a target folder. no ?

Besides that, i find that to be a nice way to handle anonymisation through annotation, it feel simple to export and apply to another project.

@mazsudo
Copy link
Contributor Author

mazsudo commented Mar 3, 2020

I'm not sure it's possible (or the goal), but it would be awesome to run this command in a production env, and instead of executing thoose queries it will create an anonymized dump of the DB in a target folder. no ?

not the purpose here but your thought kind of already exists here :
https://github.com/ifsnop/mysqldump-php
https://github.com/machbarmacher/gdpr-dump

@mazsudo mazsudo force-pushed the add-anonymize-feature branch from 26da8ec to 373df0c Compare March 3, 2020 16:40
Annotations/AnonymizedEntity.php Outdated Show resolved Hide resolved
Annotations/AnonymizedEntity.php Outdated Show resolved Hide resolved
Annotations/AnonymizedEntity.php Outdated Show resolved Hide resolved
Annotations/AnonymizedProperty.php Outdated Show resolved Hide resolved
Annotations/AnonymizedProperty.php Outdated Show resolved Hide resolved
Meta/AnonymizedMetadataBuilder.php Outdated Show resolved Hide resolved
QueryBuilder/AnonymizedQueryBuilder.php Outdated Show resolved Hide resolved
QueryBuilder/AnonymizedQueryBuilder.php Outdated Show resolved Hide resolved
QueryBuilder/AnonymizedQueryBuilder.php Outdated Show resolved Hide resolved
Meta/AnonymizedMetadata.php Show resolved Hide resolved
Annotations/AnonymizedEntity.php Outdated Show resolved Hide resolved
Annotations/AnonymizedEntity.php Outdated Show resolved Hide resolved
Annotations/AnonymizedProperty.php Outdated Show resolved Hide resolved
Annotations/AnonymizedProperty.php Outdated Show resolved Hide resolved
QueryBuilder/AnonymizedQueryBuilder.php Show resolved Hide resolved
QueryBuilder/AnonymizedQueryBuilder.php Outdated Show resolved Hide resolved
QueryBuilder/AnonymizedQueryBuilder.php Outdated Show resolved Hide resolved
} elseif ($anonymizedProperty->isComposed()) {
$composedFieldParts = $anonymizedProperty->explodeComposedFieldValue();

return sprintf('%s = concat(concat("%s", %s), "%s")', $anonymizedProperty->getColumnName(),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes for example I'm pretty sure the concat expression varies by platform.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 seems ok at least for mysql & posgresql. WDYT?

QueryBuilder/AnonymizedQueryBuilder.php Outdated Show resolved Hide resolved
@mazsudo mazsudo force-pushed the add-anonymize-feature branch 2 times, most recently from 7131a13 to a1cdb88 Compare March 4, 2020 12:39
@mazsudo mazsudo force-pushed the add-anonymize-feature branch from a1cdb88 to 12699cf Compare March 4, 2020 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants