This software anonymises data inside text files and in sheet files. It recognises and removes various sorts of personally identifiable information (PII). Each removed part is replaced / obfuscated with a suitable generic text, depending on the type of removed data.
Currently English and Russian languages are supported. Russian works both with Cyrillic and Latin characters.
The language is automatically detected. In case of sheet files, the language of each cell is detected separately. Therefore multi-language sheet files are supported as well.
Currently supported sheet file formats are CSV-files, TSV-files, Excel files (XLSX only), and OpenDocument Sheet files (ODS).
Example input and output copied to an annotated PDF file: Anonymisation example 1.pdf
Example input and output file pairs for TXT and CSV file formats in English language, and TXT file format in Russian language with Cyrillic and Latin alphabet:
- data/test_input_en.txt - data/example_output_en.txt
- data/test_input_en.csv - data/example_output_en.csv
- data/test_input_ru_cyr.txt - data/example_output_ru_cyr.txt
- data/test_input_ru_lat.txt - data/example_output_ru_lat.txt
This software uses a combination of Named Entity Recognition (NER) and regular expressions to perform its function.
The configuration options can be found in the file Anonymiser.ini
python Anonymiser.py "input_file.txt"|"input_file.csv"|"input_file.tsv"|"input_file.xlsx"|"input_file.ods" ["output_file.txt"|"output_file.csv"|"output_file.tsv"|"output_file.xlsx"|"output_file.ods"]
The user provided files are expected to be in the same folder as the main Python script, unless an absolute path is provided. If run without arguments then sample files in the data folder are used. If the user provides input file name but no output file name then the output file name will be calculated as input filename + _anonymised + .input filename extension.
If the CSV file parsing fails or the CSV output seems to have wrong structure, please check and adjust the CSV parsing settings in Anonymiser.ini. More concretely, CsvDelimiter, CsvQuoteChar, CsvDoubleQuote, and CsvEscapeChar parameters may need adjustment.
Ready to use. Is actively developed further.