remove normalizer processor (#400)

* remove normalizer processor * remove pygrok dependency * add regex dependency * update changelog --------- Co-authored-by: ekneg54 <[email protected]>
fkie-cad · Jun 24, 2024 · 1fec611 · 1fec611
1 parent a60bed5
commit 1fec611
Show file tree

Hide file tree

Showing 42 changed files with 46 additions and 3,076 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,12 +1,13 @@
 ## Upcoming Changes
 
-
 ## next release
+### Breaking
 
-This release limits the maximum python version to `3.12.3` because of the issue
+* This release limits the maximum python version to `3.12.3` because of the issue
 [#612](https://github.com/fkie-cad/Logprep/issues/612).
+* Remove `normalizer` processor, as it's functionality was replaced by the `grokker`, `timestamper`
+and `field_manager` processors
 
-### Breaking
 ### Features
 ### Improvements
 
@@ -417,7 +418,6 @@ In case of positive detection results, rule attributions are now inserted in the
 * Bump `requests` to `>=2.31.0` to circumvent `CVE-2023-32681`
 * Include a lucene representation of the rule filter into the predetector results. The
 representation is not completely lucene compatible due to non-existing regex functionality.
-* Remove direct dependency of `python-dateutil`
 
 ### Bugfix
 

diff --git a/doc/source/user_manual/configuration/processor.rst b/doc/source/user_manual/configuration/processor.rst
@@ -24,7 +24,6 @@ Processors
 .. automodule:: logprep.processor.key_checker.processor
 .. automodule:: logprep.processor.labeler.processor
 .. automodule:: logprep.processor.list_comparison.processor
-.. automodule:: logprep.processor.normalizer.processor
 .. automodule:: logprep.processor.pre_detector.processor
 .. automodule:: logprep.processor.pseudonymizer.processor
 .. automodule:: logprep.processor.requester.processor

diff --git a/logprep/connector/file/input.py b/logprep/connector/file/input.py
@@ -210,7 +210,7 @@ class Config(Input.Config):
 
         logfile_path: str = field(validator=file_validator)
         """A path to a file in generic raw format, which can be in any string based
-        format. Needs to be parsed with normalizer or another processor"""
+        format. Needs to be parsed with dissector or another processor"""
 
         start: str = field(
             validator=[validators.instance_of(str), validators.in_(("begin", "end"))],

diff --git a/logprep/processor/normalizer/__init__.py b/logprep/processor/normalizer/__init__.py