Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update add_field_to function for improved error handling #696

Merged
merged 38 commits into from
Nov 14, 2024

Conversation

dtrai2
Copy link
Collaborator

@dtrai2 dtrai2 commented Nov 6, 2024

  • rename add_field_to to add_fields_to
  • add_fields_to always raises FieldExistsWarning on failure
  • add_fields_to accepts a batch of fields in form of a dict
  • add_fields_to can take a Rule as argument now, needed to raise proper FieldExistsWarning; is optional because of use of add_fields_to outside from processors, e.g. InputConnectors
  • Refactor add_fields_to function for clarity and consistency.
  • Update unit tests to reflect new exception handling logic.
  • Update code and tests to utilize new add_fields_to (especially processors)

ToDo:

  • Add tests for adding multiple fields with add_fields_to method
  • rethink usage of _write_target_field in abc/processor.py

@dtrai2 dtrai2 added the enhancement New feature or request label Nov 6, 2024
@dtrai2 dtrai2 requested a review from djkhl November 6, 2024 14:16
@dtrai2 dtrai2 self-assigned this Nov 6, 2024
@dtrai2 dtrai2 force-pushed the dev-revise-add-field-to-error-handling branch from 53b7ff9 to 2fbfa7e Compare November 11, 2024 07:54
@dtrai2 dtrai2 requested a review from ekneg54 November 11, 2024 12:04
@dtrai2 dtrai2 marked this pull request as ready for review November 11, 2024 12:11
Copy link
Collaborator

@ekneg54 ekneg54 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thank you for your contribution. please have a look on my remarks.

logprep/abc/input.py Outdated Show resolved Hide resolved
logprep/processor/base/exceptions.py Show resolved Hide resolved
logprep/util/helper.py Outdated Show resolved Hide resolved
logprep/util/helper.py Outdated Show resolved Hide resolved
logprep/util/helper.py Outdated Show resolved Hide resolved
tests/unit/util/test_helper_add_field.py Outdated Show resolved Hide resolved
tests/unit/util/test_helper_add_field.py Outdated Show resolved Hide resolved
tests/unit/util/test_helper_add_field.py Outdated Show resolved Hide resolved
tests/unit/util/test_helper_add_field.py Outdated Show resolved Hide resolved
tests/unit/util/test_helper_add_field.py Outdated Show resolved Hide resolved
@dtrai2 dtrai2 force-pushed the dev-revise-add-field-to-error-handling branch from 032a95c to 4de86e2 Compare November 12, 2024 09:23
- add CHANGELOG.md
- remove duplicate test
- write documentation
- Refactored helper functions to consistently raise exceptions.
- Improved error handling by eliminating silent failures.
- Updated related unit tests to expect raised exceptions.
- highlight breaking change in CHANGELOG.md
- Renames `overwrite_output_field` to `overwrite_target_field` in multiple files.
- Ensures consistency in method signatures
- Ensure document state is asserted correctly after exceptions are raised.
- Update test name and add a comment for clarity.
- Merges target and content arguments into a single field argument.
- Replace `add_batch_to` with `add_field_to` throughout code.
- Update helper functions to streamline `add_field_to` usage.
- Add tests to cover adding multiple fields at once.
- Include scenarios for overwriting, extending lists, and raising warnings.
- Corrected 'delimiter' typo across test and implementation files.
- Updated changelog to reflect this fix.
@dtrai2 dtrai2 force-pushed the dev-revise-add-field-to-error-handling branch from 4d5dbf8 to 827b7f4 Compare November 13, 2024 10:46
@dtrai2 dtrai2 requested a review from ekneg54 November 13, 2024 11:55
- Simplified function name to better reflect its purpose.
- Updated all instances where the function is invoked to maintain consistency.
Copy link
Collaborator

@ekneg54 ekneg54 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to me

@ekneg54 ekneg54 merged commit 205045e into main Nov 14, 2024
13 checks passed
@ekneg54 ekneg54 deleted the dev-revise-add-field-to-error-handling branch November 14, 2024 07:57
dtrai2 added a commit that referenced this pull request Dec 10, 2024
- this fixes a FieldExist warning which was thrown when processing times of a rule where already present in the event
- recreates previous behavior before #696 was merged
ekneg54 pushed a commit that referenced this pull request Dec 11, 2024
…725)

* use _add_field_to_silent_fail in APPEND_TO_EVENT metrics

- this fixes a FieldExist warning which was thrown when processing times of a rule where already present in the event
- recreates previous behavior before #696 was merged

* add CHANGELOG.md entry
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants