Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for SMTP TLS reports #453

Merged
merged 26 commits into from
Feb 19, 2024
Merged

Add support for SMTP TLS reports #453

merged 26 commits into from
Feb 19, 2024

Conversation

seanthegeek
Copy link
Contributor

@seanthegeek seanthegeek commented Dec 27, 2023

This is a draft PR. I need some help working on it.

@seanthegeek seanthegeek changed the title Add support for SMTP TLS reports #71 Add support for SMTP TLS reports Dec 27, 2023
Copy link

codecov bot commented Dec 27, 2023

Codecov Report

Attention: 134 lines in your changes are missing coverage. Please review.

Comparison is base (100f12e) 58.42% compared to head (009f1d6) 54.29%.
Report is 2 commits behind head on master.

Files Patch % Lines
parsedmarc/__init__.py 24.29% 134 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #453      +/-   ##
==========================================
- Coverage   58.42%   54.29%   -4.13%     
==========================================
  Files          11       11              
  Lines        1347     1501     +154     
==========================================
+ Hits          787      815      +28     
- Misses        560      686     +126     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@seanthegeek seanthegeek marked this pull request as ready for review January 2, 2024 21:30
@seanthegeek
Copy link
Contributor Author

seanthegeek commented Jan 2, 2024

Ok. I think initial SMTP TLS report support is done, including JSON, CSV, Elasticsearch, and Splunk output, but I haven't done much testing yet

Other steps that are needed:

  • Add SMTP TLS report output support to the Kafka client. @mikesiegel, can you look into this?
  • Add SMTP TLS report output support to the LogAnalytics client. @rubeste, can you help with this?
  • Add SMTP TLS report output support to the S3 client. @tom-henderson, can you help with that?
  • Add SMTP TLS report output support to the Syslog client. @chris-y, can you help with this?
  • Add SMTP TLS report dashboards for Splunk. I should be able help with that once I get more failure examples.
  • Add SMTP TLS report to the Kibana dashboard. I haven't had ELK set up for a while, so if someone else could contribute dashboard changes would be awesome.
  • Add SMTP TLS report support to the Grafana dashboard.
  • Add SMTP TLS report samples and tests

@seanthegeek
Copy link
Contributor Author

I was able to reuse existing code to create SMTP TLS report output support for Kafka, S3, Log Analytics, and Syslog. I have not tested any of these and testing would be appreciated.

@rubeste
Copy link
Contributor

rubeste commented Jan 3, 2024

@seanthegeek I'll see if I can test it for Log Analytics. But I first need an SMTP TLS report.

parsedmarc/loganalytics.py Outdated Show resolved Hide resolved
@mikesiegel
Copy link
Contributor

Hi @seanthegeek

I haven't worked in a Kafka environment/dealing with DMARC since 2020. I can reach out to some old coworkers though and see if they can help out.

@tom-henderson
Copy link
Contributor

Hi @seanthegeek it's been a while since I've looked at this but the changes in s3.py look good to me 👍

@chris-y
Copy link
Contributor

chris-y commented Jan 8, 2024

As far as I can tell the syslog looks ok, I'm not able to test though.

@rubeste
Copy link
Contributor

rubeste commented Jan 10, 2024

@seanthegeek Created a Pull request #459 with some fixes and documentation.

@rubeste
Copy link
Contributor

rubeste commented Jan 10, 2024

I have also tested the application and it works now.

@seanthegeek seanthegeek merged commit b808850 into master Feb 19, 2024
1 of 3 checks passed
@seanthegeek seanthegeek deleted the smtp-tls branch February 19, 2024 23:46
seanthegeek added a commit that referenced this pull request Feb 20, 2024
- Add support for SMTP TLS reports (PR #453 closes issue #71)
- Do not replace content in forensic samples (fix #403)
- Pin `msgraph-core` dependency at version `0.2.2` until Microsoft provides better documentation (PR #466 Close [#464](#464))
- Properly handle base64-encoded email attachments (PR #453)
- Do not crash when attempting to parse invalid email content (PR #453)
- Ignore errors when parsing text-based forensic reports (PR #460)
- Add email date to email processing debug logs (PR #462)
- Set default batch size to 10 to match the documentation (PR #465)
- Properly handle none values (PR #468)
- Add Gmail pagination (PR #469)
- Use the correct `msgraph` scope (PR #471)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants