Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EPSS enricher #1439

Merged
merged 5 commits into from
Dec 17, 2024
Merged

EPSS enricher #1439

merged 5 commits into from
Dec 17, 2024

Conversation

daynewlee
Copy link
Contributor

@daynewlee daynewlee commented Nov 12, 2024

Epss enricher should be able to update, parse EPSS data and enrich vulnerability report with the data

@daynewlee daynewlee force-pushed the yli3/epssUpdater branch 6 times, most recently from e876194 to 32941b1 Compare November 13, 2024 18:49

const (
// Type is the type of data returned from the Enricher's Enrich method.
Type = `message/vnd.clair.map.vulnerability; enricher=clair.epss schema=https://csrc.nist.gov/schema/nvd/feed/1.1/cvss-v3.x.json`
Copy link
Contributor Author

@daynewlee daynewlee Nov 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

converting CSV file to json, not sure what the Type should look like.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we need to point to an actual schema here

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any suggestions? Do we need to define a schema in the enrichment package?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if there is an equivalent for EPSS. @crozzy what do you suggest to do here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be the JSON schema of the EPSSItem when it is marshaled but I'm not entirely sure where would be a good place for it, any ideas @hdonnay ?

@daynewlee daynewlee force-pushed the yli3/epssUpdater branch 3 times, most recently from f5058ef to 7fbe5d3 Compare November 14, 2024 22:01
@daynewlee daynewlee changed the title EPSS updater EPSS enricher Nov 15, 2024
@daynewlee daynewlee force-pushed the yli3/epssUpdater branch 2 times, most recently from 34f4e08 to 5f69717 Compare November 17, 2024 21:32
@daynewlee daynewlee marked this pull request as ready for review November 17, 2024 21:33
@daynewlee daynewlee requested a review from a team as a code owner November 17, 2024 21:33
@daynewlee daynewlee requested review from crozzy, RTann and jvdm and removed request for a team November 17, 2024 21:33
@daynewlee daynewlee changed the title EPSS enricher ROX-27045: EPSS enricher Nov 19, 2024
@daynewlee daynewlee changed the title ROX-27045: EPSS enricher EPSS enricher Nov 19, 2024
@daynewlee daynewlee requested a review from dcaravel November 20, 2024 00:47
@daynewlee
Copy link
Contributor Author

As of Nov 20th, we decide to include model version and score date in the enrichment data

enricher/epss/epss.go Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
go.mod Outdated Show resolved Hide resolved
enricher/epss/epss.go Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss_test.go Show resolved Hide resolved
@daynewlee daynewlee force-pushed the yli3/epssUpdater branch 3 times, most recently from 9525df0 to 6bdb39d Compare November 22, 2024 16:38
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved

const (
// Type is the type of data returned from the Enricher's Enrich method.
Type = `message/vnd.clair.map.vulnerability; enricher=clair.epss schema=https://csrc.nist.gov/schema/nvd/feed/1.1/cvss-v3.x.json`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we need to point to an actual schema here

enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
@crozzy
Copy link
Contributor

crozzy commented Dec 4, 2024

Most everything I brought up seems to be addressed (apart from the new comments)

enricher/constants.go Outdated Show resolved Hide resolved
enricher/constants.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Show resolved Hide resolved

const (
// Type is the type of data returned from the Enricher's Enrich method.
Type = `message/vnd.clair.map.vulnerability; enricher=clair.epss schema=https://csrc.nist.gov/schema/nvd/feed/1.1/cvss-v3.x.json`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if there is an equivalent for EPSS. @crozzy what do you suggest to do here?

enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss_test.go Outdated Show resolved Hide resolved
@daynewlee daynewlee force-pushed the yli3/epssUpdater branch 9 times, most recently from b41b7b1 to 9fa70d3 Compare December 6, 2024 16:13
@daynewlee daynewlee requested review from RTann and crozzy December 6, 2024 16:52
enricher/epss/epss.go Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Show resolved Hide resolved
enricher/epss/epss.go Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
Int("count", len(ret)).
Msg("decoded enrichments")

if !errors.Is(err, io.EOF) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just confirming, we expect an EOF once we exhaust all records?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I think so.

@daynewlee daynewlee force-pushed the yli3/epssUpdater branch 2 times, most recently from e7cf505 to f93b034 Compare December 9, 2024 23:06
@daynewlee daynewlee requested a review from RTann December 9, 2024 23:19
Copy link
Contributor

@crozzy crozzy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of small things

enricher/epss/epss.go Outdated Show resolved Hide resolved
enricher/epss/epss.go Outdated Show resolved Hide resolved
@daynewlee daynewlee merged commit 82f8e60 into quay:main Dec 17, 2024
6 checks passed
@daynewlee daynewlee deleted the yli3/epssUpdater branch December 17, 2024 18:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

3 participants