Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NETOBSERV-1471 add gRPC write stage #621

Merged
merged 2 commits into from
Mar 12, 2024
Merged

NETOBSERV-1471 add gRPC write stage #621

merged 2 commits into from
Mar 12, 2024

Conversation

jpinsonneau
Copy link
Collaborator

@jpinsonneau jpinsonneau commented Mar 4, 2024 •
edited
Loading

Description

This PR adds gRPC write stage to FLP.
The alternative tcp PR is also available

Dependencies

n/a

Checklist

If you are not familiar with our processes or don't know what to answer in the list below, let us know in a comment: the maintainers will take care of that.

  • Will this change affect NetObserv / Network Observability operator? If not, you can ignore the rest of this checklist.
  • Is this PR backed with a JIRA ticket? If so, make sure it is written as a title prefix (in general, PRs affecting the NetObserv/Network Observability product should be backed with a JIRA ticket - especially if they bring user facing changes).
  • Does this PR require product documentation?
    • If so, make sure the JIRA epic is labelled with "documentation" and provides a description relevant for doc writers, such as use cases or scenarios. Any required step to activate or configure the feature should be documented there, such as new CRD knobs.
  • Does this PR require a product release notes entry?
    • If so, fill in "Release Note Text" in the JIRA.
  • Is there anything else the QE team should know before testing? E.g: configuration changes, environment setup, etc.
    • If so, make sure it is described in the JIRA ticket.
  • QE requirements (check 1 from the list):
    • Standard QE validation, with pre-merge tests unless stated otherwise.
    • Regression tests only (e.g. refactoring with no user-facing change).
    • No QE (e.g. trivial change with high reviewer's confidence, or per agreement with the QE team).

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Mar 4, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@jpinsonneau jpinsonneau mentioned this pull request Mar 4, 2024
Closed
11 tasks
@jpinsonneau jpinsonneau changed the title NETOBSERV-1471 add gRPC write stage WIP NETOBSERV-1471 add gRPC write stage Mar 4, 2024
@jpinsonneau jpinsonneau mentioned this pull request Mar 4, 2024
Merged
@codecov Codecov
Copy link

codecov bot commented Mar 4, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 61.86047% with 82 lines in your changes are missing coverage. Please review.

Project coverage is 67.07%. Comparing base (ab65bc4) to head (bc514f2).
Report is 11 commits behind head on main.

Files Patch % Lines
...kg/pipeline/write/grpc/genericmap/genericmap.pb.go 52.08% 44 Missing and 2 partials ⚠️
pkg/api/write_grpc.go 18.18% 6 Missing and 3 partials ⚠️
pkg/pipeline/write/grpc/server.go 73.33% 6 Missing and 2 partials ⚠️
...peline/write/grpc/genericmap/genericmap_grpc.pb.go 80.00% 5 Missing and 1 partial ⚠️
pkg/pipeline/write/write_grpc.go 80.00% 4 Missing and 2 partials ⚠️
pkg/pipeline/write/grpc/client.go 68.75% 4 Missing and 1 partial ⚠️
pkg/pipeline/pipeline_builder.go 0.00% 2 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #621      +/-   ##
==========================================
+ Coverage   66.97%   67.07%   +0.09%     
==========================================
  Files         104      110       +6     
  Lines        7446     7674     +228     
==========================================
+ Hits         4987     5147     +160     
- Misses       2162     2214      +52     
- Partials      297      313      +16
Flag Coverage Δ
unittests 67.07% <61.86%> (+0.09%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jpinsonneau jpinsonneau marked this pull request as ready for review March 6, 2024 11:28
@jpinsonneau jpinsonneau changed the title WIP NETOBSERV-1471 add gRPC write stage NETOBSERV-1471 add gRPC write stage Mar 6, 2024
jotak
jotak reviewed Mar 11, 2024
View reviewed changes
pkg/pipeline/write/grpc/server.go
@@ -0,0 +1,77 @@
package grpc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like this grpc server is only used for test, so it could be moved under the test package? Or is it meant to be imported from other projects?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed I'm reusing this into CLI:
https://github.com/netobserv/network-observability-cli/blob/c7615024c987813d3368921f99c598edafd4feb8/cmd/flow-capture.go#L89-L94

We do the same in FLP from eBPF agent gRPC definition:

flowlogs-pipeline/pkg/pipeline/ingest/ingest_grpc.go

Lines 48 to 49 in df307e0

collector, err := grpc.StartCollector(netObserv.Port, flowPackets,
grpc.WithGRPCServerOptions(grpc2.UnaryInterceptor(instrumentGRPC(metrics))))

@jotak
Copy link
Member

jotak commented Mar 12, 2024

Thanks @jpinsonneau
/lgtm

@openshift-ci openshift-ci bot added the lgtm label Mar 12, 2024
@jpinsonneau
Copy link
Collaborator Author

I'm merging this since only CLI will us it for now; no impact on existing features

@openshift-ci OpenShift CI
Copy link

openshift-ci bot commented Mar 12, 2024

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot bot merged commit ddc1f67 into netobserv:main Mar 12, 2024
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jotak jotak jotak left review comments

@msherif1234 msherif1234 Awaiting requested review from msherif1234

@OlivierCazade OlivierCazade Awaiting requested review from OlivierCazade

Assignees

@jotak jotak

Labels
approved lgtm
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@jpinsonneau @jotak @msherif1234 @OlivierCazade