Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connector request service and mapping problem selectors #6514

Open
wants to merge 27 commits into
base: next
Choose a base branch
from

Conversation

pubmodmatt
Copy link

@pubmodmatt pubmodmatt commented Jan 6, 2025

Initial implementation of a Connector Request Service, which handles individual requests to an Apollo Connector.

The Request and Response types for this new service contain information about the Connector, as well as how the request or response was mapped, including any mapping problems.

The Connector Service makes one or more calls to the Connector Request Service, and the Connector Request Service in turn makes one call to the underlying connector transport, currently HTTP through the HTTP Client Service.

The transport details are separated out into separate TransportRequest and TransportResponse enum variants. Currently, there is only an Http variant, but this allows for easily expanding to more transport variants in the future without modifying the Request and Response types.

The Connector Request Service contributes a connector_request_service method to the PluginPrivate trait. This method is implemented by the telemetry and authentication plugins.

New selectors are added to the telemetry plugin to allow selection of mapping problems from the request and response.

Existing connectors selectors have been re-implemented at the connector_request_service level. This resolves an issue where using the HTTP response selectors would previously cause the data for the last HTTP response to be added to the connect span. The response data will now correctly be added to each connect_request span.

Standard events are now emitted correctly and contain the http.request.body (for request events) and http.response.body (for response events).


Checklist

Complete the checklist (and note appropriate exceptions) before the PR is marked ready-for-review.

  • Changes are compatible1
  • Documentation2 completed
  • Performance impact assessed and acceptable
  • Tests added and passing3
    • Unit Tests
    • Integration Tests
    • Manual Tests

Exceptions

Note any exceptions here

Notes

Footnotes

  1. It may be appropriate to bring upcoming changes to the attention of other (impacted) groups. Please endeavour to do this before seeking PR approval. The mechanism for doing this will vary considerably, so use your judgement as to how and when to do this.

  2. Configuration is an important part of many changes. Where applicable please try to document configuration examples.

  3. Tick whichever testing boxes are applicable. If you are adding Manual Tests, please document the manual testing (extensively) in the Exceptions.

@svc-apollo-docs
Copy link
Collaborator

svc-apollo-docs commented Jan 6, 2025

✅ Docs preview has no changes

The preview was not built because there were no changes.

Build ID: 1d47657cb547d8dafd21dad4

@router-perf
Copy link

router-perf bot commented Jan 6, 2025

CI performance tests

  • connectors-const - Connectors stress test that runs with a constant number of users
  • const - Basic stress test that runs with a constant number of users
  • demand-control-instrumented - A copy of the step test, but with demand control monitoring and metrics enabled
  • demand-control-uninstrumented - A copy of the step test, but with demand control monitoring enabled
  • enhanced-signature - Enhanced signature enabled
  • events - Stress test for events with a lot of users and deduplication ENABLED
  • events_big_cap_high_rate - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity
  • events_big_cap_high_rate_callback - Stress test for events with a lot of users, deduplication enabled and high rate event with a big queue capacity using callback mode
  • events_callback - Stress test for events with a lot of users and deduplication ENABLED in callback mode
  • events_without_dedup - Stress test for events with a lot of users and deduplication DISABLED
  • events_without_dedup_callback - Stress test for events with a lot of users and deduplication DISABLED using callback mode
  • extended-reference-mode - Extended reference mode enabled
  • large-request - Stress test with a 1 MB request payload
  • no-tracing - Basic stress test, no tracing
  • reload - Reload test over a long period of time at a constant rate of users
  • step-jemalloc-tuning - Clone of the basic stress test for jemalloc tuning
  • step-local-metrics - Field stats that are generated from the router rather than FTV1
  • step-with-prometheus - A copy of the step test with the Prometheus metrics exporter enabled
  • step - Basic stress test that steps up the number of users over time
  • xlarge-request - Stress test with 10 MB request payload
  • xxlarge-request - Stress test with 100 MB request payload

@pubmodmatt pubmodmatt force-pushed the pubmodmatt/connectors/request_service branch from 476883d to bf8cd27 Compare January 8, 2025 21:07
@pubmodmatt pubmodmatt self-assigned this Jan 9, 2025
@pubmodmatt pubmodmatt changed the title PoC implementation of connector request service and mapping problem selectors Connector request service and mapping problem selectors Jan 10, 2025
@pubmodmatt pubmodmatt marked this pull request as ready for review January 11, 2025 01:40
@pubmodmatt pubmodmatt requested review from a team as code owners January 11, 2025 01:40
@pubmodmatt pubmodmatt removed request for a team January 11, 2025 01:40
Copy link
Contributor

@garypen garypen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to setup a sync up meeting to bring this in line with the back-pressure changes we are making to the router for 2.0.

I don't think it's a huge amount of work, but talking through the required changes and answering questions about why the changes are important will help us make faster progress.

Copy link

mergify bot commented Jan 17, 2025

Merge Protections

Your pull request matches the following merge protections and will not be merged until they are valid.

🟠 ❄️ Ensure queue is not active [Scheduled Freeze]

Waiting for scheduled freeze to end.

A freeze on the repository is scheduled for the following reason: Ensure queue is not active

  • current-datetime < 2025-01-17T23:40:20.870992[Europe/London]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants