Skip to content

Commit

Permalink
ref(nextjs): Use integration to add request data to transaction events (
Browse files Browse the repository at this point in the history
#5703)

In most of our Node-based SDKs, we use domains to prevent scope bleed between requests, running the entire wrapped event-handling process inside a single domain. This works because in those cases, there is only one request-handling entry point for us to wrap, so we can stick the entire thing in a single `domain.run()` call and (more or less) rely on the fact that we've got a single, consistent, unique-to-that-request `Scope` object that we're dealing with. We've followed this pattern in the nextjs SDK as well, both in `instrumentServer` and `withSentry`. The new auto-wrapping of data-fetching functions presents a challenge, though, because a single request may involve a number of our wrapped functions running at different points in the request handling process, with no shared (little S) scope between them. While we still use domains when wrapping the data fetchers, a single request may pass through multiple domains during its lifecycle, preventing us from using any given domain as a universal `Scope`-carrier for a single `Scope` instance.

One place where this becomes is problem is our use of `addRequestDataToEvent`, which up until now we've just been calling inside a single-request event processor added to the current `Scope`. In this system, each event processor holds a reference to its particular `req` object. In the case of the data-fetchers, however, the `Scope` instance to which we might add such an event processor isn't the same as the one which will be active when the event is being processed, so the current way doesn't work. But given that the only way in which our current, single-request-specific event processors differ is by the request to which they hold a reference, they can be replaced by a single, and universal, event processor, as long as we can access `req` a different way besides keeping it in a closure as we do now.

This PR does just that (that is, switches to using a single event processor) for transaction events. First, a reference to `req` is stored in the transaction's metadata (which is then available to event processors as `sdkProcessingMetadata`). Then a new default integration, `RequestData`, pulls `req` out of the metadata and uses it to add a `request` property to the event. 

Notes:

- The options API for the new integration is inspired by, but different from, the options API for our Express request handler. (When we work on cleaning up the request data utility functions as part of fixing #5718, we might consider bringing those options in line with these.) The primary differences are:
  - The options have been (almost entirely) flattened. (It never made sense that inside of options for request data, you had a `request` key holding a subset of the options.) Now everything is at the same level and only takes a boolean. The one exception is `user`, which can still take a boolean or a list of attributes.
  - In the handler options, `transaction` can either be a boolean or a `TransactionNamingScheme`. In the integration, it can no longer be a boolean - events are going to have transactions, one way or another, so we shouldn't let people set it to `false`. Further, since it's now not about whether `transaction` is or isn't included, it's been moved out of `include` into its own `transactionNamingScheme` option. (Note that the new option is not yet used, but will be in future PRs.)
  - `method` has also been moved out of include, and has been hardcoded to `true`, since it's an integral part of naming the request. We currently include it in the transaction name, regardless of the setting, so again here, letting people set it to `false` makes no sense.

- Though `req` has been added to transaction metadata everywhere we start transactions, the existing domain/`Scope`-based event processors haven't yet been removed, because this new method only works for transactions, not errors. (Solving that will be the subject of a future PR.) The existing processors have been modified to not apply to transaction events, however.

- Though we may at some point use the `RequestData` integration added here in browser-based SDKs as well, both in this PR and in near-term future PRs it will only be used in a Node context. It's therefore been added to `@sentry/node`, to prevent a repeat of the dependency injection mess [we just undid](#5759).

- No integration tests were added specifically for this change, because the existing integration tests already test that transaction events include a `request` property, so as long as they continue to pass, we know that using the integration instead of an event processor is working.

Ref: #5505
  • Loading branch information
lobsterkatie authored Sep 19, 2022
1 parent c072b89 commit 5660d9f
Show file tree
Hide file tree
Showing 12 changed files with 235 additions and 14 deletions.
22 changes: 14 additions & 8 deletions packages/nextjs/src/config/wrappers/wrapperUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,11 @@ export function callTracedServerSideDataFetcher<F extends (...args: any[]) => Pr

requestTransaction = newTransaction;
autoEndTransactionOnResponseEnd(newTransaction, res);

// Link the transaction and the request together, so that when we would normally only have access to one, it's
// still possible to grab the other.
setTransactionOnRequest(newTransaction, req);
newTransaction.setMetadata({ request: req });
}

const dataFetcherSpan = requestTransaction.startChild({
Expand All @@ -118,14 +122,16 @@ export function callTracedServerSideDataFetcher<F extends (...args: any[]) => Pr
if (currentScope) {
currentScope.setSpan(dataFetcherSpan);
currentScope.addEventProcessor(event =>
addRequestDataToEvent(event, req, {
include: {
// When the `transaction` option is set to true, it tries to extract a transaction name from the request
// object. We don't want this since we already have a high-quality transaction name with a parameterized
// route. Setting `transaction` to `true` will clobber that transaction name.
transaction: false,
},
}),
event.type !== 'transaction'
? addRequestDataToEvent(event, req, {
include: {
// When the `transaction` option is set to true, it tries to extract a transaction name from the request
// object. We don't want this since we already have a high-quality transaction name with a parameterized
// route. Setting `transaction` to `true` will clobber that transaction name.
transaction: false,
},
})
: event,
);
}

Expand Down
3 changes: 3 additions & 0 deletions packages/nextjs/src/index.server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,9 @@ function addServerIntegrations(options: NextjsOptions): void {
});
integrations = addOrUpdateIntegration(defaultRewriteFramesIntegration, integrations);

const defaultRequestDataIntegration = new Integrations.RequestData();
integrations = addOrUpdateIntegration(defaultRequestDataIntegration, integrations);

if (hasTracingEnabled(options)) {
const defaultHttpTracingIntegration = new Integrations.Http({ tracing: true });
integrations = addOrUpdateIntegration(defaultHttpTracingIntegration, integrations, {
Expand Down
5 changes: 4 additions & 1 deletion packages/nextjs/src/utils/instrumentServer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -244,7 +244,9 @@ function makeWrappedReqHandler(origReqHandler: ReqHandler): WrappedReqHandler {
const currentScope = getCurrentHub().getScope();

if (currentScope) {
currentScope.addEventProcessor(event => addRequestDataToEvent(event, nextReq));
currentScope.addEventProcessor(event =>
event.type !== 'transaction' ? addRequestDataToEvent(event, nextReq) : event,
);

// We only want to record page and API requests
if (hasTracingEnabled() && shouldTraceRequest(nextReq.url, publicDirFiles)) {
Expand Down Expand Up @@ -276,6 +278,7 @@ function makeWrappedReqHandler(origReqHandler: ReqHandler): WrappedReqHandler {
// like `source: isDynamicRoute? 'url' : 'route'`
// TODO: What happens when `withSentry` is used also? Which values of `name` and `source` win?
source: 'url',
request: req,
},
...traceparentData,
},
Expand Down
5 changes: 4 additions & 1 deletion packages/nextjs/src/utils/withSentry.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,9 @@ export const withSentry = (origHandler: NextApiHandler): WrappedNextApiHandler =
const currentScope = getCurrentHub().getScope();

if (currentScope) {
currentScope.addEventProcessor(event => addRequestDataToEvent(event, req));
currentScope.addEventProcessor(event =>
event.type !== 'transaction' ? addRequestDataToEvent(event, req) : event,
);

if (hasTracingEnabled()) {
// If there is a trace header set, extract the data from it (parentSpanId, traceId, and sampling decision)
Expand Down Expand Up @@ -90,6 +92,7 @@ export const withSentry = (origHandler: NextApiHandler): WrappedNextApiHandler =
metadata: {
dynamicSamplingContext: traceparentData && !dynamicSamplingContext ? {} : dynamicSamplingContext,
source: 'route',
request: req,
},
},
// extra context passed to the `tracesSampler`
Expand Down
70 changes: 70 additions & 0 deletions packages/nextjs/test/config/wrappers.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
import * as SentryCore from '@sentry/core';
import * as SentryTracing from '@sentry/tracing';
import { IncomingMessage, ServerResponse } from 'http';

import {
withSentryGetServerSideProps,
withSentryServerSideGetInitialProps,
// TODO: Leaving `withSentryGetStaticProps` out for now until we figure out what to do with it
// withSentryGetStaticProps,
// TODO: Leaving these out for now until we figure out pages with no data fetchers
// withSentryServerSideAppGetInitialProps,
// withSentryServerSideDocumentGetInitialProps,
// withSentryServerSideErrorGetInitialProps,
} from '../../src/config/wrappers';

const startTransactionSpy = jest.spyOn(SentryCore, 'startTransaction');
const setMetadataSpy = jest.spyOn(SentryTracing.Transaction.prototype, 'setMetadata');

describe('data-fetching function wrappers', () => {
const route = '/tricks/[trickName]';
let req: IncomingMessage;
let res: ServerResponse;

describe('starts a transaction and puts request in metadata if tracing enabled', () => {
beforeEach(() => {
req = { headers: {}, url: 'http://dogs.are.great/tricks/kangaroo' } as IncomingMessage;
res = {} as ServerResponse;

jest.spyOn(SentryTracing, 'hasTracingEnabled').mockReturnValueOnce(true);
});

afterEach(() => {
jest.clearAllMocks();
});

test('withSentryGetServerSideProps', async () => {
const origFunction = jest.fn(async () => ({ props: {} }));

const wrappedOriginal = withSentryGetServerSideProps(origFunction, route);
await wrappedOriginal({ req, res } as any);

expect(startTransactionSpy).toHaveBeenCalledWith(
expect.objectContaining({
name: '/tricks/[trickName]',
op: 'nextjs.data.server',
metadata: expect.objectContaining({ source: 'route' }),
}),
);

expect(setMetadataSpy).toHaveBeenCalledWith({ request: req });
});

test('withSentryServerSideGetInitialProps', async () => {
const origFunction = jest.fn(async () => ({}));

const wrappedOriginal = withSentryServerSideGetInitialProps(origFunction);
await wrappedOriginal({ req, res, pathname: route } as any);

expect(startTransactionSpy).toHaveBeenCalledWith(
expect.objectContaining({
name: '/tricks/[trickName]',
op: 'nextjs.data.server',
metadata: expect.objectContaining({ source: 'route' }),
}),
);

expect(setMetadataSpy).toHaveBeenCalledWith({ request: req });
});
});
});
2 changes: 2 additions & 0 deletions packages/nextjs/test/index.server.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -150,8 +150,10 @@ describe('Server init()', () => {

const nodeInitOptions = nodeInit.mock.calls[0][0] as ModifiedInitOptions;
const rewriteFramesIntegration = findIntegrationByName(nodeInitOptions.integrations, 'RewriteFrames');
const requestDataIntegration = findIntegrationByName(nodeInitOptions.integrations, 'RequestData');

expect(rewriteFramesIntegration).toBeDefined();
expect(requestDataIntegration).toBeDefined();
});

it('supports passing unrelated integrations through options', () => {
Expand Down
3 changes: 2 additions & 1 deletion packages/nextjs/test/utils/withSentry.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ describe('withSentry', () => {
});

describe('tracing', () => {
it('starts a transaction when tracing is enabled', async () => {
it('starts a transaction and sets metadata when tracing is enabled', async () => {
jest
.spyOn(hub.Hub.prototype, 'getClient')
.mockReturnValueOnce({ getOptions: () => ({ tracesSampleRate: 1 } as ClientOptions) } as Client);
Expand All @@ -118,6 +118,7 @@ describe('withSentry', () => {

metadata: {
source: 'route',
request: expect.objectContaining({ url: 'http://dogs.are.great' }),
},
},
{ request: expect.objectContaining({ url: 'http://dogs.are.great' }) },
Expand Down
3 changes: 2 additions & 1 deletion packages/node/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ export type {
} from '@sentry/types';
export type { AddRequestDataToEventOptions } from '@sentry/utils';

export type { TransactionNamingScheme } from './requestdata';
export type { NodeOptions } from './types';

export {
Expand Down Expand Up @@ -47,7 +48,7 @@ export {
export { NodeClient } from './client';
export { makeNodeTransport } from './transports';
export { defaultIntegrations, init, defaultStackParser, lastEventId, flush, close, getSentryRelease } from './sdk';
export { addRequestDataToEvent, extractRequestData } from './requestdata';
export { addRequestDataToEvent, DEFAULT_USER_INCLUDES, extractRequestData } from './requestdata';
export { deepReadDirSync } from './utils';

import { Integrations as CoreIntegrations } from '@sentry/core';
Expand Down
1 change: 1 addition & 0 deletions packages/node/src/integrations/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ export { LinkedErrors } from './linkederrors';
export { Modules } from './modules';
export { ContextLines } from './contextlines';
export { Context } from './context';
export { RequestData } from './requestdata';
125 changes: 125 additions & 0 deletions packages/node/src/integrations/requestdata.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
// TODO (v8 or v9): Whenever this becomes a default integration for `@sentry/browser`, move this to `@sentry/core`. For
// now, we leave it in `@sentry/integrations` so that it doesn't contribute bytes to our CDN bundles.

import { EventProcessor, Hub, Integration } from '@sentry/types';

import {
addRequestDataToEvent,
AddRequestDataToEventOptions,
DEFAULT_USER_INCLUDES,
TransactionNamingScheme,
} from '../requestdata';

type RequestDataOptions = {
/**
* Controls what data is pulled from the request and added to the event
*/
include: {
cookies?: boolean;
data?: boolean;
headers?: boolean;
ip?: boolean;
query_string?: boolean;
url?: boolean;
user?: boolean | Array<typeof DEFAULT_USER_INCLUDES[number]>;
};

/** Whether to identify transactions by parameterized path, parameterized path with method, or handler name */
transactionNamingScheme: TransactionNamingScheme;

/**
* Function for adding request data to event. Defaults to `addRequestDataToEvent` from `@sentry/node` for now, but
* left injectable so this integration can be moved to `@sentry/core` and used in browser-based SDKs in the future.
*
* @hidden
*/
addRequestData: typeof addRequestDataToEvent;
};

const DEFAULT_OPTIONS = {
addRequestData: addRequestDataToEvent,
include: {
cookies: true,
data: true,
headers: true,
ip: false,
query_string: true,
url: true,
user: DEFAULT_USER_INCLUDES,
},
transactionNamingScheme: 'methodpath',
};

/** Add data about a request to an event. Primarily for use in Node-based SDKs, but included in `@sentry/integrations`
* so it can be used in cross-platform SDKs like `@sentry/nextjs`. */
export class RequestData implements Integration {
/**
* @inheritDoc
*/
public static id: string = 'RequestData';

/**
* @inheritDoc
*/
public name: string = RequestData.id;

private _options: RequestDataOptions;

/**
* @inheritDoc
*/
public constructor(options: Partial<RequestDataOptions> = {}) {
this._options = {
...DEFAULT_OPTIONS,
...options,
include: {
// @ts-ignore It's mad because `method` isn't a known `include` key. (It's only here and not set by default in
// `addRequestDataToEvent` for legacy reasons. TODO (v8): Change that.)
method: true,
...DEFAULT_OPTIONS.include,
...options.include,
},
};
}

/**
* @inheritDoc
*/
public setupOnce(addGlobalEventProcessor: (eventProcessor: EventProcessor) => void, getCurrentHub: () => Hub): void {
const { include, addRequestData } = this._options;

addGlobalEventProcessor(event => {
const self = getCurrentHub().getIntegration(RequestData);
const req = event.sdkProcessingMetadata && event.sdkProcessingMetadata.request;

// If the globally installed instance of this integration isn't associated with the current hub, `self` will be
// undefined
if (!self || !req) {
return event;
}

return addRequestData(event, req, { include: formatIncludeOption(include) });
});
}
}

/** Convert `include` option to match what `addRequestDataToEvent` expects */
/** TODO: Can possibly be deleted once https://github.com/getsentry/sentry-javascript/issues/5718 is fixed */
function formatIncludeOption(
integrationInclude: RequestDataOptions['include'] = {},
): AddRequestDataToEventOptions['include'] {
const { ip, user, ...requestOptions } = integrationInclude;

const requestIncludeKeys: string[] = [];
for (const [key, value] of Object.entries(requestOptions)) {
if (value) {
requestIncludeKeys.push(key);
}
}

return {
ip,
user,
request: requestIncludeKeys.length !== 0 ? requestIncludeKeys : undefined,
};
}
4 changes: 2 additions & 2 deletions packages/node/src/requestdata.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ const DEFAULT_INCLUDES = {
user: true,
};
const DEFAULT_REQUEST_INCLUDES = ['cookies', 'data', 'headers', 'method', 'query_string', 'url'];
const DEFAULT_USER_INCLUDES = ['id', 'username', 'email'];
export const DEFAULT_USER_INCLUDES = ['id', 'username', 'email'];

/**
* Options deciding what parts of the request to use when enhancing an event
Expand All @@ -25,7 +25,7 @@ export interface AddRequestDataToEventOptions {
};
}

type TransactionNamingScheme = 'path' | 'methodPath' | 'handler';
export type TransactionNamingScheme = 'path' | 'methodPath' | 'handler';

/**
* Sets parameterized route as transaction name e.g.: `GET /users/:id`
Expand Down
6 changes: 6 additions & 0 deletions packages/types/src/transaction.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import { DynamicSamplingContext } from './envelope';
import { MeasurementUnit } from './measurement';
import { ExtractedNodeRequestData, Primitive, WorkerLocation } from './misc';
import { PolymorphicRequest } from './polymorphics';
import { Span, SpanContext } from './span';

/**
* Interface holding Transaction-specific properties
*/
Expand Down Expand Up @@ -145,7 +147,11 @@ export interface TransactionMetadata {
*/
dynamicSamplingContext?: Partial<DynamicSamplingContext>;

/** For transactions tracing server-side request handling, the request being tracked. */
request?: PolymorphicRequest;

/** For transactions tracing server-side request handling, the path of the request being tracked. */
/** TODO: If we rm -rf `instrumentServer`, this can go, too */
requestPath?: string;

/** Information on how a transaction name was generated. */
Expand Down

0 comments on commit 5660d9f

Please sign in to comment.