Skip to content

Implemented tracing support for Windows with ETW. #39

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Implemented tracing support for Windows with ETW. #39

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .npmignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
.lock-wscript
*.node
.lock-wscript
*~
.#*
*#
Expand Down
250 changes: 229 additions & 21 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@
This extension allows you to create native DTrace providers for your
Node.js applications. That is, to create providers and probes which
expose information specific to your application, rather than
information about the node runtime.
information about the node runtime. The extension also supports Win32
and uses ETW WinAPI to implement similar functionality provided by
DTrace on non-win platforms. That is why using this addon under Win32
is different from the other platforms.

You could use this to expose high-level information about the inner
workings of your application, or to create a specific context in which
Expand All @@ -19,13 +22,79 @@ need not run as root.

$ npm install dtrace-provider

*Windows-only:*

The ETW code uses some C++11 features and requires at least VS2012 for it.
To specify the version, set the appropriate environment variable.

GYP_MSVS_VERSION=2012

It must be set before invoking npm install or creating a project.

Creating the Visual Studio project files is done using node-gyp. This is a
node add-on for creating the appropriate builds for your installation.
To create a Visual Studio solution:

node-gyp configure

The Windows implementation supports Node.js 0.11 as well as previous releases.
To build the addon for a release version below 0.11, uncomment

#define BUILD_PRE011_COMPATIBILITY

in

src/etw_win/v8_compatibility.h

## PLATFORM-SPECIFIC INFORMATION

On the system API level, providers are created and controlled differently
on Windows and the other supported platforms.
For the platforms supporting DTrace the user needs to supply the names for
the provider and the module to create the provider, while Windows requires
a GUID which is also used to control the tracing session.

To handle this difference, there are two ways to create the provider.
Both options can be used for all of the supported platforms.

*Option 1 (compatibility way):*

var d = require('dtrace-provider');
var dtp = d.createDTraceProvider("provider", "module");

The createDTraceProvider function takes the name of the provider as the
first argument; the second argument is an optional module name. Both
arguments are strings. This is a compatibility function left to allow
the existing node apps using this extension to be able to work without
modifying their code.

*Option 2 (cross-platform way):*

var d = require('dtrace-provider');
var provider = d.createTraceProvider({provider_name: 'provider', module_name: 'module', guid: '5A391F32-A079-42E1-97B1-8A45295AA1FD'});

The createTraceProvider function takes an object with the above-written
properties. The user can be specific about what information they want to
supply to the function. In the situation when the information required
by the platform has not been passed in, it will be generated from what
has been given.

That is why the extension also exposes two helper functions: guidFromNames
and namesFromGuid exposed on Windows and the other platforms respectively. These
functions use the same algorithms that are used by createTraceProvider and
can provide the neccessary information to control the session.

See the following files for the sample code.

test_crossplatform/guid-from-names.js
test_dtrace/names-from-guid.js

## EXAMPLE

Here's a simple example of creating a provider:

var d = require('dtrace-provider');

var dtp = d.createDTraceProvider("nodeapp");
var dtp = d.createTraceProvider({provider_name: 'nodeapp'});
var p1 = dtp.addProbe("probe1", "int", "int");
var p2 = dtp.addProbe("probe2", "char *");
dtp.enable();
Expand All @@ -48,11 +117,23 @@ or via the probe objects themselves:
return ["hello, dtrace via probe", "foo"];
});

This example creates a provider called "nodeapp", and adds two
The example above creates a provider called "nodeapp", and adds two
probes. It then enables the provider, at which point the provider
becomes visible to DTrace.

The probes are then fired, which produces this output:
The probes are then fired.

## GENERAL INFORMATION

Available on all of the supported platforms argument types are "int", for
integer numeric values, "char *" for strings, and "json" for objects rendered into
JSON strings. Arguments typed as "json" will be created as "char *" probes in
the provider, but objects passed to these probe arguments will be automatically
serialized to JSON before being passed forward to the system API.

## DTRACE INFORMATION

Firing the probes from the example above produces this output:

$ sudo dtrace -Z -n 'nodeapp*:::probe1{ trace(arg0); trace(arg1) }' \
-n 'nodeapp*:::probe2{ trace(copyinstr(arg0)); }'
Expand All @@ -67,14 +148,8 @@ enabled. This means you can do more expensive work to gather arguments.

The maximum number of arguments supported is 32.

Available argument types are "int", for integer numeric values,
"char *" for strings, and "json" for objects rendered into JSON strings.

Arguments typed as "json" will be created as "char *" probes in
DTrace, but objects passed to these probe arguments will be
automatically serialized to JSON before being passed to DTrace. This
feature is best used in conjunction with the json() D subroutine, but
is available whether or not the platform supports it.
The JSON serialization feature is best used in conjunction with the json()
D subroutine, but is available whether or not the platform supports it.

# create a json probe:

Expand All @@ -90,6 +165,135 @@ is available whether or not the platform supports it.
dtrace: description 'nodeapp$target:::j1' matched 0 probes
CPU ID FUNCTION:NAME
0 68712 j1:j1 bar


## ETW INFORMATION

**Controlling the session**

Before the probes are fired the user must start a tracing session. It
can be done with session controllers like logman. After they have been
fired, the session must be stopped. Starting/stopping the session
requires the administrator privileges.

To start tracing

logman start mytrace -p {GUID} -o mytrace.etl -ets

To stop tracing

logman stop mytrace -ets


The guid may not always be known. In this case, the guidFromNames
function must be used. The following example demonstrates it.

There is an application where the provider has been created
without specifying the GUID:

var extension = require('dtrace-provider');
var provider = extension.createTraceProvider({provider_name: 'nodeapp'});
//var provider = extension.createDTraceProvider('nodeapp'); //An alternative way to create the provider.

We need to call guidFromNames before running this application.
The code snippet below demonstates how it can be done:

var extension = require('dtrace-provider');
var generated_guid = extension.guidFromNames('nodeapp');
console.log('generated guid = ' + generated_guid);

This code will print the GUID generated from the specified name.
The output will look like:

generated guid = FAC03859-DB22-3EC7-B1B5-4E6E99A4B546

Then the session can be started with

logman start mytrace -p {FAC03859-DB22-3EC7-B1B5-4E6E99A4B546} -o mytrace.etl -ets

After the session has been started, the original application
can be executed and all the probes fired will be recorded by ETW.

The guidFromNames function allows the optional module name to be passed
as well. For example:

var extension = require('dtrace-provider');
var generated_guid = extension.guidFromNames('nodeapp', 'my_module');
var provider = extension.createTraceProvider({provider_name: 'nodeapp', module_name: 'my_module'});
//var provider = extension.createDTraceProvider('nodeapp', 'my_module'); //An alternative way to create the provider.

This works for as long as there is a one-to-one match for the names.
It is guaranteed that the GUID will always be same for same strings, i.e.
the generation algorithm is deterministic.

**Consuming events**

After the session has been stopped, an etl file will appear. It can be
converted to a human-readable XML with tracerpt. The output will
contain the information about the probes and raw binary data.

To convert trace output to xml file (data will show names and values)

tracerpt mytrace.etl

To convert trace output to csv file (data will show names and values)

tracerpt mytrace.etl -of CSV

The other way of consuming events is using tools that allow a manifest
file to be supplied. The manifest contains information about the data
types of the probes added and allows the raw binary data to be parsed
correctly. The manifest is updated in runtime whenever a new probe is
created with the addProbe function and it remains valid after each update.

Some of these tools:

PerfView - http://www.microsoft.com/en-us/download/details.aspx?id=28567
LinqPad with Tx - http://tx.codeplex.com

For the example script the PerfView output will look like:

ThreadID="6,772" FormattedMessage="Node.js probe" Argument1="1" Argument2="2"
ThreadID="6,772" FormattedMessage="Node.js probe" Argument1="hello, dtrace via provider"


The manifest file is unique for each provider. It is created in the
process working directory with its name given according to the pattern:

provider_GUID.man

For the 'nodeapp' provider name it will look like:

provider_FAC03859-DB22-3EC7-B1B5-4E6E99A4B546.man

**Windows-only features**

The extension provides a variety of types under Windows in addition to
the types supported on both platforms.
Those types are:
int8, int16, int32, int64, uin8, uint16, uint32, uint64 and wchar_t *

Where 'int32' is an alias for 'int'.

When the extension is used under Windows, the user can be specific
about the ETW descriptor for the events (probes).

var provider = extension.createTraceProvider({provider_name: 'my_provider', module_name: 'my_module', guid: '5A391F32-A079-42E1-97B1-8A45295AA1FD'});
var desc = [10, 0, 0, 0, 0, 0, 0];
var probe = provider.addProbe('event1', desc1, 'char *', 'int16');

The array matches the EVENT_DESCRIPTOR structure.

http://msdn.microsoft.com/en-us/library/windows/desktop/aa363754%28v=vs.85%29.aspx

When the probes are created without the descriptor array, the event id
is incremented for each new probe; the other fields are zero-initialized.

The descriptor array will be ignored on all of the supported platforms,
except Windows. Passing the types supported only under Windows
on the other platforms will result in their fallback to the dtrace types.
For example, "uint16" will be handled as "int" and "wchar_t *" will be
handled as "char *".

## PLATFORM SUPPORT

Expand All @@ -98,15 +302,19 @@ Mac OS X and Solaris-like systems such as Illumos or SmartOS. As more
platform support is added to libusdt, those platforms will be
supported by this module. See libusdt's status at:

https://github.com/chrisa/libusdt#readme
https://github.com/chrisa/libusdt#readme

FreeBSD is supported in principle but is restricted to only 4 working
arguments per probe.

Platforms not supporting DTrace (notably, Linux and Windows) may
The Win 32/64 implementation doesn't use libusdt and relies directly
on WinAPI.

Platforms not supporting DTrace (notably, Linux) may
install this module without building libusdt, with a stub no-op
implementation provided for compatibility. This allows cross-platform
npm modules to embed probes and include a dependency on this module.
implementation provided for compatibility. The Windows implementation
doesn't require libusdt either. This allows cross-platform npm modules to
embed probes and include a dependency on this module.

GNU Make is required to build libusdt; the build scripts will look for
gmake in PATH first, and then for make.
Expand All @@ -123,18 +331,18 @@ unless probes are placed in particularly hot code paths.

The source is available at:

https://github.com/chrisa/node-dtrace-provider.
https://github.com/chrisa/node-dtrace-provider.

For issues, please use the Github issue tracker linked to the
repository. Github pull requests are very welcome.

## RUNNING THE TESTS

$ npm install
$ sudo ./node_modules/.bin/tap --tap test/*.test.js
$ npm install
$ sudo ./node_modules/.bin/tap --tap test_dtrace/*.test.js

## OTHER IMPLEMENTATIONS

This node extension is derived from the ruby-dtrace gem, via the Perl
module Devel::DTrace::Provider, both of which provide the same
functionality to those languages.
functionality to those languages.
40 changes: 30 additions & 10 deletions binding.gyp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,14 +7,14 @@

'targets': [
{
'target_name': 'DTraceProviderBindings',
'target_name': 'TraceProviderBindings',
'sources': [
'dtrace_provider.cc',
'dtrace_probe.cc',
'dtrace_argument.cc'
'src/dtrace/dtrace_provider.cc',
'src/dtrace/dtrace_probe.cc',
'src/dtrace/dtrace_argument.cc'
],
'include_dirs': [
'libusdt'
'libusdt'
],
'dependencies': [
'libusdt'
Expand All @@ -30,25 +30,45 @@
'inputs': [''],
'outputs': [''],
'action_name': 'build_libusdt',
'action': [
'action': [
'sh', 'libusdt-build.sh'
]
]
}]
}
]
}],
['OS=="win"', {

# If we are on Windows, attempt
# to build the ETW provider extension.

'targets': [
{
'target_name': 'TraceProviderBindings',
'sources': [
'src/etw_win/etw_provider.cc',
'src/etw_win/v8_provider.cc',
'src/etw_win/v8_probe.cc',
'src/etw_win/manifest_builder.cc'
],
'link_settings': {
'libraries': [ "rpcrt4.lib" ]
}
}
]
},

# If we are not on the Mac or Solaris, DTrace is unavailable.
# If we are not on the Mac, Solaris or Windows, tracing is unavailable.
# This target is necessary because GYP requires at least one
# target to exist.

{
'targets': [
{
'target_name': 'DTraceProviderStub',
'target_name': 'TraceProviderStub',
'type': 'none'
}
]
}]
]
}
}
Loading