Skip to content
Dave Cottlehuber edited this page Apr 4, 2016 · 6 revisions

Capturing traces

Typically you'll use ngrep to capture packets. You'll need to ensure the correct interface & ports are chosen, and then

sudo ngrep -texd en0 port 7777 -O output.pcap &
swift --debug --progress --tracker server:7777 --hash 00alreadyhashed00 -o /tmp/
sudo kill ngrep

You can use wireshark to view these traces at your leisure.

Adding traces

Clone the wiki repo, add any traces you're interested to the /traces/ directory and push. There's no way to view the directory listing directly, so add a note here.

If you have traces from multiple endpoints (e.g. several peers) you can merge these together using either tcpslice *.pcap -w full.pcap, which is available in FreeBSD ports, or mergecap -w full.pcapng *.pcap which comes with wireshark.

Use the perl pcap2erl script to convert pcap format into an erlang array to include in tests or programmatic generation of traffic.

Tracing in Erlang/OTP

The BEAM has extremely powerful trace and diagnostic functionality built in, exposed by a number of very useful libraries, of most interest are redbug and recon. The examples below cover recon specifically. While this is not a full introduction to tracing Erlang applications, just enough to get you started, you can find an excellent grounding in Operations OTP-style in Erlang in Anger.

What's in a trace?

A trace consists of 4 things:

  • groups of processes (a list of pids, a specific pid, all processes, existing, or new processes)
  • a trace specification (modules, functions, arguments)
  • how long, or how many events, to collect
  • to where the output should be sent

In most cases we'll simply use the console as output so we can ignore the latter.

Processes

As a lazy developer, I'll trace on all processes, both across and inside modules. If you compose your modules nicely, leaving out the scope tuple entirely, or {scope, global}, simply tracks the API calls across/between modules. {pid, all} takes alternative atoms existing, and new which should be obvious, or you can supply a list of specific pids. Note that tracing will only be enabled for modules that have already been loaded into the VM - your code & apps must be present. return and stack are the return value, assuming the function didn't crash, and the stack backtrace.

Options = [{pid, all}, {scope, local}, stack, return].

What You Want??

I typically want to see return values from functions:

Stack = fun(_) -> return_trace() end.

NB for elixir use fn x -> :recon.return_trace end.

And also a list of modules I'm interested in. To keep things simple I use a list of the modules themselves and assume I'm tracing all functions within that module. Refer to recon docs for more complicated specifications & adjust to taste.

Modules = lists:map(fun(M) -> {M, '_', Stack } end,
  [channel_sup, channel_worker, peer_sup, peer_worker, ppspp_channel,
    ppspp_chunk, ppspp_datagram, ppspp_handshake, ppspp_have, ppspp_message,
    ppspp_options, swarm_sup, swarm_worker, swirl, swirl_app, swirl_sup]).

When to Stop

Traces, if left running, can explode mailboxes and bring down your VM, or at best cause IO to plummet. We limit these then either by the number of messages / traces received, or by time duration.

%% 100 traces or 10 seconds, whichever comes first
Scope = {100, 10000}.

Trace On!

%% start tracing
recon_trace:calls(Modules, Scope, Options).
%% do your thing here ... woah ...
%% clear all traces if required
recon_trace:clear().

Interesting PPSPP packet traces