Skip to content

Supported Trace Formats

Shane Alcock edited this page Jan 10, 2019 · 16 revisions
Format URI Input Output Notes
Live pcap interface pcapint:<interface> Yes Yes
pcap trace file pcapfile:<filename> Yes Yes
pcag-ng trace file pcapng:<filename> Yes Yes Read support added in libtrace 4.0.2, write support added in libtrace 4.0.6
ERF trace file erf:<filename> Yes Yes
DAG device dag:<device location> Yes Yes Requires DAG hardware capture card and libraries
Native Linux interface int:<interface> Yes Yes Only available on Linux
Native Linux interface (ring buffers) ring:<interface> Yes Yes Only available on Linux -- requires Libtrace 3.0.16 or later
Intel Data Plane Development Kit (DPDK) dpdk:<PCI address> Yes Yes Only available on Linux -- experimental (requires additional setup)
Native BSD interface bpf:<interface> Yes No Only available on BSD or Mac OS X
TSH trace file tsh:<filename> Yes No
FR+ trace file fr+:<filename> Yes No
Legacy DAG ATM trace file legacyatm:<filename> Yes No
Legacy DAG POS trace file legacypos:<filename> Yes No
Legacy DAG Ethernet trace file legacyeth:<filename> Yes No
Legacy DAG NZIX trace file legacynzix:<filename> Yes No
ATM cell header trace file atmhdr:<filename> Yes No
RT network protocol rt:<host>:<port> Yes No
nDAG protocol ndag:<interface>,<multicast group>,<port> Yes No
nDAG protocol via DPDK dpdkndag:<PCI address> Yes No Only available on Linux -- experimental (requires additional setup)

Libtrace input and output sources are specified using URIs which describe both the format and location of the trace, interface or device in question. To do so, simply append the location of the trace file, interface or device to the URI for the appropriate format as specified in the table above.

For example, pcapfile:sample.pcap.gz describes the PCAP trace file called sample.pcap.gz. dag:/dev/dag0 describes the DAG device present at /dev/dag0. int:eth0 is the URI for the Linux interface eth0.

URIs that include a filename usually accept '-' to represent stdin, e.g. erf:- will read ERF records from stdin.

Per format notes

PCAP

  • There are three pcap formats supported by libtrace: pcapfile, pcapint and pcap. Generally you want to use pcapfile if reading or writing pcap trace files and pcapint if reading or writing from a live pcap interface.
  • pcapfile and pcapint have been implemented directly within libtrace, whereas pcap uses the libpcap API. We recommend against using pcap wherever possible.
  • Most pcap-based utilities, including tcpdump, tcptrace and snort, do not support compressed trace files so you may need to decompress the trace first. For example, "zcat foo.pcap.gz | tcpdump -r-" will do this for tcpdump.

PCAP NG

  • pcapng refers to the next generation pcap format that is used as the default output format by Wireshark.

DAG capture card (dag:)

  • Requires the Dag API from Endace.
  • Anyone experiencing problems with using DAG cards for libtrace may wish to read over our DAG Notes, which describes the libtrace DAG support in more detail.

ERF Trace file (erf:)

  • Supports a maximum of 4 interfaces, by convention 0 for outgoing, 1 for incoming, 2 for other, and 3 is unused.
  • There is at least one traceset which is known to have an ERF type of "0", just to be annoying, libtrace attempts to account for this.
  • Does NOT require the dag API to read erf (and compressed erf) traces.

Native Linux interface (int:)

  • Currently supports a fast way of retrieving the timestamp of the packet from kernel space using ancillary data.
  • Supports uploading bpf filters into the kernel since 3.0.5+
  • Supports only incoming and outgoing direction, attempts to set other directions will fail. (3.0.3+)
  • Currently the default if you just provide an interface name for the URI without explicitly defining the format.

Native Linux interface (ring:)

  • Provides much better performance than int:. To use, you must provide the full URI when describing the input source (e.g. "ring:eth0", rather than just "eth0").
  • Supports all the same features as int:

Intel Data Plane Development Kit (dpdk:) - This format is currently experimental

  • Requires Intel DPDK to be installed and kernel modules to be loaded
  • Better performance than int: and ring: under the correct conditions.
  • Binds to a single CPU core and polls on a buffer, this means waiting for a packet uses 100% CPU on that bound core.
  • You can only use a single instance of the dpdk: format at a given time - that is you can listen to a single interface or write to a single interface but not both.

For more information please read over the DPDK Notes, which describes the libtrace DPDK support in more detail.

BSD BPF Interface (bpf:)

  • Currently supports capture, but not writing.

RT protocol

  • libtrace does not (yet?) support RT output.
  • The protocol specification may eventually be made available so people can write their own servers.

Network DAG (nDAG)

  • libtrace does not support nDAG output.
  • This format is designed to receive ERF records over a network from a host running an nDAG multicaster (software currently in development and not yet available publicly).
  • nDAG works best when used with the parallel API.

Network DAG over DPDK (dpdkndag:)

  • Capture only, no output support.
  • This format allows users to use a DPDK-capable interface to intercept and decode ERF records being exported by an nDAG multicaster.
Clone this wiki locally