Skip to content
This repository has been archived by the owner on Nov 15, 2023. It is now read-only.

OpenTestPoint

Jump to bottom
Steven Galgano edited this page Apr 10, 2017 · 2 revisions

All of the tutorial demonstrations, with the exception of 0, 4 and 7, are provisioned to use the OpenTestPoint data collection framework. OpenTestPoint provides mechanisms to uniformly collect, distribute and access information from running experiments.

OpenTestPoint defines a set of APIs for writing plugin measurement collectors called probes. Probes can collect any type of measurement: routing information, process utilization, emulator statistics, shared code SDR model statistics, etc. The framework uses measurement specifications defined using Google Protocol Buffers coupled with Python introspection to create an environment for generic analysis of real-time and post-processed data.

OpenTestPoint Background

OpenTestPoint and EMANE

Let's take a look at how we use OpenTestPoint EMANE probes.

  1. Take a look at the directory listing in Demonstration 1

    [me@host emane-tutorial]$ cd 1
    [me@host 1]$ ls otestpoint* probe-*
otestpoint-broker.xml      otestpoint-recorder1.xml
otestpointd10.xml          otestpoint-recorder2.xml
otestpointd1.xml           otestpoint-recorder3.xml
otestpointd2.xml           otestpoint-recorder4.xml
otestpointd3.xml           otestpoint-recorder5.xml
otestpointd4.xml           otestpoint-recorder6.xml
otestpointd5.xml           otestpoint-recorder7.xml
otestpointd6.xml           otestpoint-recorder8.xml
otestpointd7.xml           otestpoint-recorder9.xml
otestpointd8.xml           probe-emane-physicallayer.xml
otestpointd9.xml           probe-emane-rfpipe.xml
otestpointd.xml.template   probe-emane-virtualtransport.xml
otestpoint-recorder10.xml

    These are the files used to configure OpenTestPoint.

    Let's take a look at an OpenTestPoint daemon configuration: otestpointd1.xml.

    <otestpoint id="node-1" discovery="node-1:8881" publish="node-1:8882">
  <probe configuration="probe-emane-physicallayer.xml">
    <python module="otestpoint.emane.physicallayer" class="PhysicalLayer"/>
  </probe>
   <probe configuration="probe-emane-rfpipe.xml">
     <python module="otestpoint.emane.rfpipe" class="RFPipe"/>
   </probe>
   <probe configuration="probe-emane-virtualtransport.xml">
     <python module="otestpoint.emane.virtualtransport" class="VirtualTransport"/>
   </probe>
</otestpoint>

    Here we are going to load three plugins, one for each NEM layer contained in node-1's emulator instance.

    • otestpoint.emane.physicallayer.PhysicalLayer
    • otestpoint.emane.rfpipe.RFPipe
    • otestpoint.emane.virtualtransport.VirtualTransport

    The id attribute corresponds to a unique tag that will identify all measurements as being from node-1. The discovery attribute is the ZeroMQ REQ/RES endpoint used for measurement discovery requests. The publish attribute is the ZeroMQ PUB endpoint used for publishing and subscribing measurement data.

  2. Lets take a look at the input configuration for the otestpoint.emane.physicallayer.PhysicalLayer probe contained in probe-emane-physicallayer.xml.

    <probe-emane-PhysicalLayer address='localhost'>
  <probes>
    <EMANE.PhysicalLayer.Counters.General enable="yes"/>
    <EMANE.PhysicalLayer.Tables.Events enable="yes"/>
    <EMANE.PhysicalLayer.Tables.Status enable="yes"/>
  </probes>
</probe-emane-PhysicalLayer>

    The three measurement types produced by this plugin map to the statistics and statistic tables made available by the emulator physical layer:

    • EMANE.PhysicalLayer.Counters.General
    • EMANE.PhysicalLayer.Tables.Events
    • EMANE.PhysicalLayer.Tables.Status

    The OpenTestPoint EMANE probes connect to the emulator using the Control Port.

    Each of these measurements corresponds to a Google Protocol Buffer message.

    message Measurement_emane_physicallayer_counters_general
{
  message Description
  {
    required string name = 1 [default = "Measurement_emane_physicallayer_counters_general"];
    required string module = 2 [default = "otestpoint.emane.physicallayer"];
    required uint32 version = 3 [default = 1];
  }
  // do not set description
  optional Description description = 1;

  optional float avgdownstreamprocessingdelay0 = 2 ;
  optional double avgprocessapiqueuedepth = 3 ;
  optional double avgprocessapiqueuewait = 4 ;
  optional double avgtimedeventlatency = 5 ;
  optional double avgtimedeventlatencyratio = 6 ;
  optional float avgupstreamprocessingdelay0 = 7 ;
  optional uint64 numdownstreambytesbroadcastgenerated0 = 8 ;
  optional uint64 numdownstreambytesbroadcastrx0 = 9 ;
  optional uint64 numdownstreambytesbroadcasttx0 = 10 ;
  optional uint64 numdownstreambytesunicastgenerated0 = 11 ;
  optional uint64 numdownstreambytesunicastrx0 = 12 ;
  optional uint64 numdownstreambytesunicasttx0 = 13 ;
  optional uint64 numdownstreampacketsbroadcastdrop0 = 14 ;
  optional uint64 numdownstreampacketsbroadcastgenerated0 = 15 ;
  optional uint64 numdownstreampacketsbroadcastrx0 = 16 ;
  optional uint64 numdownstreampacketsbroadcasttx0 = 17 ;
  optional uint64 numdownstreampacketsunicastdrop0 = 18 ;
  optional uint64 numdownstreampacketsunicastgenerated0 = 19 ;
  optional uint64 numdownstreampacketsunicastrx0 = 20 ;
  optional uint64 numdownstreampacketsunicasttx0 = 21 ;
  optional uint64 numupstreambytesbroadcastrx0 = 22 ;
  optional uint64 numupstreambytesbroadcasttx0 = 23 ;
  optional uint64 numupstreambytesunicastrx0 = 24 ;
  optional uint64 numupstreambytesunicasttx0 = 25 ;
  optional uint64 numupstreampacketsbroadcastdrop0 = 26 ;
  optional uint64 numupstreampacketsbroadcastrx0 = 27 ;
  optional uint64 numupstreampacketsbroadcasttx0 = 28 ;
  optional uint64 numupstreampacketsunicastdrop0 = 29 ;
  optional uint64 numupstreampacketsunicastrx0 = 30 ;
  optional uint64 numupstreampacketsunicasttx0 = 31 ;
  optional uint64 numtimesyncthresholdrewrite = 32 ;
  optional uint64 processedconfiguration = 33 ;
  optional uint64 processeddownstreamcontrol = 34 ;
  optional uint64 processeddownstreampackets = 35 ;
  optional uint64 processedevents = 36 ;
  optional uint64 processedtimedevents = 37 ;
  optional uint64 processedupstreamcontrol = 38 ;
  optional uint64 processedupstreampackets = 39 ;
}

message Measurement_emane_physicallayer_tables_events
{
  message Description
  {
    required string name = 1 [default = "Measurement_emane_physicallayer_tables_events"];
    required string module = 2 [default = "otestpoint.emane.physicallayer"];
    required uint32 version = 3 [default = 1];
  }
  // do not set description
  optional Description description = 1;

   optional MeasurementTable antennaprofileeventinfotable = 2 ;
   optional EventReceptionTable eventreceptiontable = 3 ;
   optional MeasurementTable locationeventinfotable = 4 ;
   optional PathlossEventTable pathlosseventinfotable = 5 ;
}

message Measurement_emane_physicallayer_tables_status
{
  message Description
  {
    required string name = 1 [default = "Measurement_emane_physicallayer_tables_status"];
    required string module = 2 [default = "otestpoint.emane.physicallayer"];
    required uint32 version = 3 [default = 1];
  }
  // do not set description
  optional Description description = 1;

   optional MeasurementTable broadcastpacketaccepttable0 = 2 ;
   optional MeasurementTable broadcastpacketdroptable0 = 3 ;
   optional MeasurementTable unicastpacketaccepttable0 = 4 ;
   optional MeasurementTable unicastpacketdroptable0 = 5 ;
   optional PhysicalLayerReceivePowerTable receivepowertable = 6 ;
}

    Each message contains a Description:

    message Description
{
  required string name = 1 [default = "Measurement_emane_physicallayer_counters_general"];
  required string module = 2 [default = "otestpoint.emane.physicallayer"];
  required uint32 version = 3 [default = 1];
}

    The name and module components are used by the Python portions of the OpenTestPoint framework to dynamically create and interact with measurement objects.

  3. So let's see how this all fits together by looking at a running demonstration 1.

    If we ssh to node-1 and do a process listing we can see what OpenTestPoint components are running on the node.

    [me@node-1 ~]$ ps ax
  PID TTY      STAT   TIME COMMAND
    1 pts/0    S+     0:00 /usr/sbin/init.lxc --name node-1 --lxcpath /var/lib/l
    5 ?        Ss     0:00 /usr/sbin/sshd -o PidFile=/home/emane/emane-tutorial/
    7 ?        Ssl    0:00 emane platform1.xml -r -d -l 3 -f /home/emane/emane-t
   16 ?        Ssl    0:00 emaneeventd -r -d eventdaemon1.xml -l 3 -f /home/eman
   29 ?        S<s    0:00 gpsd -P /home/emane/emane-tutorial/1/persist/1/var/ru
   31 ?        Ssl    0:00 otestpoint-recorder otestpoint-recorder1.xml -d -l 3 
   44 ?        Ssl    0:00 otestpointd otestpointd1.xml -d -l 3 -f /home/emane/e
   61 ?        Sl     0:00 otestpoint-probe
   62 ?        Ss     0:00 olsrd -f routing1.conf
   64 pts/0    S+     0:00 mgen input mgen output /home/emane/emane-tutorial/1/p
   72 ?        Sl     0:00 otestpoint-probe
   80 ?        Sl     0:00 otestpoint-probe
   86 ?        Ss     0:00 sshd: emane [priv]
   89 ?        S      0:00 sshd: emane@pts/1
   90 pts/1    Ss     0:00 -bash
  107 pts/1    R+     0:00 ps ax

    The otestpointd process manages each specified plugin, instantiating each within its own isolated otestpoint-probe process. See Probe Management for more information.

    The otestpoint-recorder process subscribes to all available OpenTestPoint measurements and records them locally. See Recorders for more information.

    otestpoint-recorder1.xml:

    <otestpoint-recorder file="persist/1/var/log/otestpoint-recorder.data">
  <testpoint publish="node-1:8882"/>
</otestpoint-recorder>

  4. Looking on the host we find a single OpenTestPoint component running.

    [me@host 1]$ ps ax | grep otestpoint-broke[r]
 6511 ?        Ssl    0:00 otestpoint-broker otestpoint-broker.xml -d -l 3 -f persist/host/var/log/otestpoint-broker.log --pidfile persist/host/var/run/otestpoint-broker.pid --uuidfile persist/host/var/run/otestpoint-broker.uuid

    otestpoint-broker.xml:

    <otestpoint-broker discovery="localhost:9001" publish="localhost:9002">
  <testpoint discovery="node-1:8881" publish="node-1:8882"/>
  <testpoint discovery="node-2:8881" publish="node-2:8882"/>
  <testpoint discovery="node-3:8881" publish="node-3:8882"/>
  <testpoint discovery="node-4:8881" publish="node-4:8882"/>
  <testpoint discovery="node-5:8881" publish="node-5:8882"/>
  <testpoint discovery="node-6:8881" publish="node-6:8882"/>
  <testpoint discovery="node-7:8881" publish="node-7:8882"/>
  <testpoint discovery="node-8:8881" publish="node-8:8882"/>
  <testpoint discovery="node-9:8881" publish="node-9:8882"/>
  <testpoint discovery="node-10:8881" publish="node-10:8882"/>
</otestpoint-broker>

    The otestpoint-broker brokers one or more otestpointd and/or otestpoint-broker endpoints to provide centralized endpoints for client tools to subscribe to measurements from multiple nodes. See Brokers for more information.

  5. Query node-1 from the host to discovery what measurements are available.

    [me@host 1]$ otestpoint-discover node-1:8881
tcp://10.99.0.1:8882
EMANE.PhysicalLayer.Counters.General.node-1
EMANE.PhysicalLayer.Tables.Events.node-1
EMANE.PhysicalLayer.Tables.Status.node-1
EMANE.RFPipe.Counters.General.node-1
EMANE.RFPipe.Tables.Events.node-1
EMANE.RFPipe.Tables.Neighbor.node-1
EMANE.RFPipe.Tables.Status.node-1
EMANE.VirtualTransport.Counters.General.node-1
EMANE.VirtualTransport.Tables.Events.node-1
EMANE.VirtualTransport.Tables.Status.node-1

  6. Now try the same command but point at the broker using localhost:9001.

    [me@host 1]$ otestpoint-discover localhost:9001
tcp://[::1]:9002
EMANE.PhysicalLayer.Counters.General.node-1
EMANE.PhysicalLayer.Counters.General.node-10
EMANE.PhysicalLayer.Counters.General.node-2
EMANE.PhysicalLayer.Counters.General.node-3
EMANE.PhysicalLayer.Counters.General.node-4
EMANE.PhysicalLayer.Counters.General.node-5
EMANE.PhysicalLayer.Counters.General.node-6
EMANE.PhysicalLayer.Counters.General.node-7
EMANE.PhysicalLayer.Counters.General.node-8
EMANE.PhysicalLayer.Counters.General.node-9
EMANE.PhysicalLayer.Tables.Events.node-1
EMANE.PhysicalLayer.Tables.Events.node-10
EMANE.PhysicalLayer.Tables.Events.node-2
EMANE.PhysicalLayer.Tables.Events.node-3
EMANE.PhysicalLayer.Tables.Events.node-4
EMANE.PhysicalLayer.Tables.Events.node-5
EMANE.PhysicalLayer.Tables.Events.node-6
EMANE.PhysicalLayer.Tables.Events.node-7
EMANE.PhysicalLayer.Tables.Events.node-8
EMANE.PhysicalLayer.Tables.Events.node-9
EMANE.PhysicalLayer.Tables.Status.node-1
<...truncated for listing...>

    Using the broker endpoint you now have access to the measurements of all nodes.

  7. Take a look at some measurements using otestpoint-dump.

    [me@host 1]$ otestpoint-dump localhost:9002 EMANE.PhysicalLayer.Tables.Status
[1470160620] node-4/0 4c02c843-5e10-46db-bc04-ec07543ec753
otestpoint.emane.physicallayer Measurement_emane_physicallayer_tables_status v1 1423 bytes
EMANE.PhysicalLayer.Tables.Status.node-4
[] broadcastpacketaccepttable0
|NEM|Num Pkts Tx|Num Bytes Tx|Num Pkts Rx|Num Bytes Rx|
|1  |0          |0           |15403      |5471083     |
|2  |0          |0           |15401      |5470371     |
|3  |0          |0           |15386      |5460718     |
|4  |15437      |5469379     |0          |0           |
|5  |0          |0           |15373      |5455192     |
|6  |0          |0           |15394      |5455725     |
|7  |0          |0           |60         |21408       |
|8  |0          |0           |61         |21490       |
|9  |0          |0           |60         |20372       |
--
[] broadcastpacketdroptable0
|NEM|Out-of-Band|Rx Sensitivity|Propagation Model|Gain Location|Gain Horizon|Gain Profile|Not FOI|Spectrum Clamp|
|1  |0          |0             |41               |0            |0           |0           |0      |0             |
|2  |0          |0             |41               |0            |0           |0           |0      |0             |
|3  |0          |0             |45               |0            |0           |0           |0      |0             |
|5  |0          |0             |45               |0            |0           |0           |0      |0             |
|6  |0          |0             |43               |0            |0           |0           |0      |0             |
|7  |0          |15312         |43               |0            |0           |0           |0      |0             |
|8  |0          |15319         |41               |0            |0           |0           |0      |0             |
|9  |0          |15306         |40               |0            |0           |0           |0      |0             |
|10 |0          |0             |8126             |0            |0           |0           |0      |0             |
--
[] receivepowertable
| NEM   | Frequency Hz | Rx Power dBm | Last Packet Time |
| 1     | 2347000000   | -73.1999969482 | 1470160619.53    |
| 2     | 2347000000   | -69.9000015259 | 1470160619.91    |
| 3     | 2347000000   | 0.0          | 1470160619.89    |
| 5     | 2347000000   | 0.0          | 1470160619.08    |
| 6     | 2347000000   | -68.6999969482 | 1470160620.02    |
| 7     | 2347000000   | -174.600006104 | 1470160620.03    |
| 8     | 2347000000   | -175.0       | 1470160620.03    |
| 9     | 2347000000   | -175.199996948 | 1470160619.52    |
--
[] unicastpacketaccepttable0
|NEM|Num Pkts Tx|Num Bytes Tx|Num Pkts Rx|Num Bytes Rx|
--
[] unicastpacketdroptable0
|NEM|Out-of-Band|Rx Sensitivity|Propagation Model|Gain Location|Gain Horizon|Gain Profile|Not FOI|Spectrum Clamp|
--


[1470160620] node-3/0 129532e9-b207-4a3b-a4ba-8a0010fd012f
otestpoint.emane.physicallayer Measurement_emane_physicallayer_tables_status v1 1423 bytes
EMANE.PhysicalLayer.Tables.Status.node-3
[] broadcastpacketaccepttable0
|NEM|Num Pkts Tx|Num Bytes Tx|Num Pkts Rx|Num Bytes Rx|
|1  |0          |0           |15403      |5471083     |
|2  |0          |0           |15401      |5470371     |
|3  |15434      |5464422     |0          |0           |
|4  |0          |0           |15391      |5465823     |
|5  |0          |0           |15373      |5455192     |
|6  |0          |0           |15394      |5455725     |
|7  |0          |0           |62         |22152       |
|8  |0          |0           |64         |22376       |
|9  |0          |0           |62         |21196       |
--
[] broadcastpacketdroptable0
|NEM|Out-of-Band|Rx Sensitivity|Propagation Model|Gain Location|Gain Horizon|Gain Profile|Not FOI|Spectrum Clamp|
|1  |0          |0             |45               |0            |0           |0           |0      |0             |
|2  |0          |0             |45               |0            |0           |0           |0      |0             |
|4  |0          |0             |46               |0            |0           |0           |0      |0             |
|5  |0          |0             |45               |0            |0           |0           |0      |0             |
|6  |0          |0             |43               |0            |0           |0           |0      |0             |
|7  |0          |15310         |43               |0            |0           |0           |0      |0             |
|8  |0          |15316         |41               |0            |0           |0           |0      |0             |
|9  |0          |15304         |40               |0            |0           |0           |0      |0             |
|10 |0          |0             |8126             |0            |0           |0           |0      |0             |
--
[] receivepowertable
| NEM   | Frequency Hz | Rx Power dBm | Last Packet Time |
| 1     | 2347000000   | -69.5        | 1470160619.53    |
| 2     | 2347000000   | 0.0          | 1470160619.91    |
| 4     | 2347000000   | 0.0          | 1470160619.96    |
| 5     | 2347000000   | -69.5        | 1470160619.08    |
| 6     | 2347000000   | -72.5999984741 | 1470160620.02    |
| 7     | 2347000000   | -174.600006104 | 1470160620.03    |
| 8     | 2347000000   | -174.800003052 | 1470160620.03    |
| 9     | 2347000000   | -175.199996948 | 1470160619.52    |
--
[] unicastpacketaccepttable0
|NEM|Num Pkts Tx|Num Bytes Tx|Num Pkts Rx|Num Bytes Rx|
--
[] unicastpacketdroptable0
|NEM|Out-of-Band|Rx Sensitivity|Propagation Model|Gain Location|Gain Horizon|Gain Profile|Not FOI|Spectrum Clamp|
--
<...truncated for listing...>

    otestpoint-dump is a command line tool for subscribing and printing measurements.

  8. You can develop your own probes, analysis and smart-scenario applications that process OpenTestPoint measurements in a variety of interesting ways.

    Follow the OpenTestPoint project for more information.

Copyright (c) 2014-2018 - Adjacent Link LLC, Bridgewater, New Jersey
Except where otherwise noted, content on this site is licensed under a Creative Commons Attribution 4.0 International License.

Clone this wiki locally