Rawnetworkinterface stream

[JoshuaWatt]

Description

Adds support for live streaming packets from a RawNetworkInterfaceDriver. This can be useful for writing tests that parse packets in real time looking for a specific pattern, for example:

import dpkt
    
drv = target.get_driver("RawNetworkInterfaceDriver")
    
with drv.record("-", timeout=60) as p:
    pcap = dpkt.pcap.Reader(p.stdout)
    for timestamp, buf in pcap:
        eth = dpkt.ethernet.Ethernet(buf)

        if _some_criteria_:
            break
    else:
        assert False, "Packet not found!"

Checklist

• Documentation for the feature
• Tests for the feature

• The arguments and description in doc/configuration.rst have been updated

• Add a section on how to use the feature to doc/usage.rst

• Add a section on how to use the feature to doc/development.rst

• PR has been tested

• Man pages have been regenerated

[JoshuaWatt]

Of note, we've had a local driver that was very similar to RawNetworkInterfaceDriver for some time that was used to stream packets; the intention of this patch is to bring it in line with the functionality in that driver so we can eliminate it. For posterity, here is our driver:

import attr
import subprocess
import dpkt
from contextlib import contextmanager

from labgrid import target_factory
from labgrid.driver import Driver


@target_factory.reg_driver
@attr.s(eq=False)
class InterfaceCaptureDriver(Driver):
    """Driver to obtain tcpdump output from device's network"""

    bindings = {
        "iface": {"RemoteNetworkInterface", "USBNetworkInterface", "NetworkInterface"},
    }

    @contextmanager
    def live_capture(self, timeout=None):
        """
        Perform live capture of traffic from device's network.

        :param int timeout: When specified, performs capture for ``timeout`` seconds. \
            Defaults to `None`.

        :yields: A `dpkt.pcap.Reader` object of the captured network traffic
        """
        extra = getattr(self.iface, "extra", {})
        command = ["sudo", "tcpdump", "-i", self.iface.ifname, "-U", "-s0", "-w", "-"]

        if timeout is not None:
            command.extend(["-G", str(timeout), "-W", "1"])

        if "proxy_required" in extra:
            command = ["ssh", extra["proxy"], "--"] + command

        with subprocess.Popen(command, stdout=subprocess.PIPE) as p:
            yield dpkt.pcap.Reader(p.stdout)

[codecov]

Codecov Report

Attention: Patch coverage is 14.28571% with 12 lines in your changes are missing coverage. Please review.

Project coverage is 62.7%. Comparing base (722838b) to head (d68686a).
Report is 9 commits behind head on master.

❗ Current head d68686a differs from pull request most recent head fdeb30b. Consider uploading reports for the commit fdeb30b to get more accurate results

Files	Patch %	Lines
labgrid/driver/rawnetworkinterfacedriver.py	14.2%	12 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff            @@
##           master   #1320     +/-   ##
========================================
+ Coverage    62.2%   62.7%   +0.5%     
========================================
  Files         164     163      -1     
  Lines       12191   12008    -183     
========================================
- Hits         7584    7534     -50     
+ Misses       4607    4474    -133     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[Bastian-Krause]

Thanks! Looks good overall.

In the commit message of d68686a ("rawnetworkinterfacedriver: Implement live streaming"):

-with drv.record("-", timeout=60) as p
+with drv.record("-", timeout=60) as p:

[Bastian-Krause]

@JoshuaWatt I'm interested in using the changes in this PR in our network tests. Could you have a look at the review comments? If you don't have the time to integrate the changes, I'd be willing to take this over.

[JoshuaWatt]

@JoshuaWatt I'm interested in using the changes in this PR in our network tests. Could you have a look at the review comments? If you don't have the time to integrate the changes, I'd be willing to take this over.

I'll take a look this week. I was AFK for EOSS

[Bastian-Krause]

Thanks for addressing the feedback. The changes look good to me. I'll try to test this within the next weeks and add my approval then.

[JoshuaWatt]

Any update on this?

[Bastian-Krause]

Sorry for the delay. I've now tested this successfully.

I'm basically using it the same way as suggested, but with scapy instead of dpkt:

from scapy.all import PcapReader

drv = target.get_driver("RawNetworkInterfaceDriver")

with drv.record(None, count=1, timeout=5) as record:
        pcap = PcapReader(record.stdout)
        packet = pcap.next()
        ...

[JoshuaWatt]

Ping

[jluebbe]

Looks good otherwise.

[Bastian-Krause]

@JoshuaWatt Are you still interested in this or should we take this over? I'm basing more and more network tests upon this PR, so I'd like to see this merged rather sooner than later.

[JoshuaWatt]

@JoshuaWatt Are you still interested in this or should we take this over? I'm basing more and more network tests upon this PR, so I'd like to see this merged rather sooner than later.

Ya, Sorry. I've been really busy with other stuff lately. Feel free to take it over if you need it

[Bastian-Krause]

@JoshuaWatt Will do, thanks for your work!

[Bastian-Krause]

Rebased on latest master.