Modes supported #10

mrbluecoat · 2024-07-27T16:43:53Z

Out of curiosity, which modes does durdur support? https://man.archlinux.org/man/xdp-loader.8.en#-m,_--mode

(my use case requires SKB mode)

boratanrikulu · 2024-07-27T16:57:37Z

It uses default one - Generic (SKB) mode;

type XDPOptions struct {
	// Program must be an XDP BPF program.
	Program *ebpf.Program

	// Interface is the interface index to attach program to.
	Interface int

	// Flags is one of XDPAttachFlags (optional).
	//
	// Only one XDP mode should be set, without flag defaults
	// to driver/generic mode (best effort).
	Flags XDPAttachFlags
}

const (
	// XDPGenericMode (SKB) links XDP BPF program for drivers which do
	// not yet support native XDP.
	XDPGenericMode XDPAttachFlags = 1 << (iota + 1)
	// XDPDriverMode links XDP BPF program into the driver’s receive path.
	XDPDriverMode
	// XDPOffloadMode offloads the entire XDP BPF program into hardware.
	XDPOffloadMode
)

mrbluecoat · 2024-07-27T17:07:58Z

hmm.. any ideas why I'm getting this then?

# durdur attach --interface enp0s6
Error: attach: failed to attach link: create link: invalid argument

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP group default qlen 1000
    link/ether 02:00:17:xx:xx:xx brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.19/24 metric 100 brd 10.0.0.255 scope global enp0s6
       valid_lft forever preferred_lft forever
    inet6 fe80::17ff:xxxx:xxxx/64 scope link 
       valid_lft forever preferred_lft forever

# uname -a
Linux mynode 6.5.0-1026-oracle #26~22.04.1-Ubuntu SMP Mon Jun 24 14:22:18 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

boratanrikulu · 2024-07-27T17:15:25Z

Not sure 🤔 never tested with ARM

How did you build the application?

mrbluecoat · 2024-07-27T17:30:44Z

Not sure 🤔 never tested with ARM

How did you build the application?

#9 (comment)

boratanrikulu · 2024-07-27T18:47:43Z

Looks similar cilium/ebpf#783

boratanrikulu · 2024-07-27T18:49:49Z

Can you check this doc https://trying2adult.com/what-is-xdp-and-how-do-you-use-it-in-linux-amazon-ec2-example/

mrbluecoat · 2024-07-27T18:56:37Z

Unfortunately, no luck:

# ip link set dev enp0s6 mtu 3818
# ip a | grep enp0s6.*3818
2: enp0s6: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 3818 qdisc mq state UP group default qlen 1000
# durdur attach --interface enp0s6
Error: attach: failed to attach link: create link: invalid argument

mrbluecoat · 2024-07-27T18:57:42Z

Not sure if it helps, but https://github.com/ahsifer/goxdp is working (even without the MTU modification)

mrbluecoat · 2024-07-27T19:05:54Z

P.S. If you want to test on ARM, Oracle offers free instances: https://mrbluecoat.blogspot.com/2021/06/oracle-cloud-arm-most-generous-free-tier.html

boratanrikulu · 2024-07-27T19:20:52Z

interesting

Can you try to change eBPF-go version in go.mod file v0.12.3
and run these commands

go mod tidy
make build
sudo ./build/durdur attach -i enp0s6

module github.com/boratanrikulu/durdur

go 1.21

toolchain go1.22.5

require (
	github.com/cilium/ebpf v0.12.3
	github.com/frankban/quicktest v1.14.5
	github.com/urfave/cli/v2 v2.23.7
	golang.org/x/exp v0.0.0-20230224173230-c95f2b4c22f2
)

require (
	github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
	github.com/google/go-cmp v0.6.0 // indirect
	github.com/kr/pretty v0.3.1 // indirect
	github.com/kr/text v0.2.0 // indirect
	github.com/rogpeppe/go-internal v1.11.0 // indirect
	github.com/russross/blackfriday/v2 v2.1.0 // indirect
	github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
	golang.org/x/sys v0.20.0 // indirect
)

mrbluecoat · 2024-07-27T21:42:11Z

Sorry, no luck:

# cat go.mod 
module github.com/boratanrikulu/durdur

go 1.21

toolchain go1.22.5

require (
        github.com/cilium/ebpf v0.12.3
        github.com/frankban/quicktest v1.14.5
        github.com/urfave/cli/v2 v2.23.7
        golang.org/x/exp v0.0.0-20230224173230-c95f2b4c22f2
)

require (
        github.com/cpuguy83/go-md2man/v2 v2.0.2 // indirect
        github.com/google/go-cmp v0.6.0 // indirect
        github.com/kr/pretty v0.3.1 // indirect
        github.com/kr/text v0.2.0 // indirect
        github.com/rogpeppe/go-internal v1.11.0 // indirect
        github.com/russross/blackfriday/v2 v2.1.0 // indirect
        github.com/xrash/smetrics v0.0.0-20201216005158-039620a65673 // indirect
        golang.org/x/sys v0.20.0 // indirect
)
# go mod tidy
go: downloading github.com/frankban/quicktest v1.14.5
# make build
go generate ./internal/generated...
Compiled /tmp/durdur/internal/generated/bpf_bpfeb.o
Stripped /tmp/durdur/internal/generated/bpf_bpfeb.o
Wrote /tmp/durdur/internal/generated/bpf_bpfeb.go
Compiled /tmp/durdur/internal/generated/bpf_bpfel.o
Stripped /tmp/durdur/internal/generated/bpf_bpfel.o
Wrote /tmp/durdur/internal/generated/bpf_bpfel.go
CGO_ENABLED=0 go build -o build/durdur ./cmd/durdur
# ./build/durdur attach -i enp0s6
Error: attach: create link: invalid argument

boratanrikulu · 2024-07-27T21:53:01Z

you have root while running the program, right?

I set up ARM CI here: https://github.com/boratanrikulu/durdur/actions/runs/10126646992/job/28003538415

Linux vm0dcmer 6.5.0-1023-azure #24~22.04.1-Ubuntu SMP Thu Jun 13 03:54:29 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux

Attach/Detach just works fine. But drop not working. Probably some bugs in eBPF code.

mrbluecoat · 2024-07-27T22:02:33Z

Turns out MTU 3818 was still too high. cilium/cilium#25453 (comment) indicated 3498 which worked!

mrbluecoat · 2024-07-27T22:05:32Z

The default MTU is set to 9001 on the ena driver. Given XDP buffers are linear, they operate on a single page. A driver typically reserves some headroom for XDP as well (e.g. for encapsulation purpose), therefore, the highest possible MTU for XDP would be 3498.

https://docs.cilium.io/en/stable/network/kubernetes/kubeproxy-free/#nodeport-xdp-on-aws

boratanrikulu · 2024-07-27T22:08:10Z

Aha, nice!

What about Drop. Is it working for you?

sudo ./build/durdur attach -i enp0s6
sudo ./build/durdur drop --src "169.155.49.112"
ping -c 1 -I enp0s6 169.155.49.112 # should show 100% packet loss
sudo ./build/durdur detach

mrbluecoat · 2024-07-27T22:11:43Z

Yes, appears to be (using the new MTU and the new go.mod). I'll do a bit more testing..

mrbluecoat · 2024-07-27T22:22:51Z

Awesome, it passed the stress test! <3

# PIDMAX=$(cat /proc/sys/kernel/pid_max)
# 
# echo "fs.file-max = $PIDMAX" >> /etc/sysctl.d/local.conf
# echo "fs.nr_open = $PIDMAX" >> /etc/sysctl.d/local.conf
# sysctl -p /etc/sysctl.d/local.conf
# ulimit -n $PIDMAX
# ulimit -l unlimited
# sed -i "s/# End of file//" /etc/security/limits.conf
# printf "\n* - nofile $PIDMAX\nroot - nofile $PIDMAX\n" >> /etc/security/limits.conf
# printf "\n* - memlock unlimited\nroot - memlock unlimited\n" >> /etc/security/limits.conf
# printf "\nulimit -n $PIDMAX\nulimit -l unlimited\n" >> ~/.bashrc
# 
# durdur attach --interface enp0s6
# 
# wget -q https://github.com/dibdot/DoH-IP-blocklists/raw/master/doh-ipv4.txt
# while read line; do durdur drop --src $(echo "$line" | awk '{print $1}'); done < doh-ipv4.txt
# 
# durdur drop --src $(dig +short example.com)
# 
# durdur list all | jq length
2016
# ping -c1 example.com
PING example.com (93.184.215.14) 56(84) bytes of data.

--- example.com ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms
# ping -c1 google.com
PING google.com (74.125.202.113) 56(84) bytes of data.
64 bytes from io-in-f113.1e100.net (74.125.202.113): icmp_seq=1 ttl=61 time=55.8 ms

--- google.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 55.816/55.816/55.816/0.000 ms
#

boratanrikulu · 2024-07-27T22:24:44Z

Nice, but somehow it's not working with arm ci 😄 #11

mrbluecoat · 2024-07-27T22:25:39Z

LOL, I'll reopen so you can figure that one out 😄

boratanrikulu · 2024-07-27T22:36:11Z

Looks like working with lo interface but not working with ens3 interface 😄

mrbluecoat · 2024-07-27T22:45:07Z

Also, drop works for IP addresses (src) but not DNS:

# durdur list dns
{"example.com":4}
# ping -c1 -I enp0s6 example.com
PING example.com (93.184.215.14) from 10.0.0.19 enp0s6: 56(84) bytes of data.
64 bytes from 93.184.215.14 (93.184.215.14): icmp_seq=1 ttl=58 time=8.68 ms

--- example.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 8.682/8.682/8.682/0.000 ms
# durdur drop --src $(dig +short badexample.com)
# ping -c1 -I enp0s6 badexample.com
PING badexample.com (173.236.193.195) from 10.0.0.19 enp0s6: 56(84) bytes of data.

--- badexample.com ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0ms

mrbluecoat · 2024-07-27T22:46:55Z

Looks like working with lo interface but not working with ens3 interface 😄

What's the MTU on ens3?

boratanrikulu · 2024-07-27T22:50:52Z

Also, drop works for IP addresses (src) but not DNS:

Can you test DNS like this, without using DNS cache;

dig @8.8.8.8 bora.sh +short

DNS/DROP only working for DNS record checkings

boratanrikulu · 2024-07-27T22:51:01Z

What's the MTU on ens3?

1400

mrbluecoat · 2024-07-27T22:56:24Z

Can you test DNS like this, without using DNS cache;

Oh, I see.

# dig @8.8.8.8 +short example.com
;; communications error to 8.8.8.8#53: timed out

I guess this is only useful if the server has no upstream recursive DNS servers, right? (e.g. exclusively localhost DNS authoritative server)

boratanrikulu · 2024-07-27T23:12:11Z

❯ sudo ./build/durdur drop --dns "bora.sh"
❯ dig @127.0.0.1 bora.sh +short
;; communications error to 127.0.0.1#53: timed out
❯ sudo ./build/durdur undrop --dns "bora.sh"
❯ dig @127.0.0.1 bora.sh +short
172.67.137.35
104.21.26.157

I guess should be fine with local dns servers too. But DoH is not supported, if you have enabled DoH inside your local DNS server, durdur can't catch it.

mrbluecoat · 2024-07-27T23:17:02Z

If durdur is running on a proxy and I want to block a domain but not the IP (e.g. for situations where a shared host is using a single IP for hosting multiple domains) is it possible? I see the dig is blocked but curl can still access the website and devices connecting to the proxy server can still access the website.

For example, durdur drop --dns 'example.com' doesn't work but echo '0.0.0.0 example.com' >> /etc/hosts does.

boratanrikulu · 2024-07-27T23:24:03Z

It should be working with durdur too

❯ sudo ./build/durdur list all
{"127.0.0.8":3,"bora.sh":4}
❯ curl bora.sh
curl: (6) Could not resolve host: bora.sh

Are you using DoH?

boratanrikulu · 2024-07-27T23:29:56Z

❯ curl https://www.google.com -I 2&>1 | head -n1
HTTP/2 200 
❯ sudo ./build/durdur drop --dns "www.google.com"
❯ curl https://www.google.com -I
curl: (6) Could not resolve host: www.google.com

boratanrikulu · 2024-07-27T23:32:18Z

Maybe there are some logic issues, not sure

if (ip->protocol == IPPROTO_UDP)
{
	struct udphdr *udp;
	if (data + sizeof(struct ethhdr) + sizeof(struct iphdr) + sizeof(struct udphdr) > data_end)
	{
		return XDP_PASS;
	}

	udp = data + sizeof(struct ethhdr) + sizeof(struct iphdr);
	if (udp->source == bpf_htons(53))
	{
		if (data + sizeof(*eth) + sizeof(*ip) + sizeof(*udp) + sizeof(struct dnshdr) > data_end)
		{
			return XDP_PASS;
		}

		struct dnshdr *dns = data + sizeof(*eth) + sizeof(*ip) + sizeof(*udp);
		if (dns->opcode == 0) // it's a dns query.
		{
			void *query_start = (void *)dns + sizeof(struct dnshdr);

			struct dnsquery query;
			if (!parse_query(data_end, query_start, &query))
			{
				return XDP_PASS;
			}

			long *pkt_count = bpf_map_lookup_elem(&drop_dns, &query.name);
			if (pkt_count)
			{
				printk("[BLOCK] DNS QUERY TO %s", &query.name);
				__sync_fetch_and_add(pkt_count, 1);
				return XDP_DROP;
			}
			printk("[ALLOW] DNS QUERY TO %s", &query.name);
		}
	}
}

boratanrikulu · 2024-07-27T23:36:37Z

if (udp->source == bpf_htons(53))

Do you use different port for your local DNS server?

mrbluecoat · 2024-07-27T23:37:57Z

# curl -si https://www.google.com | head -n1
HTTP/2 200 
# durdur drop --dns "www.google.com"
# durdur list dns
{"example.com":0,"www.google.com":0}
# curl -si https://www.google.com | head -n1
HTTP/2 200 
# nslookup www.google.com
Server:         127.0.0.53
Address:        127.0.0.53#53

Non-authoritative answer:
www.google.com  canonical name = forcesafesearch.google.com.
Name:   forcesafesearch.google.com
Address: 216.239.38.120
Name:   forcesafesearch.google.com
Address: 2001:4860:4802:32::78

mrbluecoat · 2024-07-27T23:39:18Z

I'm using Tailscale, so that may be affecting it... https://tailscale.com/kb/1054/dns?tab=linux

mrbluecoat · 2024-07-27T23:41:39Z

dig @8.8.8.8 bora.sh +short

As a completely random aside, I see you're developing an observability and monitoring agent at Sematext. Is durdur the spiritual successor to https://github.com/sematext/oxdpus ?

boratanrikulu · 2024-07-27T23:44:30Z

No it's not. These are different projects. Durdur is not directly related to Sematext

boratanrikulu · 2024-07-27T23:45:02Z

I'm using Tailscale, so that may be affecting it... https://tailscale.com/kb/1054/dns?tab=linux

Maybe they have DoH enabled. I'm not sure if we can catch DNS in that case

mrbluecoat · 2024-07-27T23:46:20Z

I'm using Tailscale, so that may be affecting it... https://tailscale.com/kb/1054/dns?tab=linux

I figured it out. For IP blocking (src), I have to attach to the WAN NIC but for DNS blocking I have to attach to the tailscale0 virtual interface:

# ip a | grep tailscale
3: tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 xdpgeneric/id:4117 qdisc fq_codel state UNKNOWN group default qlen 500
    inet 100.103.191.142/32 scope global tailscale0

Makes sense in hindsight. Can I run two instances of durdur concurrently on different interfaces?

boratanrikulu · 2024-07-27T23:49:19Z

Right now we don't have support for multiple interfaces. But I guess it would be easy to add that feature.

mrbluecoat · 2024-07-27T23:53:10Z

That would be awesome. A comma-separated option would be fine for my needs, like: https://github.com/ahsifer/goxdp/blob/main/README.md?plain=1#L75-L78

mrbluecoat closed this as completed Jul 27, 2024

mrbluecoat reopened this Jul 27, 2024

boratanrikulu mentioned this issue Jul 27, 2024

Multiple interfaces support #12

Open

boratanrikulu closed this as completed Jul 28, 2024

Modes supported #10

Modes supported #10

Comments

mrbluecoat commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024 • edited Loading

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024 • edited Loading

mrbluecoat commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024 • edited Loading

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024 • edited Loading

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024 • edited Loading

boratanrikulu commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024 • edited Loading

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024 • edited Loading

boratanrikulu commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024

mrbluecoat commented Jul 27, 2024 •

edited

Loading

boratanrikulu commented Jul 27, 2024 •

edited

Loading

boratanrikulu commented Jul 27, 2024 •

edited

Loading

mrbluecoat commented Jul 27, 2024 •

edited

Loading

mrbluecoat commented Jul 27, 2024 •

edited

Loading

boratanrikulu commented Jul 27, 2024 •

edited

Loading

mrbluecoat commented Jul 27, 2024 •

edited

Loading