Please make tcpdump on 'wan' interface and examine with wireshark.
Reject sends packet bigger than SYN back, eso bad if you have asymmetrical internet like cable or dsl, you might want to drop wan input too. It does not influence established connections or anything else explicitly allowed in other rules.
Can you produce domain.xml from your vm? I dont quite get what is virtio bridge - is it outside vm or inside vm?

Please try:
nft 'insert rule inet fw4 handle_reject limit rate over 1000/second burst 50 packets counter drop'
And observe counter in nft list ruleset
It will not persist over reboot or firewall restart, like reconfig from luci web.

This is limiting ICMP generation roughly guided by kernel defaults:
sysctl net.ipv4.icmp_ratelimit net.ipv6.icmp.ratelimit net.ipv4.icmp_msgs_burst net.ipv4.icmp_msgs_per_sec

EDIT: fix formatting to alow copypasta commands.

Basically test:
1/ confirm device and internet works fine with DROP
2/ Enable REJECT on WAN, wait a minute and confirm that pings are lost
3/ insert extra rule
4/ wait another minute and check if device works good again
5a/if yes, confirm that new rule counter grew and new rule should take fame
5b/if not - ainsert new rule - reduce rate and burst 10x, of 2 prepended rules 1st will work i.e one prepended last.

While at it you can have all linux bridges with virtio-net adapters, even trunking VLAN-s in physical adapters to stuff more isolated networks if switch can support it.

Thank you for the fast response.

I now suspect this to be routing issue at my provider.

With logging enabled on the wan zone (with the default fw rules) I see a lot of random packages.
Many:
reject wan out: IN=eth3 OUT=eth3 MAC=...
and fewer:
drop wan in: IN=eth3 OUT= MAC=...

The reject packages have DST not only of the IP range of my modems current network, but also of other ranges of my provider.

OpenWrt default setting of reject do apparently just seem to reach the rate limit under this packet spam conditions.

Does rate limiting your ICMP output rate solve the connection quality issue? (DROP in 2 places is also completely valid solution, im just assessing if "common case" can be stabilised)

That looks like a IPv6 only rule, I use IPv4.

If you look at that chain you see it does reset of tcp and default icmp4/icmp6 action for all other types.

        chain handle_reject {
                limit rate over 1000/second burst 50 packets counter packets 0 bytes 0 **drop**
                meta l4proto tcp **reject** with tcp **reset** comment "!fw4: Reject TCP traffic"
                **reject** comment "!fw4: Reject any other traffic"
        }

Understand. I mixed it up with the already existing rule set that also uses the same limits.

I used a fresh OpenWrt image to test this to prevent any other of my changes influencing it.

Using this extra rule makes no difference. Also its counter stays at 0.

64 bytes from 8.8.8.8: seq=119 ttl=120 time=8.404 ms
^C
--- 8.8.8.8 ping statistics ---
124 packets transmitted, 41 packets received, 66% packet loss
round-trip min/avg/max = 8.194/37.563/1073.108 ms

chain handle_reject {
                limit rate over 1000/second burst 50 packets counter packets 0 bytes 0 drop
                meta l4proto tcp reject with tcp reset comment "!fw4: Reject TCP traffic"
                reject comment "!fw4: Reject any other traffic"
        }

Edit: Same with this rules. With wan Forward reject I have package loss, with wan Forward drop it works fine. But this lower limits cause some package drop now.

nft 'insert rule inet fw4 handle_reject limit rate over 100/second burst 5 packets counter drop'

nft 'insert rule inet fw4 handle_reject limit rate over 10/second burst 1 packets counter drop'

burst 5 is default, one of 3 '1000' sysctl settings is millisecs between generated packets i.e 1/second burst 5 packets
10/sec generated dozen hits/h
1/sec thousands/h
(just counter without drop to measure)
If you look at forward-wan chain+jumps it quite directly hits reject terminal chain. Some rules have counters - all zeroes dxcept last jump and new rules.

Ping @jow-

@PixelOfDeath it will apply only to new clean installations, keep your DROP rule as you figured out to transfer it via config backups.

I think I now have a working hypothesis what happens.
I obviously get incorrect packages send to my MAC. I'm sure this is a configuration error on my providers end.
When OpenWrt sends a reject packet for a packet/IP that it should not have received in the first place, there must be a rule on one of the provider routers that does detect that I respond for packages/IPs that I should have nothing to do with. And then for a very short time (~1 Second?) all packages from and/or to me get dropped.

A simple test where each 20 second one reject package causes my connection to drop:
Default settings, expect that wan Input drop and only wan Forward is left to reject

root@OpenWrt:~# nft 'insert rule inet fw4 handle_reject limit rate over 3/minute burst 1 packets counter drop'
root@OpenWrt:~# ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: seq=0 ttl=120 time=8.529 ms
64 bytes from 8.8.8.8: seq=1 ttl=120 time=8.861 ms
64 bytes from 8.8.8.8: seq=2 ttl=120 time=8.857 ms
64 bytes from 8.8.8.8: seq=3 ttl=120 time=9.286 ms
64 bytes from 8.8.8.8: seq=4 ttl=120 time=9.013 ms
64 bytes from 8.8.8.8: seq=5 ttl=120 time=9.264 ms
64 bytes from 8.8.8.8: seq=6 ttl=120 time=8.605 ms
64 bytes from 8.8.8.8: seq=7 ttl=120 time=8.846 ms
64 bytes from 8.8.8.8: seq=8 ttl=120 time=9.333 ms
64 bytes from 8.8.8.8: seq=9 ttl=120 time=9.137 ms
64 bytes from 8.8.8.8: seq=10 ttl=120 time=8.786 ms
64 bytes from 8.8.8.8: seq=11 ttl=120 time=8.241 ms
64 bytes from 8.8.8.8: seq=12 ttl=120 time=22.412 ms
64 bytes from 8.8.8.8: seq=13 ttl=120 time=12.809 ms
64 bytes from 8.8.8.8: seq=14 ttl=120 time=13.356 ms
64 bytes from 8.8.8.8: seq=15 ttl=120 time=10.094 ms
64 bytes from 8.8.8.8: seq=16 ttl=120 time=8.831 ms
64 bytes from 8.8.8.8: seq=17 ttl=120 time=12.634 ms
64 bytes from 8.8.8.8: seq=18 ttl=120 time=9.542 ms
64 bytes from 8.8.8.8: seq=19 ttl=120 time=8.436 ms

64 bytes from 8.8.8.8: seq=21 ttl=120 time=8.392 ms
64 bytes from 8.8.8.8: seq=22 ttl=120 time=9.828 ms
64 bytes from 8.8.8.8: seq=23 ttl=120 time=9.738 ms
64 bytes from 8.8.8.8: seq=24 ttl=120 time=12.441 ms
64 bytes from 8.8.8.8: seq=25 ttl=120 time=9.286 ms
64 bytes from 8.8.8.8: seq=26 ttl=120 time=10.555 ms
64 bytes from 8.8.8.8: seq=27 ttl=120 time=18.915 ms
64 bytes from 8.8.8.8: seq=28 ttl=120 time=8.971 ms
64 bytes from 8.8.8.8: seq=29 ttl=120 time=8.565 ms
64 bytes from 8.8.8.8: seq=30 ttl=120 time=9.218 ms
64 bytes from 8.8.8.8: seq=31 ttl=120 time=7.847 ms
64 bytes from 8.8.8.8: seq=32 ttl=120 time=16.414 ms
64 bytes from 8.8.8.8: seq=33 ttl=120 time=8.647 ms
64 bytes from 8.8.8.8: seq=34 ttl=120 time=12.151 ms
64 bytes from 8.8.8.8: seq=35 ttl=120 time=23.497 ms
64 bytes from 8.8.8.8: seq=36 ttl=120 time=9.212 ms
64 bytes from 8.8.8.8: seq=37 ttl=120 time=8.738 ms
64 bytes from 8.8.8.8: seq=38 ttl=120 time=11.812 ms
64 bytes from 8.8.8.8: seq=39 ttl=120 time=10.038 ms

64 bytes from 8.8.8.8: seq=41 ttl=120 time=10.006 ms
64 bytes from 8.8.8.8: seq=42 ttl=120 time=8.822 ms
64 bytes from 8.8.8.8: seq=43 ttl=120 time=10.787 ms

Note that the default settings with wan Input reject + wan Forward drop work perfectly fine and my connection never stops working. Even when sending reject responses for the traditional port scanners/viruses/whatever that is hammering my IP.

VM sends rejects on packets that switches leak to your connection wire and probably your provider has some punisher script for OOB resets sent...
You see the glitch like every 20s now, rhyming with 3/min.

(leave at "DROP", the global limit on nft-generated packets is other discipline, ill do an independed try on that, some places have limit some none)

Summary - when something enabled promiscuous mode (eg ip link set wan promisc on) - nftables firewalls takes misdirected packets and "rejects" them with a spoofed packet. Also worth enabling rp_filter on wan interface, as by default it is possible to drop ICMP errors/tcp ressets to inside LAN by spofing packets from it via wan.