-
Notifications
You must be signed in to change notification settings - Fork 174
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Intel igb/em Interfaces Broken on 2.6/22.01+ #67
Comments
Can confirm its not working after an upgrade. Following the troubleshooting instructions shows that the modules have loaded. PFatt logs dont show anything. |
Can confirm it is broken for me as well, running supplicant |
Same issue. Not grabbing DHCP. Edit: I am using the WPA supplicant method. |
Want to confirm that reverting to 2.5.2 or 21.05.2 immediately restores internet for me after setting everything back up. |
Yes, It was an absolute pain in the a**, but restoring to 21.05.2 immediately fixed it for me. IPv6 wouldn't grab for an hour or so but finally started working. |
Also, I posted on the Netgate Forums. If anyone else wants to add anything over there here is the link. https://forum.netgate.com/topic/169882/22-01-2-6-0-upgrade-broke-dhcp-on-wan-interface-with-custom-startup-script |
I am having the same problem and now my WireGuard and other tools don't work and can't get them to work. |
Yep - supplicant not working for me either. The last time a new version of pfsense broke pfatt Matt Johnson submitted this issue to pfsense redmine. Should we do that here? Here is the issue that originated the whole thing. |
It also broke mine after update. Per the docs, I ran "tcpdump -ei ONT_IF" and "tcpdump -ei RG_IF", which should filter and capture link layer information (2), on my interfaces and captured 0 packets from RG_IP and only the bridged DHCP traffic on the ONT_IF interface. I reset netgraph, which removes the hooks, rebooted the gateway and modem with tcpdump running and captured 0 packets from the interfaces. Before removing the netgraph hooks, the only traffic I seen on any of the three interfaces, was the DHCP request on the ngeth0 virtual interaface, and the bridged ONT_IF interface. So the DHCP requests are still getting to the correct interface. The fact that tcpdump doesn't see any traffic makes me think its being filtered, like promisc mode isn't allowing EAPOL 802.1X traffic to be capture, and there fore is not bridged. No authentication mean no DHCP response. IMO I've moved to inline behind the gateway until this can be figured out. I would be willing to test once a day. |
Okay, had some success today based on info I gathered from all the various discussions online. I think it is something to do with the em(4) driver. Do all of you having issues have Intel NIC's? I put together a test pfSense server from a bunch of spare parts and it worked right away on the latest release. After digging, I couldn't get any Intel NIC to work. Using what I had around (a few crappy USB dongles worked and old PC's with integrated NICs) I had success with everything not Intel GbE. When I re-upgraded my main pfSense box I was able to move my WAN link to an SFP slot (with RJ45 Module) with some success. I say "some" because all my SFP/RJ45 modules are 10GB and they do not negotiate well with the ONT. Something interesting for me, if_em.ko is present in /boot/kernel on 2.6.0 but wasn't in my previous version of pfSense. My knowledge is limited but I am not sure where the driver was located in the previous version? Anyone smarter than me know? Some Useful Links: |
I think this is going somewhere because I've tried multiple different boxes but they're all Intel Nics, when I get off work I will try a couple USB dongle's to see if it gets traffic that way. |
The USBs work for me but are slow. Download is like 100m, upload is better at around 400m. I have a 1G SFP that should get here tomorrow. Really hoping that talks better with the ONT then the 10G did. |
For USBs to work at 1 gig speeds you have to have 3.1 USB port or better. For FreeBSD, I am using a box equivalent to the netgate 1541 Same everything but a lot more powerful. Let me know how it goes with the other Nics. |
Will do! If it helps I'm using the XG-1537 so USB3.0 |
Is the usb dongles 3.0, when I was using usb in past it worked great I was able to get full 1gb speeds out of my usb ports. If the usb is 3.0 then I don't know why I am getting full 1gb speeds. But I did downgrade back to 2.5.2 now WireGuard don't work on 2.5.2. |
Yep! |
Nothing useful to add here but I can confirm I'm using an Intel NIC with the em driver. Neither tethered or supplicant working for me on 22.1 but supplicant is working on 21.7.8 em0: <Intel(R) 82583V> port 0xe000-0xe01f mem 0xdf500000-0xdf51ffff,0xdf520000-0xdf523fff irq 16 at device 0.0 on pci1 I'm on a Protectli FW6D |
I am using an Intel NIC but with the IGB driver. |
And is it working or not because my system is using igb drivers too and mine is not working |
My knowledge on FreeBSD is limited but I believe igb uses the em(4) driver. All the common Intel cards fall under it (I350, 82575, etc.) |
Not working |
If you look at the if_igb.ko driver in /boot/kernel it just is a shortcut to if_em.ko. I think at one point the two intel drivers merged. https://www.intel.com/content/www/us/en/download/15187/intel-network-adapter-gigabit-base-driver-for-freebsd.html?wapkw=i350%20freebsd |
Okay, I got everything up and working on my regular Intel NIC. I’m not the biggest expert here so bear with me. Through troubleshooting I was able to get every non-Intel NIC to authenticate and pull DHCP. After more testing all igb(4) driver-based cards failed. In the /boot/kernel folder I noticed if_igb.ko is simply a shortcut to the em(4) driver (if_em.ko). I am guessing FreeBSD is using this combined driver from intel? https://www.intel.com/content/www/us/en/download/15187/intel-network-adapter-gigabit-base-driver-for-freebsd.html Alternatively, I found this driver that appears to be for igb(4) separately, and it seems newer. https://www.intel.com/content/www/us/en/download/14610/intel-network-adapter-driver-for-82575-6-and-82580-based-gigabit-network-connections-under-freebsd.html?wapkw=i350%20freebsd I downloaded a FreeBSD-12.3 VM, its related source code (amd64), and complied the separate igb(4) driver. I loaded my newly compiled if_igb.ko into the /boot/modules folder with chmod 555 permissions. Next, I added the following two lines to my /boot/loader.conf file to supersede the included driver. if_igb_load="YES" Rebooted and everything came up just fine! Feel free to use my compiled if_igb.ko if you don’t want to build your own. Also, for reference here is my pfatt script if anyone needs a reference. A few notes:
|
Interesting that the intel igb driver works. I searched for bugs on the FreeBSD buglist and found this... https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=260068 Looks like it might be related? Issues with vlan tagging. Was introduced in 13.0 and 12.3... recently fixed in the stable branches, so the timing lines up. |
Some comments and feedback in testing so far:
|
Could explain why we are passing 802.1x not pulling DHCP on VLAN 0. I'll add it to my redmine issue on pfSense. If anyone else has success can they go on and comment. Hopefully we get some traction! https://redmine.pfsense.org/issues/12821?next_issue_id=12820 |
I am testing now reimaging since wiregraud is broke in my install right now. |
i am testing now reimaging since wiregraud is broke in my install right now.
will do internet going out for a bit to update and bring system online. |
Good point on the .local, will adjust that. For my kldstat does just this portion work for ya or do you want the whole output? 3 1 0xffffffff83cfb000 35e08 if_igb.ko (/boot/modules/if_igb.ko) |
I've been too busy to formally submit any bugs. The FreeBSD upstream fixes are all over the place, I'm with you, hard to follow. I re-enabled vlanhwtso and my setup still works fine. Confirming your experience. For reference, I have the vlanhwtag vlanhwfilter enabled on my LAN em0 Intel interface with multiple VLANS in OPNSense and it works fine. I do think it is an igb related FreeBSD driver bug that these HW vlan offloading options don't work. Additionally it looks like the 22.1.4 codebase for the e1000 driver (igb and em in Freebsd) remains unchanged with a Dec 22, 2021 modified date. So, these disabling options are still needed for the time being. |
Tested on pfSense Plus 22.05 and getting the following: Fatal error: Uncaught Error: Call to undefined function pfSense_ngctl_attach() in Command line code:1 I also tried running the PHP code directly without the variables and it threw the same errors to console. I'll attach this to the redmine as well. |
The pfSense_ngctl_attach() and pfSense_ngctl_detach() php modules are being removed in pfSense Plus 22.05 and pfSense CE 2.7, which is why this command is failing on the latest builds. These commands are apparently no longer needed as all interfaces will always be a part of netgraph in the newer release, so these lines should be able to be commented out. I'll test with these lines removed in the next few days. If it works without them, I'll create a fork and/or open an issue with the project to correct it. |
Support newer versions of pfSense without pfSense_ngctl_attach. Relates to MonkWho#67
A bit different than the Intel issue being discussed previously but that one's actually easy to fix. Added a pull that supports old and newer versions of pfSense while people update. Currently running opnsense so I haven't tested it past confirming it parses correctly but its pretty straight forward. If you want to give it a shot it'd be appreciated. |
Thank you for the patch neclimdul! I've tested with the patched script. My testing can be found here: https://redmine.pfsense.org/issues/12821#note-14 Seems that on internal builds of pfSense Plus 22.05 there is still an issue with DHCP over VLAN0 on the included Intel driver. I've raised the flag on this internally so hopefully there will be a fix soon. Even if it isn't, though, things should keep humming along for non-Intel NICs or with the custom driver, which I'll compile a new one of and post here from my build environment. |
FYI to anyone not watching the redmine or who stumbles on this bug report, the issue ONLY affects igb/em interfaces. Intel ix or igc are not affected and likely Realtek, Broadcom, or any other NIC should be fine as well. For igb/em interfaces, just compile a driver from the Intel provided driver above and load in the driver with the loader.conf.local override. |
@ChronicledMonocle would you then suggest those of us with Realtek, Broadcom etc chippers create a different bug report then? |
What kind of NIC do you have and what issue are you running into? I'm running this script on a box with ix and igc interfaces without issues and unaltered just fine. |
22.05 has been released. Has anyone been able to test the final release version yet? |
I have. Still broken for igb/em interfaces. Works fine on other types of interfaces I've tested. |
The base hasn’t changed from FreeBSD 12.3, so does the existing driver solution for 22.01 work? In short, can you safely direct update if you are on 22.01 to 22.05? |
If the major kernel version hasn't changed, should be fine. |
Rolled the dice and updated. So far so good. |
Working fine with 22.05 no problems with both types |
For those already using the customer driver, when you upgrade the custom driver stays in place. The root issue is not solved yet but has been marked in Redmine for the "next" release. https://redmine.pfsense.org/issues/12821?next_issue_id=12820 If you do a clean install to 22.05 the issue will probably come back. |
I installed opnsense 22.07 and this issue still persists with the Intel interfaces. I changed the config to a Realtek interface and then the script worked. |
@neydah700 you saved the day my friend! 2.6 was my first attempt with pfSense. I was about to give up. The above did the trick. I'm up and running and my ATT 1Gb has never been faster. Thank you! |
We might finally be able to close this issue out. https://redmine.pfsense.org/issues/12821?next_issue_id=12820 |
Having issues with ixRecently updated and all hell broke loose. Not using I did a backup and clean reinstall + restore, however, now earlyshellcmd is not included, so script can't auto exec on startup, tried symlinking After applying PR #73 the issue appears to be at uname -a
Error:
Log:
sysctl dev.ix.0
|
@anthonywww there's no reason to use the tethered method anymore, nor supplicant with netgraph/switch. pfSense includes the necessary patches in wpa_supplicant and dhclient already. All you need are the certificates. |
bump #67 (comment) |
If you still want to use the tethered method and not any of the other workarounds you don't need, and probably shouldn't use, netgraph anymore. There is functionality built into pfsense now. https://docs.netgate.com/pfsense/en/latest/recipes/authbridge.html |
Ah yes, I've seen that article, though it doesn't appear to work for the CE (non-Plus) version of PfSense unfortunately. Is there a guide for the CE version? |
If there is a guide I never found it. I decided to use the supplicant method since pfsense now supports it in all versions. |
The dhcp lease for connections is not handed through to the ngeth0 interface properly. There isn't any real "errors" in the logs.
If you try to run the script manually after boot you get "ngctl: send msg: File exists"
Logs from pfatt.log:
2022-02-14 14:36:56 :: [pfatt.sh] :: pfSense + AT&T U-verse Residential Gateway for true bridge mode
2022-02-14 14:36:56 :: [pfatt.sh] :: Configuration:
2022-02-14 14:36:56 :: [pfatt.sh] :: ONT_IF: igb0
2022-02-14 14:36:56 :: [pfatt.sh] :: RG_IF: igb1
2022-02-14 14:36:56 :: [pfatt.sh] :: RG_ETHER_ADDR: [MY MAC HERE]
2022-02-14 14:36:56 :: [pfatt.sh] :: attaching interfaces to ng_ether... OK!
2022-02-14 14:36:56 :: [pfatt.sh] :: building netgraph nodes...
2022-02-14 14:36:56 :: [pfatt.sh] :: creating ng_one2many... 2022-02-14 14:37:00 :: [pfatt.sh] :: pfSense + AT&T U-verse Residential Gateway for true bridge mode
I am not running wpa_supplicant mode.
The text was updated successfully, but these errors were encountered: