2 Raspberry Pi 3 B+ at different locations red DHCP stopped working permanently

nax_za · 22 January 2021 08:24

Hi,

I decided several months back that I would like to have an open-source firewall between my ISP and my networks that I controlled.

I wanted this at my office and home.

So I purchased 2 Raspberry Pi 3 B+ with a TP-Link USB to Ethernet and set up the built-in Ethernet as my red interface with DHCP and the TP-Link a static IP connected to my router and made ti the gateway for my network.

This setup worked really well for a long while with little to no issues.

Then in the space of a week, both home and office Pi’s stopped getting DHCP IPs on the red interface after a power cycling.

Since then, I have tried everything I know to try and figure out what is going on here but nothing works.

I even tried on a fresh install on another memory card to going back to older version, trying both core-151 and 149.

Nothing seems to be working. And if I bypass IPFire and connect other devices directly to the ISP they get an address without any issues.

Below is an extract from my DHCP on the one Pi.

21:37:44    dhcpcd[1551]    : dhcpcd-9.1.2 starting
21:37:44    dhcpcd[1553]    : DUID 25:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:ce
21:37:45    dhcpcd[1553]    : red0: IAID 25:b4:95:ce
21:37:45    dhcpcd[1553]    : red0: adding address fexx::xxxx:xxxx:xxxx:xxce
21:37:45    dhcpcd[1553]    : ipv6_addaddr1: Permission denied
21:37:45    dhcpcd[1553]    : red0: carrier lost
21:38:15    dhcpcd[1553]    : timed out
21:38:15    dhcpcd[1553]    : dhcpcd exited
21:39:51    dhcpcd[1516]    : dhcpcd-9.1.2 starting
21:37:44    dhcpcd[1553]    : DUID 25:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:ce
21:39:51    dhcpcd[1518]    : red0: waiting for carrier
21:40:21    dhcpcd[1518]    : timed out
21:40:21    dhcpcd[1518]    : dhcpcd exited
21:43:19    dhcpcd[2011]    : dhcpcd-9.1.2 starting
21:43:19    dhcpcd[2013]    : DUID 25:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:ce
21:43:20    dhcpcd[2013]    : red0: waiting for carrier
21:43:50    dhcpcd[2013]    : timed out
21:43:50    dhcpcd[2013]    : dhcpcd exited
21:44:49    dhcpcd[2350]    : dhcpcd-9.1.2 starting
21:44:49    dhcpcd[2352]    : DUID 25:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:ce
21:44:49    dhcpcd[2352]    : red0: waiting for carrier
21:45:19    dhcpcd[2352]    : timed out
21:45:20    dhcpcd[2352]    : dhcpcd exited
21:46:03    dhcpcd[2788]    : dhcpcd-9.1.2 starting
21:46:03    dhcpcd[2790]    : DUID 25:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:ce
21:46:03    dhcpcd[2790]    : red0: waiting for carrier
21:46:33    dhcpcd[2790]    : timed out
21:46:33    dhcpcd[2790]    : dhcpcd exited
22:02:14    dhcpcd[2413]    : dhcpcd-9.1.2 starting
22:02:14    dhcpcd[2415]    : DUID 25:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:ce
22:02:15    dhcpcd[2415]    : red0: waiting for carrier
22:02:45    dhcpcd[2415]    : timed out
22:02:45    dhcpcd[2415]    : dhcpcd exited
22:04:25    dhcpcd[2882]    : dhcpcd-9.1.2 starting
22:04:25    dhcpcd[2884]    : DUID 25:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:ce
22:04:25    dhcpcd[2884]    : red0: waiting for carrier
22:04:56    dhcpcd[2884]    : timed out
22:04:56    dhcpcd[2884]    : dhcpcd exited

Any guidance as to what is going on here would be greatly appreciated. If I can’t get this working I will need to move to OpenWRT for my firewall something I am trying to avoid because IPFire is the better firewall.

Thank you.

bbitsch · 22 January 2021 10:50

dhcpdc is waiting for a carrier, means the ethernet interface isn’t up.
Did you check the direct connection with the same cable?
Is the interface coming up, if you change the association ( internal=green, USB=red )?

nax_za · 22 January 2021 16:33

I have to say I am now even more confused.

I first tried ethtool -r red0. This gave an error. I then start trying different combinations of things.

If I have Ethernet and TP-Link (USB) plugging I see a lot of this in the messages log.

kernel: lan78xx 1-1.1.1:1.0 red0: Failed to read register index 0x00000120. ret = -110
kernel: lan78xx 1-1.1.1:1.0 red0: Failed to read stat ret = -110
kernel: lan78xx 1-1.1.1:1.0 red0: kevent 4 may have been dropped
kernel: lan78xx 1-1.1.1:1.0 red0: Failed to write register index 0x00000410. ret = -110
kernel: lan78xx 1-1.1.1:1.0 red0: kevent 4 may have been dropped
last message repeated 5038 times
kernel: lan78xx 1-1.1.1:1.0 red0: Failed to write register index 0x0000000c. ret = -110
kernel: lan78xx 1-1.1.1:1.0 red0: link reset failed (0)
kernel: lan78xx 1-1.1.1:1.0 red0: Failed to write register index 0x000004a0. ret = -110
kernel: lan78xx 1-1.1.1:1.0 red0: kevent 4 may have been dropped
last message repeated 5038 times

If I only have the TP-Link USB plugged in and connected as RED it comes up but with the wrong lease.

If I statically configure the TP-Link USB to green, connected to a switch, and leave RED unplugged. It works, I can SSH between it and my laptop.

If I unplug the TP-Link and connect only the Ethernet, it does the same thing. It comes up but keeps complaining about a lease being offered on the green interface and ends up with an IP on the 10.0.0.x range.

So if I manually configure the RED interface to use the IP that was offered, it works. I have internet and can ping and resolve IP addresses.

If I then plugin the TP-Link USB everything stops working until I unplug the USB and restart networking.

So in summary:

I can statically configure the TP-Link USB or the Ethernet I can it getting working but only on its own.
When ether is configured on its own to use DCHP, it fails, complaining about leases being offered on the wrong interface for the wrong network range.
If both are plugged in and configured in any way nothing works and I get read and write register index errors.

On a different note, I can plug the TP-Link into my Arch laptop and it works, no issues, it gets a DHCP lease and all is good.

I have never seen something so odd in all my life. And I am seeing the same thing on 2 different devices on to different networks and locations within a week of each other.

rodneyp · 22 January 2021 20:39

I’ve used various machines runing IPFire that uses USB-Ethernet as one zone. Some issues:

commercial modem/routers having Gb sockets might not work error-free with 10/100 devices connected
if RED link is dropped, for any reason, it might not come back up sucessfully
others report USB3 adapters going into power-saving mode

I suggest you try a different configuration, not too different from your original and install IPFire to a fresh SD card for it:

make the onboard Gb RED - for reliability of connection
set RED as STATIC - you are aware that DNS & Gateway settings are then mandatory

A modem/router should have at least 253 usable IP addresses. Note that you are assiging one as STATIC, not as fixed lease by the modem/router. The latter would not reduce delays in establishing a connection

nax_za · 25 January 2021 15:23

Thanks for the tips, it helped more than I am willing to admit to .

I have been able to re-instant one of the Pi’s back into service and still working on the second.

I am suspicious of the second device may be experiencing a POE surge (if that is possible) that caused some damage, because the second device is behaving very differently.

I have also learned a few things. Some things I seem to need to learn more than once and stubbornly keep forgetting

Some tips I learned using a Pi and IPFire

Always, always check your network cable before spending time troubleshooting. I can’t be 100% sure but one of the cables had a broken clip that I was not aware of. I don’t know if it played a role but I swapped out the cable and then started having better luck with troubleshooting.
DHCP on the built-in interface is fine but always set a static IP on a USB network interface. It’s a lot more stable.
Double check what interfaces you configure to the red, green, blue and orange aliases. It’s easy to get them the wrong way round.
If you plug in, change, or plug out a USB interface always restart the device and then run setup.
Don’t assume your ISP router is using NAT with DHCP using a local/private address space.
Don’t assume you can statically set an IP on red bypassing DHCP. An ISP can require a lease.
If you having DHCP issues you can try adding noarp; to /var/ipfire/dhcpd/dhcpd.conf.local to turn off arp conflict checking see Arch link below for more info.

Ref: dhcpcd - ArchWiki