I have a semi-bricked IPFire box

Hey guys.

Alright so, I have had this one box running IPFire for close to three years now. It’s awesome!
Unfortunately a couple of nights ago, I decided to take it from its Core 154 version to Core 158. So I went over the the pakfire tab on the WebUI, clicked update and away it went.
I kept my eye on it while I did other things, and it was progressing smoothly like usual. At one point I noticed it was stuck applying 157. I figured I’d let it just sit there and do whatever it needed to do, so I went to bed. I woke up the next day, to the same screen and I thought “uh oh”.
I opened a browser on a different device, pointed it to my IPFire’s webUI and to my horror I was met with a “PR_CONNECT_RESET_ERROR”.
Now, here’s the funky part: Some features seem to work. I can still log in at the box itself. It will do port forwarding, it will do DHCP, it will do its intrusion detection/prevention stuff. It will not display the WebUI, VPN stopped working and it will not resolve DNS stuff. Now, thankfully I have a separate device for DNS, so my network is not down, but I feel like it’s going to die any day now.
I tried to sort of force the update again buy manually telling it that it was back on Core 156, and then trying to run pakfire from the terminal, and that’s how I discovered that it wasn’t resolving things. It could not resolve any of the update servers. I tried researching the hostnames and IPs of the servers as much as I could and added them manually so it could resolve, and some of them actually worked and downloaded some stuff, but something always fails.
Any idea on how I could “Repair” my firewall? Should I just bite the bullet and re-do it? I tried restoring a backup config from a few days back, but it keeps failing with some “access denied” errors (while running as root ??) so I dunno. ¯_(ツ)_/¯

Please read the thread trouble-when-upgrading-from-core-157-to-core-158. At a first glance, your problems seem to be similar.

Please run at the root directory ‘/’ the command

find -type d -perm 700

and post the output here. This will show all directories with a permission 0700.
Some of them may have a wrong permission.

1 Like

Oh man! This was probably posted after my initial investigation. Thank you for pointing it out. I have some reading to do. :grinning:

Hi,

welcome to the IPFire community. :slight_smile:

Indeed, this looks as you are suffering from the same file permissions error (which we suspect is actually tar bug, fixed in this commit, so it cannot happen in future releases again). Thanks to this change, Core Update 158 now fixes any faulty permissions for good measure - installations already upgraded remain unchanged.

If the thread linked by @ewald applies to you as well, please confirm so. To avoid duplicates, I will then close this thread, and ask for posting further questions in the original one. :slight_smile:

Thanks, and best regards,
Peter Müller

Hi Peter,

Thank you for this. I am almost certain this is the same issue, but I will not know for sure until I can have some downtime tonight to check and reboot if necessary. I’ll be sure to update this thread if it is indeed the same bug.

1 Like

oh man. There’s like 4 pages of 700 perm directories.

pretty much all of /var, all of /etc a lot of /lib almost all /usr, /boot… What a mess. lol

I’ll start fixing these and report back.

Read thru the other thread for the chmod command:

chmod -v 755 /usr /usr/bin /usr/lib /usr/sbin /var /var/ipfire

This might make things easier!

2 Likes

ok so I did do that, it brought my WebUI back, and vpn. Unfortunatelly DNS is still broken.

If I try to update I get “Giving up: There was no chance to get the file <filename.db> from any available server. There was an error on the way, please fix it”

If I try to nslookup anything, it comes back with "connection timed out, no servers found.

Oddly enough, everything else on the network, for the most part resolves just fine.

Alright, I’m getting close to just giving up. So I tried the following out of desperation:

I manually changed the DNS on my ipfire from localhost to my other DNS appliance. I rebooted, and was able to update and upgrade to 158.

I’m about to restart. If I never make it back, you guys can assume I gave everything up and moved to a deserted island to live off of the land.

Wish me luck

1 Like

Welp. That did not do it.

Half my stuff can’t resolve anything now. At least I can get to the UI now so I can back up my config easily.

I guess I’ll be pulling an all-nighter tonight, reinstalling ipfire from scratch.

See ya later!

When you say it is broken, what error messages are you finding in the log if you grep for unbound?

Are you using your ISP’s DMS servers or have you added some other servers on the DMS servers menu page?

Does the Status line at the top of the page say Working in green or Broken in red?

If you press Check DNS Servers is the status of any of the DNS servers showing up as OK in green or is their a red broken against all of them?

If you hold your mouse pointer for a short period over the red broken status for each of the DNS servers that are not working what error message is displayed?

Alright. Next chapter in the saga:

I backed up my config.

I reinstalled IPFire. Easy peasy lemon squeezy.

I restored my DHCP settings, then I restored my firewall rules. Turned on intrusion prevention and geo blocking. Everything good up to that point.

I start going around and making sure that all my servers and devices can talk to each other. Everything is just honky-dory, and I started thinking “Well, you know? That wasn’t so bad!” … and that, ladies and gentlemen, is where I went wrong.

I proceeded to re-do my VPN. Generated Certs, created a connection for my laptop thinking I may continue this remotely. So, I hop on my hotspot to test it… no dice. I say to myself: “Dude, you’re tired, sleep-deprived and under caffeinated. You did something wrong.” So I start over. Nothing.

So I figured, I don’t really need the VPN this minute. I’ll just get the web proxy back online, and call it a night. Except when I turn it on, DHCP breaks (!!) .

So there. I need a drink.

Hey. Thanks for all that, but I’ve already wiped and started over.

To answer your questions though:

I do have a separate DNS appliance. The funny thing is that IPFire did assign my DNS server as part of the DHCP lease to all the clients. On the clients themselves, DNS would break intermitently, but curiously enough, the DNS request wasn’t even making it to the DNS server. Every time it worked, I could see it in the DNS server logs, weather it was blocked or not. But when the client failed to resolve something, it wouldn’t even show in the logs. Not IPFire’s, not the DNS.

I did try to set a whole bunch of different DNS servers as a test. And DNS was green and working.

The problem was that IPFire itself wouldn’t be able to resolve anything, until I manually changed the resolv.conf file to point away from itself, THEN I could resolve and update stuff. But even after updating stuff was still wonky (and yes, that is a technical term) so that’s why I decided to reinstall.