After upgrade to release 180, pakfire does not work

moisesbites · 3 November 2023 22:10

Hello guys!

After to upgrade to release 180, pakfire don’t work anymore. The dns of ipfire.org answer with two addresses, ipv6 and ipv4. The firewall host don’t get pakfire updates anymore.

Is it possible to make dns of ipfire.org to answer only ipv4?

If ipv6 don’t work in firewall, so, please change the dns record to answer only ipv4 to ipfire.org hosts.

From green or from firewall, no one of ipfire.org are available to access.

So, considering the context of this issue, if it is possible, I appreciate if you would to delete the AAAA records in the Dns of ipfire.org.

Thank you.

[root@ipfire ~]# nslookup www.ipfire.org
Server:		127.0.0.1
Address:	127.0.0.1#53

Non-authoritative answer:
www.ipfire.org	canonical name = fw01.ipfire.org.
Name:	fw01.ipfire.org
Address: 81.3.27.38
Name:	fw01.ipfire.org
Address: 2001:678:b28::

[root@ipfire ~]# ping www.ipfire.org
PING fw01.ipfire.org (81.3.27.38) 56(84) bytes of data.
64 bytes from fw01.ipfire.org (81.3.27.38): icmp_seq=1 ttl=48 time=246 ms
64 bytes from fw01.ipfire.org (81.3.27.38): icmp_seq=2 ttl=48 time=245 ms
64 bytes from fw01.ipfire.org (81.3.27.38): icmp_seq=3 ttl=48 time=245 ms
^C
--- fw01.ipfire.org ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2001ms
rtt min/avg/max/mdev = 245.262/245.352/245.500/0.105 ms
[root@ipfire ~]# curl -vv https://www.ipfire.org
* processing: https://www.ipfire.org
*   Trying 81.3.27.38:443...
* connect to 81.3.27.38 port 443 failed: Connection timed out
* Failed to connect to www.ipfire.org port 443 after 15518 ms: Couldn't connect to server
* Closing connection
curl: (28) Failed to connect to www.ipfire.org port 443 after 15518 ms: Couldn't connect to server
[root@ipfire ~]# pakfire update

Giving up: There was no chance to get the file "2.27-x86_64/lists/server-list.db" from any available server.
There was an error on the way. Please fix it.

Giving up: There was no chance to get the file "2.27-x86_64/lists/server-list.db" from any available server.
There was an error on the way. Please fix it.
MIRROR ERROR: Could not find or download a server list
[root@ipfire ~]#

bonnietwin · 5 November 2023 14:17

I just ran pakfire update on my Core Update 180 IPFire and everything worked fine.

From your error message I would expect that the issue is not related to the ipv6 address also being provided (because IPFire-2.x will ignore that) but more likely a dns issue.

On your Domain Name System WUI page is the overall status showing Working in green or Broken in red.

What are the statuses of the individual dns servers if you press the Check DNS Servers button.

moisesbites · 5 November 2023 19:06

Hello Adolf, thank you for your answer.

In fact, I rolled back the update to version 179. In the clean installation the problem remains. Restoring backup of ipfire, the problem remains. I tried direct access through the Internet operator’s modem, and the address https://www.ipfire.org is accessible. However, behind ipfire, through ipfire, it does not access at all, neither from within the host firewall, nor from another host on the internal network.

Above I send some screen shot of the dns condiguration and it’s status. And I send the screen shots of pakfire status e update status.

I don’t know what’s is happening.
Apparently the problem is IPv6. Or the access being exchanged during the route for an IPv6. Some tests I did with curl always tried to access via IPv6. I had a similar problem with the configuration of another website that responded via DNS with IPv6. Talking to the supplier, he switched to IPv4 exclusively and the website was accessible again.

I am accessing this forum via cell phone directly through the 4g connection. From within the network behind ipfire I cannot access it.

bonnietwin · 5 November 2023 19:33

IPFire pakfire is only using ipv4. So ipfire.org dns has both ipv4 and ipv6 but pakfire code just uses ipv4.

I am using the same Core Update as you (180). My pakfire is accessing the same ipfire.org url and I don’t have any problem at all.

So I am not convinced that it is down solely to the ipv6 address being provided by ipfire.org. If it was then I would expect everyone to be having the same problem.

I also see that your last server list, core list and packages list update times are 19666 days ago which is back in 1970. It looks like those times are going back to the Unix Epoch - 1st January 1970.

What do you find for the lists by running
ls -hal /opt/pakfire/db/lists/

I get the following

ls -hal /opt/pakfire/db/lists/
total 24K
drwxr-xr-x 2 root root 4.0K Nov  5 15:12 .
drwxr-xr-x 7 root root 4.0K Jun 26 14:16 ..
-rw-r--r-- 1 root root  903 Nov  5 15:12 core-list.db
-rw-r--r-- 1 root root 5.2K Nov  5 15:12 packages_list.db
-rw-r--r-- 1 root root 2.1K Nov  5 15:12 server-list.db

It would be interesting to see what is shown in that directory for your system.

moisesbites · 6 November 2023 00:00

With a clean instalation, after that, with restored backup:

[root@ipfire ~]# ls -hal /opt/pakfire/db/lists/
total 8.0K
drwxr-xr-x 2 root root 4.0K Sep 19 08:30 .
drwxr-xr-x 7 root root 4.0K Nov  3 13:02 ..

Where I could get that files present in your list?

bonnietwin · 6 November 2023 07:18

I wanted to see what that list looked like on the system that had the problems.

As you have now done a fresh install plus a restore, what happens if you run the pakfire list update from the WUI Pakfire page. Do you still have the problem?

moisesbites · 6 November 2023 13:39

The problem is the same. None of the sites of ipfire.org domain are accessible by firewall or from green network.

Recently my ISP is providing ipv6. I think, maybe, this could be the problem.

Directly, from the modem of my ISP, connected using a laptop, for example, the ipfire.org domain is accessible. Behind the ipfire firewall, connected to the same modem, using the same laptop,the domain isn’t accessible.

The only difference is ipv6 now in the modem. The red ip is ipv4. The internal network between modem and firewall is ipv4. The modem doesn’t have bridge mode. The ISP doesn’t provide this resource.

Before I did the upgrade from 179 to 180 pakfire was working and ipfire was acessible.

But , a new and clean instalation of 179 doesn’t solve this Issue.

I

bbitsch · 6 November 2023 13:44

No bridge mode means you have double NAT in your network.
Can you ping ipfire.org?
What is the output of traceroute www.ipfire.org?
What is the output of pakfire update?

moisesbites · 6 November 2023 19:04

[root@ipfire ~]# nslookup www.ipfire.org
Server:		127.0.0.1
Address:	127.0.0.1#53

Non-authoritative answer:


www.ipfire.org	canonical name = fw01.ipfire.org.
Name:	fw01.ipfire.org
Address: 81.3.27.38
Name:	fw01.ipfire.org
Address: 2001:678:b28::

[root@ipfire ~]# ping www.ipfire.org
PING fw01.ipfire.org (81.3.27.38) 56(84) bytes of data.
64 bytes from fw01.ipfire.org (81.3.27.38): icmp_seq=1 ttl=48 time=250 ms
64 bytes from fw01.ipfire.org (81.3.27.38): icmp_seq=2 ttl=48 time=250 ms
64 bytes from fw01.ipfire.org (81.3.27.38): icmp_seq=3 ttl=48 time=250 ms
64 bytes from fw01.ipfire.org (81.3.27.38): icmp_seq=4 ttl=48 time=250 ms
^C
--- fw01.ipfire.org ping statistics ---
5 packets transmitted, 4 received, 20% packet loss, time 4000ms
rtt min/avg/max/mdev = 249.770/249.933/250.072/0.110 ms
[root@ipfire ~]# curl -v https://www.ipfire.org
* processing: https://www.ipfire.org
*   Trying 81.3.27.38:443...
* connect to 81.3.27.38 port 443 failed: Connection timed out
* Failed to connect to www.ipfire.org port 443 after 15283 ms: Couldn't connect to server
* Closing connection
curl: (28) Failed to connect to www.ipfire.org port 443 after 15283 ms: Couldn't connect to server
[root@ipfire ~]# curl -v http://www.ipfire.org
* processing: http://www.ipfire.org
*   Trying 81.3.27.38:80...
* connect to 81.3.27.38 port 80 failed: Connection timed out
* Failed to connect to www.ipfire.org port 80 after 15613 ms: Couldn't connect to server
* Closing connection
curl: (28) Failed to connect to www.ipfire.org port 80 after 15613 ms: Couldn't connect to server
[root@ipfire ~]# pakfire update





DOWNLOAD ERROR: There was no chance to get the file "2.27-x86_64/lists/server-list.db" from any available server.
May be you should run "pakfire update" to get some new servers.

and, when pakfire or curl are trying connect, the connections have SYN_SENT status e just it.

without pakfire, I don’t have traceroute to install.

but from a desktop behind firewall:


desktop:~$ traceroute www.ipfire.org
traceroute to fw01.ipfire.org (81.3.27.38), 64 hops max
  1   ------------  0,184ms  0,184ms  0,109ms 
  2   ------------  0,808ms  0,549ms  0,496ms 
  3   ------------  4,432ms  3,905ms  3,617ms 
  4   100.120.70.161  5,574ms  4,904ms  4,902ms 
  5   100.120.23.51  19,266ms  18,476ms  18,948ms 
  6   100.120.25.204  20,206ms  19,874ms  * 
  7   100.120.25.82  37,763ms  37,090ms  * 
  8   45.238.97.194  39,510ms  38,930ms  38,919ms 
  9   200.16.69.44  78,418ms  77,793ms  77,407ms 
 10   200.16.69.2  142,694ms  142,295ms  141,742ms 
 11   *  *  * 
 12   146.247.201.94  244,632ms  244,075ms  243,822ms 
 13   212.112.170.209  238,548ms  238,025ms  237,714ms 
 14   194.182.97.134  243,220ms  242,298ms  242,624ms 
 15   194.182.97.241  246,027ms  245,595ms  245,254ms 
 16   77.243.32.58  243,696ms  243,274ms  243,245ms 
 17   213.146.68.155  247,601ms  246,870ms  246,967ms 
 18   195.47.229.54  248,235ms  247,917ms  248,097ms 
 19   *  *  * 
 20   *  *  * 
 21   *  *  * 
 22   *  *  * 
 23   *  *  * 
 24   *  *  * 
 25   *  *  * 
 26   *  *  * 
 27   *  *  * 
 28   *  *  * 
 29   *  *  * 
 30   *  *  * 
 31   *  *  *

another thing, in a new instalation booting from network accessing boot.ipfire.org, the address is accessible and the boot works fine.

bbitsch · 6 November 2023 20:02

Looks like problems with HTTP(S) connections.
Is the proxy enabled?
Any firewall rules, which deny http(s) from the IPFire system?

Are curl invocations to other locations than ipfire.org functioning?

moisesbites · 6 November 2023 21:36

hello Bernhard,

“ipfire.org” and its subdomains don’t work with https or http from green network or firewall host.

I tested a new clean instalation of 178, 179 and 180 versions with default configuration. None got www.ipfire.org or pakfire.ipfire.org.

Using Dns over tls with 8.8.8.8, 1.1.1.1 and synchronized ntp , the ipfire.org is inaccessible from firewall or green network.

Every other sites with ipv4 and ipv6 works correctly.

Sites with only ipv6, don’t work.

ipfire.org is accessible from my cell phone with 4g, and connected directly on modem. But it is not accessible from green network, or from firewall host.

tphz · 7 November 2023 20:50

In a fresh installation of IPFire, the tracepath command is available
e.g. tracepath ipfire.org

edit

Usage
  tracepath [options] <destination>

Options:
  -4             use IPv4
  -6             use IPv6
  -b             print both name and ip
  -l <length>    use packet <length>
  -m <hops>      use maximum <hops>
  -n             no dns name resolution
  -p <port>      use destination <port>
  -V             print version and exit
  <destination>  dns name or ip address

moisesbites · 10 November 2023 14:53

Connected on ipfire host by ssh:

[root@ipfire ~]# tracepath pakfire.ipfire.org
 1?: [LOCALHOST]                      pmtu 1500
 1:  gateway                                               0.908ms 
 1:  gateway                                               0.678ms 
 2:  201.88.216.1                                        190.061ms 
 3:  100.120.70.161                                        6.174ms 
 4:  100.120.31.223                                       21.620ms 
 5:  100.120.20.150                                       19.515ms 
 6:  no reply
 7:  45.238.97.194                                        40.512ms 
 8:  200.16.69.44                                         82.455ms asymm 10 
 9:  200.16.69.2                                         148.729ms asymm 11 
10:  no reply
11:  146.247.201.94                                      258.436ms asymm 17 
12:  212.112.170.209                                     246.031ms asymm 17 
13:  194.182.96.33                                       253.301ms asymm 20 
14:  194.182.97.154                                      247.205ms asymm 20 
15:  77.243.32.58                                        253.216ms asymm 18 
16:  et-0-0-5.hamb2pe2de.gc-net.eu                       253.517ms asymm 15 
17:  fr4a.h.as24679.net                                  257.107ms asymm 16 
18:  no reply
19:  no reply
20:  no reply
21:  no reply
22:  no reply
23:  no reply
24:  no reply
25:  no reply
26:  no reply
27:  no reply
28:  no reply
29:  no reply
30:  no reply
     Too many hops: pmtu 1500
     Resume: pmtu 1500

Using a desktop connected direct to modem:

~$ tracepath pakfire.ipfire.org
 1?: [LOCALHOST]                      pmtu 1500
 1:  _gateway                                              0.816ms 
 1:  _gateway                                              1.016ms 
 2:  201.88.216.1                                          4.710ms 
 3:  100.120.70.111                                        5.181ms 
 4:  100.120.23.53                                        22.957ms 
 5:  100.120.25.202                                       30.410ms 
 6:  100.120.22.141                                      121.375ms 
 7:  45.238.97.194                                        37.315ms asymm  9 
 8:  200.16.69.44                                         78.374ms asymm 10 
 9:  200.16.69.2                                         143.551ms asymm 11 
10:  sem resposta
11:  146.247.201.94                                      251.067ms asymm 17 
12:  212.112.170.209                                     246.464ms asymm 17 
13:  194.182.96.33                                       246.848ms asymm 20 
14:  194.182.97.154                                      241.894ms asymm 20 
15:  77.243.32.58                                        240.995ms asymm 18 
16:  et-0-0-5.hamb2pe2de.gc-net.eu                       251.292ms asymm 15 
17:  fr4a.h.as24679.net                                  247.523ms asymm 16 
18:  sem resposta
19:  sem resposta
20:  sem resposta
21:  sem resposta
22:  sem resposta
23:  sem resposta
24:  sem resposta
25:  sem resposta
26:  sem resposta
27:  sem resposta
28:  sem resposta
29:  sem resposta
30:  sem resposta
     Too many hops: pmtu 1500
     Resumir: pmtu 1500

Now, using curl, I think that the web server, or the reverse proxy, is receiving a connecting with IPv4 but is changing to IPv6 in answer if is supported by modem or is not answering in ipv4 when ipv6 is presente. But ipfire don’t know IPv6, so the server is inaccessible.

From a desktop connect direct to modem, the connection on IPv4 was change to IPv6:

~$ curl -v https://ipfire.org
*   Trying 2001:678:b28:::443...
* TCP_NODELAY set
*   Trying 81.3.27.38:443...
* TCP_NODELAY set
* Connected to ipfire.org (2001:678:b28::) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: CN=www.ipfire.org
*  start date: Nov  7 23:25:39 2023 GMT
*  expire date: Feb  5 23:25:38 2024 GMT
*  subjectAltName: host "ipfire.org" matched cert's "ipfire.org"
*  issuer: C=US; O=Let's Encrypt; CN=R3
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x55d586315050)
> GET / HTTP/2
> Host: ipfire.org
> user-agent: curl/7.68.0
> accept: */*
* Connection #0 to host ipfire.org left intact

Connected behind ipfire, same desktop:

~$ curl -v https://ipfire.org
*   Trying 81.3.27.38:443...
* TCP_NODELAY set
*   Trying 2001:678:b28:::443...
* TCP_NODELAY set
* Immediate connect fail for 2001:678:b28::: A rede está fora de alcance
*   Trying 2001:678:b28:::443...
* TCP_NODELAY set
* Immediate connect fail for 2001:678:b28::: A rede está fora de alcance
*   Trying 2001:678:b28:::443...
* TCP_NODELAY set
* Immediate connect fail for 2001:678:b28::: A rede está fora de alcance
*   Trying 2001:678:b28:::443...
* TCP_NODELAY set
* Immediate connect fail for 2001:678:b28::: A rede está fora de alcance
*   Trying 2001:678:b28:::443...
* TCP_NODELAY set
* Immediate connect fail for 2001:678:b28::: A rede está fora de alcance

Connected on a ipfire terminal by ssh:

[root@ipfire ~]# curl -v https://www.ipfire.org
* processing: https://www.ipfire.org
*   Trying 81.3.27.38:443...
* connect to 81.3.27.38 port 443 failed: Connection timed out
* Failed to connect to www.ipfire.org port 443 after 15380 ms: Couldn't connect to server
* Closing connection
curl: (28) Failed to connect to www.ipfire.org port 443 after 15380 ms: Couldn't connect to server

I did a TCP package capture from my desktop. For some reason, the ipfire web server answer with ipv6 and not ipv4. The IP address of my desktop is ipv4 and ipv6, gave from modem. I think that ipv6 is preferred from ipfire web server. But I`m not sure/.

Other websites with address only IPv4 works fine. And with ipv4 and ipv6 answer correctly with ipv4 or ipv6 and are accessible. Only ipfire is not accessible.

I think the problem is the DNS or the webserver, that anwser with two addresses, ipv4 and ipv6, and prefer ipv6.

tphz · 10 November 2023 15:27

edit

For testing

uncheck DNS 185.49.141.37 , 46.182.19.48

obraz823×60 17.3 KB
change ‘Protocol for DNS qeries’ to UDP
(Don’t forget to click “Save”)
reboot IPFire
repeat connection tests with ipfire.org

Additional information about dns2.digitalcourage.de

dns2.digitalcourage.de

Virtual machine for all DNS varieties:

     Server location: external, German data center
     IPV4: 46.182.19.48 (Please no longer record in new configurations!)*
     IPV6: 2A02: 2970: 1002 :: 18 (please no longer record in new configurations!)*
     Port: 53 (unencrypted DNS)
     Port: 853 (encrypted DNS, but please use dns3.digitalcourage.de)
     SHA256 SPKI Pinset of the server (tls_pubkey_pinset): v7rm6otqqd3x/wbsdhdzjidg+utmzvnox3jq3vi8tgu =
     Common name of the certificate (tls_auth_name): dns2.digitalcourage.de
     Exhibitor of the certificate: Let's Encrypt
     No logging - only in the event of a fault

* DNS2.digitalcourage.de now sends over 2000 successful answers per second at peak times. Unfortunately, much more power will not be possible on this server. We thank you for the trust and ask not to enter this server in new configurations.

For technical reasons, this server uses a different IP in the backend (for the resolver), which is why the following IP is displayed for usual DNS leak tests: 46.182.18.9 / dns2.digitalcourage.de

https://digitalcourage.de/support/zensurfreier-dns-server

moisesbites · 10 November 2023 22:13

Hi iptom,

I did several tests, connected via cell phone, which has ipv6 and ipv4, via modem with ipv4 and ipv6, and via ipfire server. I made the DNS changes you asked me to make. I restarted ipfire. The conclusion I reached after testing other sites is that the problem is in the configuration of the web server or in the reverse proxy that serves the address ipfire.org. I tried to access via cell phone and via modem, which are from different telephone operators, without going through ipfire, and for example, with curl -4 the web server does not respond. If I do curl -6 the ipfire.org web server responds correctly.

I honestly think the problem is with the web server or its access infrastructure.

bonnietwin · 10 November 2023 22:30

If it was then I would expect a large number of people complaining about the web server not working and there aren’t. There has to be some other factor that is interacting so that only the ipv6 response is being provided to your IPFire.

You originally mentioned the problem occurred when you upgraded from CU179 to CU180 but that when you went back to CU179 it was also experiencing the same problem. That tells me that something else changed at around the same time making a previously working combination (CU179) stop working.

moisesbites · 11 November 2023 00:35

Hi Adolf,

My desktop connected to the Internet through my 4G cell phone, connecting to the web server with the curl -4 command does not access www.ipfire.org. If I use curl -6 I usually go to www.ipfire.org. Since I’m not using anything from the ipfire firewall, I think this has nothing to do with the ipfire firewall, but rather with the ipfire.org web server configuration.

Connected to the internet directly by modem, of another company different of my cell phone, the results are the same.

So I think something has changed in the ipfire.org server configuration in the last 15 or 20 days with the ipv6 configuration.

I appreciate everyone’s help. I think that only evaluating the configuration of the web server or the web server infrastructure can solve the problem.

From my cell phone by 4G:

With ipv6:

With ipv4:

Because that I think the problem is the infra or the web server of ipfire.org.

eth0 · 14 November 2023 08:46

I think Bernhard already figured out that the problem is the port 443, not the IP.

I guess telnet is not included by default in IPFire, but could you test the following on your PC:

telnet ipfire.org 443
telnet 81.3.27.38 443

You should get a blinking underscore indicating a succesful connection.

moisesbites · 9 February 2024 19:04

After many tests and a long time waiting a solution, I found the workaround.

First, there is a router in the way from my ISP to IPfire.org that have a IPv6 and it is not working with packfire.

I don’t get https://ipfire.org/ from my firewall host. Only from my mobile by 5G connection.

I configured some mirror ftp.gwdg.de and packfire is working now.

I don’t know why IPfire.org don’t pass through IPv6 router or don’t get IPv6 from IPfire.org. But I think this is the problem. Mirrors with ipv4 route don’t have problem.

Another sites with IPv6 works fine.