Update-accelerator returns TCP_DENIED/403 for fedora-mirror

Networking Web Proxy
1

Hi,
i’m using the update-accelerator extension in the web proxy for linux clients and it works fine for debian and ubuntu systems.
But for fedora the proxy returns a 403-error to the client when i try to update with “dnf update” and the proxy is set in “/etc/dnf/dnf.conf”.

I can see this 403-error also in the proxy logs in IPFire. It looks like the proxy gets this 403-error from the fedora mirrors.

/var/log/squid/access.log:
TCP_DENIED/403 3678 CONNECT mirrors.fedoraproject.org:443 - HIER_NONE/- text/html

Is anybody using the update-accelerator with fedora? Does anybody can help?
Regards, Dieter

2

I use it with Fedora 36 and Fedora 37. Not experienced this issue with either. This for both my green running transparent proxy and blue running non-transparent. Is this an ongoing issue or maybe the particular mirror was just temporarily down?

3

Thanks for your answer. It is an ongoing issue and the fedora-mirror isn’t down. I can update my systems without the IPFire as dnf-proxy.

4

To explain, what i already did:

I’ve added the ip-address for the IPFire proxy to the package-manager-config:

/etc/dnf/dnf.conf
proxy=http://192.168.xx.yy:800

192.168.xx.yy is the IPFire IP-address

When i try to update the fedora-packes on the client i get the following error:

bash-5.2# dnf update
Fedora 37 - x86_64 - Updates                                                                    0.0  B/s |   0  B     00:00    
Errors during downloading metadata for repository 'updates':
  - Curl error (56): Failure when receiving data from the peer for https://mirrors.fedoraproject.org/metalink?repo=updates-released-f37&arch=x86_64 [Received HTTP code 403 from proxy after CONNECT]
Error: Failed to download metadata for repo 'updates': Cannot prepare internal mirrorlist: Curl error (56): Failure when receiving data from the peer for https://mirrors.fedoraproject.org/metalink?repo=updates-released-f37&arch=x86_64 [Received HTTP code 403 from proxy after CONNECT]

Log-entries on IPFire:

/var/log/squid/access.log
TCP_DENIED/403 3678 CONNECT mirrors.fedoraproject.org:443 - HIER_NONE/- text/html

/var/log/squid/cache.log
Mar 25 09:22:24 squid-asnbl-helper[23418] WARN: Destination 'mirrors.fedoraproject.org' exceeds ASN diversity threshold (8 > 5), possibly Fast Flux: [3701, 15456, 16509, 21785, 22753, 36850, 54455, 61317]
Mar 25 09:22:24 squid-asnbl-helper[23418] INFO: Denying access to possible Fast Flux destination 'mirrors.fedoraproject.org'

When i remove the proxy from /etc/dnf/dnf.conf i can update my system with dnf update

5

I’ve tried to replicate this but cannot. My proxy settings are distributed through IPFIRE WPAD via DHCP server.

Even disabling that and manually setting up the proxy on my Fedora 37 laptop, I cannot reproduce this error.

6

I am not sure, but as far as i know WPAD is used for setting the web-proxy, for example for web-browser. I am using IPFire as Web-Proxy for browsing, also with fedora-systems.

But i want to use the Web-Proxy-extension “update-accelerator”, to store packages (.deb and .rpm) on IPFire for installing or updating my several Systems. So IPFire gets every package only one time from the web, and can deliver them multiple times to the clients.
To do that i have to define the proxy for apt in /etc/apt/apt.conf.d/ (.deb for Debian and Ubuntu) and dnf in /etc/dnf/dnf.conf (.rpm for Fedora). I am using the update-accelerator for Debian und Ubuntu for a while, without problem, and want to use it now also for Fedora-Updates (.rpm packages). This is the function, where i have the problem.

1 Like
7

Are you using the proxy in transparent mode? Are these systems on the same subnet, firewall group or is there anything different between them other than distro? Any differences in DNS settings between the Ubuntu and Fedora systems? Any locally installed VPN in use when experiencing this error?

If you are using proxy in transparent mode then no local settings or changes are needed on the clients. I have many wired systems on my network using my Green with transparent proxy. Not one has any local settings all all, be they Fedora, Ubuntu, Debian, Mint, Gentoo or other cache when updating.

I’m fairly sure update accelerator requires transparent proxy, but don’t quote me on that. There are only few laptops on my Blue non-transparent subnet (one is Fedora 37) but as far as I know these do not cache. That said, this laptop also does not experience the DNF issue you are describing after testing with the same setup as best I could based on description.

8

A lot of questions, but thanks a million for your help. I try to answer them all:

Are you using the proxy in transparent mode?

No. With transparent mode the web-proxy can’t be used for secure sessions (https://), but nearly all sessions to the web are secure. So the web-proxy is from my point of view only useful with manual configuration at the clients or using WPAD with DHCP. This is the reason why i am running the proxy not in transparent mode.

Are these systems on the same subnet, firewall group or is there anything different between them other than distro?

The anwser is yes for all points. The debian- and fedora-clients are running as virtual systems on the same client (i am using Qubes OS) and for the connection to IPFire they have all the same connection and therefore the same IP-address, the same subnet, the same route, the same firewall-entries.

Any differences in DNS settings between the Ubuntu and Fedora systems?

No, they are using the same DNS-server (IPFire).

Any locally installed VPN in use when experiencing this error?

No

I’m fairly sure update accelerator requires transparent proxy,

I took again a look at the documentation https://wiki.ipfire.org/configuration/network/proxy/update_accelerator. There is no information, if transparent proxy is required or not.

But i’ve tested it. There is no difference when setting the transparent mode at the web-proxy. I got the same error 403.

But i found another useful information in the documentation. Update-accelerator can’t be used for https-urls. So i’ve changed all addresses for fedora mirrors (in /etc/yum.repos.d/) from https to http.

I tested it with transparent mode in the web-proxy on IPFire on and off and got in both cases again the error 403 on the client. On IPFire in the squid-access-log there is also the message with the 403 error:

/var/log/squid/access.log
TCP_DENIED/403 3870 GET http://mirrors.fedoraproject.org/metalink?repo=fedora-37&arch=x86_64 - HIER_NONE/- text/html

And also the same messages in squid-cache-log:

/var/log/squid/cache.log
Mar 26 10:28:34 squid-asnbl-helper[7505] WARN: Destination 'mirrors.fedoraproject.org' exceeds ASN diversity threshold (8 > 5), possibly Fast Flux: [3701, 15456, 16509, 21785, 22753, 36850, 54455, 61317]
Mar 26 10:28:34 squid-asnbl-helper[7505] INFO: Denying access to possible Fast Flux destination 'mirrors.fedoraproject.org'

At this point (i wrote this text parallel while doing the tests) i recognized, that the last message in the cache log says, squid is denying the access to the fedora mirrors. To be sure, that this isn’t done from url-filter i set squid-clamav and urlfilter in the web-proxy to off. After changing that, i had no more error 403. Then i activated first sqid-clamav, tested and next activated urlfilter and tested again.
Now, i can’t reproduce the error! And i can see in the squid-access-log, that the rpm-files are downloaded now.

But i am now in the same configuration as at the beginning:

  • IPFire: Web-Proxy is started and not in transparent mode. Squid-clamav, urlfilter and update-accelerator are all active.
  • Client: In the file /etc/dnf/dnf.conf the proxy is set. But dnf update is working now on the fedora-client without error message.

strange

9

URL filter makes sense in blocking the site even though it should not have been unless explicit rule was set. Also why I could not reproduce; I don’t use it. Possible the filter list was corrupt and was replaced when disabled then enabled.

You have sorted it out which is the important bit.

1 Like
10

This is the needed hint. There is a protection against Fast Flux hosts that is often used by malware to use a botnet to spread the files.

mirrors.fedoraproject.com use to many changing IP’s in dns so this check is triggered.
I will contact Peter for a solution.
blog.ipfire.org - Feature Spotlight: Weaponising IPFire Location to proactively detect Fast Flux setups

6 Likes
11

Thank you Arne for your help. I didn’t remember this blog-article, but i read it, when it was published.

The solution (or maybe a workaround) is mentioned in this article. When i include the url mirrors.fedoraproject.org in the whitelist of the urlfilter, then the dnf update is running with proxy set and without error.
What is interesting: I don’t have to activate the whitelist, it is enough to include the url.