Can not get non-Transparent web proxy to work

@jon This is the summary of this post; you can’t have everything working by just setting the firewall, that’s only when you use transparent proxy. You need to instruct any and all clients, automatically or manually to use the proxy, and whether you do it automatically or manually, it depends both on the OS and the client.

My understanding is that there are two separate issues:

A) correctly setup the squid proxy;
B) block at the firewall level any attempt to get on red from anyone in your network on the web ports.

IF you have both A and B, AND you manually set the browser of your laptop or phone to use http://ipfire.localdomain:800 and https://ipfire.localdomain:800 as proxies for port 80 and 443, THEN you must be able to browse internet, ELSE you have a miss-configured squid.

Another way to say the same thing is: IF you have A and B, UNLESS the clients are instructed to use the proxy, your clients wont access the web.

As far as I know, there are three methods to do so:

  1. manual configuration of the client;
  2. automatic configuration of the client by looking at the OS system settings; this works IF the OS is being manually configured, OR it is using proxy.pac, directly or by following the DHCP setting as suggested in the wiki; <— this is my system (OS manually configured), see below
  3. automatic configuration by the client, using proxy.pac directly.

In my situation, for mobile devices, setting manually the proxy at the OS level in the WIFI configuration, pretty much works out of the box for any APP working on those devices except few specialized apps that do not handle a proxy server well.

Similarly but more tiresome, on MacOS I also had to set:

  1. in the network configuration to use the proxy, and
  2. in the client (Firefox), to use the proxy system settings.

This combination works well. Auto-discovery by using proxy.pac never worked for me. Not even once, on Android, iOS and MacOS.

I should have added this in the first post. The client proxy is enabled and it seems to be working. And yes, this could be the problem. Since the client proxy seems to be working I figured I’d try the firewall rules side.

On MacOS I look at the scutil --proxy command:

This is all proxy disabled:

MBP6:~ jcmurphy$ scutil --proxy
<dictionary> {
  ExceptionsList : <array> {
    0 : *.local
    1 : 169.254/16
  }
  FTPPassive : 1
  HTTPEnable : 0
  ProxyAutoDiscoveryEnable : 0
}

and this is with Auto Proxy Discovery enabled:

MBP6:~ jcmurphy$ scutil --proxy
<dictionary> {
  ExceptionsList : <array> {
    0 : *.local
    1 : 169.254/16
  }
  FTPPassive : 1
  HTTPEnable : 0
  ProxyAutoConfigEnable : 1
  ProxyAutoConfigURLString : http://192.168.66.1:81/proxy.pac
  ProxyAutoDiscoveryEnable : 1
}

I saw your comment about Auto-discovery not working so I’ll check that first!

Out of curiosity - how do you block port 80 & 443 in the firewall rules?

Referring also to my firewall rule mentioned above I block all traffic for forward traffic as a general rule (see firewall options). Open is only mail and ICMP. So no traffic via port 80 o 443 is routed.

All traffic from clients go through squid automatically by the firewall itself.

On the outgoing firewall rules the rule mentioned above is working.
All traffic from client to port 80 or 443 goes now through the proxy.

On the client side proxy is manually activated in the OS (Opensuse) and in the browser (firefox)

Th Wpad rules worked on my Laptop (Opensuse) and are activated for the blue network. With other OSs I have no experience.

1 Like

I first created a group including every network (blue, green, OpenVPN), then this group goes to the source, NAT is unchecked, destination is RED, service group is WEB (group that includes both TCP web ports 80 and 443), reject. This is the last rule of the firewall and any exception goes above, for example the machine where I use services that do not deal well with a proxy.

1 Like

@jon what happens if you disable transparent proxy? Can your client use it successfully in this case?

The transparent proxy has been disabled since I started my experimenting. All of my experiments are on the BLUE network. ( I’ll do GREEN once I figure out what is not working. )

In the non-transparent mode

Depending on which firewall rule I am experimenting with - sometimes things work and sometimes not. I am having trouble figuring out if I am fighting against the Squid Proxy cache or fighting the browser cache or fighting a Firewall rule.

This is my current rule (current experiment!). I think it is similar to your description in Post #7. I wasn’t sure if your Firewall Rule (FR) RED is Standard RED of Firewall RED. So I just picked one.

That’s correct. The group Web_proxy is composed of TCP 80 and 443, right? If yes, your rule is identical to mine.

Yes it is.

Maybe you can figure out if the client is connecting to the proxy by looking at squid/access.log

 tail -f /var/log/squid/access.log

while you connect with the client’s browser to a website

Thanks. That’s been running since the beginning also.

(Browsers are WAY too busy when they should be sitting idle…)

what was done to setup the HTTPS side of the proxy server?

Allowed SSL ports, one per line

443 # https
563 # snews

I have Authentication method set to none, in case this is the problem.

1 Like

Mine is already set-up that way:

Was anything added to the pac/wpad file? I see this:

function FindProxyForURL(url, host)
{
if (
     (isPlainHostName(host)) ||
     (isInNet(host, "127.0.0.1", "255.0.0.0")) ||
     (isInNet(host, "192.168.6.0", "255.255.255.0")) ||
     (isInNet(host, "192.168.66.0", "255.255.255.0")) ||
     (isInNet(host, "10.10.10.0", "255.255.255.0")) ||
     (isInNet(host, "169.254.0.0", "255.255.0.0"))
   )
     return "DIRECT";

 else

if (
     (isInNet(myIpAddress(), "192.168.6.0", "255.255.255.0"))
   )
     return "PROXY 192.168.6.1:800";

 else

if (
     (isInNet(myIpAddress(), "192.168.66.0", "255.255.255.0"))
   )
     return "PROXY 192.168.66.1:800";
}

Mine is the same…

This is mine:

function FindProxyForURL(url, host)
{
if (
     (isPlainHostName(host)) ||
     (isInNet(host, "127.0.0.1", "255.0.0.0")) ||
     (isInNet(host, "10.1.1.0", "255.255.255.0")) ||
     (isInNet(host, "10.1.3.0", "255.255.255.0")) ||
     (isInNet(host, "10.1.2.0", "255.255.255.0")) ||
     (isInNet(host, "169.254.0.0", "255.255.0.0"))
   )
     return "DIRECT";

 else

if (
     (isInNet(myIpAddress(), "10.1.1.0", "255.255.255.0")) ||
     (isInNet(myIpAddress(), "10.1.4.0", "255.255.255.0"))
   )
     return "PROXY 10.1.1.1:800";

 else

if (
     (isInNet(myIpAddress(), "10.1.3.0", "255.255.255.0"))
   )
     return "PROXY 10.1.3.1:800";
}

1 Like

they look very similar except for the IP address. Thanks for posting!

Just wanted to add an update… I was getting frustrated with setting up the proxy so I figured I’d walk away for a few days and do something else.

I have the non-transparent (conventional) proxy running and things still don’t seem to be running correctly. I was hoping updating the CU 171 (test) might change something but still no joy.

I am guessing I have something configured incorrectly but I do not know what.

Right now browsing the Internet is painfully slow. To load a new web page takes about 30 seconds. If I enable transparent mode all of the slow disappears. I think it is my firewall rule but only because that is a big unknown for me.

My current firewall rule is in Post 9.

Can someone post a picture/screenshot of their entire Firewall Rule for blocking port 80 & Port 443?

I did the opposit. I do only allow a handfull of ports that are needed. Why should I keep every windows and door open? That’s just a security risk. So my default outgoing firewall settings is set to “blocked”.

I have 4 subnets for each green and blue.

  1. is for stationary knows and trustfull infrastructure network members without internet access
  2. is for stationary knows and trustfull enduser network members that must use the non-transparent proxy (otherwise no internet access)
  3. is for mobile and trustfull enduser network members (may or may not use the non-transparent proxy, transparent proxy working for internet access)
  4. is for any new/unknows network members (no internet access)

“Trustfull network members” are defined in the firewall hosts, therefore IP and MAC need to fit.

For a working internet access, it needs only rules for ICMP, NTP, DNS, HTTP and HTTPS (+ in our case ports for Email because of Thunderbird/Outlook):

1 Like

Hi @jon.

I have it this way:

I have it in non-transparent mode and it works like a shot.

I hope that sooner rather than later, you find a solution. :+1:

Greetings.

@jon there is one thing that I do not understand with your setting. Transparent mode works only for unencrypted traffic. What happens to the https traffic when you enable transparent proxy, it still goes through the proxy or it goes directly? Because if it is the latter you then not only activate transparent proxy, but also you disable the firewall rule in post 9. Do I understand your setting correctly?

This is the direction I am slowly heading. But without a basic understanding of the proxy I am very reluctant to do FORWARD Blocked and OUTGOING Blocked.

Currently these are set to FORWARD Allowed and OUTGOING Allowed. And this may be the crux of the issue! With one rule hopefully blocking 80 & 443.

This is more advanced than my network. I just have one subnet for green and a separate subnet for blue. Green is a wired ethernet (no WiFi). Blue is mostly wireless (WiFi).

Thank you for posting!

2 Likes