Filtering madness

Networking Web Proxy
1

Hello people.

As I have no complains with the IPF in general, the filtering bussines is a total madness. There’s no way to get some unified results in different machines to be able to draw some sane conclusions.
All my hosts are DHCP, none fixed. The DNS is configured to force all traffic through firewall - following wiki. I have Talos and emerging threats lists activated with only the rules they come with selected. For IP address lists I selected just bogons, both emerging lists and I guess nothing more. The web proxy non-transparent on both green and blue - configured on hosts machines not through DHCP wpad. For URL filter only ads, malware, phising and stalkware.
Now… what’s happening is maddening and hilarious in the same time. Depending on browser brave/ firefox and hosts the results are not the same. For instance I can access youtube on an android phone but only on firefox (and not on brave). On an apple machine yt is not reachable at all - firefox or brave.
Yt is one example, there’s much more - various benign media portals, blogs, tech sites etc.
I have an appletv (on blue) in which wifi works but only locally - it is able to stream from a media server (on green) but no internet whatsoever. No proxy configured on that particular device, but the amount of traffic that appears as forwardfw in firewall log for this IP is absolutely insane - mostly 443 apple/ amazon IPs (different topic - though I would like to filter some of it).
The main question being… where/ how can I check the blocked traffic to have at least some indication of who is blocking what to obtain a better understanding on how the filtering works and why the filtering is so randomly from a host/browser to another… and finally to be able to fine tune a bit the lists. For instance the SARG/ Proxy reports shows some denied traffic for the apple machine IP (not appleTV), but youtube is not there (at all!). The proxy logs shows all traffic, without any specifications. The only log that speaks clearly is IPS log viewer - it names the lists that deny the hit and the reason for which is doing so.

Could anybody enlighten me a bit? Thanks.

2

IPFire’s filtering can be visible in the logs if you enable them.

But each OS ( Linux, Android, Windows, MacOS, iOS … ) you use behind can also have its own filtering rules, as can different browsers (Chrome, Safari, Brave, Firefox, Opera, etc.), which you can also configure.

If you really want to monitor the traffic of each device, you can use tcpdump (add-on to install) and wireshark on your PC/Mac to analyze the traffic.

1 Like
3

Just for the record, IPFire also has the Tshark package.

Regards

2 Likes
4

„IPFire’s filtering can be visible in the logs if you enable them.”

I have them as they came on install - enabled. But I guess I need to learn more to understand them properly, as I intend to stay on IPF.
Some of these logs, for instance proxy logs, don’t specify what the data listed there means (and wiki is hellova vague, too - loosing point in many more topics).
Proxy logs says websites matching selected criteria. OK… meaning? In the end I deduced indirectly that those are blocked websites - an info not specifically enunciated. But if so, why youtube doesn’t appear among them every time is blocked on a host?
To free access to YT I added the youtube.com on URL filter configuration custom whitelist - I guess it requires a reboot as it is still blocked (if so it would be nice to specify that too).
Now… this URL Filer Configuration page have ticked just ads, malware and phishing. Exactly which one of those blocks YT only on cromium based browser?

I cannot express my confusion clearer then that… and I hope you understand my complain.

„But each OS ( Linux, Android, Windows, MacOS, iOS … ) you use behind can also have its own filtering rules, as can different browsers (Chrome, Safari, Brave, Firefox, Opera, etc.), which you can also configure.”

This is a truism even for my non-kungfu tek mind. What I don’t get is exactly why browsers that work similarly (I mean they really work based on ublock or other and on) change their behaviour if I load a malware/ stalkware/whareverware filter list at router level. And if they do so, where can I find more info to correct the issue?

„If you really want to monitor the traffic of each device, you can use tcpdump (add-on to install) and wireshark on your PC/Mac to analyze the traffic.”

I know, but bear in mind than I… and probably many like me… am a regular civilian who stares at a wireshark screen as a dog to a matrix screensaver. I am a wui user, coping to advance.
So, please, when you answer, try not to offer a solution that stands for you, but for who is asking it. Generalisations don’t serve either.
I noticed around that people are flocking away from ubiquous brands that sell plug and play IT security products for home and SOHO consumption - as they are abandoned in a couple of years in a state of swiss cheese of bugs… so this info here may serve to others like me.

But… hey… thank you, I really appreciate you take time to answer. For now I’ll follow the links you indicated.

5

To master this problems, you must be aware of the basic data flows and filtering mechanisms.

  • IPFire is the device for WAN access of all local devices
    • It allows connections initiated from local networks and blocks connections coming from outside.
    • Name resolution is offered by the DNS part to internal systems.
    • Web acccess ( HTTP/HTTPS ) is provided by the proxy
  • Further allowance / deny of network traffic with the WAN is handled by the firewall. This module works on IP level. ( URLs are resolved by the device with means of the DNS server )
    • on this level IP Block Lists are active
  • For web traffic the proxy contains the module URLFilter. This handles all HTTP(S) packets.

IPFire is not just a plug and play solution which gives you the result a manufacturer thinks is good for you.
IPFire gives you a frame work, that lets you shape the device to your needs. But this demands some basic knowledge of networking and security modules.
The development team tries to put its profund knowledge into the management web user interface and into the documentation, the wiki.

The IPFire system tries to inform the devices in the local network about the policies. This mainly done by the DHCP lease answer

  • DNS server to use
  • proxy for web access; through the WPAD mechanism

Devices in the local network should obey these informations, but not all OSs do this fully. Even for browsing ( web access ) this may be different. Default configs for browsers are generated from system-local information.
To circumvent this problem you either can check the config of each device or try forcing by IPFire internal mechanisms.
But there is no guarantee for a full success. This brings us back to the basic knowledge part.

BR,
Bernhard

1 Like
6

Sorry,
You didn’t really specify your level of networking knowledge, but a minimum is preferable for using IPFire.

No, The “Proxy-Logs” also lists, if the proxy is activated, all calls to websites of the connected clients.

This is specified in the wiki page when you use the online help button
help-browser

Youtube is not blocked by URL Filter unless you specify otherwise.

Try transparent proxy and check your YouTube access

maybe this can help you for use non-transparent

1 Like