It sucks ACL’s did not work
Is this the right way round.
acl streaming dst [-n] 192.168.60.218/24 … # http://name-changed.s3 Amazonaws [slow]
I am worried about two things:
-
http://name-changed.s3.amazonaws.compoints to thousands of IP addresses. -
and this statement in the squid wiki:
But I won’t know unless I try! I’ll let you know.
Jon
May have it backwards
acl streaming dst http://name-change.s3 [-n] Amazonaws …# 192.168.69.218/24 [slow]
I am curious - what is the [-n] ?
-n Disable lookups and address type conversions. If lookup or
conversion is required because the parameter type (IP or
domain name) does not match the message address type (domain
name or IP), then the ACL would immediately declare a mismatch
without any warnings or lookups.
From the link above.
I forgot to mention: there are usually two lines (minimum). The 2nd line is the directive (or the action done when the first line is found).
What is the directive that goes with this ACL?
IDK.
Examples:
acl macaddress arp 09:00:2b:23:45:67
acl myexample dst_as 1241
acl password proxy_auth REQUIRED
acl fileupload req_mime_type -i ^multipart/form-data$
acl javascript rep_mime_type -i ^application/x-javascript$
Something like one of those?
acl netflix dstdomain .netflix.com
redirector_access deny netflix
What would that look like.
Acl some service dstdomain .name-changed.s3 amazonaws.com
redirector_access deny some service
Is Squid gaurd installed by default?
Just for the record, this is what I had tried that did not work:
# from https://serverfault.com/questions/163130/best-way-to-bypass-squid-for-certain-sites
acl streaming dstdomain name-changed.s3.amazonaws.com
cache deny streaming
# http://www.squid-cache.org/Doc/config/always_direct/
acl streaming dstdomain name-changed.s3.amazonaws.com
always_direct allow streaming
# from https://wiki.squid-cache.org/SquidFaq/SquidAcl#How_do_I_implement_an_ACL_ban_list.3F
acl streaming dstdomain dst name-changed.s3.amazonaws.com
http_access deny streaming
# causes TCP_DENIED 403 PUT http://name-changed... HIER_NONE/- text/htm
1 Like
What device?
Have you tried a different device?
Apple TV vs fire tv vs ROKU ?
Looks like fire tv has proxy settings!
Found this on Roku. This article is hard to follow.
But. Roku perhaps a VPN from blue to red. To bypass proxy.
This does not solve the Squid problem.
But maybe a workaround.
The device is a streaming DVR. It records over-air-tv shows (via outside antenna) to a local HD.
I am 95% sure the code in Post 40 will be A-OK.
I expanded it using an ipset and location and ASN. More testing!
The text there is pretty clear and it makes a lot of sense. How can any acl bypass a request if you have already committed it to squid? As the text says, once given to it, squid has either to service it or to fail it. It can’t be bypassed because it has no mechanism to give back the traffic to the router and say “please take back this traffic and contact instead the server directly”. Any bypass has to happen before, at the iptables level so that the traffic is never sent to squid in the first place.
The transparency happens at layer 3, while the acl operate at the application layer, way above the protocol stack.
At least, this is how I understand it.
2 Likes
Don’t know my networking layers that well.
Could the order of processing be a problem?
Firewall rule before proxy?
Without knowing the details of how this complex system works, it is possible to be extremely wrong. I do not know anything about how squid works. I only make conclusions on a very approximate mental model. If my premises are wrong, my conclusion is wrong.
Having said that, yes I believe this is exactly the problem. The routing table assigns any packet destined to port 80 to squid by rewriting the packets so that they are destined to port 3129. This happens before anything else. Next squid sees these packets and has to decide what to do with them. My assumption is that squid at this point has no mechanism to send back those packets to iptables and ask it to rewirte it back to port 80 so that it can be handled directly by the browser or the streaming software.
As long as the streaming software was compatible to squid, everything worked well. As soon as squid changed something (or the streaming software changed something) the system got broken. At this point squid has to be bypassed. My hypothesis is that this has to happen before the packets are given to squid, because the caching software has no mechanism to get out of the loop by himself. Once those packets are in its memory, it has to service it (failing the streaming) or drop it (failing the streaming).
My assumptions is also that squid has a mechanism to interact at the protocol (I mean, application) layer with its clients. For example, say firefox uses squid by virtue of being configured to do so. Here squid has a way to communicate back to the client a message like “for this ip address, please retrieve the information directly” and therefore an acl table stating “do not cache this ip” will work.
1 Like
FYI -
This is working for me to bypass the squid.
iptables -t nat -N BYPASS
iptables -t nat -A CUSTOMPREROUTING -s 192.168.60.218/32 -p tcp -m tcp --dport 80 -j BYPASS
iptables -t nat -A BYPASS -d 52.216.0.0/15 -j ACCEPT
iptables -t nat -A BYPASS -d 54.231.0.0/16 -j ACCEPT
.
I am testing an ipset with ASN version also:
ASN='AS16509v?'
iptables -t nat -N BYPASS
iptables -t nat -A CUSTOMPREROUTING -s 192.168.60.218/32 -p tcp -m tcp --dport 80 -j BYPASS
iptables -t nat -A BYPASS -m set --match-set ${ASN} dst -j ACCEPT
2 Likes
Great job @jon if only because you didn’t give up. I think it would be nice if what you learned here could be somehow integrated in the wiki.
2 Likes
Thanks!
Yes, I plan to document!
And I’ll document the ipset / ASN version. I used location → ipset → iptables match-set
The location package makes this VERY easy!
ASN=16509
location list-networks-by-as --format=ipset --family=ipv4 ${ASN} > "/etc/ipset/AS${ASN}.ipset"
ipset restore < "/etc/ipset/AS${ASN}.ipset"
.
and the above feeds this…
iptables -t nat -N BYPASS
iptables -t nat -A CUSTOMPREROUTING -s 192.168.60.218/32 -p tcp -m tcp --dport 80 -j BYPASS
iptables -t nat -A BYPASS -m set --match-set "AS${ASN}v?" dst -j ACCEPT
EDIT : updated code blocks
2 Likes
@jon well done. This goes beyond squid and transparent proxy, it will end up being useful even only for how to use iptables. Congratulations.
2 Likes