Easy home transparent proxy
Everyone in South Africa wants to save a little more bandwidth, as low traffic caps are the rule of the day (esp if you are hanging off an expensive 3G connection).
While the "correct" thing to do is to use wpad autodetection, and thus politely request that users use your proxy, this isn't always an option:
- Firefox doesn't Autodetect Proxies by default
- Autodetection doesn't behave well for many roaming users (firefox should talk to network-manager)
- Many programs simply don't support wpad.
- Your upstream ISP transparently proxies anyway (the norm in ZA), so it's not like we have any end-to-endness to protect.
So, here's how you do it:
- Lets assume your network is 10.1.1.0/24, and the squid box is 10.1.1.1 on eth0
- Install squid (
aptitude install squid), configure it to have a reasonably large storage pool, give it some sane ACLs, etc. - Add
http_port 8080 transparenttosquid.conf(orhttp_port 10.1.1.1:8080 transparentif you are using explicithttp_portoptions) invoke-rc.d squid reload- Add the following to your iptables script:
If you run squid on your network's default gateway, then you are done. Otherwise, if you have a separate router, you need to do the following on the router:
- Add a new
transproxtable to/etc/iproute2/rt_tables, i.e.1 transprox - Pick a new netfilter MARK value, i.e. 0x04
- Add the following to the router's iptables script:
iptables -t mangle -F PREROUTING
iptables -t mangle -A PREROUTING -i br-lan -s ! 10.1.1.1 -d ! 10.1.1.0/24 -p tcp --dport 80 -j MARK --set-mark 0x04
ip route del table transprox
ip route add default via 10.1.1.1 table transprox
ip rule del table transprox
ip rule add fwmark 0x04 pref 10 table transprox
- Done: test and tail your squid logs
The reason we use iproute rules rather than iptables DNAT is that you lose destination-IP information with a DNAT (like the envelope of an e-mail).
An alternative solution is to run tinyproxy on the router (with the transparent option, enabled in ubuntu but not debian), use the REDIRECT rule above on the router, to redirect to the tinyproxy, and have that upstream to the squid. But tinyproxy requires some RAM, and on a WRT54 or the likes, you don't have any of that to spare...
Should you need to temporarily disable this for any reason:
- With all-in-one-router:
iptables -t nat -F PREROUTING - With the separate router:
iptables -t mangle -F PREROUTING
Comments
I must admit to disliking
I must admit to disliking transparent proxies with a passion, if only because they’re difficult to bypass and the ones I hit seem to cache DNS failures for hours. It’s also impossible to fake DNS entries (as I occasionally have to do while I wait for it to propagate) to test vhosts and such. The transparent thing also feels dishonest somehow.
On the other hand, I don’t really have a better solution.
Oops
Oops, forgot to provide details. The above comment is me. :-)
Yes, I also don’t like
Yes, I also don’t like them, but they save you real bandwidth and are better than blocking port 80. Automatic Proxy Detection is badly supported in clients, unfortunately.
You drop Squid’s negative DNS cache time down to a second or two. (I normally set it to 5s)
I feel the pain about the fake DNS entries. However, if you are on SAIX DSL (or 3G), you are transparently proxied anyway, and so you aren’t losing anything. Possible workarounds:
I don’t have guests on my
I don’t have guests on my network often enough for the transparent thing to make sense. The DNS thing is only really an issue for proxies beyond my control. I have a couple of test.jerith.za.net domains set up to get around this. I guess my real reason for disliking them is the magic element. I have no idea how many transproxies my request might be traversing and it makes debugging the aforementioned thingies awfully tricky.
Polite proxies
Polite proxies add X-Forwarded-For.
The thing is a transparent proxy is the only sure-fire way to catch Windows machine’s automatic updates (short of running a WUS server), and lots of sofware simply doesn’t do WPAD correctly (or at all).
The other issue with WPAD is every time you move network, your browser will cache the previous WPAD for a bit (you can mitigate this with SwitchProxy in Firefox). Firefox should integrate with NetworkManager (and Windows) in this respect. It irritates users.
The problem is that WPAD (and esp static proxies) are great solutions for desktops, but not so good for highly-mobile laptops.
Then again, you dial a VPN whenever you sit down, so you are prepared to go through extra shit :-)
Pingback
Post new comment