Local only DSL

Update: Debian/Ubuntu version

I've finally jumped onto the local only DSL bandwagon. If you haven't done it yet, it's a great way to save some bucks. The idea is that you get a local only account like this, which costs a fraction per GiB compared to normal account. Then you get your router to connect to both simultaneously, and route intelligently between them.

Most ADSL routers won't let you connect 2 concurrent ADSL connections on the same ATM circuit. The solution is to use a separate modem and router. I'm using a basic Billion modem, in bridged mode, and a WRT54GL, running OpenWRT/kamikaze, as the router.

OpenWRT doesn't support 2 PPPoE connections out of the box, but I've found the problems, and got a few changes committed upstream, that solve them:

The firewall (/etc/init.d/firewall) needs to be modified with "WAN=ppp+" somewhere near the top, so that it masquerades all the ppp connections. This was a hack, apparently the firewall is being re-written soon.

There is also a bug that resets existing PPPoE connections on a ethernet interface when you fire up a new connection. This will apparently be fixed by the future interface aliasing support. For now, I just hacked around it in /lib/network/config.sh:

prepare_interface() {
        local iface="$1"
        local config="$2"

        #SR: We don't want to reset any pppoe connections
        config_get proto "$config" proto
        [ "$proto" = "pppoe" ] && return 0

and /sbin/ifdown:

config_get ifname "$cfg" ifname
config_get device "$cfg" device

[ ."${proto%oe}" == ."ppp" ] && device=
[ ."$device" != ."$ifname" ] || device=
for dev in $ifname $device; do
        ifconfig "$dev" 0.0.0.0 down >/dev/null 2>/dev/null
done

I got my local routes list from cocooncrash's site (he gets them from local-route-server.is.co.za, aggregates them, and publishes every 6 hours). OpenWRT already has a static routing configuration system, but it's very verbose. So I wrote my own, adding the new configuration option routefile. I used these hotplug scripts to set up routing and source routing, with the help of iproute2:

$ ipkg install ip

$ mkdir /etc/routes
$ wget http://mene.za.net/za-routes/latest.txt -O /etc/routes/zaroutes

You'll probably want to update that route file regularly. I don't run cron on my WRT54GL, so I do it manually. Up to you.

/etc/config/network:

# ...local interfaces...

#### WAN configuration
config interface    wan
        option ifname   "eth0.1"
        option proto    pppoe
        option username "xxxxx@international.co.za"
        option password "xxxxxx"
        option routefile "/etc/routes/exceptions"
        option defaultroute 1

config interface    localdsl
        option ifname   "eth0.1"
        option proto    pppoe
        option username "xxxxx@local.co.za"
        option password "xxxxxx"
        option routefile "/etc/routes/zaroutes"
        option defaultroute 0

/etc/iproute2/rt_tables:

#
# reserved values
#
255     local
254     main
253     default
0       unspec
#
# local
#
1   wan
2   localdsl

/etc/hotplug.d/iface/20-split-routes:

case "$ACTION" in
  ifup)
    . /etc/functions.sh
    include /lib/network
    scan_interfaces
    config_get routefile "$INTERFACE" routefile

    # Does this interface have custom routes?
    if [ -e "$routefile" ]; then

      # Add routes for this interface
      cat "$routefile" | while read route; do
        ip route add "$route" dev "$DEVICE"
      done  

      # Set up source routing
      peer=`ip addr show dev $DEVICE | sed -n '/inet/ s#.* peer \([0-9.]*\)/.*#\1# p'`
      address=`ip addr show dev $DEVICE | sed -n '/inet/ s/.* inet \([0-9.]*\) .*/\1/ p'`

      ip route add "$peer" dev "$DEVICE" src "$address" table "$INTERFACE"
      ip route add default via "$peer" table "$INTERFACE"
      ip rule add from "$address" table "$INTERFACE"
    fi

    # Make sure this interface is present in all the custom routing tables:
    route=`ip route show dev "$DEVICE" | awk '/scope link  src/ {print $1}'`
    awk '/^[0-9]/ {if ($1 > 0 && $1 < 250) print $2}' /etc/iproute2/rt_tables | while read table; do
      ip route add "$route" dev "$DEVICE" table "$table"
    done
    ;;
esac

/etc/hotplug.d/net/20-split-routes:

case "$ACTION" in
  register)
    . /etc/functions.sh
    include /lib/network
    scan_interfaces

    # If this interface doesn't have a link local route, we don't need to bother
    route=`ip route show dev "$INTERFACE" | awk '/scope link  src/ {print $1}'`
    [ ."$route" = ."" ] && return 0

    # Add this interface's route to all custom routing tables
    awk '/^[0-9]/ {if ($1 > 0 && $1 < 250) print $2}' /etc/iproute2/rt_tables | while read table; do
      ip route add "$route" dev "$INTERFACE" table "$table"
    done
    ;;
esac

Now, lastly, it won't bring up both interfaces by default. That will be fixed by aliasing in the future, but for now:

/etc/init.d/network-multiwan:

#!/bin/sh
ifup wan
ifup localdsl
$ chmod 755 /etc/init.d/network-multiwan
$ ln -s ../init.d/network-multiwan /etc/rc.d/S49network-multiwan

That's it, and it's working beautifully :-)

What is source routing, people ask? The problem is that your router now has 2 WAN IP addresses. IP1 is used for local traffic, and IP2 for international. So if somebody in ZA tries to connect to IP2, the reply (local destination) will go out of Interface 1. The ISP on the other end of Interface 1 will drop this reply, because it's not coming from IP1.

Source routing tells the router that replies must go out of the same interface that the request came in on. I'm doing it by creating separate routing tables for traffic origionating from each WAN interface.

Gotchas

  • If you are using 2 different ISPs (say SAIX international and IS local), you must make sure that DNS queries get routed out the right interface. SAIX won't accept queries on their servers from IS, and vice versa.
  • SAIX Web proxies, Mail servers, and News servers don't accept traffic from local accounts. (especially from another ISP)

Comments

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.

I'm a noob when it comes to Li

I'm a noob when it comes to Linux (always been on the Microsoft side)... but your script gives me the functionality that I'm looking for. However I'm battling to figure out where to put all this code in... could you maybe point me in the right direction... please. Thanks.

I specifically wrote it at a h

I specifically wrote it at a high level, because it involves a bit of hacking around with OpenWRT, and there are many easy ways to screw your self up :-)

But I've got friends who followed this step by step, and succeeded, so it's perfectly do-able.

Give it a try ;-)

Hi Thanks for the response...

Hi Thanks for the response... I have given it a go but I'm not sure where to add the last script... is this done in some startup script or something? or do I just run this ont he router once and it creates all the files it needs by itself?

When I say "$ cat something" I

When I say "$ cat something" I mean I'm showing you the contents of a file. You have to create that file, with that contents.

My current set up runs tomato

My current set up runs tomato 1.10 on WRT54GL in Wireless bridge mode to connect my main computer set to DG834 which handles ADSL conn.

I'm trying to configure my network to be able to switch int/local traffic just as described in this blog.

Some questions :
Can WRT54GL with OpenWRT connect to DG834 in wireless bridge mode and establish multiple adsl connections?

Which version of OpenWRT should one download?
Does the official release 7.0.9 contain changes discussed in your blog?

No, most routers can't be made

No, most routers can't be made to make multiple PPPoE connections. But if they provide "bridged mode" (i.e. modem only, not router) then you can make multiple PPPoE connections from another machine connected to the modem.

Great post. I followed your gu

Great post. I followed your guide and noticed you have and noticed an error with
$ wget http://alm.za.net/ip/localroutes4.txt -O - | awk '{print $4}' > /etc/routes/localroutes
which should actually be
$ wget http://alm.za.net/ip/localroutes4.txt -O - | awk '{print $4}' > /etc/routes/zaroutes

Also it looks like the routes aren't inserted correctly.
See my routes table here http://pastebin.com/f5c701131.
Is the gateway supposed to be 0.0.0.0?
Accessing international sites work correctly, seems this routing table will not allow me to access local. How can i fix this?

Will there be any use for -> option routefile "/etc/routes/exceptions"?

Again thanks for getting this post out :)

peterpan: correct about the fi

peterpan: correct about the filename.

Yes, the gateway should be 0.0.0.0. It's a point-to-point connection, so the packets only have one place to go to.

Yes, the exceptions are things like NNTP-servers, proxies, etc. that SAIX don't want local-only users to be able to access. If you use IS for your local, you should also include SAIX's DNS resolvers in exceptions.

Thanks for the response tumble

Thanks for the response tumbleweed. Im using Webafrica prepaid 1 GB ADSL with 10 GB IS from Webafrica.

I tried adding what i think are saix DNS servers (168.210.2.2, 196.14.239.2) to the exceptions file, but that prevented any DNS requests from going through.

So back to my problem, i can access any international sites but i cant access any local sites... What am i doing wrong?

peterpan: Those or the IS reso

peterpan: Those or the IS resolvers, you can find the SAIX reslvers listed here: http://www.saix.net/cgi-bin/saix_dns.pl

As to your problem, your routing table looks fine. Is your firewall blocking it?

I can't say what you are doing wrong, without more debugging information.

Thanks 4response tumbleweed, I

Thanks 4response tumbleweed, I made successfully multiple PPPoE connection when going via cable directly from XP PC to the DG834 in "bridged (modem only)" mode.
However when attempting to route trough WRT54GL that is bridged to the DG834 (in modem only) connections fail.

Any how, looks that OpenWRT can be worked like a typical Linux box, so things make bit more sense. Perhaps my project of hopping onto local only DSL bandwagon will yield faster results than ICASA & Telscum reaching consumer beneficial agreements.

I really got inspired by your life story, and how you became the Linux geek. :))=- Hopefully one day ill be able to at least understand more if not stand shoulder to shoulder in CLUG.......

DblD: anything will move faste

DblD: anything will move faster than ICASA.

Thanks, and good luck, Linux is fun :-)

I'm sorry your connections are failing, it's probably something simple, find me on IRC / jabber, and I'll see if I can help.

Pingback

[...] the savvy users out there use hacks like mine to least-cost-route local traffic over cheaper IS “Local-Only” accounts (like these). [...]

lol... Why not just use Route

lol... Why not just use Route Sentry with RASPPPoE?? RASPPPoE allows u to dial another PPPoE connection and Route Sentry will split it... MUCH easier

Good Advisor: The thought of h

Good Advisor: The thought of having a router running Windows gives me the heebie-jeebies. In fact, the thought of any of my machines running Windows isn't particularly pleasant either.

The idea of this implementation is that it works transparently for my entire network. My router runs OpenWRT, so it makes sense to implement this all on my router. Nobody had ever tried to do multiple PPPoE connections on OpenWRT before, so I had to get a few things changed. This is a documentation of that process.

Pingback

[...] local Only. November 18, 2007 After reading Stephano Rivera’s (tumbleweed) post about local only ADSL accounts I had decided to implement the idea on my home network, which has an overworked 400mhz [...]

I'm looking at a bit of a diff

I'm looking at a bit of a different scenario - multiple ADSL accounts (one for each user) on one ADSL line. Still want to take advantage of the firewall features / traffic shaping (either via a WRT54GL or a self-built Linux / BSD box).

Anyone know if this is possible, and how to implement it? Perhaps using a distro such as SmoothWall Express or pfSense.

The rumour mills told me that

The rumour mills told me that you can only run 4xPPPoE connections on a single line. But I have no idea of the validity of that statement. Still one PPPoE connection per user sounds overly complex, unless you have a good argument for it.

As to distros, if it's possible in one distro, it's possible in another. It's quite simple to source route traffic, and can be worked out from my examples above.

Traffic shaping is a little more complex, if you want to shape all the connections together...

It's mostly to to ensure fairn

It's mostly to to ensure fairness - one user can't blame another for using his / her bandwidth (the max number accounts wouldn't exceed 3 or 4 anyway).

From what I have been reading, it would be quite a complex solution if you wanted to do traffic shaping on each PPPOE connection. Instead, it may be better to run your router in pure bridge mode and make a PPPOE connection on each PC, or run your router in half-bridge mode (PPPOE-Relay option required). This of course, removes any possibilty of unified traffic shaping and would require a software firewall on each PC (right?).

Either way, it seems it may be better just to use one ADSL account at a time (either via the router or a gateway / proxy box), and then do logging / accounting of traffic to determine per user usage.

pete, what I was describing in

pete, what I was describing in my reply to you is a solution where the router (be it the modem, or separate, using a full-bridged modem) terminates all the PPPoE connections. It would then be possible to shape, although tricky as hell.

It sounds like what you want is to put your modem in full-bridged mode, and then just start an individual PPPoE connection from each PC, that's it. Really easy, no accounting necessary. But forget shaping :-)

half-bridged mode isn't really relevant to your problem, as I've never heard of a router that can only half-bridge more than a single connection.

If you want to do accounting, get a single account, and either roll your own accounting, or use something like IPcop with built-in accounting and capping.

upping the signal strength

With OpenWRT/kamikaze on the wrt54gl router, is it possible to increase the signal strength so that a few walls in the way and a little distance don’t have such an effect on the network speed?

I think signal strength can

I think signal strength can be increased (using iwpriv). But it’s unlikely to help you much unless you increase the strength on all devices. You’d get better results by using more directional antennas.

paid to implement?

does anyone in cape town area want to implement this for me? (maybe a little extra features) im willing to pay for labour and hardware. you can get me on skype: haxoses thanks

made it work with dsl + eth :)

Thank you for the tutorial. It was very useful to configure my 2 links :)

Great Work!

It’s great to see some homebrewed solutions that deal with our (quite rare) internet issues, even better that they are for my favourite firmware flavour!

I’ve thought about using something similar to to reduce bandwidth costs for clients who have VPN to their branches as well as internet access over the same line, but never got around to look at it.

Kudos to you sir.

not working for me

Let me start off by saying i’m a complete n00b when it comes to routing tables.

I followed your guide, including modifying wget command as mentioned by peterpan, but if I run an ifconfig it seems all traffic is being routed though my international line, any idea on how to troubleshoot this?

Here’s the output of route :

Destination Gateway Genmask Flags Metric Ref Use Iface 41.245.64.1 * 255.255.255.255 UH 0 0 0 ppp0 165.146.172.1 * 255.255.255.255 UH 0 0 0 ppp1 192.168.1.0 * 255.255.255.0 U 0 0 0 br-lan default dsl-245-64-01.t 0.0.0.0 UG 0 0 0 ppp0

not sure what I’ve missed, it might be worth mentioning that i’ve been using this router with OpenWrt and pppoe configured before I applied the steps in the guide, perhaps I should go back to a vanilla Kamikaze and start again from there?

not working for me

sorry, route output got a bit messed up in the previous post, let's try again:

Destination Gateway Genmask Flags Metric Ref Use Iface

41.245.64.1 * 255.255.255.255 UH 0 0 0 ppp0

165.146.172.1 * 255.255.255.255 UH 0 0 0 ppp1

192.168.1.0 * 255.255.255.0 U 0 0 0 br-lan

default dsl-245-64-01.t 0.0.0.0 UG 0 0 0 ppp0

Fixed

Oops, a simple copy-paste error got me there all looking good now.

Thanks Tumbleweed!

wireless setup

Great post, i flashed from tomato 1.17 to Kamikaze 7.09… had a bit of trouble… i upgraded using this (openwrt-brcm-2.4-squashfs.trx) file in tomatos upgrade page…

i had to go into failsafe mode… telnet & setup a password, then ssh… took a while to figure out the settings (i should have printed the trouble shooting guide like it says…)

now i have wired internet working…

root@OpenWrt:~# cat /etc/config/network | tail -n 8
#### WAN configuration
config interface        wan
        option ifname   "eth0.1"
        option proto    pppoe
        option username "<my-saix-username>" # i only have 1 adsl account for now... ill order a IS later...
        option password "<my-password>"

how do i start stop connections? i wish the openwrt howto docs were more like gentoo’s…

anyway, here’s my wireless config

root@OpenWrt:~# cat /etc/config/wireless
config wifi-device  wl0
        option type     broadcom
        option channel  5

config wifi-iface
        option device   wl0
        option network  lan
        option mode     ap
        option ssid     my-network-name
        option encryption wpa2
        option key my-secret-password

i can see my wireless network… but my laptops fails to connect. what have i done wrong?

all i want to do for now is have the wired & wireless network bridged… so they can obviously share the internet connection & i can do networking… intelligent dynamic routing can come later :)

also, im really not sure what i want my :

option mode 
option encryption

to look like.. i want wpa2 encription, but can i put ‘psk2’ in the same line?

thanks

@OuZo

You start and stop connections like debian: ifup fooand ifdown foo.

Change your encryption type from wpa2 to psk2. wpa is for the entrprisey radius use of wpa.

Mode should be ap yes. It means access-point mode, rather than client mode or ad-hoc mode.

In many cases ad-hoc mode is technically more efficient than access-point mode (clients can talk directly to each other), but you’re limited to 11Mbps, and most operating systems favour access points to ad-hoc networks.

Pingback

[...] An example of a route server in South Africa is local-route-server.is.co.za, which is used by geeks to get an up-to-date list of “local” IP networks by running the show ip bgp Cisco command. A popular use for this information is “split routing” - a trick you might use if you have more than one Internet connection, especially if one of them is local-only (split routing is described here and here). [...]

Pingback

[...] Vote Local only DSL | Tumbleweed Rants [...]

Kamikaze 8

Hi what about updating this for kamikaze 8?

Thanks

re: Kamikaze 8

Yes, I will get to it at some point :-)

Just shout

I would really like that. I'm not very familiar with openwrt configuration but would like to help to get it done quicker. Just shout!

I have this going on kamakaze

I have this going on kamakaze 8.09. Its mostly the same, but I added the following to /etc/config/firewall to get the firewall to play:

config zone
option name localdsl
option input REJECT
option output ACCEPT
option forward REJECT
option masq 1

config forwarding
option src lan
option dest localdsl
option mtu_fix 1

Solution from Club Meet

I enjoyed your talk at Clug last night, but lost my bag and diary somewhere along the way. Could you post the two Ubuntu apps I need to download to make my Hardy compatible - ip2 and something else?

Split routing and FreeBSD

Hi there, your post was quite an inspiration to get a script like you have working on FreeBSD. Its been 3 interesting days and it was worth it! It was a total rewrite / new application.

I would like to know if there a reason why you included the the local routes into your firewall and not into the routing table ?

Thanks
Wynand

FreeBSD

Um, the routes are completely inside the routing table. The only changes to the firewall were to not be specific to ppp0

Kamikaze 8.09?

anyone managed to get this to work on Kamikaze 8.09?

So far i have had no success at all and this would be very helpful.

Someone please help?

8.09

You are going to have to work it out from first principles, I'm afraid. Start with the basic split routing logic and keep fixing things until it works.

I haven't played with 8.09 yet, and I'm not likely to in the near future.

i'm actually using 8.09.1 the

i'm actually using 8.09.1 the very latest one.

I have tried with the changes to the network script except the instructions are very minimal and i can't seem to work things out without getting syntax errors :(

dual wan on 8.09.1

I Successfully installed openwrt and configured dual wan in 8.09.1 - simply following the steps tumbleweed outlined, as well adding:

config zone
option name localdsl
option input REJECT
option output ACCEPT
option forward REJECT
option masq 1

config forwarding
option src lan
option dest localdsl
option mtu_fix 1

to /etc/config/firewall as adrian mentioned. Luci (the Web UI) also works with the multiple interfaces

I follow the instructions but

I follow the instructions but it seems to continually fail :(

Is there any way you could pastebin your configs and i could compare what mine look like once i change them?

:(

after following this to the T all that seems to happen is, when i try to start either interface, it takes down the currently connected interface first.

That was fixed by one of the

That was fixed by one of the patches listed at the beginning

?

Do you mean the /lib/network/config.sh section?
----
prepare_interface() {
local iface="$1"
local config="$2"

#SR: We don't want to reset any pppoe connections
config_get proto "$config" proto
[ "$proto" = "pppoe" ] && return 0
----
mine varies slighltly, should i change it to exactly what yours is?

----
prepare_interface() {
local iface="$1"
local config="$2"
local vifmac="$3"
local proto

#SR: We don't want to reset any pppoe connections
config_get proto "$config" proto
[ "$proto" = "pppoe" ] && return 0

:(

wish i could get this to work :(

Hei, great topic! and are

Hei, great topic!

and are you still keep up with openwrt?

I have try your method to hack the config.sh and ifdown.sh and find that the pppoe will still reset.

when "ifup wan1" and let it connected and then "ifup wan2" , pppd will hangup the connection immediately like this:
Mar 3 15:54:32 OpenWrt daemon.notice pppd[10723]: Modem hangup
Mar 3 15:54:32 OpenWrt daemon.info pppd[10723]: Connect time 4.5 minutes.
Mar 3 15:54:32 OpenWrt daemon.info pppd[10723]: Sent 9421 bytes, received 23251 bytes.
Mar 3 15:54:32 OpenWrt daemon.debug pppd[10723]: Script /etc/ppp/ip-down started (pid 11538)
Mar 3 15:54:32 OpenWrt daemon.notice pppd[10723]: Connection terminated.
Mar 3 15:54:35 OpenWrt daemon.info pppd[11628]: Plugin rp-pppoe.so loaded.
Mar 3 15:54:35 OpenWrt daemon.notice pppd[11628]: pppd 2.4.4 started by root, uid 0

that another pppd session starts, the first one hangup.
is this the bug of pppd and any suggestion?

I am using openwrt backfire 10.03.1, but I prefer to use your method than macvlan.
Please give some comments. Thanks a lot

Another problem

Another problem:

when the 2 pppoe are connecting from one wan port, No echo-requests can be received, ever tried to sent the value to 50.

Mar 4 10:34:07 OpenWrt daemon.info pppd[10991]: No response to 50 echo-requests
Mar 4 10:34:07 OpenWrt daemon.notice pppd[10991]: Serial link appears to be disconnected.
Mar 4 10:34:07 OpenWrt daemon.info pppd[10991]: Connect time 10 minutes.

But there is no problem with 2 pppoe connections from 2 different wan ports, each one can receive the echo-reply.

Maybe this cause by the same MAC address?

Post new comment

The content of this field is kept private and will not be shown publicly.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.