Network emulation for app testing.

I’ve been contracted to develop a WAN emulator for the testing of a teleconferencing-type application. The app runs on mobile devices and the expectation is that it will run most of the time on the cellular network. The app is designed for transmitting information from the field to a central location, so upload/uplink bandwidth is critical. Previously, we found the uplink bandwidth on the mobile carriers in Christchurch ranges from marginal to terrible.

So we’ve decided to turn the question around and measure how well the app performs for given amounts of bandwidth (and in the presence of other network properties like delay, etc.).

To do this I’ve taken the original network configuration:


and replaced the big scary uncontrollable internet blob in the middle with a Linux PC.


The major (and unavoidable) change is that the first hop from the mobile device is over Wifi, rather than 3G. Does that matter? Almost certainly yes. But, since I don’t have a test cell site sitting in the lab, I can’t see any way around it.

In the middle I’ve put a PC which acts as a wireless access point and router. Traffic between the the right and left sides of the diagram can be shaped using the network emulation (netem) layer in the Linux networking stack. At least that’s the plan.

In somewhat more detail, the PC is set up as…


The PC has three ethernet NICs, the built-in plus an Intel Pro/1000MT dual-port card. The PC also has a dual-band 3×3 802.11n wifi card from TP-Link.

The built-in NIC is connected to the greater internet and provides the main point of access to the machine.

Internally I’ve created two bridges, “outside” and “inside”. Each is assigned the x.x.x.1 address for a different subnet. The outside bridge contains one of the Intel NICs and the Wifi adaptor. The inside bridge contains the other Intel NIC.

This appears in my /etc/network/interfaces as:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

iface eth1 inet manual
iface wwan0 inet manual

auto br-outside
iface br-outside inet static
bridge_ports wlan0 eth2

auto br-inside
iface br-inside inet static
bridge_ports eth1

In addition, the host PC:

  • Provides an access point on the wifi adaptor using hostapd.
  • Provides DHCP and DNS to the two test networks using dnsmasq.
  • Uses iptables to NAT traffic on the test networks to the outside world (iptables config provided at the end of the post).

Finally, I’m running two VirtualBox virtual machines, one on each of two test networks. They’re very simple Ubuntu Server installations, mostly so I can run quick validation tests without having both PCs running — or in case I need to give one of the PCs back. Part of the experiment will involve testing the VMs versus the “real” PCs to see if there is a significant performance difference. The two VMs use VirtualBox’s “bridged adaptor” to connect to the relevant bridges.

n.b. I’m running Ubuntu Server 13.04 on everything (the main PC, the test PCs, the VMs), with the “stock” versions of all the relevant software as provided from Ubuntu. That means VirtualBox 4.2.10 for now…

For reference, I’ve done the NAT configuration by hand in /etc/rc.local:

# Clear any existing iptables
iptables -F

# Block most incoming traffic
iptables -I INPUT -i lo -j ACCEPT
iptables -A INPUT -i eth0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --dport ssh -j ACCEPT
iptables -A INPUT -i eth0 -j DROP

# Configure basic NAT from br-inside and br-outside to the outside world on eth0
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -A FORWARD -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i br-inside -o eth0 -j ACCEPT
iptables -A FORWARD -i br-outside -o eth0 -j ACCEPT

exit 0