Network emulation for app testing.

I’ve been contracted to develop a WAN emulator for the testing of a teleconferencing-type application. The app runs on mobile devices and the expectation is that it will run most of the time on the cellular network. The app is designed for transmitting information from the field to a central location, so upload/uplink bandwidth is critical. Previously, we found the uplink bandwidth on the mobile carriers in Christchurch ranges from marginal to terrible.

So we’ve decided to turn the question around and measure how well the app performs for given amounts of bandwidth (and in the presence of other network properties like delay, etc.).

To do this I’ve taken the original network configuration:


and replaced the big scary uncontrollable internet blob in the middle with a Linux PC.


The major (and unavoidable) change is that the first hop from the mobile device is over Wifi, rather than 3G. Does that matter? Almost certainly yes. But, since I don’t have a test cell site sitting in the lab, I can’t see any way around it.

In the middle I’ve put a PC which acts as a wireless access point and router. Traffic between the the right and left sides of the diagram can be shaped using the network emulation (netem) layer in the Linux networking stack. At least that’s the plan.

In somewhat more detail, the PC is set up as…


The PC has three ethernet NICs, the built-in plus an Intel Pro/1000MT dual-port card. The PC also has a dual-band 3×3 802.11n wifi card from TP-Link.

The built-in NIC is connected to the greater internet and provides the main point of access to the machine.

Internally I’ve created two bridges, “outside” and “inside”. Each is assigned the x.x.x.1 address for a different subnet. The outside bridge contains one of the Intel NICs and the Wifi adaptor. The inside bridge contains the other Intel NIC.

This appears in my /etc/network/interfaces as:

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto eth0
iface eth0 inet dhcp

iface eth1 inet manual
iface wwan0 inet manual

auto br-outside
iface br-outside inet static
bridge_ports wlan0 eth2

auto br-inside
iface br-inside inet static
bridge_ports eth1

In addition, the host PC:

  • Provides an access point on the wifi adaptor using hostapd.
  • Provides DHCP and DNS to the two test networks using dnsmasq.
  • Uses iptables to NAT traffic on the test networks to the outside world (iptables config provided at the end of the post).

Finally, I’m running two VirtualBox virtual machines, one on each of two test networks. They’re very simple Ubuntu Server installations, mostly so I can run quick validation tests without having both PCs running — or in case I need to give one of the PCs back. Part of the experiment will involve testing the VMs versus the “real” PCs to see if there is a significant performance difference. The two VMs use VirtualBox’s “bridged adaptor” to connect to the relevant bridges.

n.b. I’m running Ubuntu Server 13.04 on everything (the main PC, the test PCs, the VMs), with the “stock” versions of all the relevant software as provided from Ubuntu. That means VirtualBox 4.2.10 for now…

For reference, I’ve done the NAT configuration by hand in /etc/rc.local:

# Clear any existing iptables
iptables -F

# Block most incoming traffic
iptables -I INPUT -i lo -j ACCEPT
iptables -A INPUT -i eth0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -i eth0 -p tcp --dport ssh -j ACCEPT
iptables -A INPUT -i eth0 -j DROP

# Configure basic NAT from br-inside and br-outside to the outside world on eth0
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -A FORWARD -i eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i br-inside -o eth0 -j ACCEPT
iptables -A FORWARD -i br-outside -o eth0 -j ACCEPT

exit 0

Sierra Wireless 3G/4G modem drivers for Linux 3.8

I recently forward-ported the latest Sierra Wireless Linux kernel modules to kernel 3.8 to get an Aircard 320U working under Ubuntu 13.04. Along the way I also ported to 3.6 for Ubuntu 12.10 and the Raspberry Pi.

It’s on github at

It’s not brilliant kernel hacking, but it got me up and running.

Here’s a bit from the readme:

I recently purchased a Sierra Wireless (now Netgear) Aircard 320U USB 3G/4G modem for some testing. I selected the Aircard 320U based partly on its technical specs (compatibility with the major New Zealand providers), as well as the fact they actually bother to provide support for Linux.

The unit is supported by two kernel modules. sierra is a USB-serial module, which enumerates a number of /dev/ttyUSB* serial devices. These allow direct communication with the modem. For example, I’m using /dev/ttyUSB2 to send AT-style commands to the modem.

The sierra_net module appears as a USB-network driver. As such, the modem appears directly as a network device (wwan0 for me) without any additional mucking about with PPP, etc.

Sierra provides instruction for building their kernel source. Unfortunately, the last entry is for “v3.2_1740” for kernel 3.0.x. As I’m using a relatively recent Ubuntu installation, I’m running a 3.8 kernel.

The Linux 3.8 kernel provides the sierra (no version number) and sierra_net (v2.0) drivers, but I was unable to get these drivers to enumerate the Aircard 320U. The tarball from Sierra lists sierra v.1.7.40 and sierra_net v.3.2, but out of the box the Sierra code would not build under kernel 3.8.x (changing USB APIs, I imagine).

It seemed like a case of the kernel containing old code which was being ported forward with each new kernel rev, while the vendor was providing updated code, but not keeping up with kernel dev. Seemed like a straightforward call to forward-port the driver, which you see here.

I should disclaim I didn’t do this using my deep knowledge of the Linux kernel APIs, but by a “monkey-see-monkey-do” port between the Sierra code and the current kernel code. It works for me, on Ubuntu 13.04, with an Aircard 320U. Maybe it will work for you too?

I was briefly testing under Ubuntu 12.10, so I also ported to 3.6. I’ve also been able to use this code to run the Sierra on a RaspberryPi running Raspbian (kernel 3.6.y). Actually building the kernel modules was mildly inconvenient, instructions can be found on the internet.

Open Source computer vision “applications”

I’ve been having a bit of browse through the forest of Open Source computer vision “applications” as of late. I quote “application” to mean packages with a high level of integration — either end-user applications, or high-level libraries which are one step from a demo application — in contrast to say, OpenCV or ANN which are very much at the nuts and bolts level.

I’ve made no effort to evaluate these packages, just to call out that they exist. I’m ambivalent to academic work which leads to large monolithic packages — I guess I just want things to be a bit more atomic so the quality visualizer from this project can be mated to the descriptor code from that project etc without completely knocking the two apart.

But on the other hand, these are wonderful repositories for knowledge and code — code for constructing and decomposing fundamental matrices, bundle adjustment, etc. There’s already far too much re-re-re-engineering of algorithms — although I’m as guilty as others, as re-implementation is a great way to learn an algorithm inside and out.

Of course, at this point I should mention OpenSLAM which aims to be a repository (in the philosophical and literal sense) for SLAM algorithms and test datasets.

  • Insight3d is a full 3D reconstruction application. Images in one side, point cloud (and geometric model?) out the other.
  • PTAM (Parallel Tracking and Mapping) is a SLAM-like toolkit/application designed for augmented reality, coming out of the Machine Vision group at Oxford.

These are a bit more like libraries, but I’ll call them out:

  • Christopher Zach has published Simple Sparse Bundle Adjustment (SSBA). He’s also provided a full Structure for Motion toolkit.
  • SBA is another sparse bundle adjustment toolkit “from Manolis Lourakis at the Institute of Computer Science of the Foundation for Research and Technology – Hellas (FORTH) in Heraklion, Crete, Greece.”

A small MSP430 board

I couple of months back I acquired a TI Launchpad, their loss-leader development board for their MSP430 family of microprocessors. I’ve had a long-term interest in the MSP430 as an alternative in the realm of 8-/16- bit micros.

Out of the box, I was able to the set up the mspgcc toolchain, and the other installation details were amalgamated from the interwebs. I believe the msp toolchain is now in the Ubuntu repositories, so all of the compilation la-di-da is a thing of the past.

Sadly, I was without an application.

The next week my coworker asked me where he could find a signal generator — I made the mistake of inquiring and it turned out he needed a 30Hz (later 25Hz) synchronization pulse for a cluster of computer vision cameras. Clearly a signal gen would be massive overkill (and would tie up the generator for months to come), so I offered to make a signal generator for him.

My first version was built on the Launchpad, and worked fine. Last week I finally got around to constructing a self-contained version.

Total time to first prototype was perhaps four hours, though I went through three iterations of the clocks — first using the internal very-low power oscillator (VLO), then the internal digitally-controlled oscillator (DCO), and finally using an external 32.768 kHz watch crystal. Each step was an attempt to find a little bit more timing accuracy.

For the curious, the schematics are here

It’s not the minimum number of components to run an MSP430, but close enough. Nor is it the smallest MSP that would do that job. But the G2211 came with the Launchpad.

The USB port only provides power (it’s convenient and there’s no chance of getting the voltage or polarity wrong). The C connecting reset to ground is recommended, but I’m not sure it’s mandatory. It should provide a little highpass filtering on the reset line. As built there’s no in-circuit programming, hence the socketed processor. It wouldn’t be too hard to add an ISP but I didn’t see the point.

The code is as follows (or here)

 * 25Hz flasher.  Ripped unashamedly from the Launchpad demo app.
 * This version is targeted at the 430G2211 with a watch crystal

//#include   <-taken care of by includeing io.h and setting -mmcu=msp430x2012 in cflags

#define     LED0                  BIT0
#define     LED1                  BIT6
#define     LED_DIR               P1DIR
#define     LED_OUT               P1OUT

// The Launchpad has two LEDs
// but the app board has just one
#undef      TWO_LEDS

#define     TA_OUT0               BIT2
#define     TA_DIR                P1DIR

void main(void)
  WDTCTL = WDTPW + WDTHOLD;                 // Stop WDT

  TA_DIR |= TA_OUT0;
  P1SEL  |= TA_OUT0;                        // Should set Pin 1.2 to TimerA out1

#ifdef TWO_LEDS
  LED_DIR |= LED0 + LED1;
  LED_OUT &= ~LED1;
  LED_DIR |= LED0;
  LED_OUT |= LED0;                          // To enable the LED toggling effect
  // Use a 32.768 KHz watch crystal
  BCSCTL2 = SELM_3 | SELS;                   // MCLK = SMCLK = LFXT1CLK
  BCSCTL3 &= ~(LFXT1S_3);                    // Clear LFXT1Sx for 32678 crystal on LFXT1
  BCSCTL3 |= XCAP_3;                      // Set ~12 pF capacitance 

  // As set up, MC_1 means count to CCR0
  // OUTMOD_7 means reset at CCR1, set at CCR0
  // ISR make the LEDs behave similarly -- RED on at CCR1, GREEN on at CCR0
  // Uses toggle mode, OUT1 toggles every time TimerA == TACCR1
  // And TimerA is in Up mode, so it counts 0->TACCR0, then resets
  // So the half-period of the output is set by TACCR0
  // 32768 / 655 = 50.02
  // So ~655 for 25MHz
  TACCR0 = 655;                             // By calibration
  TACTL = TASSEL_1 |  MC_1;                  // TACLK = SMCLK, Up Mode
  TACCTL1 = OUTMOD_4;                // Toggle mode
  TACCR1 = 0;

  WRITE_SR(GIE);                     // Enable interrupts

  /* Main Application Loop */
  while(1) {
    __bis_SR_register(LPM3_bits + GIE);          // LPM0 with interrupts enabled


interrupt(TIMERA0_VECTOR) ta0_isr(void)
  static unsigned int count = 0;


  if( ++count >= 25 ) {
#ifdef TWO_LEDS
    LED_OUT ^= (LED0 | LED1);
    LED_OUT ^= LED0;
    count = 0;


Not the tidiest code in the world. The 25Hz square wave is driven by the TimerA, while the flashing LEDs (the Launchpad board has two blinkenlights, while my board has just one) are driven my a more conventional counter-in-an-ISR. I’m quite sure I could fix it up so the LED is controlled by a timer as well, but I haven’t bothered.

Disabling shutdown when the power button is pressed

I run a pretty sparse Linux installation on my desktop. It’s based on Xubuntu 11.04 (right now), but typically I use musca in preference to the full xfce4 experience.

By default, if no other power management software (like xfce4-power-manager) is running, pressing the power button will initiate a machine shutdown. I turn my desktop off most evenings, but when I do leave it on, I’d always stumble in the next morning, stab the power button … and watch it shutdown.

Here’s the quick fix. When the power button is pressed, acpid loads the event at /etc/acpi/events/powerbtn. By default this calls the script /etc/acpi/

This script checks to see if any power management programs (again, gnome-power-manager, kpowersave, xfce4-power-manager, etc) are running. If they are, it just exits. If not, it gets to the last line:

# If all else failed, just initiate a plain shutdown.
/sbin/shutdown -h now "Power button pressed"

Well there’s the problem. I commented out the shutdown, reloaded acpid (this probably isn’t necessary), and away I went.

Reconsidering my foolishness

I took a few minutes to try option B on this serial-to-network problem.

Here’s a much simpler option.

On the Ministation, run a script like (this is the lightly tarted up version):



stty -F $PORT 57600

while [ true ];
sleep 1

The client netcat (nc) will die if it can’t connect to the server, hence the while loop. The sleep prevents flapping in that case. [Though as I discovered below, this only works as long as bits are coming into the serial port…]

And on the other end:



nc -kl $PORT >> $OFILE

Time to try it out, I think…


Here’s an even more tarted up version which lets you send data back out the serial port by poking it to the server through a FIFO.




stty -F $PORT 57600

while [ true ];
sleep 1




if [ ! -p $OUTFIFO ]
mkfifo $OUTFIFO
tail -f $OUTFIFO | nc -kl $PORT >> $OFILE

This seems pretty robust to most combinations of server or client problems except if the client can’t reach the server and there’s no data coming in the serial port. The client’s nc will die but the associated cat won’t die till it tries to write to the broken pipe.

Does this matter? Yes. If the client starts and can’t connect to the server, the nc will die, and if the widget is mis-configured to not output any data on the serial port, the (FIFO)->(server’s nc)->(client’s nc)->(serial port) redirection won’t work because the client’s nc is dead, but because the cat is still running, the while(1) won’t cycle. There’s probably a clever way around this, but I’ve definitely reached the limits of my shell scripting abilities.

Of course, if you have a network connection you could always SSH to the client and poke the serial port manually.

Ah, and these scripts will require old-skool logfile rotation:

cp ofile.txt ofile.txt.2 && cat /dev/null > ofile.txt

I assume logrotate can handle that elegantly.

2nd side-note. The Ministation SDK doesn’t build busybox’s stty by default. You’ll probably want that for setting the baud rate on the serial port. It’s easily added by changing the relevant line in conf/xs2/busybox.config to:



For the weather buoy project we need a way to take data in over our Ministation’s serial port and push it upstream over the network. As the ministation is pretty limited in flash and ram, I assumed a compiled (as opposed to scripted) solution would be best.

I think my ideal system would be store-and-forward, with the serial port as a “push,” something along the lines of:

  1. Open the serial port.
  2. Forever:
    1. Read an buffer the data.
    2. When you have one “line” of data, try to poke it out over a network interface.
    3. If you succeed, forget that line of data.
    4. If you fail, save it to try next time. When you reach some maximum amount of backlog, start dropping the older messages.

Where the other end of the conversation would be a network-to-disk server:

  1. Open a network port
  2. As data comes in, write it to a file.

It sounds like a simple brief, but I couldn’t find a canned solution which does the job.

One obvious choice is ser2net, though it’s actually designed the other way around. As a “server,” it listens on a network connection (many, actually), opening serial ports as necessary when a connection comes in.

I’m a bit rusty on Linux network programming, so as an interim, I thought I’d try a serial-to-syslog gateway, operating as above but pushing data to syslog (who can then handle the network forwarding).

It’s not a terrible solution, with a couple of caveats:

  • It only really works because our data is read-only, and it’s NMEA-like strings, so it’s ASCII, nicely delimited into lines. Syslog wouldn’t work with binary data, I don’t think.
  • The Ministation SDK runs busybox’s syslogd, which only supports UDP forwarding, not TCP. OK, so it would only provide one additional ACK of security, not true store and forward, but it’s would be a start.

On the other end of the connection, I’ll have a full-fat computer running rsyslogd which can handle a bit of store and forward to servers off in the cloud.

In any case, I wrote a ser2syslog. I won’t pretend it’s polished, but works for me. As per the README, I was originally going to build it from the framework of ser2net, but given ser2net is a mature, featureful product, and arranged backwards (as above), I stripped 95% of the ser2net code out. I believe (hope!) the copyright and attribution are correct.

Next up is testing it on a Ministation.


Hm. I just realized I could do the same thing very simply on the command line. Something like:

$ cat /dev/ttyS0 | logger -p -t “/dev/ttyS0”

And I could probably achieve what I need network-wise with netcat. Sigh….

PhD Proposal

As part of my studies, I’m required to submit a research proposal at about 6 months into my program. I did that long ago, then subsequently binned that topic.

Since January, I’ve been working on a new proposal, which I’ve posted here.

I’ll fully admit it’s too vague, and too long. But it’s also submitted and out of my hair.

Building custom Ubiquiti firmwares

As noted previously, I’m mucking about with a Ubiquiti Ministation2 as a networked SBC for a project. The Ministation2 is Linux-based, and Ubiquiti provide an SDK for rolling your own firmware. I wanted a mechanism for introducing my own customizations while also keeping the tarball downloads from Ubiquiti at arms length.

After a bit of iteration, I ended up with the a project I’ve posted at Github. It’s a kind of reverse-patch-and-overlay environment, and no, I’m not a shell and Makefile genius, so I accept there’s lots of room for improvement.

Basically, you check out the the Remix,

$ git clone git://


$ cd Ubiquiti-SDK-Remix
$ ./

This will download the SDK from Ubiquiti (the version is currently hardcoded in, expand it on top of itself, and patch the SDK’s files as appropriate. You end up with a hybrid of the original SDK with the Remix’s improvements.

The Remix won’t check that you have the relevant tools, so make sure you’ve installed the MIPS toolchain and other dependencies. Then:

$ make clean xs2

as before.

The remix currently includes just one new package — ser2net-2.7 (it builds, but totally untested on the Ministation), but it provides a good template for how to include other packages in apps/local/.

Custom firmware on the Ubiquiti Ministation

We’re using a Ubiquiti Ministation as wifi-enabled SBC for a weather station project. There’s a lot to like, and a few things lacking. It has a single 802.11b/g interface, a single ethernet port, and a single console/serial port. Like? Most everything. Dislike? The single serial port, as we want to use it as a serial-to-network gateway, which means sacrificing the console. It also runs a custom Linux installation, which we can hack it to our devious needs.

Instructions for rebuilding the system are out there, but pretty brief. Here’s my experiences.

Ubiquiti provide the SDK as a tarball, as well as a MIPS toolchain as a i386 deb. That’s fine with me as I’m running Ubuntu, though sadly I’m running 64-bit, so I built up a small VirtualBox machine for development. Not optimal, but nothing wrong with it. It’s noted here that you could also build the MIPS compiler with buildroot. your mileage may vary.


  1. Download the SDK and toolchain from here, selecting the Ministation2 as the model. As of right now, the current firmware is v3.6.1.
  2. Install the toolchain on a i386 Debian/Ubuntu system.
  3. You’ll also need a few other packages. I got this list from here, so I can’t verify it’s correctness.

    $ sudo apt-get install sharutils fakeroot zlib1g-dev patch lzma flex bison

    If a package is missing, you’ll notice pretty quickly when the compile blows up.

  4. Unpack the SDK someplace convenient:

    $ mkdir ministation
    $ cd ministation
    $ tar -xjvf ~/SDK.UBNT.v3.6.1.4873.tar.bz2
    $ cd SDK.UBNT.v3.6.1.4873

First, let’s see how the SDK works out of the box. Note, I already have my ministation up, running, and configured properly. So what happens when I “upgrade” my ministation with a filesystem I’ve built myself.

First, build it:

$ make xs2

And go get a refreshing drink.

The resulting package will end up in a directory with a name like rootfs/XS2.ar2316.v3.6.1.unknown.110707.1210 which gives the Ubiquiti model (XS2), the chip (ar2316), version number (v3.6.1) and a date/timestamp (110707.1210). The unkown is where Ubiquiti puts a patch number (like 4873 in this case), not sure why that doesn’t make it through.

Conveniently, they also symlink in rootfs/XS2.ar2316.v3.6.1.latest which points to the latest build.

In that directory are a number of intermediate results, as well as a few .bin packages:

$ ls -al
total 38976
drwxr-xr-x 2 aaron aaron 4096 2011-07-14 16:17 .
drwxr-xr-x 14 aaron aaron 4096 2011-07-14 16:22 ..
-rw-r–r– 1 aaron aaron 678439 2011-07-14 16:17 bzImage
-rwxr-xr-x 1 aaron aaron 2605056 2011-07-14 16:17 squashfs.bin
-rwxr-xr-x 1 aaron aaron 493554 2011-07-14 16:17 vmlinux.lzma
-rwxr-xr-x 1 aaron aaron 25055859 2011-07-14 16:17 vmlinux.notstripped
-rw-r–r– 1 aaron aaron 3099018 2011-07-14 16:17 XS2.ar2316.v3.6.1.unknown.110714.1614-8M.bin
-rw-r–r– 1 aaron aaron 3099018 2011-07-14 16:17 XS2.ar2316.v3.6.1.unknown.110714.1614.bin
-rw-r–r– 1 aaron aaron 4858567 2011-07-14 16:17 XS2.ar2316.v3.6.1.unknown.110714.1614.debug_binaries.notstripped.tgz

Comparing XS2.ar2316.v3.6.1.unknown.110714.1614.bin to the “stock” Ubiquiti filesystem:

$ ls -l
total 9856
-rw-r–r– 1 amarburg amarburg 3244841 2011-07-19 10:50 MiniStation2-v3.6.1.build4866.bin

my version is a bit smaller. Hmm. Well, never mind.

Install the new version of the s/w through the the web interace. Luckily, the Ministation warns me I’m about to break things:

Well, let’s go. I monitored the process on the serial console (which is a separate topic).

Writing ‘FIS directory ‘ to /dev/mtd4(FIS directory ) … [[1300398.770000] Restarting system.
+Ethernet eth0: MAC address 00:15:6d:a4:14:ed
IP:, Gateway:
Default server:

RedBoot(tm) bootstrap and debug environment [ROMRAM]
Ubiquiti Networks certified release, version 0.9.00483.1103151313 – built 13:14:44, Mar 15 2011

Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc.

Board: Ubiquiti Ubiquiti AR2315/6/7/8 based board (Ubiquiti PicoStation2 detected)

Arch: ar2316
RAM: 0x80000000-0x82000000, [0x80040b80-0x80fe1000] available
Flash: 0xbfc00000-0xbfff0000, in 64 blocks of 0x00010000 bytes each
== Executing boot script in 1.000 seconds – enter ^C to abort
RedBoot> cache off
RedBoot> fis load -d -e kernel
Trying LZMA decompression…
Image loaded from 0x80041000-0x801df23c
Entry point: 0x80196040, address range: 0x80042000-0x801bf000
RedBoot> go
[ 0.000000] CPU revision is: 00019064
[ 0.000000] Primary instruction cache 16kB, physically tagged, 4-way, linesize 16 bytes.
[ 0.000000] Primary data cache 16kB 4-way, linesize 16 bytes.
[ 0.000000] Linux version 2.4.27-ubnt0 (aaron@moa) (gcc version 3.3.3) #2 Tue Jul 19 10:55:22 NZST 2011

Much to my surprise, it worked. And, much to my surprise, my configuration has been maintained. Sweet.