Sunday, August 18, 2013

Reverse engineering network traffic with a Raspberry Pi

Sometimes it's useful to monitor traffic between a client and the server for debugging or to figure out how something works. In the past I've used a Linksys WRT54gl router which can be flashed to run OpenWRT. These devices don't have much free space so recently I've started using a Raspberry Pi for the same purpose. It's much more compact and more capable.

To create a Wifi hotspot with a Raspberry Pi, I followed the instructions at Adafruit. The system needs a dhcp server to give out addresses to wifi clients, hostapd to authenticate clients and some iptables commands to nat the addresses of TCP requests going through the device.

The Wifi USB dongle I purchased is an Edimax Nano USB WiFi adapter (EW-7811Un) which cost $35 over the counter. These seem to work well with the Pi's limited power budget and are capable of being a hotspot.

My network is called "honeypot".

The simplest way to snoop on what an app is doing is with tcpdump. I monitor traffic on the wireless interface and display the first part of each packet in ascii. The command is "sudo tcpdump -i wlan0 -A". Here's what I see on opening a TV guide app:

You can see they are talking to flurry and crashlytics, but the display is not very convenient for seeing what's going on.

A more friendly way to watch HTTP traffic is to use a proxy such as mitmproxy. Then set the proxy on the device to point to it. Now in our terminal we see a nice summery of each request.

Pressing return lets us drill in to the request and see the headers nicely formatted:

Pressing tab switches over to the response:

Lots of interesting things to learn in there! I typically then use the MacOS JSON Edit program for exploring json data (there's lots of tools around for this).

iOS apps seem to honour the proxy setting but I find that often Android apps do not, or only do some. It's possible to use iptables to set up a transparent proxy. I've got this mostly working although most https requests mostly seem to fail, here's my magic incantation:

# start a transparent proxy
sudo sysctl -w net.ipv4.ip_forward=1
# clean old firewall
sudo iptables -F
sudo iptables -X
sudo iptables -t nat -F
sudo iptables -t nat -X
sudo iptables -t mangle -F
sudo iptables -t mangle -X

# nat on the local lan
sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
sudo iptables -A FORWARD -i eth0 -o wlan0 -m state --state RELATED,ESTABLISHED -j ACCEPT

# forward all requests to the proxy
sudo iptables -t nat -A PREROUTING -i wlan0 -p tcp --dport 80 -j REDIRECT --to-port 8080
sudo iptables -t nat -A PREROUTING -i wlan0 -p tcp --dport 433 -j REDIRECT --to-port 8080

mitmproxy -T --host

So, a $38 raspberry pi plus a $35 Edimax wifi dongle, makes a pretty nice Wifi hotspot with advanced network inspection capabilities.


Anonymous said...

Thanks for posting the article, but you got swindled on the edimax. It's 10$ on Amazon.

Peter Marks said...

Ha, wow $10 for the Edimax? I'll have a look around - thanks.

Anonymous said...

Very interesting. In the past when I have wanted to analyze the network traffic of some mobile application I am testing (or some malware), I have used the same router (WRT54GL with DD-WRT installed).

While as you mentioned, the on-board memory for the router is quite small to store the dumps, there are two ways to by-pass that.

1. Mess with the IP Tables to forward all traffic to another computer on the network. This is what I have actually done.

2. DD-WRT supports SMB (CIFS), so it is possible to maintain a location on another computer on the network to save the traffic. I have not done this before so it is just brain storming and what I have seen other people doing.

However, in the end you mentioned that the Pi is more compact which is very true. This means that now I can have two different hardware to analyze network traffic. Thank you for sharing :D

Anonymous said...

Good article, Thanks

Jason Scatena said...

Thanks for the article, I recently used this as the basis for a research project in which we built an AP that can save snapchats sent or received from a connected device. I used a script based largely of yours for this purpose(properly credited in the report).

In using your script however I noticed why you are having issues with SSL, it lloks like you accidently wrote 433 as the port for HTTPS when it is actually 443. Hope that helps.

Thank you again for the post