How do I track public bandwidth usage on EC2 instances or Elastic IP’s?

Question

I am looking into finding a way to track public bandwidth usage on a per-instance or per elastic IP basis. Amazon does not seem to offer these metrics. You can get total in/out bandwidth through their reporting mechanisms, but this includes private network bandwidth, and is account wide. You can use cloudwatch to gather more in depth metrics, but they also lump public and private bandwidth together. We are looking into rolling our own, but your servers are built with one interface, and any elastic IPs are NAT’d to that interface. Since everything goes through one interface, it is all lumped together.

Does anyone have any suggestions? Have you ever encountered a similar issue? That is a linux server environment with one interface from which you had to determine public bandwidth usage.

Answer 1

Answering an old question for the benefit of Googlers.

We encountered a similar problem, and "solved" it using iptables counters, making us of the fact that all outgoing traffic that is private will be on a 10.0.0.0/8 IP address, with the remainder being public traffic. You can also track input for other purposes; only outgoing public traffic is charged, of course.

So, create some counters:

   iptables -A INPUT -s 0.0.0.0/0    --> Total incoming traffic
   iptables -A INPUT -s 10.0.0.0/8   --> private incoming  traffic
   iptables -A OUTPUT -d 0.0.0.0/0   --> Total outgoing traffic
   iptables -A OUTPUT -d 10.0.0.0/8  --> private outgoing traffic

Check counters:

   iptables -nv -L INPUT --> counters about incoming traffic
   iptables -nv -L OUTPUT --> counters about outgoing traffic

NOTE: When you use the values, you get private and TOTAL: so to get public, subtract private from Total before using it for anything.

You can also zero out the counters if you don't want to report cumulative bandwidth:

   iptables --zero INPUT  --> clear counter
   iptables --zero OUTPUT --> clear counter

The following is an (ugly) bash script that will push this information out to Ganglia, assuming you created the counters already:

 #!/bin/bash
 OUTPUT_PUBLIC=`sudo iptables -nvx -L OUTPUT | head -3 | tail -1 | tr -s [:blank:] |cut -d' ' -f3`
 OUTPUT_PRIVATE=`sudo iptables -nvx -L OUTPUT | tail -1 | tr -s [:blank:] |cut -d' ' -f3`
 let OUTPUT_PUBLIC=$OUTPUT_PUBLIC-$OUTPUT_PRIVATE
 sudo iptables --zero INPUT
 sudo iptables --zero OUTPUT

 gmetric -n "public_outbound_traffic" -v $OUTPUT_PUBLIC -t uint32 -u "bytes"
 gmetric -n "private_outbound_traffic" -v $OUTPUT_PRIVATE -t uint32 -u "bytes"

Run this in a cronjob, just make sure that the cronjob frequency matches up with your ganglia reporting frequency (or otherwise handle possible mismatches).

Hope this helps someone.