Your download activity in one file!

I started downloading a very big set of files today on bittorrent (using the simple and elegant Transmission Bittorrent Client that comes with Ubuntu).  As I was watching the download speed indicator fluctuate up and down, the data analyst in me started stirring.  I thought it would be neat if I could find a program that would output my download speed over time to a dataset that I could then analyze for trends, averages, highs and lows.

I inputted the term 'bandwidth' in Synaptic Package Manager to see what kinds of programs were available for this purpose.  I found some simple command line programs such as nload, bmon, and iftop.  All of them provide text-mode screens that help you monitor your bandwidth, but none of them allow you to output download speed over time to a plain text file.

After examining the man (manual) pages for each of these programs, I noticed a commonality: each of these programs sample bandwidth information from a file named dev in the /proc/net/ directory on my computer.  Curious for more info, I navigated to that directory and opened up the file.  Here's an example of what it looks like:

If you've spent any time messing around with your network connection settings in Ubuntu (or most other operating systems) then you'll recognize most of the row titles in the above image.  Now, my computer is only connected to the net through a wireless connection to my Router/ADSL Modem.  Therefore, I don't expect any significant data in rows other than the one titled wlan0.

Next is the neat step: Linux repeatedly updates the dev file with the total number of bytes and packets received since boot-up.  All you need to do to find out the download speed in bytes/second is:

• Open the file
• Copy the numeric value just to the right of wlan0, under the bytes column
• Wait a second
• Do the same thing with the next value
• Subtract the two!

Of course nobody in their right minds would want to do this manually, but the implementation of the above algorithm is very simple in Python:

def get_bytes():
    dev = open('/proc/net/dev','rb').read()
    import re
    pat = re.compile('wlan0:\s*([0-9]+)\s*')
    return int(pat.search(dev).group(1))

def get_kbps():
    import time
    bytes1 = get_bytes()
    time.sleep(1)
    result = round(((get_bytes() - bytes1)/1000.0),1)
    return '%3.1f' % result

The first function above simply opens the dev file and returns the numeric value beside wlan0.  The second function uses the first function to get the bytes value two times and calculates Kilobytes/second (it's a smaller number than bytes/second).

I'm now using these functions to calculate Kilobytes/second every 3 seconds for a total of 4 hours to get my dataset!  If you'd like to use these functions just remember that if you're using a wired ethernet connection, change wlan0 to eth0 and you will be able get your data.