Application performance metrics for Python

Question

We're writing a Python web-application, where we'd like to know general-ballpark-figures about what our users are doing.

As an example, we have a bunch of legacy-modes for setting the used locale; url-parameter, using different cookies in different formats, Accpet-Language-headers &c. We would really like to measure this in some way (but logging it is simply too much data).

I've tried looking for libraries like Metrics for Node.js, but I can't seem to find anything for Python. Right now, I think we can make do with a bunch of UNIX-load-like counters. (But more is always better, right?)

Have I missed some obvious library or some smart technique somewhere?

Answer 1

Just found the PyCounters project, which looks just like what I needed to begin with.

Answer 2

A Python project called Graphite coupled with a Node.js project called Statsd can do exactly the thing you want. Many times people use these projects for software performance metrics and business metrics as well.

Incidentally there are hosted app versions that provide a lot of the same functionality as Graphite and Statsd. My company's product, Instrumental, will do the metric hosting frontend and graph generation for you; in Python, you'd use a Python statsd client in concert w/ our Statsd to Instrumental proxy to report your metrics to our service.

Since the accepted answer was doing file-based logging (by default at least:) ), I'll note the transportation layer is interesting among these different things: A statsd server (whether the official or our proxy) uses UDP to send your metric information to the server. You can also contact Graphite or Instrumental directly using a line based TCP protocol; other commercial services out there like librato Metrics or Stathat take in your metrics using HTTP messages. Depending on the scale of messages you're sending, any one of these tools may be more or less attractive to you.