What does all of the data in a Google App Engine HTTP log mean? For example, in the following (anonymised) log:
107.10.42.191 - foobiz [10/May/2011:17:26:28 -0700] "GET /page.html HTTP/1.1" 500 2297 "http://www.example.com/home.html" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4,gzip(gfe),gzip(gfe),gzip(gfe)" "www.example.com" ms=364 cpu_ms=23 api_cpu_ms=0 cpm_usd=0.001059
I understand most of the columns, can you help fill in columns 2 and 14?
- IP Address: 107.10.42.191
-
Just a hyphen, or something more?: -
- Logged in user: foobiz
- Request Time: [10/May/2011:17:26:28 -0700]
- HTTP Request: "GET /page.html HTTP/1.1"
- HTTP Response Status Code: 500
- HTTP Response Size in Bytes: 2297
- Referring Page: "http://www.example.com/home.html"
- Browser Info: "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4,gzip(gfe),gzip(gfe),gzip(gfe)"
- Host: "www.example.com"
- Total Time: ms=364
- CPU Time: cpu_ms=23
- API Time: api_cpu_ms=0
- What is this?: cpm_usd=0.001059
I know there is a similar question on SO, but it seems outdated and wasn't really answered.
Logs are in Apache Combined Log Format, with some additional fields. The fields, in order, are:
- Client's IP address (107.10.42.191)
- The RFC1413 identity of the client (in practice, always '-')
- The userid determined by HTTP authentication ('foobiz')
- The timestamp of the request ('[10/May/2011:17:26:28 -0700]')
- The first line of the request, containing request method, path, and HTTP protocol version ("GET /page.html HTTP/1.1")
- The status code returned by the server (500)
- The size in bytes of the response (2297)
- The referer path ("http://www.example.com/home.html")
- The user-agent ("Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-us) AppleWebKit/533.19.4 (KHTML, like Gecko) Version/5.0.3 Safari/533.19.4,gzip(gfe),gzip(gfe),gzip(gfe)")
- The hostname
- The wallclock milliseconds required to fulfill the request
- The CPU milliseconds required to fulfill the request
- The CPU milliseconds spent by API calls
- An estimate of the cost of 1000 requests like this, in USD.
This question has been answered here: GAE/J request log format breakdown
cpm_usd is the estimated cost of 1000 similar requests in US dollars.