Google Analytics - Getting raw data logs

Question

I have an app that sends data to Google Analytics. I am interested in accessing and storing this data on a Hadoop cluster. I am guessing this raw data will be in the form of logs. In particular, I would like to see the user_id, the searches made by the user and the search option that he/she decided to pay for on the app.

How can I do this? I am completely new to GA and I was not the one who set up GA for the app. I am just trying to see if there is a way through which I can access this raw data.

Would like to add that I cannot use Big Query since we do not have access to it. And the folks who set up GA are not interested in upgrading to Universal Analytics.

Any help/thoughts/suggestions are appreciated.

Answer 1

There is no way to get the logs, but ..

The Google Analytics API will let you extract your data out of the system.

There are limits to what you can do:

1. you are limited to 7 dimensions and 10 metrics per requests.
2. There is also a quota of 10k requests per day per profile (view).
3. some of the information you are talking about is not available. Unless the Google Analytics account is set up correctly.
4. The data will still be aggregated in one way or another. The smallest time unit available in the API is minutes, so you will not be able to get raw data with timestamps, for example.

It may be good to note that a professional Google Analytics customer could export the raw data from GA to Big Query. Exporting data from BigQuery is free of charge, but storage and query processing is priced based on usage.

Premium analytics at a reasonable price for one flat annual fee of $150,000

Answer 2

since we're supposed to answer the original question, there is no way to get actual raw Google Analytics logs other than by duplicating the server call system.

In other words, you need to use a modified copy of the analytics.js script to point to a hosted webserver that can collect server calls.

Long story short, you want your site to capture hits to http://www.yourdatacollectionserver.com/collect?v=1&t=pageview[...] instead of http://www.google-analytics.com/collect?v=1&t=pageview[...]

This is easily deployed using a tag manager such as Google's GTM, along with normal Google Analytics tags.

That will effectively create log entries in your web server which you can process using an ETL or Snowplow or Splunk or your favorite Python/perl/Ruby text parsing engine.

It is then up to you to process the actual raw logs into something manageable. And before you ask, this is not retroactive.