Best way to count page view in Plone

Question

I'm working in a case require fairy complex use of page views.

Each content object will have page view, and this should be accessed easily so that we can do various things related to it (sorted on catalog result, display, calculate popular metter ...). The closest equivalent is the youtube video's view.

I'm thinking of some possible ways to implemented this:

1. Use annotation storage & indexer to create portal_catalog index & metadata.

2. Use indexer only (either use volatile attribute or update index base on the previous index) so that we don't have to write a frequently changed data twice. Page view is only store in object brain.

3. Use a relational database. Then how can we make it work with portal_catalog ?

4. Use a wrapper layer before Plone to do analytics and get desired data through some API. This scarify flexibility but helps reduce much work on Plone side (write event subscriber, check session, cookie ...), and the performance should be better ?

Your ideas/experience on this ?

Answer 1

We have used an external log analyser for a client project (a large private intranet). Architecture:

• A js library adds a 'web bug', an empty gif with additional query parameters, loaded from a dedicated nginx server.
• A log processor picks up the nginx logs, rotates them, and parses the lines into a database, counting access together with the additional metadata. The entries in the db include the UID of the content, among other interesting angles.
• The site has read-only access to the same database, to make stats queries.

Page counts are then easy, just query the database for the right UID. Ranked lists are not much harder; query the statistics, then use the UIDs to attach catalog data to the result set.

The biggest problem we face now is a lack of data warehousing know-how (turning individual access rows in the database into efficient aggregates), and we are looking into retooling this setup to use Piwik as the statistics engine instead.

We cannot use Google Analytics in this particular case, but if you do not have such a restriction, I'd certainly would advise you to look into collective.googleanalytics and see if you can make it fit your use case.