Elastic Search : How to get most researched terms

Question

i m implemeting elasticsearch in a symfony2 project with fos_elastica.

everythings works fine ( indexing data, updating, etc.)

i m currently looking for user behavior analysis : i would like to get the 10 most user searches or keywords in order to re-query it .

for example :

if 45% of searches are about yellow balloons and 45% are about red balloons, i would like to suggest on my homepage some yellow or red balloons

firstly, i was thinking about creating a symfony2 entity to save user search with a timestamp then compute last 1000 searches to get the most famous keywords. although it would surely work , that would be resource killer.

i was wondering if elasticsearch is able to provide these and how to implement it .

i ve read that i could create an index to store my user queries ( and that would be awsome, cause i could use facets to compute them really easily ) , but i don t know how to do save it directly in elastic search from symfony2 without an dedicated entity.

Answer 1

Okay, i finally got it !

here are the different steps :

1) create a new index in config.yml with a specific mapping for your keywords search

in config.yml

indexes:
    your_index:
        types:
            search:
                mappings:
                    value: {type:string}
                    date : {type:date}
                    provider: acme\AppBundle\Service\SearchProvider

2) create a new class SearchProvider in Service directory

in acme\Appbundle\Service\SearchProvider

<?php


namespace acme\AppBundle\Service;

use FOS\ElasticaBundle\Provider\ProviderInterface;
use Elastica\Type;
use Elastica\Document;

class SearchProvider implements ProviderInterface
{
protected   $searchType;
private     $search;

public function __construct(Type $searchType)
{
    $this->searchType = $searchType;
}

// the function you will call from your service
public function add( $search )
{
    $this->search = $search;
    $this->populate();
}

/**
 * Insert the repository objects in the type index
 *
 * @param \Closure $loggerClosure
 * @param array    $options
 */
public function populate(\Closure $loggerClosure = null, array $options = array())
{
    if ($loggerClosure) {
        $loggerClosure('Indexing users');
    }

    $date  = time();

    $document = new Document();
    $document->setData(array('value' => $this->search, 'date' => $date ) );
    $this->userType->addDocuments(array($document));
    $this->userType->getIndex()->refresh();
}
}

3) create a new service declaration in your service.yml

services:
acme.search_provider:
    class: acme\AppBundle\Service\SearchProvider
    arguments:
        - @fos_elastica.index.recetas.search
    tags:
        - { name: fos_elastica.provider, index: your_index, type: search }

4) call your service to store new searches like this

$this->get("acme.search_provider")->add("kapoue"); 

kapoue will be added to the searches.

5) get all the search keywords and rank it with aggregation

    $es                 = $this->get('fos_elastica.index.acme.search');
    $query              = new \Elastica\Query();

    $aggregation        = new \Elastica\Aggregation\Terms("top_hits");
    $aggregation->setField('value');
    $aggregation->setSize( 3 );

    $query->addAggregation($aggregation);

    $result             = $es->search($query);
    $mostResearched     = $result->getAggregation("top_hits");

    print_r ( $mostResearched ); die();