How to provide highlighting with Spring data elasticsearch

Question

it seems that SpringData ES don't provide classes to fetch highlights returned by ES. Spring Data can return Lists of Objects but the highlights sections in the Json returned by ES is in a separated part that is not handled by the "ElasticSearchTemplate" class.

Code example :-

QueryBuilder query = QueryBuilders.matchQuery("name","tom"); 
SearchQuery searchQuery =new NativeSearchQueryBuilder().withQuery(query).
                               with HighlightFields(new Field("name")).build();
List<ESDocument> publications = elasticsearchTemplate.queryForList
                                                (searchQuery, ESDocument.class);

I might be wrong, but I can't figure out to do only with SpringDataES. Someone can post an example of how we can get highlights with Spring Data ES ?

Thanks in advance !

Answer 1

From the test cases in spring data elasticsearch I've found solution to this :

This can be helpful.

@Test
public void shouldReturnHighlightedFieldsForGivenQueryAndFields() {

    //given
    String documentId = randomNumeric(5);
    String actualMessage = "some test message";
    String highlightedMessage = "some <em>test</em> message";

    SampleEntity sampleEntity = SampleEntity.builder().id(documentId)
            .message(actualMessage)
            .version(System.currentTimeMillis()).build();

    IndexQuery indexQuery = getIndexQuery(sampleEntity);

    elasticsearchTemplate.index(indexQuery);
    elasticsearchTemplate.refresh(SampleEntity.class);

    SearchQuery searchQuery = new NativeSearchQueryBuilder()
            .withQuery(termQuery("message", "test"))
            .withHighlightFields(new HighlightBuilder.Field("message"))
            .build();

    Page<SampleEntity> sampleEntities = elasticsearchTemplate.queryForPage(searchQuery, SampleEntity.class, new SearchResultMapper() {
        @Override
        public <T> Page<T> mapResults(SearchResponse response, Class<T> clazz, Pageable pageable) {
            List<SampleEntity> chunk = new ArrayList<SampleEntity>();
            for (SearchHit searchHit : response.getHits()) {
                if (response.getHits().getHits().length <= 0) {
                    return null;
                }
                SampleEntity user = new SampleEntity();
                user.setId(searchHit.getId());
                user.setMessage((String) searchHit.getSource().get("message"));
                user.setHighlightedMessage(searchHit.getHighlightFields().get("message").fragments()[0].toString());
                chunk.add(user);
            }
            if (chunk.size() > 0) {
                return new PageImpl<T>((List<T>) chunk);
            }
            return null;
        }
    });

    assertThat(sampleEntities.getContent().get(0).getHighlightedMessage(), is(highlightedMessage));
}

Answer 2

Spring Data Elasticsearch 4.0 now has the SearchPage result type, which makes things a little easier if we need to return highlighted results:

This is a working sample:

    String query = "(id:123 OR id:456) AND (database:UCLF) AND (services:(sealer?), services:electronic*)"
    
    NativeSearchQuery searchQuery = new NativeSearchQueryBuilder()
            .withPageable(pageable)
            .withQuery(queryStringQuery(query))
            .withSourceFilter(sourceFilter)
            .withHighlightFields(new HighlightBuilder.Field("goodsAndServices"))
            .build();
    
    
    SearchHits<Trademark> searchHits = template.search(searchQuery, Trademark.class, IndexCoordinates.of("trademark"));
    SearchPage<Trademark> page = SearchHitSupport.searchPageFor(searchHits, searchQuery.getPageable());
    return (Page<Trademark>) SearchHitSupport.unwrapSearchHits(page);

And this would be the response from Page object in json:

{
    "content": [
        {
            "id": "123",
            "score": 12.10748,
            "sortValues": [],
            "content": {
                "_id": "1P0XzXIBdRyrchmFplEA",
                "trademarkIdentifier": "abc234",
                "goodsAndServices": null,
                "language": "EN",
                "niceClass": "2",
                "sequence": null,
                "database": "UCLF",
                "taggedResult": null
            },
            "highlightFields": {
                "goodsAndServices": [
                    "VARNISHES, <em>SEALERS</em>, AND NATURAL WOOD FINISHES"
                ]
            }
        }
    ],
    "pageable": {
        "sort": {
            "unsorted": true,
            "sorted": false,
            "empty": true
        },
        "offset": 0,
        "pageNumber": 0,
        "pageSize": 20,
        "unpaged": false,
        "paged": true
    },
    "searchHits": {
        "totalHits": 1,
        "totalHitsRelation": "EQUAL_TO",
        "maxScore": 12.10748,
        "scrollId": null,
        "searchHits": [
            {
                "id": "123",
                "score": 12.10748,
                "sortValues": [],
                "content": {
                    "_id": "1P0XzXIBdRyrchmFplEA",
                    "trademarkIdentifier": "abc234",
                    "goodsAndServices": null,
                    "language": "EN",
                    "niceClass": "2",
                    "sequence": null,
                    "database": "UCLF",
                    "taggedResult": null
                },
                "highlightFields": {
                    "goodsAndServices": [
                        "VARNISHES, <em>SEALERS</em>, AND NATURAL WOOD FINISHES"
                    ]
                }
            }
        ],
        "aggregations": null,
        "empty": false
    },
    "totalPages": 1,
    "totalElements": 1,
    "size": 20,
    "number": 0,
    "numberOfElements": 1,
    "last": true,
    "first": true,
    "sort": {
        "unsorted": true,
        "sorted": false,
        "empty": true
    },
    "empty": false
}

Answer 3

Actually, you could do the following, with a custom ResultExtractor:

QueryBuilder query = QueryBuilders.matchQuery("name", "tom"); 
SearchQuery searchQuery = new NativeSearchQueryBuilder()
                           .withQuery(query)
                           .withHighlightFields(new Field("name")).build();
return elasticsearchTemplate.query(searchQuery.build(), new CustomResultExtractor());

And then

public class CustomResultExtractor implements ResultsExtractor<List<MyClass>> {

private final DefaultEntityMapper defaultEntityMapper;

public CustomResultExtractor() {
    defaultEntityMapper = new DefaultEntityMapper();
}


@Override
public List<MyClass> extract(SearchResponse response) {
    return StreamSupport.stream(response.getHits().spliterator(), false) 
        .map(this::searchHitToMyClass) 
        .collect(Collectors.toList());
}

private MyClass searchHitToMyClass(SearchHit searchHit) {
    MyElasticSearchObject myObject;
    try {
        myObject = defaultEntityMapper.mapToObject(searchHit.getSourceAsString(), MyElasticSearchObject.class);
    } catch (IOException e) {
        throw new ElasticsearchException("failed to map source [ " + searchHit.getSourceAsString() + "] to class " + MyElasticSearchObject.class.getSimpleName(), e);
    }
    List<String> highlights = searchHit.getHighlightFields().values()
        .stream() 
        .flatMap(highlightField -> Arrays.stream(highlightField.fragments())) 
        .map(Text::string) 
        .collect(Collectors.toList());
    // Or whatever you want to do with the highlights
    return new MyClass(myObject, highlights);
}}

Note that I used a list but you could use any other iterable data structure. Also, you could do something else with the highlights. Here I'm simply listing them.

Answer 4

https://stackoverflow.com/a/37163711/6643675 The first answer does works,but I found some pageable problems with its returned result,which display with the wrong total elements and toalpages.Arter I checkout the DefaultResultMapper implementation, the returned statement shoud be return new AggregatedPageImpl((List<T>) chunk, pageable, totalHits, response.getAggregations(), response.getScrollId(), maxScore);,and then it works with paging.wish i could help you guys~