How to transform Dataset[(String, Seq[String])] to Dataset[(String, String)]?

Question

Probably this's simple problem, but I begin my adventure with spark.

Problem: I'd like to get following structure (Expected result) in spark. Now I have following structure.

title1, {word11, word12, word13 ...}
title2, {word12, word22, word23 ...}

Data are stored in Dataset[(String, Seq[String])]

Excepted result I would like to get Tuple [word, title]

word11, {title1}
word12, {title1}

What I do
1. Make (title, seq[word1,word2,word,3])

docs.mapPartitions { iter =>
  iter.map {
     case (title, contents) => {
        val textToLemmas: Seq[String] = toText(....)
        (title, textToLemmas)
     }
  }
}

2. I tried use .map to transform my structure to Tuple, but can't do it.
3. I tried to iterate through all the elements, but then I can not return type

Thanks for answer.

Answer 1

This should work:

val result = dataSet.flatMap { case (title, words) => words.map((_, title)) }

Answer 2

Another solution is to call the explode function like this :

import org.apache.spark.sql.functions.explode
dataset.withColumn("_2", explode("_2")).as[(String, String)]

Hope this help you, Best Regrads.