Scala - Expanding an argument list in a pattern matching expression

Question

I'm very new to Scala and trying to use it as an interface to Spark. I'm running into a problem making a generic CSV to DataFrame function. For example, I've got a CSV with about 50 fields, the first of which are task, name, and id. I can get the following to work:

val reader = new CSVReader(new StringReader(txt))

reader.readAll().map(_ match {
  case Array(task, name, id, _*) => Row(task, name, id)
  case unexpectedArrayForm =>
    throw new RuntimeException("Record did not have correct number of fields: "+ unexpectedArrayForm.mkString(","))
})

However, I'd rather not have to hard code the number of fields needed to create a spark Row. I tried this:

val reader = new CSVReader(new StringReader(txt))

reader.readAll().map(_ match {
  case Array(args @ _*) => Row(args)
  case unexpectedArrayForm =>
    throw new RuntimeException("Record did not have correct number of fields: "+ unexpectedArrayForm.mkString(","))
})

But it just creates a Row object with a single element. How can I make it expand the args in Row(args) so that if I have an array of N elements I'll get a Row with N elements?

Answer 1

Change your input to be variable length by adding _*:

Row(args:_*)

This is what Row accepts per its apply signature.

In fact, you don't even need to do anything other than pass this in to the Row as it is already of the right sequence type.

reader.readAll().map(Row(_:_*))