I would like to create a Row
with multiple arguments without knowing their number. I wrote something like this in Scala:
def customRow(balance: Int,
globalGrade: Int,
indicators: Double*): Row = {
Row(
balance,
globalGrade,
indicators:_*
)
}
On Spark GitHub, the Row
object seems to accept the :_*
notation considering its apply
method:
def apply(values: Any*): Row = new GenericRow(values.toArray)
But at compilation time, this doesn't seem to be allowed:
Error:(212, 19) no ': _*' annotation allowed here
(such annotations are only allowed in arguments to *-parameters)
indicators:_*
What did I miss?
This minimal example may explain better why what you want to do is not allowed:
def f(a: Int, b: Int, c: Int, rest: Int*) = a + b + c + rest.sum
val ns = List(99, 88, 77)
f(11, 22, 33, 44, ns:_*) // Illegal
f(11, 22, 33, ns:_*) // Legal
f(11, 22, ns:_*) // Illegal
Basically, you can use the :_*
syntax only to pass a sequence directly as the vararg parameter rest
, but it's all-or-nothing. The sequence's items are not shared out between simple and vararg parameters, and the vararg parameter cannot gather values from both the simple arguments and the provided sequence.
In your case, you are trying to call Row
as if it had two simple parameters and then a vararg one, but that's not the case. When you create the sequence yourself, you are making it fit correctly into the signature.
Note that in dynamically-typed programming languages, this is typically not an issue. For example, in Python:
>>> def f(a, b, c, *rest):
return a + b + c + sum(rest)
>>> ns = [99, 88, 77]
>>> f(11, 22, 33, 44, *ns)
374
>>> f(11, 22, 33, *ns)
330
>>> f(11, 22, *ns)
297
Resolved it by adding an intermediary Seq :
def customRow(balance: Int,
globalGrade: Int,
indicators: Double*): Row = {
val args = Seq(
balance,
globalGrade
) ++ indicators
Row(
args:_*
)
}
But still, I do not know why it works.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With