So, while working my way through "Scala for the Impatient" I found myself wondering: Can you use a Scala for loop without a sequence?
For example, there is an exercise in the book that asks you to build a counter object that cannot be incremented past Integer.MAX_VALUE. In order to test my solution, I wrote the following code:
var c = new Counter
for( i <- 0 to Integer.MAX_VALUE ) c.increment()
This throws an error: sequences cannot contain more than Int.MaxValue elements. It seems to me that means that Scala is first allocating and populating a sequence object, with the values 0 through Integer.MaxValue, and then doing a foreach loop on that sequence object.
I realize that I could do this instead:
var c = new Counter
while(c.value < Integer.MAX_VALUE ) c.increment()
But is there any way to do a traditional C-style for loop with the for statement?
In Scala, for-loop allows you to filter some elements from the given collection using one or more if statements in for-loop. Syntax: for(i<- List if condition1; if condition2; if condition3; ...) { // code.. }
If you heard about loops in Scala, try to forget them. Try to do without them. Instead of asking “how can I loop through this”, ask “how can I transform this”. Take this one principle to heart and you'll be ahead of many Scala programmers already.
Looping in spark in always sequential and also not a good idea to use it in code. As per your code, you are using while and reading single record at a time which will not allow spark to run in parallel. Spark code should be design without for and while loop if you have large data set.
For nesting loops, we can place the second loop inside the body of the first loop. This is the traditional way and most used one too. And is done like this, loop1{ loop2{ //code to be executed… } }
In fact, 0 to N
does not actually populate anything with integers from 0
to N
. It instead creates an instance of scala.collection.immutable.Range
, which applies its methods to all the integers generated on the fly.
The error you ran into is only because you have to be able to fit the number of elements (whether they actually exist or not) into the positive part of an Int
in order to maintain the contract for the length
method. 1 to Int.MaxValue
works fine, as does 0 until Int.MaxValue
. And the latter is what your while loop is doing anyway (to
includes the right endpoint, until
omits it).
Anyway, since the Scala for
is a very different (much more generic) creature than the C for
, the short answer is no, you can't do exactly the same thing. But you can probably do what you want with for
(though maybe not as fast as you want, since there is some performance penalty).
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With