Strange behaviour of eager take

Question

Naturally lazy

D:\>6e "my @bar = 'a', 'b', 'c'; sub foo( @b ) { my $bar = 0; gather loop { print "*"; take ($bar, @b[$bar]); $bar++; last if $bar > 2; } }; .print for foo( @bar )"
*(0 a)*(1 b)*(2 c)

So far, so expected. Now, let's be eager.

D:\>6e "my @bar = 'a', 'b', 'c'; sub foo( @b ) { my $bar = 0; eager gather loop { print "*"; take ($bar, @b[$bar]); $bar++; last if $bar > 2; } }; .print for foo( @bar )"
***(3 a)(3 b)(3 c)

Why on earth are the numeric values all "3"? As if, I don't know, the closure vanished in a puff of illogic. take-rw doesn't cut it, either.

D:\>6e "my @bar = 'a', 'b', 'c'; sub foo( @b ) { my $bar = 0; eager gather loop { print "*"; take-rw ($bar, @b[$bar]); $bar++; last if $bar > 2; } }; .print for foo( @bar )"
***(3 a)(3 b)(3 c)

I can mitigate

D:\>6e "my @bar = 'a', 'b', 'c'; sub foo( @b ) { my $bar = 0; eager gather loop { print "*"; take-rw ($bar + 0, @b[$bar]); $bar++; last if $bar > 2; } }; .print for foo( @bar )"
***(0 a)(1 b)(2 c)

But why do I have to?

Edit: So, questions boils down to,

>6e "my $bar = 0; .say for gather { take ($bar,); $bar++; take ($bar,) }"
(0)
(1)

>6e "my $bar = 0; .say for eager gather { take ($bar,); $bar++; take ($bar,) }"
(1)
(1)

It is still unexplained, why there is a difference in behaviour in eager vs. lazy case.

Answer 1

Holli has accepted the answer that quoted two of Brad's comments. I deliberately kept that one terse.

But you're reading this.¹ So I'll go in the opposite direction with this answer.

($bar,...) is a list containing $bar, not $bar's value

The behavior shown in the question really is all about containers vs values, and how list literals store containers, not their values¹:

my $bar = 0;
my $list = ($bar,);
say $list[0];       # 0
$bar = 1;
say $list[0];       # 1

The laziness vs eager aspect is just things doing what they're all supposed to do. Emphasizing this first point may be enough to inspire you to focus on containers vs values. And maybe that'll lead you to quickly understand what went wrong in Holli's code. But maybe not.¹ So I'll continue.

What happens in the lazy case?

A lazy list waits until a value is demanded before attempting to produce it. Then, it just does the necessary work and pauses, yielding control, until it's asked to produce another value at some later time.

In Holli's code the for loop demands values.

The first time around the for loop, it demands a value from the lazy expression. This turns around and demands a value from the gather'd expression. The latter then computes until the take, by which time it's created a list whose first element is the container $bar. This list is the result of the take.

Then the .print prints that first list. At the moment of printing, $bar still contains 0. (The first increment of $bar hasn't yet happened.)

The second time around the for loop, the inner control structure enclosing the take (the loop) is re-entered. The first thing that happens is that $bar gets incremented for the first time. Then the loop exit condition is checked (and fails), so the second time around the loop starts. Another list is created. Then it's taked.

When the second list is printed, its first element, which is the $bar container, prints as 1, not 0, because at that point, having been incremented, $bar now contains 1.

(If Holli had written code that held onto the first list, and printed that first list again now, after having just printed the second list, they'd have discovered that the first list also now printed with a 1, no longer a 0. Because all the taked lists have the same $bar container as their first element.)

And likewise the third list.

After the third list has been printed, the for loop demands a fourth go at the gather. This re-enters the loop at the statement after the take statement. $bar gets incremented for the third time, to 3, and then the last if $bar > 2; condition triggers, exiting the loop (and thus the expression being gather'd and ultimately the whole .print for ... statement).

What happens in the eager case?

All the gathering is completed before any of the printing.

At the end of this, the for construct has a sequence of three lists. It has not yet called any .print calls. The third time around the loop in the gather has left $bar containing 3.

Next, .print is called on each of the three lists. $bar contains 3 so they all print with 3 as their first element.

Solving the problem by switching to an array

I think the idiomatic way to deal with this would be to switch from a list literal to an array literal:

[$bar, @bar[$bar]]
# instead of
($bar, @bar[$bar])

This works because, unlike a list literal, an array literal treats an element that's a container as an r-value², i.e. it copies the value contained in the container out of that container, rather than storing the container itself.

It just so happens that the value is copied into another new Scalar container. (That's because all elements of new non-native arrays are fresh Scalar containers; this one of the main things that makes an array different from a list.) But the effect in this context is the same as if the value were copied directly into the array because it no longer matters that the value contained in $bar is changing as things proceed.

The upshot is that the first element of the three arrays ends up containing, respectively, 0, 1, and 2, the three values that were contained in $bar at the time each array was instantiated.

Solving the problem by switching to an expression

As Holli noted, writing $bar + 0 also worked.

In fact any expression will do, so long as it isn't just $bar on its own.

Of course, the expression needs to work, and return the right value. I think $bar.self should work and return the right value no matter what value $bar is bound to or assigned.

(Though it does read a little strangely; $bar.self is not $bar itself if $bar is bound to a Scalar container! Indeed, in an even more counter-intuitive twist, even $bar.VAR, which uses .VAR, a method which "Returns the underlying Scalar object, if there is one.", still ends up being treated as an r-value instead!)

Does the doc need updating?

The above is an entirely logical consequence of:

• What Scalars are;

• What list literals do with Scalars;

• What lazy vs eager processing means.

If the doc is weak, it's presumably in its explanation of one of the last two aspects. It looks like it's primarily the list literal aspect.

The doc's Syntax page has a section on various literals, including Array literals, but not list literals. The doc's Lists, sequences, and arrays does have a List literals section (and not one on Arrays) but it doesn't mention what they do with Scalars.

Presumably that warrants attention.

The Lists, sequences, and arrays page also has a Lazy lists section that could perhaps be updated.

Putting the above together it looks like the simplest doc fix might be to update the Lists, sequences, and arrays page.

Footnotes

¹ In my first couple versions of this answer (1, 2, I tried to get Holli to reflect on the impact of containers vs values. But that failed for them and maybe hasn't worked for you either. If you're not familiar with Raku's containers, consider reading:

• Containers, the official doc's "low-level explanation of Raku containers".

• Containers in Perl 6, the third of Elizabeth Mattijsen's series of articles about Raku fundamentals for those familiar with Perl.

² Some of the details in Wikipedia's discussion of "l-values and r-values" don't fit Raku but the general principle is the same.