Does perl optimize foreach over a range?

Question

Given a simple program:

use strict;
use warnings;

my $count = 0;

foreach ('a' .. 'zzzzzzzzzz') {
    $count++;
}

print "$count\n";

Will perl create entire range of about 1.4+e14 elements in memory and then iterate over them or does it have any internal optimizations to simply step over one state of range operator at a time?

Well, I've actually ran that, so empirical data from top shows it is the latter (and it didn't finish yet to see data from time as well), but is it documented anywhere or where to find confirmation for that in sources?

Answer 1

From perldoc perlop:

In the current implementation, no temporary array is created when the range operator is used as the expression in foreach loops, but older versions of Perl might burn a lot of memory when you write something like [ what you wrote ]

Answer 2

Yes, the following is optimized into a counting loop:

 for (EXPRX..EXPRY) { ... }

perlop mentions this.

In the current implementation, no temporary array is created when the range operator is used as the expression in foreach loops, but [versions of Perl older than 5.6] might burn a lot of memory when you write something like this:

for (1 .. 1_000_000) {
    # code
}

You can see this using the following:

$ perl -e'
   my $n = 10_000_000;
   system("ps h -o rss $$");
   for (1..$n) { system("ps h -o rss $$"); last; }
'
1876
1944

Even within the loop, there's only 1944 KiB used. This is the same amount of that was allocated before the loop, and nowhere near enough to hold 10,000,000 scalars. On the other hand, the following slightly different code shows 385 MiB being used in the loop:

$ perl -e'
   my $n = 10_000_000;
   system("ps h -o rss $$");
   for ((), 1..$n) { system("ps h -o rss $$"); last; }
'
 1876
394724

In this latter version, all 10 million scalars are allocated.

In fact, it's not just EXPR..EXPR that's special. All of the following are implemented differently:

• for (EXPR; EXPR; EXPR) ("C-style for loop", an augmented while loop.)
  Functionally different.
• for (EXPRX..EXPRY) (A range and nothing else.)
  Incrementing counting loop.
• for (reverse CONSTX..CONSTY) (A constant range, preceded by reverse.)
  Descending counting loop over indexes of array built at compile-time.^[1]
• for (reverse EXPRX..EXPRY) (A variable range, preceded by reverse.)
  Descending counting loop.
• for (@ARRAY) (An array and nothing else.)
  Counting loop over indexes of array.
• for (reverse @ARRAY) (Reverse of an array and nothing else.)
  Descending counting loop over indexes of array.
• for (reverse LIST) (Any list that doesn't fit the above patterns.)
  Flattened list iterated from end to start.
• for (LIST) (Any list that doesn't fit the above patterns.)
  Flattened list iterated from start to end.

---

1. Everywhere but in for (CONSTX..CONSTY),

   CONSTX..CONSTY
   

   is optimized to

   my @anon;
   BEGIN { @anon = CONSTX..CONSTY; }
   @anon
   

   You can see this here:

   $ perl -e'
      BEGIN { system("ps h -o rss $$"); }
      system("ps h -o rss $$");
      exit(0);
      for (reverse 1..10_000_000) {  }
   '
    1868
   709540