Does iterating over a hash reference require implicitly copying it in perl?

Question

Lets say I have a large hash and I want to iterate over the contents of it contents. The standard idiom would be something like this:

while(($key, $value) = each(%{$hash_ref})){
   ///do something
}

However, if I understand my perl correctly this is actually doing two things. First the

%{$hash_ref}

is translating the ref into list context. Thus returning something like

(key1, value1, key2, value2, key3, value3 etc)

which will be stored in my stacks memory. Then the each method will run, eating the first two values in memory (key1 & value1) and returning them to my while loop to process.

If my understanding of this is right that means that I have effectively copied my entire hash into my stacks memory only to iterate over the new copy, which could be expensive for a large hash, due to the expense of iterating over the array twice, but also due to potential cache hits if both hashes can't be held in memory at once. It seems pretty inefficient. I'm wondering if this is what really happens, or if I'm either misunderstanding the actual behavior or the compiler optimizes away the inefficiency for me?

Follow up questions, assuming I am correct about the standard behavior.

1. Is there a syntax to avoid copying of the hash by iterating over it values in the original hash? If not for a hash is there one for the simpler array?

2. Does this mean that in the above example I could get inconsistent values between the copy of my hash and my actual hash if I modify the hash_ref content within my loop; resulting in $value having a different value then $hash_ref->($key)?

Answer 1

No, the syntax you quote does not create a copy.

This expression:

%{$hash_ref}

is exactly equivalent to:

%$hash_ref

and assuming the $hash_ref scalar variable does indeed contain a reference to a hash, then adding the % on the front is simply 'dereferencing' the reference - i.e. it resolves to a value that represents the underlying hash (the thing that $hash_ref was pointing to).

If you look at the documentation for the each function, you'll see that it expects a hash as an argument. Putting the % on the front is how you provide a hash when what you have is a hashref.

If you wrote your own subroutine and passed a hash to it like this:

my_sub(%$hash_ref);

then on some level you could say that the hash had been 'copied', since inside the subroutine the special @_ array would contain a list of all the key/value pairs from the hash. However even in that case, the elements of @_ are actually aliases for the keys and values. You'd only actually get a copy if you did something like: my @args = @_.

Perl's builtin each function is declared with the prototype '+' which effectively coerces a hash (or array) argument into a reference to the underlying data structure.

As an aside, starting with version 5.14, the each function can also take a reference to a hash. So instead of:

($key, $value) = each(%{$hash_ref})

You can simply say:

($key, $value) = each($hash_ref)

Answer 2

No copy is created by each (though you do copy the returned values into $key and $value through assignment). The hash itself is passed to each.

each is a little special. It supports the following syntaxes:

each HASH
each ARRAY

As you can see, it doesn't accept an arbitrary expression. (That would be each EXPR or each LIST). The reason for that is to allow each(%foo) to pass the hash %foo itself to each rather than evaluating it in list context. each can do that because it's an operator, and operators can have their own parsing rules. However, you can do something similar with the \% prototype.

use Data::Dumper;

sub f     { print(Dumper(@_)); }
sub g(\%) { print(Dumper(@_)); }   # Similar to each

my %h = (a=>1, b=>2);
f(%h);  # Evaluates %h in list context.
print("\n");
g(%h);  # Passes a reference to %h.

Output:

$VAR1 = 'a';           # 4 args, the keys and values of the hash
$VAR2 = 1;
$VAR3 = 'b';
$VAR4 = 2;

$VAR1 = {              # 1 arg, a reference to the hash
          'a' => 1,
          'b' => 2
        };

%{$h_ref} is the same as %h, so all of the above applies to %{$h_ref} too.

---

Note that the hash isn't copied even if it is flattened. The keys are "copied", but the values are returned directly.

use Data::Dumper;
my %h = (abc=>"def", ghi=>"jkl");
print(Dumper(\%h));
$_ = uc($_) for %h;
print(Dumper(\%h));

Output:

$VAR1 = {
          'abc' => 'def',
          'ghi' => 'jkl'
        };
$VAR1 = {
          'abc' => 'DEF',
          'ghi' => 'JKL'
        };

You can read more about this here.