Why include SvSETMAGIC() on output variables in an XSUB?

Question

Reading the perlxs documentation, I came to the section on the OUTPUT keyword:

xsubpp emits an automatic SvSETMAGIC() for all parameters in the OUTPUT section of the XSUB, except RETVAL. This is the usually desired behavior, as it takes care of properly invoking 'set' magic on output parameters (needed for hash or array element parameters that must be created if they didn't exist).

I am not sure I understand why is set magic is desired (and why it is not desired for RETVAL)? And why is set magic needed for hash and array element parameters?

Answer 1

All of Perl's data structures support magic, not just SVs (despite the name) and specifically for hashes and arrays this is the basis for things like the tie mechanism or things like fieldhash which implements an analogue of weak references at the level of hash entries.

Since the OUTPUT directive indicates which arguments would presumably be modified by the C body of the XSUB, and a variable containing set magic might be passed in, setting the value as per the typemap without invoking the set handler may result in inconsistent behaviour.

use Scalar::Util qw(weaken);

my $foo;
my $ref = \$foo;
weaken($ref);

As an example of magic, weaken decrements the reference count of $foo, and adds magic pointing back to $ref so that it is cleared it when $foo gets garbage collected.

Additionally, it also adds set magic to $ref, to tear down this back referencing, otherwise when $foo is destroyed, $ref would be cleared even though at this point it's no longer pointing at $foo.

If you use $ref as an argument, it gets aliased on the stack (which is why $_[0] is assignable):

modifies_arguments($ref);

sub modifies_arguments {
    $_[0] = "blah"; # set magic is invoked to tear down the back referencing
} 

If modifies_arguments is a pure Perl it's easy to see why this is appropriate, but the same assumptions about correctness must of course hold for XSUBs, which is why OUTPUT is used to mark which arguments will have their value set to whatever the C level argument variable had at the end of the function body, and have set magic triggered.

This does not apply to RETVAL, since that is not technically an assignment, but rather pushing a new SV onto the stack, and any set magic will be handled after the function returns by the assignment op (if any).

Answer 2

It's quite simple. Whenever you assign to a scalar, you need to call SvSETMAGIC() on it afterwards in case it has magic associated with it.

Assigning to RETVAL doesn't assign to a Perl variable, so calling SvSETMAGIC(RETVAL) (unless you've actually modified RETVAL) would be wrong. If the returned value is assigned to another scalar in the caller, then the assignment will call SvGETMAGIC on the returned value before the assignment, and SvSETMAGIC on the assigned variable after the assignment.