Raku rebless doesn't work with inherited classes anymore

Question

The code given in this thread doesn't work anymore: How can I rebless an object in Perl 6?

I wrote this piece of code last year, and it worked then. Now it doesn't:

class Person { ; }
class Woman is Person { ; }
my $tom = Person.new;
my $lisa = Woman.new;

say $tom.^name;  # -> Person
say $lisa.^name; # -> Woman

Metamodel::Primitives.rebless($tom, Woman);
# -> New type Woman for Person is not a mixin type

The error message doesn't make sense, as it is supposed to work with inherited classes. At least it was.

The documentation is not helpful; https://docs.raku.org/routine/rebless

Answer 1

it is supposed to work with inherited classes

It never was supposed to be that general. I designed that API and implemented it in the first place, and it was only ever intended as an implementation detail of mixins.

Until very recently, it was not part of the language specification test suite - and when it did become part of it, it already had its current, more restrictive, semantics. The constraints on it are important for performance reasons: when we know a type is not one that can be the target of a mixin operation, we can JIT-compile attribute accesses on that object into something much simpler (we paid an extra conditional move on every attribute access before the change, and now only have to pay it on mixin target types).

It's possible to modify the original program to work by using the MOP to construct the class. In fact, the following isn't quite the original program; I did a small tweak for the sake of showing how one can provide methods in the subclass as an anonymous role, so as to avoid too much MOP boilerplate.

class Person { method m() { "person" } }
constant Woman = do {
    my \w = Metamodel::ClassHOW.new_type(:is_mixin, :name<Woman>);
    w.^add_parent(Person);
    w.^add_role(role { method m() { "woman" } });
    w.^compose()
}
my $tom = Person.new;
my $lisa = Woman.new;

say $tom.^name;  # -> Person
say $lisa.^name; # -> Woman

say $tom.m; # person
Metamodel::Primitives.rebless($tom, Woman);
say $tom.m; # woman

While that's the most semantically direct fix to the original program, there is a shorter way: use the but operator on the Person type object to produce a mixin type and return it, and then just tweak its name to your liking:

class Person { method m() { "person" } }
constant Woman = Person but role { method m() { "woman" } }
BEGIN Woman.^set_name('Woman');

my $tom = Person.new;
my $lisa = Woman.new;

say $tom.^name;  # -> Person
say $lisa.^name; # -> Woman

say $tom.m;
Metamodel::Primitives.rebless($tom, Woman);
say $tom.m;

Which is only one line extra than the original anyway.

Answer 2

See jnthn's answer for authoritative discussion about precisely what happened to rebless and what to do about it.

it worked ... Now it doesn't .. The error message doesn't make sense ... it is supposed to work with inherited classes ... At least it was ... The documentation is not helpful

This (ultra long!) answer may be worth a read for those interested in further discussion of the principles and practice of the TDD approach that underlies work on the Raku programming language and related artifacts such as the Rakudo compiler and docs.raku.org content.

This answer is structured as specific responses to particular parts of Arne's original question and of comments they wrote in response to an earlier version of this answer. My intent was to make it more useful to Arne while, hopefully, still being useful to others.

Arne: The code given in this thread doesn't work anymore: How can I rebless an object in Raku?

I've updated the accepted answer to that SO to link to this SO.

Arne: I wrote this piece of code last year, and it worked then. Now it doesn't

The relevant change was discussed in an April 2019 commit in which jnthn wrote:

Recently, types that were the target of a rebless operation started needing to be created explicitly as mixin target types, to assist optimization. ...

In a comment 11 days ago closing rakudo GH issue "Rebless to a custom type no longer seems to work", he wrote:

You'll need to arrange to have the is_mixin named argument passed to ClassHOW.new_type ... There's no way to do that with the class syntax, thus the target type of the rebless shall have to be assembled using the MOP also.

(Click above link for notes on how to do what it suggests.)

This issue is also discussed a little further in the it worked ... it suddenly didn't ... the documentation ... should document the call section below.

Arne: it is supposed to work with inherited classes. At least it was.

roast -- the repository of all spec tests -- determines what Raku code is supposed to do. (The st of roast can be read as supposed tos.)

In another April 2019 message jnthn wrote:

There was no previous spec for Metamodel::Primitives.rebless. I've added this spectest so that now there is. This means there's now some definition of what can be expected to work.

The fact that Rakudo's behavior is spec'd by an executable test suite is a fundamental part of @Larry's approach to ensuring Raku behaves reliably^[1] and has profound implications^[2].

The impact of this change on a widely used module

Here's a snapshot of the impact of this change unfolding for the popular Inline::Perl5 module.

In April 2019, niner opened a rakudo GH issue on the impact on Inline::Perl5 and I've extracted some highlights of the exchange between niner and jnthn below.

(I've elided some stuff that was important in the original context, but distracting in the context of this SO. Please don't assume you have a complete understanding of the original conversation from this extract. If in doubt, click the link.)

niner: TBH what I do here has probably always been kinda fishy ... Could even be that ... I can get rid of [it] ... Would be nice though to keep already deployed Inline::Perl5 versions up and running.

jnthn: There was no previous spec for Metamodel::Primitives.rebless. I've added [a] spectest so that now there is. This means there's now some definition of what can be expected to work, and which Inline::Perl5 can rely on.

Since unknown named parameters are ignored, but :mixin was not required on previous Rakudo versions, then it would be possible to make a new Inline::Perl5 release that can work on previous Rakudo versions as well as the upcoming one, so there can at least be back-compat.

I don't think there's any way of keeping things working for existing Inline::Perl5 versions ...

niner: Unfortunately passing :mixin doesn't help in this case as the rebless is done on a subclass of the one created via Metamodel::Primitives.create_type. The subclass uses the normal Perl6::ClassHOW.

I'm working on a major refactor to get rid of the rebless hack in the first place. I'm reopening this issue so the release manager is aware of there being no working Inline::Perl5 on rakudo's release candidate.

jnthn: Do you create that class using the MOP? You can pass :is_mixin to Perl6::ClassHOW.new_type if so.

niner: No, it's for this situation: class Bar is Foo { }

Helping with the docs

In a comment below this answer you've written:

I can help with the documentation part

That sounds to me like a very appropriate and useful response to the issue at the heart of your SOQ. I hope that we're fortunate enough that this comes to pass.

if that helps

Imo your technical writing is excellent so I would hope that the end result of you working with others involved in improving it would be a wonderful thing.

Fundamental constraints on docs.raku.org's content

A large part of the reason I wrote the rest of this very extensive answer to such a seemingly simple question, and reinstated it after initially deleting it once Jonathan had answered it, was to discuss the principles and practice of the TDD approach that underlies work on the Raku programming language and related artifacts such as the Rakudo compiler and docs.raku.org content.

Aiui, the desirable relationship between how things are supposed to work in Raku, and how they actually work in Rakudo, and how things are supposed to be documented on docs.raku.org boils down to:

• Everything MUST be presumed to be forever subject to the fundamental nature of a volunteer project; and, within that constraint:

• Behavior in roast SHOULD be documented and other behavior SHOULD NOT.

(Given available volunteer time, interest, and consensus, exceptions are occasionally made for documenting behavior of a properly QA'd Rakudo that's not covered by roast. In current practice this seems to mean behavior of a Rakudo version in a released Rakudo Star.)

Useless documentation

The documentation is not helpful

I considered this a fair comment. All things considered, the documentation as it was when you wrote your question was not helpful.

the documentation was useless [in 2018]

This is a very different statement.

There was no roast entry covering rebless at that time.

If the docs.raku.org page on rebless had described its behavior as it was in 2018, then that would have been worse than useless because it would incorrectly suggest that the then current behavior was supported. In reality there was scope for it to break in a future version of Rakudo without a reasonable prospect the 2018 behavior would be reinstated by core devs. And indeed this came to pass: its unsupported behavior from 2018 did break, and was not reinstated.

So, given the consensus on what belongs in docs.raku.org and what doesn't (see above), the most helpful thing its rebless page could do was to either not document rebless at all or, perhaps better, include a page for it but make sure it did not describe its behavior. Which is what the situation was: the page did exist; was not directly helpful; and that was arguably better than nothing.

(It's easy to imagine things being better yet. For example, what if pages documenting functions included a percentage documenting the state of test coverage associated with that function in the version of Rakudo in the latest Rakudo Star? A 0% could immediately clue a reader in to an awareness that that function wasn't covered by roast. That said, while this doc feature is easy to imagine, who is going to implement it? It's equally easy to imagine that it could take a calendar year or more of diligent work and collaboration to usefully implement and deploy, and that folk think other things are more important.)

it worked ... it suddenly didn't ... the documentation ... should document the call

it worked

It was "luck" it worked.

it suddenly didn't work anymore

Because Rakudo was improved.

the documentation ... should document the call

As explained earlier, aiui the current community consensus and/or working practice is: the documentation SHOULD document a particular version of the call, namely the roast'd behavior for the version of Rakudo in the latest Rakudo Star; and MAY document behavior in other versions.

and not refer to something else

Aiui, the current consensus and/or working practice is that what some might consider "weak" doc contributions, eg some brief, hurriedly written content and/or links outside the docs, MAY be introduced if volunteers feel an immediate change is warranted to reflect some concern raised by a user (eg this SO) and that making the "weak" change would be better than doing nothing at all. You can of course do a PR to improve it (or to revert it if you really feel that a change is so "weak" it makes matters worse).

the reference to changes in 2019.11 is 7 months off by my count

(It's something like that by my count too, though I've seen a compiler claiming to be 2019.03.1 with the same break in behavior.^[3])

I think JJ made the doc change and he just misinterpreted jnthn's comment about how to adapt to the change. I currently think it's better than nothing but look forward to you updating it. :)

Footnotes

^[1] The following was said a few minutes after Larry had first announced the project that led to Raku in his 2000 "State of the Onion" speech:

Question: Will [Raku] have specs?

Larry: what we particularly want to stress ... is not perhaps so much the [language design] spec as developing our current regression test ... into a validation test of what the language actually means and actually go out and explore all the nooks and crannies and say, “This is [Raku], this is not [Raku],” and then we actually have a machine-readable spec. And to me that’s actually a lot more important than what the verbiage in the human readable thing says.

^[2] Of course, roast only works out well for a given user if its tests sufficiently cover the user's needs. Arne's problem demonstrates how holes in coverage can be surprising. For discussion of these holes as they stood in 2018, see On Specs, Versioning, Changes, and… Breakage. The good news is that roast is just lots of unit tests written in Raku to test that expressions or constructs with particular values do a particular thing. So it's easy for individuals or corporations to contribute new tests to improve test coverage. And it's all under version control (git), so custom downstream tags, branches and forks are viable, sustainable, and manageable. (Indeed, that's how new language versions (Christmas, Diwali, Eid(?), etc.) are managed.)

^[3] I've seen an attempt to rebless a new class created using regular newclass is oldclass syntax both work (on my laptop) and not work (on repl.it) using compilers that claim to be 2019.03.1. (Presumbly repl.it installed a version of the compiler source code, or a binary compiled from it, taken from the master head shortly after the compiler's version was updated to 2019.03.1, with the breaking change in place. I note that repl.it haven't publicized their online raku repl -- I discovered it by accident -- so there's nothing untoward about this situation but it reinforced for me the need for the $RAKU.compiler.verbose-config method used in the worked/broken outputs I just linked.)