What's the correct way to deal with multiple inheritence of modules sharing a common "ancestor" in Perl?

Question

(The Moose/Moo answer is, of course, "Roles". This question is about the general case where you want to combine two modules that are both subclasses of the same parent, assuming no Moose/Moo.)

Let's take a slightly contrived example: the modules LWP::UserAgent::Determined and LWP::RobotUA are both subclasses of LWP::UserAgent and extend it in different ways. What should I do if I want to create an object that combines the methods from both? It will, at its core, still be a LWP::UserAgent object, and the other two modules don't clash with each other, so it should be easy, right?

As far as I can tell, the correct thing to do is to create a new package which declares both of the other two as parents – use parent qw(LWP::RobotUA LWP::UserAgent::Determined) – and then create objects from that. And, indeed, if you do that, you get an object that contains the methods from both, as well as from the base class LWP::UserAgent, and almost everything works as you'd expect.

But not quite. Both LWP::UserAgent::Determined and LWP::RobotUA have default values for certain attributes that are set when the object is created if no other value is given. When combining the two, LWP::RobotUA's defaults get set, but not LWP::UserAgent::Determined's. So something must be wrong.

Here's some test code:

#!/usr/bin/env perl

use strict;
use warnings;
use 5.016;

use LWP::RobotUA;
use LWP::UserAgent::Determined;

package MyUA;
use parent qw(LWP::RobotUA LWP::UserAgent::Determined);

for my $module (qw(LWP::RobotUA LWP::UserAgent::Determined MyUA)) {
    say '# ', $module, ' #';
    my $ua = $module->new(
                          'agent' => 'Test-UA',
                          'from'  => '[email protected]',
                         );
    my $req = HTTP::Request->new(GET => 'https://www.bbc.co.uk/emp/network_status.txt');
    my $response = $ua->request($req);
    unless ($module eq 'LWP::UserAgent::Determined') {
        say 'Use sleep? : ', $ua->use_sleep() // 'not defined!';
        say 'Allowed OK? : ', $ua->rules->allowed('https://www.bbc.co.uk/') // 'not defined!';
        say 'Sites with rules: ', (defined $ua->rules()->{loc}) ? join(', ', (sort keys %{$ua->rules()->{loc}})) : 'not defined!';
    }
    unless ($module eq 'LWP::RobotUA') {
        print 'Timings: ';
        if (defined $ua->timing()) {
            say $ua->timing();
        }
        else {
            print 'Timing defaults not set! ';
            $ua->timing('1,5,10,20,60,240');
            say '...but the method works: ', $ua->timing();
        }
        say 'Retry codes: ', (defined $ua->codes_to_determinate()) ? join(', ', (sort keys %{$ua->codes_to_determinate()})) : 'not defined!';
    }
    say '#'x60;
}

This outputs:

    # LWP::RobotUA #
    Use sleep? : 1
    Allowed OK? : 1
    Sites with rules: www.bbc.co.uk:443
    ############################################################
    # LWP::UserAgent::Determined #
    Timings: 1,3,15
    Retry codes: 408, 500, 502, 503, 504
    ############################################################
    # MyUA #
    Use sleep? : 1
    Allowed OK? : 1
    Sites with rules: www.bbc.co.uk:443
    Timings: Timing defaults not set! ...but the method works: 1,5,10,20,60,240
    Retry codes: not defined!
    ############################################################

Here you can see that the methods for both modules work, but default values are not set for LWP::UserAgent::Determined's timing() or codes_to_determinate() methods when combined with LWP::RobotUA, while LWP::RobotUA's use_sleep() method is created with its default value of 1. Setting values manually works fine, however, and otherwise the combined object works as expected.

So, in summary: what's the correct way of handling this case, where you want to combine two modules that subclass a common third? Is this, in fact, correct, but I just chose an unfortunate example and LWP::UserAgent::Determined isn't well-behaved in how it sets it's defaults?

Answer 1

Your new class effectively looks like this:

package MyUA;

use parent qw(LWP::RobotUA LWP::UserAgent::Determined);

1;

Let's test it like this:

#!/usr/bin/perl

use strict;
use warnings;
use feature 'say';

use MyUA;

my $ua = MyUA->new(
  agent => 'test',
  from  => '[email protected]',
);

say ref $ua;

That tells us that we have "MyUA" object. But what is it really? What have we made?

Well, objects are build using a constructor method. That's (usually) called new(). In this case, you haven't defined a new() method in your class. So Perl will look for the method in the superclasses. It does that by searching the classes it finds in @INC and seeing if each one, in turn, contains a new() method.

Both of your superclasses have a new() method. But Perl only needs one. So when it finds one, it stops looking and calls that method. The first one it calls is the one in LWP::RobotUA (because that's the first one on the list passed to use parent). So that one gets called.

This means that what you've actually got here is an object of class LWP::RobotUA. Well, mostly. It's been blessed into the right class and if you call any methods that are in LWP::UserAgent::Determined, but not in LWP::RobotUA, it will still work. But none of the LWP::UserAgent::Determined initialisation code has been called.

And that's a pretty good demonstration of why multiple inheritance is a bad idea. It's so hard to get it right in all but the most trivial of cases.

I can't give you an answer here. Because only you know which bits of the two superclasses you need. But the solution will involve adding your own new() method to your class and probably calling the two superclass constructors from within that.

Update: Ok, I've had a closer look. And it might be easier than I thought.

LWP::RobotUA::new() makes a call to LWP::UserAgent::new() in the middle of doing various other things. But LWP::UserAgent::Determined::new() makes a call to LWP::UserAgent::new() at the start of its processing and then helpfully bundles all of its other initialisation up in a separate method called _determined_init().

So it looks like your solution could be a simple as adding a constructor method like this:

sub new {
  my $class = shift;
  my $self = $class->SUPER::new(@_);
  $self->_determined_init();

  return $self;
}

The call to $class->SUPER::new() calls LWP::RobotUA::new() because that's the first class in @INC. That, in turn, calls LWP::UserAgent::new() - so that initialisation is all done. We then just have to call _determined_init() in order to initialise the other superclass.

It seems to work in my (very basic) testing. But I'm still very dubious about multiple inheritance :-)

Update 2: Yes. ikegami is right. My solution only fixes the problems with constructing the object. I didn't look into actually using it.