Is it possible to have versioned many_many relations?

Question

I already used versioning on DataObjects when they contain a lot of content, now I'm wondering if it's possible to apply versioning to a many_many relation?

Assuming I have the following:

class Page extends SiteTree
{
    private static $many_many = array(
        'Images' => 'Image' 
    );
}

Then the ORM will create a Page_Images table for me to store the relations. In order to have a versioned relation, more tables would be required (eg. Page_Images_Live).

Is there any way to tell the ORM to create versioned relations? When looking at the above example with a Page * – * Images relation, I don't want the Image class to be versioned, but rather the relation. Eg. something like this:

Version Stage:
---
    PageA
        Images ( ImageA, ImageB, ImageC )

Version Live:
---
    PageA
        Images ( ImageA, ImageC, ImageD, ImageE )

Is that even possible out of the box?

Answer 1

I've spent a lot of time looking into this and without fundamentally modifying ManyManyList (as it doesn't expose the necessary hooks through the extension system), there isn't many choices.

I am a dessert-first kind of person, how CAN we do it?

My only suggestion to accomplish this feat is essentially a many-to-many bridge object (ie. a separate entity joining Page and Image) via $has_many though it still requires quite a bit of modification.

This is partially discussed on the forum where a solution about subverting the actual relationship by storing the versioned items against the actual object rather than in a joining table. That would work but I think we can still do better than that.

I am personally leaning towards tying the version of the relationship to the Page itself and my partial solution below covers this. Read below the fold for more info trying this as an update to ManyManyList.

Something like this is a start:

class PageImageVersion extends DataObject
{
    private static $db = array(
        'Version' => 'Int'
    );

    private static $has_one = array(
        'Page' => 'Page',
        'Image' => 'Image'
    );
}

This contains our 2-way relationship plus we have our version number stored. You will want to specify the getCMSFields function to add the right fields required allowing you to relate it to an existing image or upload a new one. I am avoiding covering this as it should be relatively straight forward compared to the actual version handling part.

Now, we have a has_many on Page like so:

private static $has_many = array(
    'Images' => 'PageImageVersion' 
);

In my tests, I also added an extension for Image adding the matching $has_many onto it as well like so:

class ImageExtension extends DataExtension
{
    private static $has_many = array(
        'Pages' => 'PageImageVersion'
    );
}

Honestly, not sure if this is necessary beyond adding the Pages function on the Image side of the relationship. As far as I can see, it won't really matter for this particular usecase.

Unfortunately, because of this way of versioning, we can't use the standard way of calling the Images, we will need to be a bit creative. Something like this:

public function getVersionedImages($Version = null)
{
    if ($Version == null)
    {
        $Version = $this->Version;
    }
    else if ($Version < 0)
    {
        $Version = max($this->Version - $Version, 1);
    }

    return $this->Images()->filter(array('Version' => $Version));
}

When you call getVersionedImages(), it will return all images that have the Version set on it aligning with the version of the current page. Also supports getting previous versions via getVersionedImages(-1) for the last version or even gets images for a specific version of the page by passing any position number.

OK, so far so good. We now need to make sure that every page write we are getting a duplicate list of images for this new version of the page.

With an onAfterWrite function on Page, we can do this:

public function onAfterWrite()
{
    $lastVersionImages = $this->getVersionedImages(-1);
    foreach ($lastVersionImages as $image)
    {
        $duplicate = $image->duplicate(false);
        $duplicate->Version = $this->Version;
        $duplicate->write();
    }
}

For those playing at home, this is where things get a bit iffy relating to how restoring previous versions of Page would affect this.

Because we would be editing this in GridField, we will need to do a few things. First is make sure our code can handle the Add New function.

My idea is an onAfterWrite on the PageImageVersion object:

public function onAfterWrite()
{
    //Make sure the version is actually saved
    if ($this->Version == 0)
    {
        $this->Version = $this->Page()->Version;
        $this->write();
    }
}

To get your versioned items displaying in GridField, you would have it set up similar to this:

$gridFieldConfig = GridFieldConfig_RecordEditor::create();
$gridField = new GridField("Images", "Images", $this->getVersionedImages(), $gridFieldConfig);
$fields->addFieldToTab("Root.Images", $gridField);

You might want to link to images directly from the GridField via GridFieldConfig_RelationEditor however this is when things get sour.

Time for the veggies...

One of the big difficulties is GridField, for both linking and unlinking these entities. Using the standard GridFieldDeleteAction will directly update the relationship without the right version.

You will need to extend GridFieldDeleteAction and override the handleAction to write your Page object (to trigger another version), duplicate every version of our versioned image object for the last version while making it skip the one you don't want in the new version.

I'll admit, this last bit is just guesswork by me. From my understanding and debugging, it should work but simply there is a lot of fiddling to get it right.

Your extension of GridFieldDeleteAction then needs to be added to your specific GridField.

This would essentially be your last step away from making this solution work. Once you have the adding, removing, duplicating, version updating part down, it really is a matter of just using getVersionedImages() to get the right images.

Conclusion

Avoid. I get why you want to do this but I really don't see a clean way of being able to handle this without a decent sized update to how many_many relationships are handled in Silverstripe.

---

But I really want it as a ManyManyList!

The changes I see required for ManyManyList are having a 3-way key (Foreign Key, Local Key, Version Key) and the various methods for adding/removing/fetching etc updated.

If there were hooks in the add and remove functions, you might be able to sneak in the functionality as an extension (via Silverstripe's extension system) and add the needed data to the extra fields that many_many relationships allow.

While I could get this happening by extending ManyManyList directly and then forcing ManyManyList to be replaced with my custom class via Object::useCustomClass, it would be even more of a messy solution.

It is simply too long/complex for me to give a full answer for a pure ManyManyList solution at this stage (though I may get back to this later and give it a shot).

---

Disclaimer: I am not a Silverstripe Core dev, there may be a neater solution to this entire thing but I simply can't see how.

Answer 2

You can define second relation with "_Live" suffix and update it when the page is published. Note: This solution stores only two versions (live and stage).

Bellow is my implementation which automatically detects whether many-many relation is versioned or not. It then handles publishing and data retrieval. All what is needed is to define one extra many-many relation with "_Live" suffix.

$page->Images() returns items according to the current stage (stage/live).

class Page extends SiteTree
{
    private static $many_many = array(
        'Images' => 'Image',
        'Images_Live' => 'Image'
    );

    public function publish($fromStage, $toStage, $createNewVersion = false)
    {
        if ($toStage == 'Live')
        {
            $this->publishManyToManyComponents();
        }

        parent::publish($fromStage, $toStage, $createNewVersion);
    }

    protected function publishManyToManyComponents()
    {
        foreach (static::getVersionedManyManyComponentNames() as $component_name)
        {
            $this->publishManyToManyComponent($component_name);
        }
    }

    protected function publishManyToManyComponent($component_name)
    {
        $stage = $this->getManyManyComponents($component_name);
        $live = $this->getManyManyComponents("{$component_name}_Live");

        $live_table = $live->getJoinTable();
        $live_fk = $live->getForeignKey();
        $live_lk = $live->getLocalKey();

        $stage_table = $stage->getJoinTable();
        $stage_fk = $live->getForeignKey();
        $stage_lk = $live->getLocalKey();

        // update or add items from stage to live
        foreach ($stage as $item)
        {
            $live->add($item, $stage->getExtraData(null, $item->ID));
        }

        // delete remaining items from live table
        DB::query("DELETE l FROM $live_table AS l LEFT JOIN $stage_table AS s ON l.$live_fk = s.$stage_fk AND l.$live_lk = s.$stage_lk WHERE s.ID IS NULL");

        // update new items IDs in live table (IDs are incremental so the new records can only have higher IDs than items in ID => should not cause duplicate IDs)
        DB::query("UPDATE $live_table AS l INNER JOIN $stage_table AS s ON l.$live_fk = s.$stage_fk AND l.$live_lk = s.$stage_lk SET l.ID = s.ID WHERE l.ID != s.ID;");
    }

    public function manyManyComponent($component_name)
    {
        if (Versioned::current_stage() == 'Live' && static::isVersionedManyManyComponent($component_name))
        {
            return parent::manyManyComponent("{$component_name}_Live");
        }
        else
        {
            return parent::manyManyComponent($component_name);
        }
    }

    protected static function isVersionedManyManyComponent($component_name)
    {
        $many_many_components = (array) Config::inst()->get(static::class, 'many_many', Config::INHERITED);
        return isset($many_many_components[$component_name]) && isset($many_many_components["{$component_name}_Live"]);
    }

    protected static function getVersionedManyManyComponentNames()
    {
        $many_many_components = (array) Config::inst()->get(static::class, 'many_many', Config::INHERITED);

        foreach ($many_many_components as $component_name => $dummy)
        {
            $is_live = 0;

            $stage_component_name = preg_replace('/_Live$/', '', $component_name, -1, $is_live);

            if ($is_live > 0 && isset($many_many_components[$stage_component_name]))
            {
                yield $stage_component_name;
            }
        }
    }
}