Create "Engine" to allow integrations to main web application?

Question

Background:

I currently have a web application based on MVC Kohana PHP framework, that allows users to sell ebooks to their customers.

The code behind the webapp is all wired together and everything but API-centric. It is running pure MVC and then using mustache for template system.

What I would like to do:

I would like to integrate various accounting services (from bigger nordic providers like e-conomic.com) but also own integrations that will let users to optimize their selling and so on.

What I would like to archieve is to make something, call it an engine, that allows functionality integrate (flexibely) into parts of the webapplication, whether its in the view-part or controller/logic.

Based on the background and looking at the technical point of view, which ways are there to do this?

My thoughts is that I need some kind of placeholders all over in different areas of the webapplication. Then I need these placeholders to work together with a "engine" that then checks integrations wants to "run" in these areas?

Would that even work? What would you do?

---

Update trying to make it more clear:

So what I would like to accomplish is to have separate functionality that would integrate into the existing main webapplication.

Let's just say I have a folder called integrations/ and in here there is two different integrations that affect different parts of the system.

The first is a Kashflow (accounting software) integration, that grabs some data from our system and send to Kashflow (API way, fine) but also inside my webapp under "orders" states whether it has synced to Kashflow yet or not. (this is the part the question is about)

Another integration could be a "Featured Ebook" integration. This simply lets you pick what product should be featured and then on the ebook store, the featured product will be highlighted with a orange border around it and some bigger text. (this is the part the question is about)

How are the bold marked working? A webshop provider like Shopify has Apps which does this, and all other SaaS with Apps have this technical solution.

I wonder is it? How can I allow separate functionality affect a base webapp?

I hope it got more clear now.

---

New update:

What I look for answer is an answer based on the above stated background, how I can implement a solution that would allow this from where I am now.

A good answer would be one that also in text / pseudo way could give a description on how one of the example plugin/integrations i mentioned could be implemented.

So how does the integration communicate with the main application, what does the main application have in order to accept/allow functionality.

Answer 1

Let me start from the very beginning.

What you are looking for is called a service layer which should be implemented in your applcaition. What it does is

Defines an application's boundary with a layer of services that establishes a set of available operations and coordinates the application's response in each operation.

Enterprise applications typically require different kinds of interfaces to the data they store and the logic they implement: data loaders, user interfaces, integration gateways, and others. Despite their different purposes, these interfaces often need common interactions with the application to access and manipulate its data and invoke its business logic. The interactions may be complex, involv-ing transactions across multiple resources and the coordination of several responses to an action. Encoding the logic of the interactions separately in each interface causes a lot of duplication.

A Service Layer defines an application's boundary [Cockburn PloP] and its set of available operations from the perspective of interfacing client layers. It encapsulates the application's business logic, controlling transactions and coor-dinating responses in the implementation of its operations.

Let me explain it simple terms so you can understand. What you 1st have to do is define a service layer in your application. Since you are going with MVC, this could be another controllers handling all the requests related to this specific task. You can have separate controllers for each couple of operations. At the end your engine will be these set of controllers.

If you are willing to go to next level you can handle all these integration via an ESB(Enterprise Service Bus).

An enterprise service bus (ESB) is a software architecture model used for designing and implementing communication between mutually interacting software applications in a service-oriented architecture (SOA). As a software architectural model for distributed computing it is a specialty variant of the more general client server model and promotes agility and flexibility with regard to communication between applications. Its primary use is in enterprise application integration (EAI) of heterogeneous and complex landscapes.

If you need more information let me know.

Update

There are well documented blog posts. Please see the links below.

• How essential is it to make a service layer?
• Service Layer Guidelines

Answer 2

Based on your update, I think you describe a good case for a web application that needs to become modular. You want to be able to easily add new modules (plugins) that give you different functionalities without having to change the application core each time.

Below is a possible solution to your challenge from conceptual point of view. My intention is to help you grasp on the idea and get you started. Keep in mind that it can be simplified further or become much more complex, depending on your needs.

---

The theoretical side of things

1. Plugins/Modules

Each plugin will enable a set of specific features and must be able to work independently from other plugins that are enabled at the moment. All plugins will have to follow a common set of rules and conventions in order to be recognised by your application. This will simplify future maintenance and extension immensely. For example, each plugin should:

• Have its own subdirectory under the Plugins/Modules folder that follows a predefined structure (e.g. Backend/Portlets/InstallScripts, etc.)

• Use separate storage sandbox in your database, dedicated only to this plugin. Take Kashflow – all tables that are used by the plugin can start with a ksflw_ prefix.

• Bring its own partial Frontend view presentation(s) (along with underlying controller logic and model) that implement specific sets of functionality (for example, display pre-selected books in orange border)

• Bring its own partial Backend view presentation(s) (along with underlying controller and model) that handle in the site backend (in the case of Kashflow you have portlet visualization that maybe renders a button to manually do synchronization, enables you to schedule one and displays the datetime of the last synchronization made)

• Have an installer script, that creates tables, inserts menu items, and initialises hook subscriptions (see next bullet)

• Initialize Hooks subscriptions – All subscribed Plugin functions get called whenever a registered event occurs somewhere in the system.

---

2. Core functionality changes

You will need new functionality in your existing application in order to start supporting plugins.

• Plugin manager – GUI that allows you to install, remove, enable/disable plugins and allow access to them for your clients.

• Partial views manager – allows users to select which partial views of which plugins get displayed in what existing placeholders (this will work in conjunction with hooks)

• Placeholders for partial views on pages in the places you want to enable your users to display plugin UI and information

• Hooks throughout the application – Whenever "interesting" event happens, the system checks if any plugins are currently subscribed to this event and calls/notifies them, then displays the result. Some examples of events that deserve Hooks might be:

  • Placeholder rendering – this will trigger all subscribed functionalities to display a frontend/backend partial view

  • Specific business events – e.g. Whenever new book is being added to the catalogue or is being sold

  • Administration menu rendering – On this event each installed plugin will select all menu items in the PLUGINNAME_AdminPluginMenu table (the plugin should have created this table at install time) and return all them to the hook for displaying.

I'm sure you'll think of other relevant events, as you know your case best of all.

---

The practical side of things (based on the second update of the question)

1. Leveraging HMVC for visualisation of partial views (widgets) inside of existing views

As I already stated earlier, Kohana supports HMVC or Hierarchical Model View Controller pattern. This means that you can have a hierarchy of controllers like so (already described in the following question):

Now, this enables you to easily call controllers from other controllers and even directly from your views! And it works wonders when you need to embed widgets.

You can make a slight modification to boostrap.ini in order to enable Routes like widget_controller/controller_action/action_parameter (this is described in detail in the tutorial I'm giving you below). Then you can have the following code inside your main view template where you want to render your orange book box:

<div class="widget_sidebar">
     <?php echo Request::factory('widget_orangebook/display/3')->execute(); ?>
</div>

When executed, this acts as a hook and will invoke the widget_orangebook controller's action_display method with parameter 3 - e.g. you want to show 3 books.

The controller's action will look something like this:

public function action_display ($number_of_books){...}

As a result inside the <div>, you will see the content of the template set by the widget_orangebook controller after execution of the action.

In a sense it gives the illusion of an AJAX partial rendering, but its being executed on the server without the extra call. It is pretty powerful, and I think this is the way to go for the cases you described.

You can see this tutorial to see a detailed description on all the modifications you need to do. It's a bit fancier - it's about rendering multiple widgets in a widget section, but it follows the same logic.

Note that if you insist on using mustache and logicless templates, you can also do this kind of Request call in the controller, set the result to a variable and then pass that variable to your mustache template.

2. Kohana Modules

Kohana supports modules, that allow you to pack up your plugins in an organized way. As you implement more complex plugins this will become important. You can see more on Kohana Modules here.