Plug-in architecture II: JavaScript Event-Driven plug-in framework

Tags: ,

This is the second article of a three-part series, read the previous and next article: Plugin architecture (I): Preliminary notes and C++ Event-Driven Plug-in Framework. (Plug-in architecture III).

This second article on plug-in architecture describes a pure JavaScript plug-in framework and give some examples of use. The final objective will be to design and build an integral client-server plug-in framework.

In respect of client side, the proposed architecture solves the problem of integrating code that extends our application made by third party developers. In our case each user should be able to import, export and share these extensions easily as well as using his own set of extensions and configurations. The suggested structure allows to import, update, configure and remove these extensions without disturbing the user experience.

What I have sometimes seen in projects I worked is that modules provided to the user are already integrated in the application and there is a configuration for each user stored in the database. This configuration tells the application which modules the user will use and how, but it does not allow the user to modify a module or to use a better module programmed by an external company. To suit the needs of the user we needed to touch the application code. This architecture proposes a solution to this problem.

This plug-in framework is user oriented, close to Mozilla or Chrome extensions idea. We are talking about using plug-ins for punctual things, to extend a functionality not to build an entire application based only on extensions or to have a big inter-plug-in dependency even if it remains possible.

Lightweight and independence of the library was a requirement, minified the library weights less than 8 KB and it does not require other libraries to work as it is written in pure JavaScript.

Maybe this architecture won't suit your needs, or there are better frameworks out there. What I present here is what suited my needs, and it is made in a minimalist way without pretension. Don't hesitate to criticize, improve or modify it.

The point of this article is not to document the libraries, but to talk about application architecture in a more general way as well as to identify and give solutions to practical cases. Anyway I recommend you read the library documentation, the following text will make references to it, you can find code and examples here.

This work was inspired by other existent plug-in architectures I described in the preliminary notes, as well as Addy Osmani Patterns For Large-Scale JavaScript Application Architecture article and Learning JavaScript Design Patterns book by the same author.

Contents

Resources

  • You can see a demo that illustrates this document here.
  • The code of the demo can be downloaded here.
  • The Plug-in Handler is included in underdog.js.
  • You can read the Plug-in Handler documentation here.

Plug-in framework

The Plug-in framework is composed of two elements: Plug-ins and a Plug-in Handler that manages the life-cycle of plug-ins during the application runtime.

The plug-in

Plug-ins are JavaScript objects with a defined structure that extend an application. In the proposed architecture they can be added from two different sources, from the application or loaded from the server, we call these last ones packaged plug-ins, their objective is to be portable and customizable. Plug-ins can also extend other plug-ins. Not all of them may be executed, some can just have useful functions or wait to be extended by others. They can communicate.

This is what a plug-in looks like:

{}

Yes, it's a simple one, it won't do anything, it will be assigned a unique id but it won't be executed as it does not have a public “run” method. And as it does not have a known id it won't be able to be extended.

Let's have a look at another one:

(function(){
  ////
  // private members
  ////
  var articleContentContainer = null;

  /**
   * gets value from input and finds
   * words that match this value and higlights them.
   */
  function highlightWord(e){
    //(...)
  };

  ////
  // public methods
  ////
  return {
    "id" : "wordhighlight.word-highlight",
    "events" : ["load-text-after"],
    "run" : function(params,configuration,eventName,success,failure){
      //(...)
    }
  };
}())

This plug-in framework is even-driven [11]. The “events” property is used to specify on which events we want the plug-in to be executed. This one will be executed when "load-text-after" event is fired, executed means its run() method will be called.

NOTE in the example above we are using function scope to simulate private members. If we didn't have private members, our plug-in will be equal to the object returned by the function. If you want you can read more about how to implement private and protected members in javascript here [8].

One more:

{
  "run" : function(params,configuration,eventName,success,failure){
    //(...)
    if (params[“x”]){
      params[“x”] = 1;
    }
    if (success){
      success({"x":x});
    }
  }
}

This one will be executed when added because it does not have an “events” property and it has a run method.

NOTE In this example run method calls the success callback function passing "x" value, plug-ins can be used as data transformers.

A plug-in may not have a run method, in this case it will be executed only if it extends a plug-in with a run method.


Extensions
Let's imagine plug-ins are all loaded at the initialization of the application. It will be easy to manage extensiveness. But in our architecture plug-ins can be lazy loaded, in that case they are only loaded and added when one of their respective events is fired, so what happens if a plug-in that extends another one is loaded before the one that is extended? What happens if a plug-in that has to be extended is loaded before its extension?
We are not forcing anyone to be loaded here.
Isn't it a weak point?

What do we mean by extension?
A plug-in can extend more than one plug-in. The plug-in that extends will integrate the public members of the extended plug-in that haven't been overridden. Once a plug-in is extended it remains inactive, other plug-ins can continue to extend it but it will not be executed again, we are supposing its extensions will do the job.

Example of plugin2 extending plugin1:

// The extended: plugin1
PH.add(function(){
    var x = 1;
    return {
        "id" : "plugin1.plugin1",
        "events" : ["event-name-1"],
        "run" : function(params,configuration){
            console.log("run of plug-in 1");
            console.log("x => ", sumToX(3));
            this.sayHello();
        },
        "sumToX" : function(a){
            x += a;
            return x;
        },
        "sayHello" : function(){
            console.log("Hello world!");
            console.log("scope => ", this);
            console.log("x => ", x);
        }
    };
}());

// The extension: plugin2
PH.add(function(){
    var x = 1;
    return {
        "id" : "plugin2.plugin2",
        "extends" : ["plugin1.plugin1"],
        "events" : ["event-name-2"],
        "run" : function(params,configuration){
            console.log("run of plug-in 2");
            console.log("x => ", sumToX(3)); // be careful here
            this.sayHello();
        },
        "multiplyByX" : function(a){
            x = x*a;
            return x;
        }
    };
}());

The second plug-in extends the first one, so the first one will never be executed again, the second will take the members that are not overridden (sumToX and sayHello).

Once the extension is done, the second plug-in will look like this:

PH.add(function(){
    var x = 1;
    return {
        "id" : "plugin2.plugin2",
        "extends" : "plugin1.plugin1"
        "events" : ["event-name-2"],
        "run" : function(params,configuration){
            console.log("run of plug-in 2");
            console.log("x => ", sumToX(3)); // be careful here
            this.sayHello();
        },
	   "multiplyByX" : function(a){
            x = x*a;
            return x;
        },
        "sumToX" : plug-in1.sumToX,
        "sayHello" : plug-in1.sayHello
    };
}());

NOTE We have to be careful with private members when extending, in this example sayHello points to the first plug-in sayHello function, same case for sumToX. When this functions will be executed by the run of plugin2, the scope of the function will be plugin2 but the “x” private variable used will be the one from plugin1.
This matters if we want to reuse x in another function, or if another plug-in extends plugin1.
To prevent this we should have used x as a public member:

"sumToX" : function(a){
            this.x += a;
            return this.x;
        },

When writing a plug-in, we should think if private members won't be a problem in case other plug-ins will want to extend it.
Also we should think if the extended plug-in needs to be loaded before its extension, for example in the case where its extension calls a function that is in the extended one (Example above, run of plugin2 calls sayHello and sumToX).
Anyway, I will recommend to use extension the less possible. It has been implemented more in the spirit of overriding functions than in the spirit of inheritance.

Allowing extension to work without the object it extends has its advantages. It weakens plug-in dependency, making the architecture a lot more flexible. Even if extension has not its extended it can still be used. And until its extension is added the extended is fully functional. We can extend up to a point at a certain time.

We will talk about server loaded plug-ins later. Let's talk a bit about the engine first. How are plug-ins managed?


The Plug-in Handler

The plug-in framework central piece is the Plug-in Handler (PH), a static class that manages the life-cycle of plug-ins during the application runtime, it loads, adds, executes and removes plug-ins. PH handles events and manages plug-in execution failures. It also manages the inter-plug-in communication and their communication with other entities as well as their extensiveness.

We can say the Plug-in Handler is a mediator:
"The mediator pattern is best introduced with a simple analogy - think of your typical airport traffic control. The tower handles what planes can take off and land because all communications are done from the planes to the control tower, rather than from plane-to-plane. A centralized controller is key to the success of this system and that's really what a mediator is.

Mediators are used when the communication between modules may be complex, but is still well defined. If it appears a system may have too many relationships between modules in your code, it may be time to have a central point of control, which is where the pattern fits in.

In real-world terms, a mediator encapsulates how disparate modules interact with each other by acting as an intermediary. The pattern also promotes loose coupling by preventing objects from referring to each other explicitly - in our system, this helps to solve our module inter-dependency issues."
Addy Osmani - Patterns For Large-Scale JavaScript Application Architecture [1,2]


Loading and adding plug-ins
PH can load packaged plug-ins from the server. They can be loaded from different given URLs (I recommend limiting the URLs to one, or two in case we want to separate the all-user plug-ins and the user plug-ins.). Plug-ins can be eager or lazy loaded. PH can also add plug-ins on-the-fly from the application.

Loading and adding plug-ins
Figure 1. Plug-in Handler loading and adding plug-ins steps.

What Plug-in Handler do is load a loader.json from each provided URL, this JSON tells the Plug-in Handler what plug-ins to load and how.

To avoid requesting loaders each time we load a page, in the proposed framework, loader.json files are cached as well as the plug-ins' configuration files using session storage for this purpose. So the second time our PH is initialize it already has these files.

If you want to know how this is implemented in our system, have a look at the documentation.

Once plug-ins are loaded and added, for quicker acces, PH has two private members: "events" and "plugins". "events" contains the names of the events and the ids of the plug-ins listening to that events; "plugins" contains the ids of the plugins that point to the plugins.

var events = {
  "init-ph-after" : ["indexandprint.add-index"],
  "load-text-after":["wordhighlight.word-highlight","betterworddefinition.word-tooltip","worddefinition.word-tooltip"]
};

var plugins = {
  "indexandprint.add-index" : pluginA,
  "wordhighlight.word-highlight" : pluginB,
  "betterworddefinition.word-tooltip" : pluginC,
  "worddefinition.word-tooltip" : pluginD
};


Event Handling and Plug-in execution: An event driven plug-in framework.
PH handles plug-in events. An event can be fired by the application by a plug-in or by any entity, when this happens PH checks if there is a plug-in that is “listening” on this event, if it is the case, PH execute the plug-in asynchronously.

Event-driven javascript plug-in framework
Figure 2. Event Handling.

NOTE JavaScript native event system is not used to not interfere. But we could imagine developing a layer that will fire plug-in events on JavaScript native events.

Events can be seen a little like eclipse extension points [9]. If there are not events fired, no plug-ins are executed, so we need to think about firing events in our application.

PH also fires events, for example when it finishes initializing, when a plug-in is removed, when a plug-in is executed.

PH native events:

Event name Description
plugin-execution-failure Fired after PH is initialized.
plugin-configuration-load-after Fired when plug-in execution has failed.
plugin-add-after Fired after a plug-in configuration has been loaded.
plugin-remove-before Fired after a plug-in is added to the Plugin Handler.
plugin-remove-after Fired before/after a plugin is removed.
x-before
x-in-process
x-after
When a plug-in is executed tree events are fired, before the run, during and after id of the plug-in + -before/-in-process/-after.

This way plug-ins life-cycle process can be extended.

Simplified implementation of the Plug-in Handler based on the described pattern:

var PH = (function(){

  var uid = -1;

  var events = {};

  var plugins = {};

  return {
    add : function(plugin){

      var pluginEvents = plugin["events"];

      if(pluginEvents){
        // add an id if plug-in does not have one.
        if (typeof plugin["id"] == "undefined"){
  				plugin["id"] = ++uid;
  			}

        plugins[plugin["id"]] = plugin;

        var len = pluginEvents.length;
        while (len--){
          var eventName = pluginEvents[len];
          if ( typeof eventName == "string" && typeof events[eventName] == "undefined" ){
            events[eventName] = [];
          }
          events[eventName].push(plugin.id);
        }

        // fire event after a plug-in is added.
        PH.fire("plugin-add-after",{"plugin":plugin});
      }
    },
    getPluginById : function(id){
			if(plugins[id]){
				return plugins[id];
			}
			return null;
		},
    fire : function(eventName,params,success,failure){
      if (events[eventName]){
        var pluginIds = events[eventName];
        var len = pluginIds.length;
        while (len--){
          var plugin = PH.getPluginById(pluginIds[len]);
          try{
            plugin.run(eventName,params,success,failure);
          } catch (err){
            // error running plug-in.
            if (failure){
              failure(params);
            }
          }
        }
      }
    }
  };
}());
Adding plug-ins and firing events:
// This plug-in is executed everytime a plug-in is added.
PH.add({
  "events" : ["plugin-add-after"],
  "run" : function(eventName,params,success,failure){
    console.log("plugin added => ", params["plugin"]);
  }
});

// This plug-in will be executed when "event-1" is fired.
PH.add({
  "events" : ["event-1"],
  "run" : function(eventName,params,success,failure){
    console.log("plugin run => ", params);
    if (success){
      success({"data":{"value":512}});
    }
  }
});

// Fire "event-1" passing some parameters to the plug-in
// and a callback function that can be called by the plug-in when executed.
PH.fire("event-1", {"data":{"param1":"hello world!"}, function(data){
  var data = data["data"];
  if (data["value"]){
    console.log("print value in fire success callback =>", data["value"]);
  };
});
The output is:
plugin added =>  Object {events: Array[1], id: 0} // run of plug-in executed on "plugin-add-after" event.
plugin added =>  Object {events: Array[1], id: 1} // run of plug-in executed on "plugin-add-after" event.
plugin run =>  Object {param1: "hello world!"}    // run of plug-in executed on "event-1" event
print value in fire success callback => 512       // fire event success callback called in the run method of plug-in executed on "event-1" event.


At this point I recommend reading about Publish–subscribe pattern [11], its advantages and disadvantages and an implementation of Publish/Subscribe pattern in javascript [12].


Managing plug-in execution failure
A plug-in will be completely removed by the PH when its execution fails. After that it won't be able to be executed nor extended again.
As seen in the table above there is an event that is fired when an execution fails. We can imaging developing a plug-in that will run in case other fail telling the server something went wrong so we can have an error log. In our model if a plug-in has implemented an “onFailure” method, this one will be called before plug-in is removed, in case we want to execute a last action.


Plug-in communication
PH handles the inter-plug-in communication, a plug-in can send messages to itself, to another plug-in or to all plug-ins. For this it will use sendMessage() for sending messages and onMessage() as listener function for recieving messages. NOTE this is very similar to events, theoretically plug-ins can also communicate by firing events, but I think it is a good idea to separate execution from communication.


Decorating plug-ins

“Classically, Decorators offered the ability to add behavior to existing classes in a system dynamically.“ Addy Osmani - Learning JavaScript Design Patterns [2]

PH dynamically adds some functions to the plug-ins like remove, removeEvents, getResourcePath etc. , a plug-in can of course override these functions and the user is given the opportunity to add new decorators that will be added to all plug-ins.


Packaged plug-ins

At this point we already know a bit about them. They contain the client side and the server side of a plug-in. The goal is to have a system that allows users to share plug-ins, to use them in different applications and to include them in an application without having to touch a single line of code. Packaged plug-ins allow this.
Packaged plug-ins have all the same file structure, they have a manifest.json file that describes them, a directory where the code is, and can have resources like images, css, json files etc.
You can have a look at the file structure and creation of packaged plug-ins in the documentation.

We have to think carefully when making a packaged plug-in, as said the purpose is to share it so other users can import them and use them in different applications. For example, maybe it won't be a good idea to use external libraries as JQuery in a plug-in, other applications or users may not want to use JQuery. Also we have to think that other users may want to extend the plug-in and care when using private variables. We have to be careful not to harm the application or other plug-ins before sharing a plug-in.

In the provided demo, you can see how four packaged plug-ins are loaded and live.


Plug-in internationalization

How do we translate texts used in our plug-ins? I suggest using plug-in configuration JSON (that are loaded at the same time as the plug-in code) or plug-in resources to store translations in a unique json or in one json per language. First solution is better if there is little amount of translation data because there won't be any extra requests to the server, but in exchange we will be retrieving unused translations. Second solution is better if the translation data is huge. I recommend reading chrome.i18n and RequireJS - Define an I18N Bundle [6,7]


Plug-ins as data transformers

In this framework plug-in run() functions can call a success or failure callback passing values, that values will be the parameter in the callbacks of the event fire. Communication messages can also be seen as a channel to send and receive transformed data.


Plug-ins as library interfaces

In JavaScript we have a lot of libraries for DOM manipulation, making AJAX requests etc. It could be interesting to use plug-ins to interface with libraries, this way it could be easier to migrate to another library without impacting our application code that will continue to interface with the same plug-in functions. Of course this may have an impact in performance.


Standardization of ids, events and data

The proposed architecture has the advantages and disadvantages of Loose coupling [13], it has the advantage of being very flexible, the entities firing events and the plug-ins don't have the obligation of knowing each other.
This is also a disadvantage, the implementation of such a structure could soon become a huge mess. Changing the id of a plug-in, its events or its data structure can have big consequences in other dependent plug-ins. Apart of limiting the dependency of plug-ins it is a good practice to standardize the naming of the ids, events and the data structure.


Proposed id naming
For packaged plugins:

package.plugin-name
Application plug-ins:
app.plugin-name


Proposed event naming

event-name-1
Event naming is more important than id naming. Giving to much specificity to a name event can be a problem for plug-in re-use. In my opinion events should be generic enough, this could be a problem if two different entities fire the same event with different meaning, but it has also the advantage that one plug-in can be used when two different sources fire the same event with the same meaning. For example if an event is named “text-load-after”, it can be fired in more than one situation, the event name does not specify what text or where the text is loaded, this information will be specified in the parameters, with this method one plug-in can be used for many different text loads.


Communication data standardization
When firing events and when receiving data from callbacks we transfer data to the plug-ins and back to the firing method callbacks. Somehow we need to structure flowing data in an uniformed way. Changing this structure can provoke errors in dependent plug-ins and in the actions taken after an event is fired.

The native Plug-in Handler events parameters have this form:

{
  “data”: {},
  [“plugin”: plugin,]
  [“eventName” : “event-name”]
}

Messages passed in the inter-plug-in communication will remain less strict as the messages are sent in a same entity or functional context.


Performance

The use of a mediator may cause problems in performance as plug-ins do not communicate directly. Also in case of using packaged plug-ins the number of requests to the server may increase. What I did is to use session storage for reducing the number of requests by caching the loader file and the plugin configuration files. Another thing that can be done is to group files in the server side to make less requests. In the client-side, we should manage the plug-in load in an intelligent way and do not abuse in the usage of plug-ins, they are not useful in all situations. I do not recommend to extend plug-ins a lot, they shouldn't be treated as classes.


Security considerations

Somehow there should be a control of quality of shared plug-ins because of their possibility of propagating quickly. They could damage the application execution, or worse damage or compromise user data. You can have same security problems as in Chrome or Mozilla extensions. I recommend reading Chrome Content Security Policy (CSP) [10] and searching for information about “chrome extensions security”.


References

In C++ Event-Driven Plug-in Framework. (Plug-in architecture III) I talk about the server side plug-in framework.