Proposal of an extension to jQuery core for async loading and module definition.

Proposal of an extension to jQuery core for async loading and module definition.

Hello friendly jQuery people!

Please excuse this rather long mail. I'm writing on it and the docs for the better part of this day. (And excuse the word "mail" and the formating in the following. I thought there was a mailing list to post it to. Now I'm feeling a little old school while pasting it in here ;)

Quicktip: There is documentation and code at https://github.com/ScheintodX/jqinit.js if you which to skip my writing and just have a peek.


I'm the author of jqinit.js and would like to propose an extension to jQuery which I think would fit very well.

My proposal would extend the scope of jQuery for an important part: Loading libraries and putting them together.

I know that there are solutions out there which are in wide use. I have looked at many of them and they didn't see them fit at least my needs.

Some of the limitations I saw:
 * Doesn't support async
 * Need to put js code in every page
 * Way to big
 * Does mostly things I don't need


jQuery is all about power in simplicity. So I intended this "Loader" to be.

This "Loader" I propose might be called at first sight "just another loader.js" but indeed it is the opposite.

Html5 gave us these nice tiny extension to the <script> tag called "defer" and "async". But you can't use it to load jQuery because every plugin to jQuery needs jQuery to be there in the first place. So no way to load jQuery *after* it's modules, isn't it?

Imagine:
 * Being able to load jQuery and dependent modules async in arbitrary order.
 * Having a well defined format for your own modules which can be loaded async
 * Being able to load modules when they are needed.
 * Don't have to care about order of initialization at all.
 * Don't have to put scripts in every html page (s. ready.js)
 * Using google's hosting for the jQuery core libs
 * Have a simple mechanism of dependency injection as a bonus.
 * Have a standard module format which encourages best practice and supports minification.
 * Having all modules accessible under one namespace
 * Doing all this in about 550 bytes (and that can still be optimized)
 * And the best: Have the "Loader" itself being loaded asynchronously.

So why did I put the word "Loader" in ampersands? Because what it does is rather the opposite of loading something. Indeed it lets all the loading do the browser which has very nifty methods of doing so. (Lookahead scanners and so on.)

What it does is simply making sure that everything which has been loaded is initialized and executed in the correct order. And all of this in just a few lines of javascript. So lets call that thing: "jQInit" for now.

So how does it look like?
-------------------------------------------------------------------------------

Simple: Put your stuff in the html header like you are used to, but add async:

    <script src="//google/jQuery" async></script>
    <script src="/js/jqinit.js" async></script>
    <script src="/js/module1.js async></script>
    <script src="/js/module2.js async></script>

Looks simple. But just to make a point, you can have written it this way, too:

    <script src="/js/module1.js async></script>
    <script src="/js/module2.js async></script>
    <script src="/js/jqinit.js" async></script>
    <script src="//google/jQuery" async></script>

or any other order you wanted. (Think about it. This is the very nature of async loading ...)

So how can a js module do this without being loaded itself? Sure there is a gotcha ... and well ... you have to do "something" of cause.

But rest assured this something are just a few bytes. So lets look at the module format:

You're of cause familiar with the simple module pattern:

    var MyModule = (function($){

        // do stuff here
        return {
            // put public stuff there
        }

    })(jQuery);

So let's extend this with dependencies and while where at it inject them as well:

    var MyModule = [ 'MyModule', 'MyOtherModule', function( MyOtherModule, $ ){

        // follow pattern

    } ];


So this defines a module with the name of 'MyModule' and an dependency on 'MyOtherModule' (and of cause jQuery, this is what this is all about).

But there is no execution of this code right there, isn't it? We'll get there in a moment. What we first need is "something" to store that stuff into. So lets create an array for that, but only if its not already there:

    var jQInit = jQInit || [];

and put our module in:

    jQInit.push( [ 'MyModule', 'MyOtherModule', function( MyOtherModule, $ ){

        // follow pattern

    } ] );

*And that's it.* Yes right. There is nothing more you have to do in a module. This one can be loaded asynchronously.


So what is missing now? The magic of cause. This comes in when we look at what jqinit.js does:

For starters jQInit itself follows normal module pattern (of cause it doesn't need to register itself):

jQInit.js:

    var jQInit = (function( _jQInit ) {

        // in case we are executed first
        _jQInit = _jQInit || [];

        // more stuff which is needed to work

        // do the magic:
        return {
            push: function( module ) {

                // Store in array
                _jQInit.push( module );

                initializeModulesIfDependenciesAreMet();
            }
        }

    } )( jQInit );

So it essentially swaps the array we used in the modules (remember: if it doesn't existed we simply created one) with its own structure and simulates the only array operation we need. So if this is executed before some of our modules are loaded the syntax to register them stays the same but now we can initialize them right away.

So why does this NEED to go into jQuery?
-------------------------------------------------------------------------------

 * First of all: It would make things simpler. Don't care about loading order. Just put the stuff in there.

 * It would make jQuery "complete": jQuery is all about simplicity. It simplified greatly the way how we manipulate DOM. But whats left out is a good way of putting together an web application. This is where this technique would step in. In an as easy way as it could get.

 * It would enhance the "experience". Everyone would like to use "async" but most don't know how and it's not trivial. (s. e.g. stackoverflow on that matter) Just follow (easy) pattern and don't care about loading order. And get module dependencies for free.

 * It would simplify jQInit. In order to work jQInit needs the modules to be in a certain form. Currently it treats jQuery special. So the best thing I came up how this could work is polling for "jQuery" to be defined. Off cause jQuery knows when it is loaded and could do the intilization right away. Further since I can't know if jQuery is loaded before jQInit I can't use it in most parts. Having the "Loader" in jQuery itself one could use jQuerys methods for some stuff which would reduce the codesize even more.

 * It would greatly enhance how jQuery plugins are written. Just by following semantics they would integrate seamlessly into this system. You could use hosted libraries and still get async loading. Your app would look like:

    <script src="//hosting//jQueryUI.js" async></script>
    <script src="//hosting//jQuery.js" async></script>
    <script src="/MyApp.js" async></script>

and thats all. No fiddling about order any more!

 * It's small. In it's current form jqinit is about 550 bytes minified but uncompressed. This still is unoptimized for size and will shrink even further if integrated in jQuery. I guess this can be reduced to about 400-450 bytes minified on top of jQuery. I argue that this won't be noticed as prolonged loading time even in slow networks. But it's effect on async loading will.

 * It won't interfere with old scripts or if you don't want to use it. (This is just a thesis right now. But at least it is something I would have it to.)

How would this look integrated in jQuery:
-------------------------------------------------------------------------------

Integrated in jQuery itself the usage could look like:

MyModule.js:

    var $ = $ || [];

    $.push( [ 'MyModule', 'MyOtherModule', function( MyOtherModule, $ ){

        // do my stuff

        // perhaps return something

    } ] );

which would only add (some neglectable amount of) bytes to one module.


State of the project:

I put together a standalone Version of jQInit and I'm using it in some of my projects. This is brand new so none of this is live. But it is working pretty well by now. You can find it on github: https://github.com/ScheintodX/jqinit.js with some more documentation. (You will find the docs familiar to this email. This is because I started writing this mail and then realized that having documentations wouldn't hurt so I ported large parts over before I continued writing here.)

I'm sure someone brighter and with more js/jQuery experience than me will come up with ways how to simplify things / make them nicer / smaller / easier / prettier / faster. I welcome all contributions to make things better.

Limitations / Possible extensions:
-------------------------------------------------------------------------------

* Currently this doesn't work for jQuery Modules like Mobile or UI. This is primary of cause because they don't follow the module syntax like it would be needed.
With some hackery it is still possible but with some limitations. I wrote a loader which uses jQuery to load them. This loader is available as jqinit.loader.js and only a few bytes long. The main drawback is that the modules are loaded after jQuery is done loading. So no async here. Of cause this situation would improve greatly if the would be integrated in jQuery and the modules would follow module semantics.

* I intentionally limited the scope of jQInit to the minimal set one would need to get things done. But I can imagine that it could provide helpful to have some means of determining the order in which document-ready stuff is executed. This could honor dependencies. On the other hand as far as I understand it $(document).ready() maintains order anyway and since execution order follows dependencies this is already what one wants.

* An addition to this something I intend to look into further is a way of passing content loaded via ajax to the modules so that they can do initialization on it, too.

* Some jQuery plugins (e.g. mobile) must be configured before they are loaded/executed. There should be a standard way for this kind of stuff. Currently one can use jqinit.loader.js to achieve this. But not async (see above).

* One other possible extension could be a simple way of loading modules while the application is already running. With the this mechanism it is already as simple as $('head').append( '<script src="/js/somescript.js"></script>' ) but it would certainly be nicer to have a function for this like: $.load( "/js/somescript.js" ); or so. But this is as easily implemented in a plugin. s. jqinit.loader.js

* Something I haven't even begun to think about is "dependency-loading". So there is currently no external definition of dependencies on a per file-basis. I like it this way because it keeps things simple but one could think about having some module-definition like: { ModA: "/js/moda.js", ModB: ... } and have jQInit load them on demand according to dependencies. This certainly makes only sense for modules which aren't loaded anyway. But it could provide useful for large apps. But I think this could be added easily as plugin if needed.

* In large apps it could be usefull to have unloading functionality. But I don't know. I don't do the large stuff.


If you can imagine putting this in jQuery I could give it a look how to integrate it in the best way and see if i can provide an patch. (This will take me some time because currently I'm clueless how jQuery's build process works.)

So sorry again for this long post but I'm rather excited ;)


Please let me know what you think!

Best regards,

Florian
~