A module loader with simple dependency management

| 4 Comments

I've written another version require2.js of my CommonJS module loader require() function. This one has two interesting features.

First, you can "pre-load" modules by mapping the module filename to the module function in the require._module_function object. If you do this, then the module will not need to be loaded. For example:

require._module_functions['math.js'] = function(require,exports,module) {
     // Code for the math module goes here
};

Second, this new version of require() has a require._print hook, which, if set to a suitable function, will print out the text of all modules it loads, wrapped in a function and assigned to the require._module_function map as above. You can even define a require._minimize hook if you want to do code minimization on your modules.

I've defined another script display_requirements.js that defines suitable _print and _minimize functions.

So, here's the upshot. For relatively simple applications that load modules statically at start up, use the require2.js script for loading modules. Its inefficient, but works well during the development phase. Then, when you're getting ready to deploy your application, load the display_requirements.js script after loading require2.js but before you actually call require() anywhere. This will cause a big chunk of code to appear at the bottom of your web page--pre-loaded minimized versions of all the modules you used. Cut-and-paste this code into a new file named requirements.js (or even paste it at the bottom of require2.js) and load the requirements.js script in place of the display_requirements.js script. Now you can continue to use require() as you have always done, but it won't have to hit the network to load your modules.

I haven't done much deployment of real-world web applications, and am not qualified to say whether a system like this would actually be helpful in practice. But it was an easy tweak to my existing code, so there it is.

4 Comments

This is exactly the direction I'm looking to take. Narwhal makes the preloaders look like:

require.register({: {
"factory": function () {

},
"depends": […]
})

These can be concatenated, or included all in one call to require.register for bundling purposes. The dependency list declaration makes it possible to implement require.async(id).when(pass, fail, progress) on the client side and do chatty async loading when necessary (lazy loading).

You'll notice that I no longer explicate require, exports, module as the arguments of the factory. The new convention is for arguments[0] to be an object of free variables for use in the module. This leaves open the possibility of constructing systems that have additional free variables, like a QUnit DSL, or build scripts, and safeguards the possibility of dependency injection eventually. This would looks like:

require.loader.load(id)({
require: require,
exports: exports,
module: module,
a: someCapabilityToInject
})

This is the nature of my proposal for module transport format.

Hi, David. I apologize for posting this question here, but I couldn't find an email address for you.

The 4th edition of your Javascript guide was published in 2002. The 5th edition was published in 2006. With 2010 nearly upon us and Javascript more important than ever on the web, are there plans for a 6th edition in the very near future?

It's good progress, but still I fear largely an academic exercise since this loader includes two fairly significant issues:

1. It still uses sync XHR. Not only is that bad practice, but it's also limited to Same Origin Policy restrictions.

2. Caching and returning the exported API to future require calls for the same module risks cross sandbox pollution since (if I understand this correctly) this script affords a global require function for all contributors to a page to use. If collaborating groups each need their own sandbox, then calling require for a known module would not re-execute the module function, but return the same API object. If group A modifies the API, group B's sandbox has been modified as well.

#1 is IMO a structural flaw of var api = require(module_id); On the one hand, sync allows subsequent code a relative guarantee of immediate API functionality, but is enabled by a known antipattern. On the other hand, async require enabled by callback/promise (e.g. when(...)) gets messy when there are multiple require calls.

#2 would be easily worked around by simply caching the module function rather than the returned API and executing that function and returning the exported API in response to each require call. Except this of course falls down when a module is required by multiple other modules within a single sandbox. The crux here is knowledge of the context of the require--was this require called from a context that already has that module? This opens up a different can of worms.

In YUI 3, by comparison, step 1 is establishment of the sandbox, and the require function is a method of that sandbox object (named "use"). use is patterned like a promise system, taking a callback function as the last argument, but also accepts n module ids so the management of multiple single require(..) expressions needn't be managed in a setTimeout driven mechanism. Missing modules are requested via script node insertion (async and xdomain) and the sandbox (YUI instance) is used as the shared exports space. The loaded module functions are cached, and are executed once per sandbox (if requested by that sandbox), so no cross contamination can take place.

Perhaps I'm too close to the forest to see the trees here, though.

The ability to register modules to avoid a resource fetch is a good call.

I don't think the print functionality is well placed hung off require, though. I can see the point of doing so as you have access to the module source from the XHR responseText. It's a best practice to combine files, and any build step that facilitates that is good.

There are considerations with manually generated rollups, though. The obvious caveat is maintainability and necessity to reroll the bundle if any constituent files change. Otherwise, including a rollup js reduces the http overhead (yay) and associated delays, but still incurs IO and parse time delays due to the blocking nature of the script node in markup. A better choice would be to have boilerplate functionality for async script loading, pointing to the rollup file.

(function (d) {
var script = d.createElement('script'),
ref = d.getElementsByTagName('script')[0];

script.src = '/path/to/rollup.js';
script.onload = fulfillRequirePromise;

ref.parentNode.insertBefore(script, ref);
})(document);

This is a browser env issue, and likely overkill for the majority case, but I thought it worth sharing :)

Leave a comment

Books

Comprehensive coverage of Ruby 1.8 and 1.9

"The New Most Important Ruby Book"
Peter Cooper,
rubyinside.com

Completely updated for Ajax and Web 2.0

"A must-have reference"
Brendan Eich,
creator of JavaScript

The classic Java quick-reference

Advertising

Pages

Hosted By

Powered by Movable Type 4.21-en