November 2009 Archives

I've written another version require2.js of my CommonJS module loader require() function. This one has two interesting features.

First, you can "pre-load" modules by mapping the module filename to the module function in the require._module_function object. If you do this, then the module will not need to be loaded. For example:

require._module_functions['math.js'] = function(require,exports,module) {
     // Code for the math module goes here
};

Second, this new version of require() has a require._print hook, which, if set to a suitable function, will print out the text of all modules it loads, wrapped in a function and assigned to the require._module_function map as above. You can even define a require._minimize hook if you want to do code minimization on your modules.

I've defined another script display_requirements.js that defines suitable _print and _minimize functions.

So, here's the upshot. For relatively simple applications that load modules statically at start up, use the require2.js script for loading modules. Its inefficient, but works well during the development phase. Then, when you're getting ready to deploy your application, load the display_requirements.js script after loading require2.js but before you actually call require() anywhere. This will cause a big chunk of code to appear at the bottom of your web page--pre-loaded minimized versions of all the modules you used. Cut-and-paste this code into a new file named requirements.js (or even paste it at the bottom of require2.js) and load the requirements.js script in place of the display_requirements.js script. Now you can continue to use require() as you have always done, but it won't have to hit the network to load your modules.

I haven't done much deployment of real-world web applications, and am not qualified to say whether a system like this would actually be helpful in practice. But it was an easy tweak to my existing code, so there it is.

I've implemented the CommonJS Modules 1.0 specification with the code in this file. It appears to pass the compliance tests when run in Firefox and Chrome on Linux, and also when run standalone in Tracemonkey, Rhino or V8.

Note that this implementation does not use the namespace probing technique I described in my previous post.

Update: One of the things that really surprised me when testing my implementation was to discover that the CommonJS spec requires (this is not explicit in the specification text, but it is explicit in the conformance tests) the require() function to return the actual exports object of a module, and not make a defensive copy of it.

Suppose a program includes module A which includes modules B and C. Module C can require B and then add, replace, or remove methods from B's API. Later, when the program includes Module B directly, it will get the modified version of B. In order to correctly use this modified module B, the programmer will have to read the documentation for module C!

Update 2: There are great comments to this post, including a link to Luke Smith's blog post that argues that the synchronous nature of CommonJS modules is not a good fit for client-side scripting. In response I wanted to make clear that I posted this code because I thought it was interesting, not because I think that client-side programmers should go out and start using it right away.

It has become common in modern JavaScript programming to use functions as namespaces. If you put your code inside a function, then your variables and functions are local to the containing function and do not clutter up the global scope.

var value = (function() {  // Wrapper function creates a local scope or namespace
    // your code goes here
    return value;  // Export a value from the namespace
)());  // Invoke the wrapper function to run your code      

Now suppose that you have some JavaScript code as a string--you've just loaded it using XMLHttpRequest, for example. You're going to evaluate the code, and you might want to evaluate it in a namespace so that it doesn't define functions and variables in the global scope. This is easy: just wrap it in a function before evaluating it. In this case, the Function() constructor is even more handy than the eval() function:

var code = ....;  // A string of JS code to evaluate
var f = new Function(code);   // Wrap it in a function
f();    // And run the function

The problem with doing this is that the function creates a sealed namespace and we can't see what is inside. If the code defines something useful like a function or a class, we can't access it, and it does us no good.

Here's a trick I've just discovered. (I'm sure someone else has thought of this, but I haven't seen it used or described elsewhere). Before you wrap your code in a function add this line to it:

return function(s) { return eval(s); };

Now, when you invoke the wrapper function, it returns this evaluator function to you. The returned function evaluates a string in the scope of the namespace, so you can use it to peek into the namespace and extract whatever values you want!

If your string of code defines a constructor function named Set() that you want to use, you can run the code in a namespace and then extract f from the namespace like this:

var code = readFile("Set.js");  // A string of JS code to evaluate
// Define and invoke a wrapper function with special suffix code.
// The return value is a namespace evaluator function and we treat
// it as a namespace object.
var setns = new Function(code + "return function(s) { return eval(s); };")(); 
var Set = setns("Set");  // Import the Set function from the namespace.
var s = new Set();  // Use the class we just imported

And what if there are 3 values you want to extract from the namespace?

// Extract an object containing 3 values from the namespace
var sets = setns('{Set:"Set", BitSet:"BitSet", MultiSet:"MultiSet"}');
var bs = new sets.BitSet();

I've defined a namespace() function for loading code and doing this kind of namespacing automatically:

Google has open-sourced the javascript library and optimizer they use in gmail and other web applications. They call it "Closure" and you can read about it here.

While the optimizer looks very cool, I think the library is most impressive. Its a really large code base, and at least some of it has been thoroughly field-tested in gmail and similar applications. You can get the code like this:

svn checkout http://closure-library.googlecode.com/svn/trunk/ closure-library-read-only

The closure library is intended to be used with the closure optimizer which removes unused code, so the APIs are broad with lots of methods--there is no sense that the closure developers were skimping on API in order to pack everything into a small download bundle. Also, and perhaps for the same reason, the code is not full of micro-optimizations. Compared to the jQuery and YUI code (for example) the Closure code is straightforward and easy to understand.