It has become common in modern JavaScript programming to use functions as namespaces. If you put your code inside a function, then your variables and functions are local to the containing function and do not clutter up the global scope.
var value = (function() { // Wrapper function creates a local scope or namespace
// your code goes here
return value; // Export a value from the namespace
)()); // Invoke the wrapper function to run your code
Now suppose that you have some JavaScript code as a string--you've just loaded it using XMLHttpRequest, for example. You're going to evaluate the code, and you might want to evaluate it in a namespace so that it doesn't define functions and variables in the global scope. This is easy: just wrap it in a function before evaluating it. In this case, the Function() constructor is even more handy than the eval() function:
var code = ....; // A string of JS code to evaluate var f = new Function(code); // Wrap it in a function f(); // And run the function
The problem with doing this is that the function creates a sealed namespace and we can't see what is inside. If the code defines something useful like a function or a class, we can't access it, and it does us no good.
Here's a trick I've just discovered. (I'm sure someone else has thought of this, but I haven't seen it used or described elsewhere). Before you wrap your code in a function add this line to it:
return function(s) { return eval(s); };
Now, when you invoke the wrapper function, it returns this evaluator function to you. The returned function evaluates a string in the scope of the namespace, so you can use it to peek into the namespace and extract whatever values you want!
If your string of code defines a constructor function named Set() that you want to use, you can run the code in a namespace and then extract f from the namespace like this:
var code = readFile("Set.js"); // A string of JS code to evaluate
// Define and invoke a wrapper function with special suffix code.
// The return value is a namespace evaluator function and we treat
// it as a namespace object.
var setns = new Function(code + "return function(s) { return eval(s); };")();
var Set = setns("Set"); // Import the Set function from the namespace.
var s = new Set(); // Use the class we just imported
And what if there are 3 values you want to extract from the namespace?
// Extract an object containing 3 values from the namespace
var sets = setns('{Set:"Set", BitSet:"BitSet", MultiSet:"MultiSet"}');
var bs = new sets.BitSet();
I've defined a namespace() function for loading code and doing this kind of namespacing automatically:
/*
* Load modules of code by enveloping them in a function and executing
* the function: then they don't pollute the global namespace (unless they
* assign to undeclared variables, but ES5 strict mode will prevent that.)
* The wrapper function we create returns an evaluator function that
* evals a string inside the namespace. This evaluator function is the
* return value of namespace() and provides read access to the symbols
* defined inside the namespace.
*/
function namespace(url) {
if (!namespace.cache) namespace.cache = {}; // First call only
if (!namespace.cache.hasOwnProperty(url)) { // Only load urls once
var code = gettext(url); // Read code from url
var f = new Function(code + // Wrap code, add a return value
"return function(s) { return eval(s); };");
namespace.cache[url] = f.call({}); // Invoke wrapper, cache evaluator
}
return namespace.cache[url]; // Return cached evaluator for this namespace
}
/* Return the text of the specified url, script element or file */
function gettext(url) {
if (typeof XMLHttpRequest !== "undefined") { // Running in a browser
if (url.charAt(0) == '#') { // URL names a script tag
var tag = document.getElementById(url.substring(1));
if (!tag || tag.tagName != "SCRIPT")
throw new Error("Unknown script " + url);
if (tag.src) return gettext(tag.src);// If it has a src attribute
else return tag.text; // Otherwise use script content
}
else { // Load file with Ajax
var req = new XMLHttpRequest();
req.open("GET", url, false); // Asynchronous get
req.send(null);
return req.responseText; // Error handling?
}
}
else if (typeof readFile == "function") return readFile(url); // Rhino
else if (typeof snarf == "function") return snarf(url); // Spidermonkey
else if (typeof read == "function") return read(url); // V8
else throw new Error("No mechanism to load module text");
}
You can also find these functions in namespace.js. I've written a simple demo. It doesn't do anything interesting--just alerts 256. But if you look at the code, you'll see that it loads and namespaces 4 chunks of code.
If you're a Python programmer you may be thinking that this technique would be useful for implementing a Python-style "from foo import bar" type module system. I've been thinking the same thing and I'm working on it (but the code isn't ready to share). The shortcoming, of course, is that there is no way to enumerate and import all the symbols defined in a namespace, so we can't simulate Python's "import *"
Great article, as usual. (I'm a huge fan.)
Given your current work, I would be curious to have your thoughts on CommonJS 's module specification[1].
[1] http://commonjs.org/specs/modules/1.0.html
It might be worth pointing out that any variable/function declarations from the string-to-eval are bound to evaluator's local scope, not scope of outer self-executing function.
This results in two things (which make this "technique" useful for inspection but somewhat limited for code creation/augmentation).
1) Declarations from within evaluator are not available to the original scope (and so to any methods that have access to that scope):
var evaluator = (function(){
var x = 1;
this.getX = function getX(){ return x; };
this.getY = function getY(){ return y; };
return function evaluator(str) {
return eval(str);
};
})();
getX(); // 1
getY(); // `ReferenceError`, `y` is not declared
evaluator('var y = 2;');
getY(); // still `ReferenceError`
2) Each evaluation is isolated from each other; declarations in one evaluation are not available to any other one:
evaluator('function foo(){}');
evaluator('typeof foo;'); // "undefined"
// or
evaluator('var bar = "baz";');
evaluator('typeof bar;'); // "undefined"
Thanks, Tobie!
I'm not sure what to make of the CommonJS modules. I'm excited that work is being done on standardizing this server-side stuff, though I'm not sure what, if any, impact it should have or will have on client-side scripting. Also, I don't understand the security issues well enough to know how the "securable" part of the specification constrains the API. So I'm not really able to offer well-informed comment.
Having said that, I'll say that I don't like the name "require" that they chose. First, it is not parallel with "exports". If they wanted to use require they should have changed exports to "provides". Second, "require" is a pretty passive verb and doesn't fit (for me) a function that actively returns an object containing the public API of the module.
kangax,
You're right that I should have mentioned that the returned evaluator function is only good for peeking inside the namespace, not for poking new stuff in there. (It is not a Ruby-style module_eval() method, for example.)
You've explained the reasons for this well. Each invocation of the evaluator function has its own transient namespace, but since nothing is returned from that namespace, it gets garbage collected and doesn't stick around. Also, in ES5 strict mode, eval() won't be allowed to declare variables and functions in the caller's scope.
David,
Yes, good point about ES5-strict eval! :)
Speaking of evaling string in a specific context, SpiderMonkey (as you probably know) allowed to do just that in recent versions (but not anymore, as it was considered a security whole, IIRC) — http://peter.michaux.ca/articles/module-pattern-provides-no-privacy-at-least-not-in-javascript-tm
On an side note, I'm not very fond of made-up terminology like namespaces ;) (unless, of course, we're talking about some syntactic or object extensions). ECMAScript is pretty clear with naming scopes as scopes and seeing something like "sealed namespaces" only confuses matters (imo). Unfortunately, that's also something I see often in many books on Javascript, where authors come up with different names to describe already-named (and well-known) ECMAScript mechanisms.
kangax,
I wouldn't say that "namespace" is made-up terminology--its a well known CS term, and I think I use it accurately here: JavaScript's function scope is serving as a namespace here.
But I see your point: if I could use the word "scope" instead that might simplify things.
I could rename my namespace() function to "evalInPrivateScope()" and refer to the returned function as a "scope probe". I like the function name "namespace", however because it works as a verb to describe the action of the function: "namespace this code". And it also works as a noun to describe the returned function. That function is an object that translates names into values, and I think it is legitimate to say that that object *is* a namespace.
Having made that argument, however, I'll say that a module-based metaphor is more appropriate than a namespace-based metaphor anyway and my preferred implementation will probably have a function named module() or from() instead of namespace().
"I'm excited that work is being done on standardizing this server-side stuff, though I'm not sure what, if any, impact it should have or will have on client-side scripting."
That pretty much summarizes my thoughts.
That said, JavaScript would really benefit from having a standardized, environment-agnostic module system.
In a similar vein, in my own home-made library, I decided, for fun/personal use only, to try a different approach to modules, and in the process completely abusing closures and scope chains (I can't say I have noticed a huge performance penalty). Given the recent work on ES5, my approach does not work for strict mode, and it is not secure (but it could be made so I suppose).
Essentially I have a base module which has something like the following:-
function base()
{
var rootNamespace = this;
var currentImportingModule;
var currentClosure = closure;
function closure(s)
{
var r;
eval(s); // sets r to something!
return r;
}
function loadModule(moduleFunction)
{
// evaluates "new moduleFunction()" in currentClosure
// within "new moduleFunction" set currentImportingModule = this
// the returned object is then added to the rootNamespace
// currentClosure is set to currentImportingModule.closure!
}
function export(o, name) // function to export
{
// code to export a function or object from a module
// this method is called within "new moduleFunction()" for each exported item
// adds it to "currentImportingModule" as set above
currentImportingModule[name] = o;
}
function inspect()
{
// code to inspect all imported modules using "currentClosure"
}
}
var root = new base();
The fun begins when we consider a module:
function module()
{
currentImportingModule = this;
function closure(s)
{
var r;
eval(s); // sets r to something!
return r;
}
export(closure);
// other exported functions and objects
}
base.importModule(module);
It has its own closure, which becomes the closure for the next module.
You end up with a nested set of modules each of which has internal closure access to all the previous closures, as well as normal namespace access!
The nesting is not a problem, as dependencies I have found are all one way if you organise it right: base->core(class construction etc)->util(strings, arrays etc)->dom->gui(widgets)->io(file handling)->myApplicationModule.
The final piece is the "inspect" function, which, using "currentClosure" can inspect the whole lot!
Thanks for sharing this Julian.
It sounds great, but without seeing your code for loadModule(), I don't quite see how it works. Also, the base() function needs to export or return something, doesn't it?
Thanks David. You are correct in that base() needs to export something. In fact it exports itself, by calling new base().
I have created a quick blog entry to set out the full code, rather than clog you up here.
http://www.baconbutty.com/blog-entry.php?id=42
I have provisionally called the scheme "nest", to distinguish it from other libraries that use the word "base".
David. Thanks for your comment on my entry. I have been looking at an XmlHttpRequest based loader, and have needed to develop a queued XmlHttpRequest system, where you can make multiple requests, but the modules are still loaded in the order you originally requested them, whatever order they come back from the server in!
@kangax
"2) Each evaluation is isolated from each other; declarations in one evaluation are not available to any other one:"
It is not impossible, to do it that. Maybe most popular approach of your words is *sandbox* produced by iframe in browser scripting environment. But how you think in clever ECMA 262-3 implementations?
var evaluator = function(){
var _evaluator = [
'(function (str){',
'evaluator = eval(_evaluator.join(""));',
'return eval(str);',
'})'
];
return eval(_evaluator.join(""));
}();
evaluator('var bar = "baz";');
evaluator('typeof bar;'); // "string"
In this example, i produce long Scope Chain. Each time when call `evaluator' i compile function expression and associate with AO/VO of current execution context via [[scope]] property of created function object. After that assign reference to that `object' on property of Global Object `evaluator'.
That isn't very good code, because much more memory consumption. Each time when i call `evaluator' AO/VO of execution context on the top of the stack, doesn't marked for garbage collection when that execution context finish.
Of course in my example your point 1 is still there. If i write something like this one:
evaluator('function f(){return bar;}');
evaluator('var bar = "baz";');
evaluator('f()'); //ReferenceError
Because `bar' is defined in one level down in the ScopeChain.
Thank you David. I appreciate the opportunity to learn from you again as I have in the past with your stellar Java and JavaScript books.
My question concerns the opening code in your article that appears to have a syntax error. Specifically, what are the parentheses surrounding the function for?
I first saw this in the JQuery module and was quite puzzled by it. I have experimented and find that my code works with or without them.
I know of only one use for parenthesis in JavaScript, and that is function invocation. I have searched your JavaScript book and haven't found it revealing.
I thought perhaps it has something to do with closures and after reading several articles on the net have, disabused myself of that notion.
So, what are they for David? Please tell me, the suspense is killing me.
Kind Regards,
Michael D. Alexander
Michael,
In this case, the parens around the function definition are an idiomatic hint that what is being assigned to the variable is not the function itself, but the result of invoking the function.
They are not required in this case, but they are required if there is no assignment. If you want to write a function expression (as opposed to a function statement) and that expression is not part of a larger statement, you have to use parentheses. Otherwise when the parser sees the function keyword, it will assume it is a statement.
It has become pretty common to use anonymous functions to turn global variables into local variables:
(function() {
// your code goes here
}());
If you don't have the parens in this case, you'll get a syntax error.
The 6th edition of my book (not sure when it will be done) will cover this programming idiom.
Thank you David,
Three more questions:
1- How would the use of parenthesis be classified in this application. For example, when used to invoke a function () are classified as 'operators'.
2- Where did you first learn about this 'idiom'.
3- Does this use of () have anything to do with closures or scoping?
-M
David, how does the syntax error show up?
I removed the variable assignment and the parenthesis surrounding the literal and placed it in-line with some other code. It still compiles and runs fine.
Also, if I place a closing semicolon after the invocation (original expression), I get an error.
Please advise.
-M
Greetings David,
I can't quite make heads or tails of your exposition. First of all there appears to be an error in your first code sample:
var value = (function() { // Wrapper function creates a local scope or namespace
// your code goes here
return value; // Export a value from the namespace
)());
I suspect you meant to close your function with a closing brace rather than a closing parenthesis:
var value = (function() { // Wrapper function creates a local scope or namespace
// your code goes here
return value; // Export a value from the namespace
}());
Then, a little further down you introduce this line:
return function(s) { return eval(s); };
Without defining "s". Unless you meant "code". Please advise.
BTW, I find it more helpful if I can get a working block of code I can run and experiment with rather than all the arm waving explanations, with variables and function appearing seemingly out of nowhere. I believe this would serve you to because it would be an error check on your article to have a working piece of code knowing how easy it is to make a mistake in ones exposition or code.
For example, does readFile make the XMLHttpRequest you mentioned earlier:
readFile("Set.js")
and are you augmenting Set.js with:
return function(s) { return eval(s); };
and can the line above be added regardless of what is presently in Set.js.
Please advise.
I realize how challenging it is to transfer what's in your head to my head through an article like this. That is why I recommend a full working code sample that includes your XMLHttpRequest and the details on "s" and "readFile".
Kind Regards,
Michael Alexander