Recently in javascript Category

If you weren't paying attention to the #teamjs hashtag on Twitter on Friday, you missed out on a lot of excitement and a great JavaScript community moment. If you were part of the fun, then you probably know all this already, and should see TeamJS Update and What's Next instead.

TeamJS is a group of JavaScript programmers and other web developers who are raising money for Feeding America as part of the Mozilla Firefox Challenge.

The Mozilla Firefox Challenge is a celebrity fundraiser, sponsored by Mozilla. Mozilla got 12 celebrities involved, donated $5,000 to their favorite cause and asked the celebrities to compete to raise more money through CrowdRise for that cause. Whichever charity raises the most money before January 11th will win an additional $25,000 from Mozilla.

But the competition isn't limited to the initial 12 celebrities: there are no celebrities behind TeamJS, and we didn't get any seed money from Mozilla. TeamJS started off as a lark to see whether the JavaScript and web developer community could donate some money and get the black-on-yellow JavaScript logo on the leaderboard among the celebrities faces.

But then something amazing happened! On Friday morning Chris Williams, the big-hearted organizer of JSConf (JavaScript's premiere conference) donated $5000 on behalf of JSConf. That put us on an even playing field with the celebrities and, when combined with donations from Wednesday and Thursday, it catapulted us into the middle of the pack.

That seed money opened the floodgates and the money began to pour in. When I left the office Friday evening, TeamJS was in second place. When I got home, we were first. The evening ended with another big $1,000 donation from Jupiter Consulting that brought us close to $11,000 in total donations. We were no longer just in first: we had a $2,000 lead and had left the celebrities in our dust. (As I write this Sunday night, we have raised $12,211 and second place is Sophia Bush with $9,990.)

It was a really amazing day, but what it means is that this is no longer a lark: we could actually win this contest. Please join us and let's make this an amazing week!

Read more about our charity, Feeding America and give whatever you can at the TeamJS page.

(If you don't live in the US, you can skip it.)

The US Census Bureau calculates that in 2008, 15% of US households were "food insecure". A slightly older version of that data is broken down by county and state in this excellent interactive map.

With continued high unemployment, that 15% figure has surely gone up since 2008. These are the households that may have to choose between food and heat this winter. If, like me, you can't even imagine what it must be like to have to make that choice, you're probably in a position to help.

Its just crazy that there's a major hunger problem in a country with huge food surpluses. That's why one of my favorite charities is Feeding America (you may have heard of them under their old name "America's Second Harvest). They're a national organization that collects surplus before it is thrown out and distributes it to a network of food banks that distribute it to people in need.

And they're very efficient at it: each dollar donated provides eight meals. And they've got very low administrative and fundraising overhead: 96% of their money goes to their hunger programs.

Please help, if you can.

And if you're going to donate money to Feeding America, a fun way to do it is to participate in the Mozilla Firefox Challenge as part of TeamJS: I'm challenging the US JavaScript and webdev community to donate more to this cause than a dozen Hollywood celebrities can raise for their causes. I think it would be great to see the bright yellow JS logo (and the Rhinoceros from my book because I think it is funny :-) in the middle of that pack of celebs. (Click on the Firefox Challenge link above for a visual of what I'm talking about.)

I've been flogging this pretty hard on Twitter (#teamjs and #fxchallenge), with fairly disappointing results. I'm not ready to give up, but I am feeling discouraged, and I could really use some help. If you think the challenge is a fun way to raise money for a good cause, please help publicize it. I don't use Facebook or Google+, so help with those networks would be especially great. If you sign up for an account on crowdrise.com, you can even join the TeamJS team.

Update: discouraged no more! Chris Williams and JSConf have made a big donation and a huge publicity push. TeamJS is now in 4th place!

And if you don't live in the US (but read this anyway) and would like to raise money for a cause that is not US-centric, you can create an account at crowdrise.com and then create your own team in the Firefox Challenge-maybe you can embarrass the US JavaScript community into giving more!

Mozilla is sponsoring The Mozilla Firefox Challenge, a celebrity fundraiser. Mozilla has donated $5000 to the favorite charity of 12 Hollywood celebrities and will donate $25,000 more to the charity of whichever celebrity can raise the most for their cause.

I want to challenge the JavaScript community, and the web developer community, and heck, the whole geek community to go head to head with the celebrities and raise more money than they can. So I've formed TeamJS to compete against the celebs in the challenge. We don't get the head start of $5k seed money, but we can still win the $25K prize!

The charity I've picked is Feeding America, which describes itself as a sort of food bank for food banks. They're a national organization that collects surplus food and distributes it, via a network of independent food banks, to those who are hungry. And they're remarkably efficient at what they do: each dollar donated provides 8 meals.

I kicked things off with a $500 donation. Please contribute what you can, and spread the word!

(Feeding America is a US charity, but the international JavaScript community is more than welcome to participate. I've asked crowdrise if they'll allow me to designate different charities on different days, so we can pick others that appeal to geeks in other parts of the world, but I don't actually know if that will be possible.

Update: crowdrise.com says that I can't switch charities once donations have been made. If you want to participate but want to give to a different charity, you can form a new team. TeamJS.eu anyone? Since crowdrise is in the US, the charities have to be registered in the US, but that still leaves lots of good ones with an international focus... )

Back in January I posted an IndexedDB example that showed how to initialize and query a database of zipcodes using the HTML5 IndexedDB API.

That API has been evolving, and Kyle Huey from Mozilla has updated my example to work with the latest version of the API, which is implemented in Firefox 10. The main difference seems to be in the way that the database is initialized. Instead of the old setVersion() magic, you now set up the database in the handler of the "upgradeneeded" event.
That's quite a bit more sensible.

Kyle's version of the demo is here. The first query you make triggers the initialization of the database. It looks like there might be a problem with that first query not succeeding. But subsequent queries do work.

Thanks Kyle!

I had a little twitter rant this morning about classes in JavaScript. (Actually, the rant was really about the attitude that seems to be behind much of the "don't add classes to JavaScript" commentary I see.) So I should probably explain in somewhat longer form why I want classes added to the language.

• The JS standard library is object oriented: we write a.map(f), not map(a,f). If the standard library defines classes, the language should make it easy to define classes.
• A number of popular JS libraries include convenience methods for defining and/or extending classes. This demonstrates to me that developers want to use classes and JavaScript, but that they want a simpler way to define them.
• Once we get used to the idea of a class keyword in the language, I think it is going to be really nice to be able to dash off little classes with that keyword, instead of the two step define the constructor and then set up its prototype object technique that we use now. In particular, I think it is going to feel really old-fashioned to define classes the old way once we've got modules in the language. I can't articulate this point well, but it is actually the one that is most compelling to me. We're getting modules in the next version of JS, and I don't think that the language will feel right without a class syntax to go along.
• The classes being proposed for JavaScript are all syntax sugar for existing patterns. Adding a class keyword will not change JS in any fundamental way, it will just make it easier to use the patterns that we already use. If you like defining your classes the old way, you'll still be able to do that. And you'll still be able to add methods to classes with C.prototype.new_method = function() {...} as you can today. As Brendan Eich would say, this is just "paving the cow paths". Here's what the classes proposal [Update link fixed now] describes as its motivation:

  ECMAScript already has excellent features for defining abstractions for kinds of things. The trinity of constructor functions, prototypes, and instances are more than adequate for solving the problems that classes solve in other languages. The intent of this strawman is not to change those semantics. Instead, it's to provide a terse and declarative surface for those semantics so that programmer intent is expressed instead of the underlying imperative machinery.

  I understand the reluctance that many JS programmers feel about adding classes. I feel it too, a little bit. A class keyword isn't "javascripty". I like, and understand the appeal of Allen Wirfs-Brock's approach: He defines extensions that make object literals more expressive, and then uses the <| and {. operators to create a simple class definition pattern. But the problem is that it is still a pattern rather than a new syntactic form. And to quote Dave Herman: "patterns are the sincerest form of feature request": if we're using patterns to define classes, then the language ought to just support classes.

  My desire to have classes in the next version of JavaScript of course has no bearing on whether we'll actually get them. The classes proposal linked above is apparently not doing well in committee, for reasons that I haven't tried to understand. There have been recent attempts to revive classes with simpler class proposals, so maybe there is still some hope.

I've recently blogged about CoverMonkey, a code coverage tool for Mozilla's SpiderMonkey JavaScript interpreter and my JavaScript HTML parser.

One of the really useful things you can do with a code coverage tool is check whether your test suites are actually exercising all of the code you're trying to test. (A hat tip here to my colleague Donovan Preston who insisted on good tests for dom.js-CoverMonkey was his idea.) There is already a very good test suite for HTML parsers and I was able to get my HTML parser to pass those tests.

That test suite is large and comprehensive, but when I ran CoverMonkey over the tests I found that there about 100 lines (out of some 5000) in my parser that did not get run by the tests. These were for pretty obscure things. Like what happens if you put a <!DOCTYPE> after <body>, or what happens if you have a NUL (\u0000) character in a <script> tag or what if you use &NotEqualTilde; character reference, which expands to two 16-bit characters rather than just one) in RCDATA state (like in a <xmp> element) ?

I ended up writing 65 test cases to improve code coverage for my parser up to almost 100%. (There is still some untested code for handling <noscript> tags when scripting is disabled because the way the test suite is set up does not allow testing with scripting disabled). Anne van Kesteren was kind enough to grant me commit rights to the html5lib repo, and my tests are now part of the suite: here and here.

I'm pretty pleased with this. It is a small thing, and there are only a handful of developers who will ever need to run these particular tests, but being able to use CoverMonkey to find and fix these few remaining holes in an otherwise excellent set of tests was pretty cool.

(If you are one of those handful of developers who might actually run these tests, note that my parser does not do validation and does not report parse errors. My tokenizer test cases include parse errors where I think they should be reported, but I can't actually verify with my own parser that there are supposed to be errors in those spots. So the tokenizer tests might have errors related to error reporting. The parser test cases also do not include any errors, but I doubt that will break things for anyone, since the errors in the parser tests aren't standardized in any way and can't really be compared.)

So, the moral of the story is: validating your test suites with a code coverage tool really works!

The project I've been working on in Mozilla's research group is dom.js: an implementation of the browser DOM in JavaScript. I've gotten much of the core DOM 4 document tree functionality implemented, and have started on DOM features defined by the HTML spec.

One of those features is the innerHTML property. And to implement it, I need an HTML parser. So I dove in and implemented one. Here are some of the things I learned:

I started work at Mozilla in May, and haven't had a chance to update my blog since then. I've been working on a project to implement the DOM in JavaScript. One of the interesting side-projects that has come out of this is a code coverage tool for SpiderMonkey (Mozilla's js interpreter). I'm calling it CoverMonkey, and have just created a repo for it on github. Take a look at some example output.

Debug builds of SpiderMonkey have a -D option that causes them to output opcode-by-opcode disassembly of every file they've run, along with execution counts for every opcode. CoverMonkey analyzes this output and displays coverage statistics and generates color-coded versions of your source code to highlight uncovered and partially covered lines.

The first big caveat about CoverMonkey is that this is not a tool you can use in the browser. It works with standalone versions of SpiderMonkey only. So it is only useful if you're using pure JavaScript. The second caveat is that you'll need a debug build of SpiderMonkey, which means that you'll probably have to build it yourself. Another, smaller caveat: CoverMonkey is a JavaScript program, but unfortunately SpiderMonkey cannot create files, so CoverMonkey requires Node.

The examples from the brand-new 6th edition of JavaScript: The Definitive Guide are available for download from O'Reilly and from Github.

These are ten of my favorites from the book. Note that many of these use new features of ES5 or of HTML5, and will only work in the newest browsers:

1) Example 1-1 is is an extended example in the first chapter of the book, intended to show readers a simple but non-trivial example of JavaScript. This is the loan calculator example from the last edition, but made much more interesting with the addition of client-side graphics, localStorage, and Ajax.

2) Example 9-7 emulates Java-style enumerated types in JavaScript. It demonstrates that JavaScript's prototype-based inheritance is so flexible that factory methods can be normal object factories or even class factories. That example is a little clearer if you look at some code that uses it.

3) Example 9-16 defines a class hierarchy of abstract and concrete Set classes. This one is a favorite because it involves data types and API design. Chapter 9 includes a number of other Set examples, too.

4) Example 9-23 demonstrates the ES5 Object.defineProperty() method and defines a convenient way to inspect and modify the attributes of the properties of an object. It may not be practical, but I think it is a beautiful hack.

5) Example 15-10 is a simple stream-like API wrapped around the innerHTML property of an element. When you're generating text (a table, for example) for display it is sometimes easier to pass each chunk that you compute to a write() method than it is to concatenate it all together and set it on innerHTML.

6) Example 21-03 is an analog clock implemented as an SVG graphic with scriptable hands. I love client-side graphics, and this is a favorite of mine because making the hands rotate is so simple with SVG transforms.

7) Example 21-06 draws a fractal Koch snowflake using the <canvas> tag. I like it because it draws the same line over and over again, but uses transformations to make the line appear at different locations, orientations and sizes. You can view the snowflakes live in your browser.

8) Example 21-13 is another graphical example: it draws sparklines. This one is a favorite just because sparklines are so cool.

9) Example 22-1 uses the HTML5 geolocation API to find out where you are then uses the Google Maps API to obtain a static map of your location. I like it because geolocation (via wifi networks) is just pure magic!

10) Example 22-15 is a long example that demonstrates the IndexedDB API. I like it because the idea of a client-side database in a web browser is crazy and cool. This one is really cutting-edge, but if you're running Firefox 4, you can try it out live.

Wow! My previous post about the piracy of my JavaScript book generated an amazing number of insightful comments (and some idiotic ones, too). I don't think I can cover this issue comprehensively, but here are a few responses.

• First of all, let me restate the main points of my earlier post: it is personally very discouraging to me to see my books pirated, and I am dismayed by how easy it has become in the last few years to find illegal copies of my books. My suggestions about how Google might alter its suggestions and search results were sincere but secondary.

• Also: I understand (and stated in the post, though not as explicitly as I could have) that piracy has less financial impact on me than other trends affecting the publishing industry: technical books in general seem less relevant today than they did in the past. Nevertheless, I get myself worked up over piracy because it feels personal.

• Mark Pilgrim wrote a prickly response to my post. He makes a few good points (I did not know, for example, that Google was already filtering terms like "rapidshare" out of its suggestions). But he also gets worked up over the fact that when I wrote my post my book was not yet actually available from the download sites. (It is now, of course.) I disagree with Mark's main point "the book is dead". The tech book industry is ailing (as my declining royalties checks illustrate), but even tech books aren't dead yet.

• I suspect that this tweet of mine about why Mark could release a free version of his book and I could not may have had something to do with the prickliness of Mark's response. In fact, a far more relevant distinction (that would not have fit into a tweet) is that Mark's book was a new one, and he had an agreement with O'Reilly that allowed him to do that, where mine was a new edition of an old book, still governed by a contract from 1997 or thereabouts. Releasing a free version of my book is simply not a legal option I had. (Had I asked for it, O'Reilly might have agreed, but it would have been their decision, not mine.) Mark: if I offended with that tweet, I apologize.

• O'Reilly ought to do a better jobs of SEO so that its ebook sales pages are at the top of the Google search results for typical pirate searches.

• No one knows how to price ebooks yet. I think that O'Reilly is actively working on this. I agree that it is absurd that Amazon's kindle price is higher than the print price, and that O'Reilly's ebook price is so high, too. I don't think I have any control over this. Note John Seifarth's comment, though: he had registered his 5th edition with O'Reilly and they offered him a $5 electronic copy of the 6th edition.

• If I were to write a completely new book, I might well try self-publishing it at a much lower price point that enabled me to retain a much larger share of the revenue. This would mean, however, that it wouldn't be professionally indexed or typeset or copyedited.

• I'm surprised how many commenters were willing to confess to book piracy on the blog of an author who is upset about book piracy. You do know that blog comment systems always log your IP address, don't you?

• I agree with many commenters that ebooks could be so much better than they are today. O'Reilly is thinking about this, I know. Being able to push updates out to ebook customers would be a really big win.

• A number of commenters suggested that I put a donation button up on my website so that people who enjoyed, but did not pay for, my books could send me money. I might try it, though there might be legal issues. And it also feels a bit like surrender, or a way of giving my blessing to piracy.

• Someone calling his or herself "Leonid Brezhnev" has drunk the pirate kool-aid and articulates it this way:

  When you steal something tangible, the owner loses what he had.

  When you copy something digital, the owner has lost nothing.

  Copyright is meaningless when there are no physical limitations. Reality is knocking at your door and taking your money - it's up to you to get another job or try to screw up other people's liberties trying to enforce an artificial law.

  Finally, I find it offensive that you claim ownership for the equivalent of one long number.

  First of all, Leonid, copyright laws predate digital media by many, many years. They prevent publishers from simply re-printing and re-releasing each other's books. There are different laws that prevent publishers from breaking into each other's warehouses and stealing books. Copyright law establishes a framework within which publishing can work as a business. Without it, there would just be a race to the bottom, margins would get very thin, and only crap would be published. Maybe the fair-use exceptions are too narrow or the copyright protection lasts for too long, but the basic idea of copyright has been around for so long because it serves an important purpose. Copyright really has nothing to do with digital vs. analog or electronic vs. physical.

  Justifying piracy by equating a book with a long number is just silly. If you want to critique copyright from a Marxist or Anarchist perspective (as a tool for wealthy publishers to oppress the masses, perhaps) you could probably do that in an intellectually consistent way, and I'd at least listen to you respectfully. But the number thing is ridiculous.

Every time a new book of mine comes out, I get myself worked up about piracy. I've been tweeting about it this time, but there are nuances that require a longer form. The title of this post is a reference to Google's suggestions to people searching for my book:

Maybe if enough people link to this post, then those search suggestions will bring would-be pirates to this page instead of to the ebook piracy sites :-) [Update: Hooray, as of tonight, for each of the search suggestions pictured above, this blog post is ranked higher than any ebook download site! Sweet symbolic victory!]

So, in no particular order, but in more than 140 characters, here are some of my thoughts about book piracy.

I'm well-aware of Tim O'Reilly's public statements about piracy. And I agree with some of them. Obscurity is a worse problem than piracy, for example. And I know that some authors have been successful at selling books even while making them free (and legal) for download. I know that some genre fiction authors have been successful by self-publishing ebooks for 99¢. (I don't know that world well enough to link to any of those authors, though.) For most of these authors, however, "success" means "I'm making more money selling books for 99¢ than I did when I sold them for $3.99". There are only a few lucky authors who can actually make a living and support a family by selling 99¢ ebooks. I don't think self-published 99¢ ebooks is the way forward for technical books.

For 15 years I've been one of those lucky authors who has been able to support himself and his family almost entirely on book royalties. But the publishing industry has been in decline and my royalties checks have decreased more-or-less steadily since the dot-com bust, and I've now decided that I need to look for a salaried job. This is kind of the end of an era for me. (So if I sound whiny, keep in mind what is at stake for me. And if I can't whine on my own blog, where can I whine? :-)

I do not know whether or to what extent piracy is responsible for my declining income. I suspect that the internet and the transition from print books to ebooks has more to do with it than piracy does. But I also suspect that piracy has a non-trivial impact, too.

But beyond the non-quantifiable financial impacts, I can report that, to me, the piracy of my books is profoundly discouraging. When my Ruby book came out in 2008 I was sad to discover that pirated copies were available within a week or so of the book's release. When my jQuery pocket reference came out earlier this year, I was shocked to discover that Google was giving the ebook download sites higher placement than reviews of the book. And now JavaScript: The Definitive Guide is out. I don't have a copy of it yet, but illegal copies are free for anyone who wants one. And Google will suggest those illegal downloads to anyone who tries to research the book (see the screenshot). I've worked really hard on this book, and I've got to say that this just feels like a kick in the gut.

I was trying to be provocative when I tweeted the question "Does Google enable piracy?" But I do think it is a valid question. If Google indexes sites like ebookee that link directly to download sites and makes it easy to find the pirated content you want and even offers suggestions on what to search for, I think there is a case to be made that they're encouraging piracy. And an important part of this is that the ease of finding illegal ebooks has removed any stigma involved. Malware sites get flagged "may harm your computer". Porn images get filtered by default in Google image search. But illegal copies of ebooks? They're just out in the open--it must be okay to download them.

I know that electronic piracy cannot be defeated. And I don't think we should (or can) lock everything down with draconian DRM. But I also think that a laissez-faire attitude toward piracy ("well, it is better than obscurity" or "its going to be pirated anyway, so you might as well just make it free") is the wrong answer. Even if publishers cannot win against pirates, they should at least fight for a stalemate rather than accept defeat.

Here are some small steps that might help:

• Google could filter its search suggestions so that they do not actively suggest piracy. I suspect that Google already filters the suggestions offered when someone enters the name of a porn star, for example. Google already has a database of copyrighted books (Google Books) so they could easily filter the suggestions offered when someone searches for a book title.

• Google could flag (without filtering) search results that are likely links to pirated content. Google already flags some results with "this site may harm your computer". Why not flag pirate sites: "Downloading content from this site may result in legal action by the copyright holder" or "Downloads from this site may be illegal". Or nice and simple: "this site may harm your karma".
  

I'll close by quoting the sidebar I titled "A Note about Piracy" in the preface of my new book:

If you are reading a digital version of this book that you (or your employer) did not pay for (or borrow from someone who did) then you probably have an illegally pirated copy. Writing the 6th edition of this book was a full-time job, and it took more than a year. The only way I get paid for that time is when readers actually buy the book. And the only way I can afford to work on a 7th edition is if I get paid for the 6th.

I do not condone piracy, but if you have a pirated copy, go ahead and read a couple of chapters. I think that you'll find that this is a valuable source of information about JavaScript, better organized and of higher quality than what you can find freely (and legally) available on the Web. If you agree that this is a valuable source of information, then please pay for that value by purchasing a legal copy (either digital or print) of the book. On the other hand, if you find that this book is no more valuable than the free information on the web, then please discard your pirated copy and use those free information sources.

Update: Thanks everyone for the kind words and the thoughtful comments! I've replied to most of your comments tonight, but I really have to get serious about writing my jsconf presentation, so I may not be able to reply to any more for a couple of days.

Update: More great comments! I'm really wishing I had a threaded comment system on this blog. Its never been an issue before

Firefox 4 partially implements the TypedArray specification, but, sadly, leaves out the DataView class. Typed arrays are cool, but they use native byte endianness, which means that you should not use them for working with data from files or from the network. The DataView class allows control over byte order, which is why it is important.

I've implemented DataView for Firefox 4, using the existing TypedArray classes that Firefox does support. (Chrome and, I assume, Webkit nightlies, already have DataView, and do not need this library.)

The DataView API is sort of sucky, so the next step is to define a useful binary API on top of DataView.

The online Rough Cut edition of JavaScript: The Definitive Guide has been updated, and is now feature complete. All the chapters, examples, and reference entries that will be in the book are finally in the rough cut. (The index has not been created yet, though).

If you're willing to order the printed book, at the cover price, for delivery in May, you can get online access to the rough cut for free right now: http://oreilly.com/catalog/9780596805531/.

One caution: the client-side reference section is not fomatted quite right in this rough cut edition: the class hierarchy information has been appended into the title of the reference entires. So when you look up the File object, for example you see a page named FileBlob (because File inherits from the Blob) And if you look up Form, the page title is "FormNode, Element" because Form inherits from Node and Element. Sorry about that!

The online Rough Cut edition of JavaScript: The Definitive Guide has been updated, and is now feature complete. All the chapters, examples, and reference entries that will be in the book are finally in the rough cut. (The index has not been created yet, though).

If you're willing to order the printed book, at the cover price, for delivery in May, you can get online access to the rough cut for free right now: http://oreilly.com/catalog/9780596805531/.

One caution: the client-side reference section is not fomatted quite right in this rough cut edition: the class hierarchy information has been appended into the title of the reference entires. So when you look up the File object, for example you see a page named FileBlob (because File inherits from the Blob) And if you look up Form, the page title is "FormNode, Element" because Form inherits from Node and Element. Sorry about that!

A few commenters on my last post were curious about my decision to include jQuery coverage in my JavaScript book. There was even a little bit of outrage on Twitter. So I thought I'd clarify:

The 6th edition of my book covers jQuery because:

• jQuery is probably the most popular framework

• it is small enough that I can cover it in one chapter

• for document manipulation, event handling, and simple animations, jQuery gets a lot of things right

One chapter of my book is dedicated to jQuery, and jQuery is discussed only in that chapter. I do not use jQuery in examples in other chapters.

The book also includes an introduction to Node as part of a chapter that also covers Rhino.

I turned in the final chunk of js:tdg6 to the O'Reilly production department yesterday. So I'm basically done with the book! (Well, okay, I still have to update the preface, check the galleys etc., but the serious writing is finally, finally done.)

Here are some stats:

• Covers: ECMAScript 5, HTML5, jQuery

• Words: 396,582

• Pages: 1024 (nice number, but the index will add another 50 or so...)

• Lines of code: ~11,500

• Chapters: 22 (plus core and client-side reference sections)

• New or completely rewritten chapters: 15

• Months of work: 21

I don't know when you'll be able to get a copy. Its out of my hands now and I don't yet know the production or printing schedule. A long book like this takes a while though.
You can buy a digital copy of the Rough Cut however.

Update: Kyle Huey of Mozilla has updated this example to work with the latest version of the API. This version works with Firefox 10.

Update, March 17th: things have gotten a lot better since I first posted this, and I've make a number of edits below. Big thanks to Ben Turner and Jeremy Orlow who explained a lot in the comments!

I've been trying to create an IndexedDB example for the next edition of my JavaScript book. IndexedDB is a promising API< strike>, but it turns out to be just barely useable right now.
I'm using Firefox 4b10 RC and Chrome 10.0.642 11dev. Here are some things I've found out. Maybe they'll help someone!

1. Both Mozilla and Chrome prefix the global indexedDB object. Use mozIndexedDB or webkitIndexedDB. Prior to FF4b9, apparently you had to use moz_indexedDB, with an underscore, but no longer.

2. Chrome also puts webkit prefixes before the other related globals IDBTransaction and IDBKeyRange. Firefox does not (currently) prefix those.

3. I can't get Chrome to insert anything into a database. Even Google's tutorial at http://www.html5rocks.com/tutorials/indexeddb/todo/ isn't working. It complains that the DB is in read only mode and won't insert. Tested on Linux and MacOS. This is presumably a regression of some sort since the tutorial must have worked on Chrome at some point. It doesn't even work if you launch chrome with the magic --unlimited-quota-for-files which you need when using the filesystem API.

4. For large databases, like my demo, you have to start Chrome with --unlimited-quota-for-indexeddb

5. Chrome's error codes are off-by-one compared to the IndexedDB spec. They still differ from the spec, but Firefox now matches Chrome for most of the codes.

6. The spec never defines what a "key path" is. The non-normative examples show it as a plain string. When creating an object store, FF requires an object like {keyPath:"property_name"} instead of just the "property_name" string. Google's examples show key paths like this, too, so they're apparently both out of sync together on this point. In comments, Ben and Jeremy explain that the spec was out of date.

7. But it gets weirder. When creating an index on an object store, Firefox wants a plain string key path, and the index won't work if you specify an object.
   Ben explains this in comments.

8. It is completely undocumented in the spec how you would express a key path for more deeply nested properties of the object being stored. An array of strings seems like the right thing to me, if anyone wants to take my advice. [Ben explains in comments that this issue has been deferred for a future revision of the spec.]

9. Finally, Firefox has diverged from the spec in one major way. There are lots of places where you make async requests in this API. The spec says that the result of your request will be in the request property of the event object that is passed to the onsuccess event handler. But Firefox has (relatively recently, I gather) moved the result value so that it is a property of the request object rather than the event object.. Instead of writing event.result, as the spec says, in Firefox you have to write request.result, or perhaps more elegantly: var result = event.result || event.target.result; Thhe spec was out of date, and Chrome has now followed Firefox. Now, you always get the result of an asynchronous operation from the result property fo the request object. The event object is no longer used.

10. Neither Firefox (See Ben's comment below for directions on accessing the Firefox UI) nor Chrome have any UI yet for managing the databases created by this API. So if you mess up a database and want to delete it, you just have to find your profile directory and delete the appropriate file or directory there. This is the command I used on my Linux machine:

    rm -rf ~/.config/google-chrome/Default/IndexedDB/*localhost*.indexeddb ~/.mozilla/firefox/*.default/indexedDB/*localhost*

11. I've just assumed that this API won't work right if you test it from a local file, so I've been running a simple node-based webserver. (That's why the database names shown above have "localhost" in their names).

My zipcode demo (written for the next edition of JavaScript: The Definitive Guide) loads a database of US postal codes and lets you search by postal code or by city name. It seems to work today in FF4b10RC1 and Chrome 11dev on Linux. I haven't tested Windows. or MacOS (The database includes latitude and longitude, so it would be sweet to hook it up to google maps!)

Update: Google's demo mentioned above can be made to work on FF4b10 by applying this patch.

Update 2: I've cleaned up my demo considerably and have updated the link above.

Update 3: Thanks to Guido Tapia who suggested in comments that I try the Chromium nightlies. This example sort of works with Chromium 11.0.652.0 (72925). The API works, so I think my example code is written correctly. But chrome's database engine can't handle all the zipcodes. It looks like it downloads one batch of a few thousand, then takes so long to process them that the next "onprogress" event gets the rest of the zipcode file. And this version of chrome can't insert 30000+ objects at once. It pegs my cpu and tries for 20 to 30 seconds then aborts the transaction. (I was surprised how slow Firefox was to insert all these zipcodes, but this version of Chrome is slower, and doesn't actually succeed...) This problem is fixed by adding the --unlimited-quota-for-indexeddb when you launch Chrome.

Web Workers are threads for client-side JavaScript. There are a lot of very cool things about them, but one of the not-so-cool things about them is that it is hard to debug code running in a worker thread. Worker's don't have access to the DOM. They can't call alert(), they can't modify the document and they don't have the console.log() API that we've all come to depend on.

I've put together a proof-of-concept implementation of console.log() for web workers. Drop the WorkerConsole.js script into your HTML file, and (in theory, anyway) your worker threads will all be able to call console.log. there are lots of caveats, though. See the comments at the top of the file.

It works by creating a MessageChannel (which means it won't work in Firefox) over which the worker can send logging messages back to the main thread. In order to create this MessageChannel, WorkerConsole.js replaces the Worker() constructor. It also forces each new Worker object to load and execute WorkerConsole.js first (to define console.log()) before executing whatever script you passed to the Worker() constructor.

You can find the code, and a very simple test at github.

I've just posted a short article about jQuery over at O'Reilly Answers. Basically, it is 5 excerpts from jQuery Pocket Reference lightly edited and jammed together into an article. I tried to select jQuery features that are useful but obscure. See what you think!

In a comment on my last post on global eval(), Tom Robinson points out that I didn't test this very well on Chrome. It turns out the Chrome defines execScript() like IE does, but that it never returns the value of the evaluated code. I just checked MSDN, and it turns out that execScript() isn't supposed to return a value.

So instead of writing this:

var geval = this.execScript || eval;

I suppose I should have just done this:

var execScript = this.execScript || eval;

That is: current versions of all major browsers either define execScript(), or have an implementation of eval() which, when called by any other name, does a global eval like execScript() does.

But notice that I say "current versions". See kangax's comment in my original post, where he argues that this isn't ready for prime time, and links to some actual tests and data.

Thanks Tom and Juriy!

And now, confession time: I wrote the original global eval post because I was joining Twitter, and I wanted to have something interesting to tweet about. Now I'm totally busted.

My new book jQuery Pocket Reference is at the printer, and is available now as an ebook through O'Reilly.

I created the fractal snow bookmarklet to publicize my other new pocket reference, so I thought I ought to do something fun to celebrate the jQuery book as well.

The result is http://jQwiky.net: a goofy wiki that allows you to edit web pages using jQuery code transmitted via Twitter. I think it ought to be fun to collaboratively build or competitively deface web pages while showing off your best jQuery one-liners.

See http://jQwiky.net for a full explanation

My new Canvas Pocket Reference is at the printer. To celebrate, I created this fun bookmarklet. Click the link to try it out on this site. If you like it, drag the link to your bookmarks toolbar and try it out on other sites. Note that uses the <canvas> tag and does not work in IE8 and before. (It should work in IE9 beta, but it doesn't, and I haven't tracked that down yet.) (It works in IE9 beta on some sites but not on others.)

Let it Snow!

The snowflakes are fractals known as Koch Snowflakes. They're drawn in <canvas> tags, and I use basic CSS animation techniques to move the snowflakes down the screen.

The fractal-drawing code is an example from the book, and it was particularly fun to write, so I'll share it below as a holiday brain-teaser. A 4th-order Koch snowflake consists of 768 connected line segments. But the function below includes only a single line drawing command: every one of those line segments is drawn with this statement:

c.lineTo(0, len);

The trick to this particular implementation is that I manipulate the canvas transformation matrix with translations, rotations and scaling and each line segment is drawn with a different transformation in effect. The code is carefully commented. Have fun figuring it out!

// Draw a level-n Koch Snowflake fractal in the context c,
// with lower-left corner at (x,y) and side length len.
function snowflake(c, n, x, y, len) {
    c.save();           // Save current transformation
    c.translate(x,y);   // Translate to starting point
    c.moveTo(0,0);      // Begin a new subpath there
    leg(n);             // Draw the first leg of the fractal
    c.rotate(-120*deg); // Rotate 120 degrees anticlockwise
    leg(n);             // Draw the second leg
    c.rotate(-120*deg); // Rotate again.
    leg(n);             // Draw the final leg
    c.closePath();      // Close the subpath
    c.restore();        // Restore original transformation

    // Draw a single leg of a level-n Koch snowflake.
    // This function leaves the current point at the end of
    // the leg it has drawn and translates the coordinate
    // system so the current point is (0,0). This means you
    // can easily call rotate() after drawing a leg.
    function leg(n) {
        c.save();               // Save current transform
        if (n == 0) {           // Non-recursive case:
            c.lineTo(len, 0);   //   Just a horizontal line
        }
        else { // Recursive case:           _  _
               //     draw 4 sub-legs like:  \/
            c.scale(1/3,1/3);   // Sub-legs are 1/3rd size
            leg(n-1);           // Draw the first sub-leg
            c.rotate(60*deg);   // Turn 60 degrees clockwise
            leg(n-1);           // Draw the second sub-leg
            c.rotate(-120*deg); // Rotate 120 degrees back
            leg(n-1);           // Third sub-leg
            c.rotate(60*deg);   // Back to original heading
            leg(n-1);           // Final sub-leg
        }
        c.restore();            // Restore the transform
        c.translate(len, 0);    // Translate to end of leg
    }
}

You can get all the code, including the animation code that makes the snowflakes fall, from github.

Little known fact: if you invoke the JavaScript eval() function through any name other than "eval", this is called an "indirect eval" and causes havoc with code optimzation, so JS interpreters perform indirect evaluation in the global scope rather than the local scope.

This is official in ES5. ES3 allows browsers to throw EvalError instead of performing an indirect eval, but as far as I know, all ES3 browsers except IE treat indirect eval as global eval, and have done so for a long time. IE has an alternative: execScript().

So here's a one-liner for global evaluation:

var geval = this.execScript || eval;

And here's some code that uses it:

var geval = this.execScript || eval;
var x = "global";
function f() { 
    var x = "local"
    console.log(geval("x"));
}
f();  // Prints "global", not "local"

jQuery has a utility function $.globalEval() that does the same thing. But jQuery's implementation is more complicated: it actually creates a <script> tag, and inserts your code into it in order to execute it globally. I'm not sure how this is different from my geval() function. But maybe jQuery does it this way because there are still some browsers out there that don't do indirect evals the way I've described here...

I have two new books coming out soon from O'Reilly:

These are available for pre-order from Amazon now, with an expected ship date of December 15th. (You can give them to all your friends for Christmas! :-).

These books are both standalone versions of chapters (and related reference-section material) from the forthcoming 6th edition of JavaScript: The Definitive Guide. That means that you can also get the same content by buying the
"Rough Cut" edition of that book.

I'm really proud of both of these books. Canvas Pocket Reference covers the <canvas> tag pretty definitively, and has a bunch of cool examples. All of the figures in the book were actually generated using a <canvas>. (But they don't have jagged lines like a screenshot might: I used a really big offscreen canvas and a transform to run my demo code at high-resolution and then used the toDataURL() method to extract a PNG image for the book.)

I love the jQuery library, but I've never been happy with the online documentation. jQuery Pocket Reference is short but thorough and explains the library more clearly (I think) than you'll find elsewhere. If you're a fan of jQuery (and even if you don't need a book about it) click on the Amazon link above so you can see a larger version of the cover. The text we put at the top of the cover, above the title, is kind of funny (and I got John Resig's permission to use it.)

I should note that you can also get these books for the same price directly from O'Reilly, and qualify for the "buy 2 get one free" offer:

• Canvas Pocket Reference
  
• jQuery Pocket Reference
  

I love the <canvas> tag, but its 2D drawing API is intentionally minimal and unfriendly, so I've written a simple wrapper library that improves it.

canto.js defines a single canto() factory function. Pass a canvas element or the id of a canvas element to this function, and it returns a Canto object to you. This Canto object is a drawing context, like the one you'd get by calling getContext('2d') on the canvas element. The Canto object implements the 2D drawing API, so your existing canvas code should work as-is, but it also implements a number of other features.

One of the most important API improvements is that all Canto methods that do not have some other return value return the Canto object. This enables method chaining. For example:

// Draw a triangle
canto("canvas_id").moveTo(100,100).lineTo(200,200,100,200).closePath().stroke();

Notice that the lineTo() method in the code above has four arguments: this is an extension that allows the single method call to draw two line segments.

Another useful API extension is that methods that actually perform drawing operations (such as stroke(), drawImage(), fillText(), etc.) accept a set of graphics attributes to be used for that one operation.

// Draw a colorful triangle with wide lines
canto("canvas_id").moveTo(100,100).lineTo(200,200,100,200).closePath().
    stroke({lineWidth: 15, strokeStyle: "red"});

If you've used SVG you may be familar with the compact path descriptions syntax used in the d attribute of the <path> element. Canto supports this syntax, both with an svgpath() method and with individual single-letter methods like M (moveto), L (lineto), C (curveto) and so on. Here's the SVG version of our triangle:

// Parse an SVG path string
canto("canvas_id").svgpath("M100 100 L200 200 100 200 Z").fill();
// Define path elements with methods with really short SVG-inspired names
canto("canvas_id").M(100,100).L(200,200,100,200).Z().stroke();

Other new API features include relative-coordinate methods like rmoveTo() and rlineTo() and a turtle graphics API. A summary of the API is in the long comment at the top of the canto.js source code.

I've set up a canto-js project at code.google.com and have released it under the MIT license. Comments, bug-reports and contributions are all welcome. Test cases would be particularly nice. I've written a few simple tests myself, and I've run Canto against Phillip Taylor's suite of canvas tests. (Results: in Firefox and Chrome, at least, Canto passes effectively all the tests that the underlying un-canto'ed 2D context pass).

Note that Canto currently uses getters and setters to handle graphics attributes: if you set the lineWidth property on your Canto object, the setter method delegates to the underlying 2D context. IE doesn't support getters and setters, so Canto does not currently work in that browser.

O'Reilly has made a draft of the 6th edition of JavaScript: The Definitive Guide available through their Rough Cuts program. This means that you can buy online access (HTML and PDF) to the current draft of the book today, and then have the print version shipped to you when it is published, for the basic cover price of the book. (See the FAQ for more about how Rough Cuts works).

Online access is through Safari, not directly through O'Reilly, which means that you need to create an account with them in order to do this. But that's a minor hassle if you want the latest version right away.

Part I of the book is finished: complete coverage of ECMAScript 5, along with completely rewritten and modernized chapters on functions, objects and classes.

Part II is being updated for HTML 5 and is still in progress. The first few chapters have been pretty much completely rewritten but a number of chapters are still missing. The Rough Cuts edition will be updated each time I finish a new chapter. If you buy it now you get ongoing access to all the updates.

I've just realized that there is one annoying JavaScript special-case that web developers no longer need to know about, and I don't need to document anymore.

As you know, JavaScript event handlers return false to cancel the browser's default action for the event. Except, that is, for the onmouseover event handler. Because of a bug in Netscape 2.0 (I think) we've always had to return true for that one. Documenting this special case has become a habit for me.

How nice, therefore, to realize that this is now irrelevant. The only default action associated with a mouseover event is to display the destination URL of a link. But to prevent phishing attacks, browsers no longer allow us to cancel this default action (and no longer allow any kind of status line scripting). Since the action can't be cancelled, the return value of onmouseover no longer matters. Hooray!

So I can now just document "return false to cancel". (Of course I still have to document e.preventDefault() for event handlers registered with addEventListener() and e.returnValue=false for event handlers registered with attachEvent(), but this is a step in the right direction.)

For a long time, IE's alternative to the CSS opacity property has been the filter property:

filter: alpha(opacity=50);

If you Google "IE8 opacity", you'll find a number of pages (including one at quirksmode.org) telling you that Microsoft removed support for opacity in a preview release of IE8, and then put it back in, but with a new syntax. These pages assert that in order to achieve the same half-transparent effect as above, you must write (note the -ms- prefix and the quoted value):

-ms-filter:"progid:DXImageTransform.Microsoft.Alpha(Opacity=50)";

As far as I can tell, however, all of these pages are based on preview releases. My (very simple) testing indicates that the old syntax continues to work in IE8, and there is no need to add a second non-standard property to our stylesheets for IE8.

This post at the IE blog backs me up on this (scroll down to the section titled "A Pragmatic Solution") Or at least I think it backs me up. The post tries to explain and justify the initial change to the filter syntax but it just doesn't make much sense.

Someone please correct me if I'm wrong. Are there versions of IE8 in the wild that don't support the old filter syntax?

I'll be speaking tonight in Vancouver, BC about the new features of ECMAScript 5.
Details here.

I'll post the slides after I give the talk.

Update: Here are the slides.

Microsoft has released a testdrive version of IE9 that includes support for:

1. addEventListener()
2. CSS3 Selectors
3. CSS border-radius style for rounded corners
4. SVG (no word on <canvas>, as far as I know, however)

Items 2, 3 & 4 are probably bigger news than item 1, but IE's lack of support for addEventListener() has always really irritated me. Finally, more than a decade after it was standardized, IE will support it. That means that by 2015 we can stop writing code that checks for addEventListener() and falls back on attachEvent().

I've written another version require2.js of my CommonJS module loader require() function. This one has two interesting features.

First, you can "pre-load" modules by mapping the module filename to the module function in the require._module_function object. If you do this, then the module will not need to be loaded. For example:

require._module_functions['math.js'] = function(require,exports,module) {
     // Code for the math module goes here
};

Second, this new version of require() has a require._print hook, which, if set to a suitable function, will print out the text of all modules it loads, wrapped in a function and assigned to the require._module_function map as above. You can even define a require._minimize hook if you want to do code minimization on your modules.

I've defined another script display_requirements.js that defines suitable _print and _minimize functions.

So, here's the upshot. For relatively simple applications that load modules statically at start up, use the require2.js script for loading modules. Its inefficient, but works well during the development phase. Then, when you're getting ready to deploy your application, load the display_requirements.js script after loading require2.js but before you actually call require() anywhere. This will cause a big chunk of code to appear at the bottom of your web page--pre-loaded minimized versions of all the modules you used. Cut-and-paste this code into a new file named requirements.js (or even paste it at the bottom of require2.js) and load the requirements.js script in place of the display_requirements.js script. Now you can continue to use require() as you have always done, but it won't have to hit the network to load your modules.

I haven't done much deployment of real-world web applications, and am not qualified to say whether a system like this would actually be helpful in practice. But it was an easy tweak to my existing code, so there it is.

I've implemented the CommonJS Modules 1.0 specification with the code in this file. It appears to pass the compliance tests when run in Firefox and Chrome on Linux, and also when run standalone in Tracemonkey, Rhino or V8.

Note that this implementation does not use the namespace probing technique I described in my previous post.

Update: One of the things that really surprised me when testing my implementation was to discover that the CommonJS spec requires (this is not explicit in the specification text, but it is explicit in the conformance tests) the require() function to return the actual exports object of a module, and not make a defensive copy of it.

Suppose a program includes module A which includes modules B and C. Module C can require B and then add, replace, or remove methods from B's API. Later, when the program includes Module B directly, it will get the modified version of B. In order to correctly use this modified module B, the programmer will have to read the documentation for module C!

Update 2: There are great comments to this post, including a link to Luke Smith's blog post that argues that the synchronous nature of CommonJS modules is not a good fit for client-side scripting. In response I wanted to make clear that I posted this code because I thought it was interesting, not because I think that client-side programmers should go out and start using it right away.

It has become common in modern JavaScript programming to use functions as namespaces. If you put your code inside a function, then your variables and functions are local to the containing function and do not clutter up the global scope.

var value = (function() {  // Wrapper function creates a local scope or namespace
    // your code goes here
    return value;  // Export a value from the namespace
)());  // Invoke the wrapper function to run your code      

Now suppose that you have some JavaScript code as a string--you've just loaded it using XMLHttpRequest, for example. You're going to evaluate the code, and you might want to evaluate it in a namespace so that it doesn't define functions and variables in the global scope. This is easy: just wrap it in a function before evaluating it. In this case, the Function() constructor is even more handy than the eval() function:

var code = ....;  // A string of JS code to evaluate
var f = new Function(code);   // Wrap it in a function
f();    // And run the function

The problem with doing this is that the function creates a sealed namespace and we can't see what is inside. If the code defines something useful like a function or a class, we can't access it, and it does us no good.

Here's a trick I've just discovered. (I'm sure someone else has thought of this, but I haven't seen it used or described elsewhere). Before you wrap your code in a function add this line to it:

return function(s) { return eval(s); };

Now, when you invoke the wrapper function, it returns this evaluator function to you. The returned function evaluates a string in the scope of the namespace, so you can use it to peek into the namespace and extract whatever values you want!

If your string of code defines a constructor function named Set() that you want to use, you can run the code in a namespace and then extract f from the namespace like this:

var code = readFile("Set.js");  // A string of JS code to evaluate
// Define and invoke a wrapper function with special suffix code.
// The return value is a namespace evaluator function and we treat
// it as a namespace object.
var setns = new Function(code + "return function(s) { return eval(s); };")(); 
var Set = setns("Set");  // Import the Set function from the namespace.
var s = new Set();  // Use the class we just imported

And what if there are 3 values you want to extract from the namespace?

// Extract an object containing 3 values from the namespace
var sets = setns('{Set:"Set", BitSet:"BitSet", MultiSet:"MultiSet"}');
var bs = new sets.BitSet();

I've defined a namespace() function for loading code and doing this kind of namespacing automatically:

Google has open-sourced the javascript library and optimizer they use in gmail and other web applications. They call it "Closure" and you can read about it here.

While the optimizer looks very cool, I think the library is most impressive. Its a really large code base, and at least some of it has been thoroughly field-tested in gmail and similar applications. You can get the code like this:

svn checkout http://closure-library.googlecode.com/svn/trunk/ closure-library-read-only

The closure library is intended to be used with the closure optimizer which removes unused code, so the APIs are broad with lots of methods--there is no sense that the closure developers were skimping on API in order to pack everything into a small download bundle. Also, and perhaps for the same reason, the code is not full of micro-optimizations. Compared to the jQuery and YUI code (for example) the Closure code is straightforward and easy to understand.

Function objects in JavaScript are callable: you can invoke them. Other, non-function objects are allowed to be callable, however. Host objects in IE (things like Window.alert()) are callable, but are not native function objects. (Other browsers implement DOM methods as true native function objects). Also, a number of browsers have followed Firefox's lead in making RegExp objects callable even though they are not functions. Although you can invoke any callable object like a function, the difference is that callable objects don't have function methods call() and apply() (and bind() in ECMAScript 5).

Today, the typeof operator returns "function" for true function objects, and returns "object" for IE's callable host objects. Most browsers return "object" for callable RegExps, but Safari returns "function", and Google Chrome is likely to follow Safari's lead on this. If you want to be sure that something is a true function (and not a regexp) you can use something like this:

function isFunction(x) { 
    return Object.prototype.toString.call(x) === "[object Function]";
}

There is not today a reliable way to write an isCallable() function, however.

Things change in the ECMAScript 5 specification. The typeof operator is required to return "function" for any native or host object that is callable. When IE implements the spec, we can expect "typeof window.alert" to evaluate to "function". The problem is that browsers like Firefox and Opera that have callable regexps are unlikely to implement the spec: the typeof operator on a regular expression will continue to return "object" for those browsers. The committee writing the specification was aware of this problem, but ran out of time to fix it.

So today typeof x === "function" is close to an isFunction() test, but it fails for regular expressions in some browsers. In ECMAScript 5, typeof will be close to an isCallable() test, but it will fail for regular expressions in some other browsers.

Fortunately, the isFunction() test above should continue to work in ECMAScript 5, and
there is a way to write a reliable isCallable() function in ECMAScript 5. It relies on the fact that the Array.prototype.forEach() method checks its argument for callability even when invoked on an empty array. (So this isCallable() function assumes that browser vendors implement the Array.forEach() method as specified.) Here it is:

Object.isCallable = function(o) {
    // Array.prototype.forEach throws TypeError for non-callable arguments
    try {
        [].forEach(o);  // o will never be invoked, but it will be tested for callabilty
        return true;
    } catch (x) {
        if (x instanceof TypeError) return false;
        else throw x;
    }
};

I was recently writing some documentation for the Array.forEach() method (part of ES5, but most browsers other than IE support it now) and worrying about the fact that there is no clean way to terminate the iteration prematurely. Nothing like the break statement, that is. If you really want to get out of the loop, the function you pass to forEach() has to throw something. And the forEach() method won't catch it for you, so you've got to write your own try block.

Then, when working with the new array predicate method Array.some(), I realized that we don't have to think of it as a predicate method. If we ignore the return value, it is an iterator method that works just like Array.forEach() except that if your function returns true (or any truthy value) then the loop terminates. So inside of the function you pass, a plain return statement with no value is like using a continue statement. And "return true" is like a break statement, causing the loop to terminate. The implicit return that occurs at the end of the function body returns undefined, which is like returning false--it keeps the loop going.

The Array.every() method is not so useful this way: you have to explicitly return a truthy value to keep the loop going, so an implicit return at the end of the function body would act like a break statement.

The problem I see with using some() in this way is a stylistic one: the name really doesn't look like the name of an iterator the way that "each" and "every" do.

Yesterday, I posted an entry about a clever way to implement string multiplication in JavaScript using Array.prototype.join()

In comments, redraiment challenged me, suggesting that an implementation based on string doubling would be more efficient. Sure, I thought, for really large values of n, but surely for small n, using the native join() method would be better, wouldn't it?

It turns out that writing a good algorithm is better than being overly clever (at least in these days of really good JIT interpreters). Here's my new string multiplication code:

String.prototype.times = function(n) {
    var s = this, total = "";
    while(n > 0) {
	if (n % 2 == 1) total += s;
	if (n == 1) break;
	s += s;
	n = n>>1;
    }
    return total;
};

By my simple benchmarks, this implementation is significantly faster than using join(), even when only multiplying by 1 or 2. I've tested it in Firefox 3.5, IE 8 and Safari 3.

In Ruby, the "*" operator used with a string on the left and a number on the right does string repetition. "Ruby"*2 evaluates to "RubyRuby", for example. This is only occasionally useful (when creating lines of hyphens for ASCII tables, for example) but it seems kind of neat. And it sure beats having to write a loop and concatenate n copies of a string one at a time--that just seems really inefficient.

I just realized that there is a clever way to implement string multiplication in JavaScript:

String.prototype.times = function(n) {
    return Array.prototype.join.call({length:n+1}, this);
};

"js".times(5)   // => "jsjsjsjsjs"

This method takes advantage of the behavior of the Array.join() method for arrays that have undefined elements. But it doesn't even bother creating an array with n+1 undefined elements. It fakes it out using and object with a length property and relies on the fact that Array.prototype.join() is defined generically. Because this object isn't an array, we can't invoke join() directly, but have to go through the prototype and use call(). Here's a simpler version that might be just as efficient:

String.prototype.times = function(n) { return (new Array(n+1)).join(this);};

When you call the Array() constructor with a single numeric argument, it just sets the length of the returned array, and doesn't actually create any elements for the array.

I've only tested these in Firefox. I'm assuming that either is more efficient than anything that involves an actual loop, but I haven't run any benchmarks.

By now, many of us have gotten used to using closures in JavaScript to define a scope that holds private variables and utility functions so that we don't have to put these in the global namespace. The idiomatic code looks like this:

(function() {
      var private_var;    // Visible only inside this function
      function helper_function() { ... }
      
      // export an object or function to the global namespace
})();  // Invoke the outer function

The outer function exists only to create a scope to hold our internal variables. It has no name and is invoked exactly once, immediately after being defined. The fact that this is a function expression (rather than a statement-like function declaration) means that we can invoke it immediately after defining it. The parentheses before the function keyword and after the closing } are required because otherwise the JavaScript interpreter would think that this was a function declaration and would complain about the missing function name.

That unusual opening parenthesis before "function" serves another important purpose in this idiom. It alerts us to the fact that this function is being used idiomatically, that it exists solely to create a scope and that it is going to be invoked immediately.

So far, so good. But I've finally gotten around to reading Douglas Crockford's JavaScript: The Good Parts and it has made me think that an additional explicit naming convention would be helpful. (As an aside, Crockford's short book is worth a read, though I find that I disagree with some of his coding conventions. I'm tempted to write a review titled "JavaScript: The Good Parts: The Good Parts"...)

When the function keyword is the first token in a new JavaScript statement, the interpreter expects to see a function declaration, not a function expression. That's why we needed the idiomatic parentheses in the code above. But when the function keyword is used as part of an assignment or as an argument to some other function, those parentheses are not required. Crockford's book includes code like this (page 37):

var myObject = function() {
    var value = 0;

    return { // 7 lines of code omitted here
    };
}();

When I first read the assignment statement it appears to me (despite the name of the variable) that it is a function value being assigned. In fact, however, the function is merely there to establish a scope for private variables. The function is invoked as soon as it is defined, and it is the return value of the function that is assigned. The problem is that I don't realize this until I read all the way down to the end of the function. Appendix E takes this to an extreme. The code begins:

var json_parse = function() {

It looks like we're creating a function and assigning it to the variable json_parse. But five pages later we see:

}();

Now we realize that the value assigned to json_parse is not the function we thought it was, but the function returned by that function.

Another example appears on page 40:

String.method('deentityify', function() {
   // 25 lines omitted
}());

It appears at first that we're passing a function as the second argument of the invocation of String.method. It is not 'till we read all the way through this function that we realize that we're passing the result of invoking the function.

So, how can this code be improved? One way would be to use the idiomatic parentheses around function expressions that are going to be immediately invoked even when they are not necessary. That would turn the code above into this:

String.method('deentityify', (function() {
   // 25 lines omitted
})());

I think that would be helpful, but I think we can do better. Function expressions are allowed to have names, and those names are only visible within the body of the function (allowing such a function to invoke itself recursively, for example). So let's say that when we're going to define a function for the purpose of creating a scope we make that explicit by giving it a dummy name like "scope" or "closure" or "invocation". This results in code like:

var myObject = function scope() {
   // code omitted
}();

String.method('deentityify', function invocation() {
   // 25 lines omitted
}());

Your thoughts are welcome in the comments. Has anyone else proposed a convention like this? What's the best name to use for these functions?

Update: I suppose we could also simply adopt a comment-based convention:

var myObject = /* return value of */ function () {
   // code omitted
}();

String.method('deentityify', /* result of */ function() {
   // 25 lines omitted
}());

Update 2: It turns out that Douglas Crockford is about 2 months ahead of me. In March he updated jslint to (optionally) issue warnings about immediate invocation of function expressions unless the entire invocation appears in parentheses, and also to warn if a function is parenthesized and is not immediately invoked. So Crockford's convention looks like the following:

var myObject = (function () {
   // code omitted
}());

String.method('deentityify', (function() {
   // 25 lines omitted
}()));

Note that Crockford wraps the invocation in parens, not just the function. That is, he uses ()) at the end instead of (the more commonly used) )(). He has said that he'll update his book to follow these conventions in the next printing.

Per a post today on the es-discuss mailing list, the next version of the JavaScript standard will be ECMAScript 5. This version was previously called ECMAScript 3.1, and is a relatively small and long-overdue update to the language. Version 4 of the standard has been in the planning stages for 10 years or more, but those plans have been scrapped. To avoid confusion, with those old plans, however, there will be no version 4 of the standard.

The comments on my last post about method chaining in JavaScript were spectacular, and I want to publicly thank all who took the time to read my code and think about it. The final version of the code (which you can see below the fold) is much stronger thanks to their comments.

In the 5th edition of my JavaScript book I made the embarrassing mistake of recommending a constructor and method chaining technique that only works for shallow class hierarchies--it works when class B extends A, for example, but not when C extends B and B extends A.

The technique I recommended was to put a superclass property in the prototype object of a class, and then to chain to a superclass constructor by calling this.superclass(). To see why this fails, imagine that we're creating an instance of class C (which extends B which extends A). The constructor C() chains to B() by calling this.superclass(). The constructor method B() is invoked on the same instance of C, however, so when it attempts to chain to A() by calling this.superclass(), it just ends up invoking itself. This incorrect chaining technique is discussed in sections 9.5.1 and 9.5.2, and is also used in example 9-10 and 9-11 at the very end of the chapter. I blogged about this mistake and a possible workaround almost two years ago.

Now, however, O'Reilly is preparing to do a reprint of the book, and I have an opportunity to fix this mistake. Below is a revised code from examples 9-10 and 9-11. I've renamed the defineClass() method to Class() and have modified it so that it automatically does constructor chaining (I was inspired by dojo for this change). More importantly, I've simplified method chaining by defining a global method named chain() for method chaining. If a method overrides a method defined by a superclass (or a "mixin" class) it can invoke that overridden class by invoking chain() like this: chain(this,arguments). (The second argument must be the arguments array of the overriding method, and the first argument must be the object on which that method was invoked.)

The code is below the fold. I think this is interesting JavaScript, and I'd love to have it checked for errors before it goes into print again... Please leave a comment if you think it could be improved! Update: comments are now closed; spammers have struck.

The <canvas> tag supports all of the normal line attributes (width, join style, cap style, miter limit) except for dashes. Given that Apple's original implementation of <canvas> seemed a little bit rushed, I have always assumed that this was a simple oversight, and that it would soon be corrected in newer implementations.

I did, however, submit a comment about it to WHAT-WG. I was quite surprised to find out that they have no intention of supporting dashed lines. I've argued and argued with Ian Hickson about this, but he is unmoved. Supporting dashed lines would be non-trivial, he believes, and there haven't been enough requests for them.

So, this post is a public service announcement. If you were assuming, as I did, that dashed lines would be supported by the official specification of the <canvas> tag, you should be aware that they will not be. If, like me, you think that it is a mistake to define a graphics library that cannot natively draw dashed lines, then you can submit a comment to that effect to whatwg@whatwg.org.

Julien Royer sent me an email today following up on this post. Try the following code in IE:

<script>
window.onload = function() {
  alert(window.event == window.event)
}
</script>

When I run this in IE6 on a Windows XP machine, I get false. This is just weird. The window.event object thinks that it is not equal to itself.

This probably does not have any impact on real-world code. I bring it up, however, because in that last post about IE and events, my commenters and I decided that the event object passed to an event handler registered with attachEvent is not the same as window.event. The code shown here calls that into question. Perhaps they are the same objects, but the weird behavior of window.event causes them to appear unequal.

In the 5th edition of my book, I propose a convention for constructor chaining:

• When you define a class B as a subclass of A, set B.prototype.superclass=A
• Then, in the constructor B(), you can chain to the superclass constructor with the natural syntax this.superclass()

Reader Bruce Wallace has just pointed out a serious flaw in this approach: you can only use it once. If you use this trick twice in the same inheritance hierarchy, you end up with infinite recursion. (If you don't see why, you can try to work out an example that includes C as a subclass of B. Or just forget it, since I think I have a better approach below.)

I'm feeling pretty embarassed to have suggested this defective method of chaining, and to have committed my suggestion to print! However, I hope to make up for it by proposing a better alternative. Here is my new convention:

• First, define the following function:

  function super(o, args) {
      if (args.callee.superclass)
  	return args.callee.superclass.apply(o, args);
  }
  

• For each constructor B that subclasses A, set B.superclass=A;
• Do the same thing if a method of B overrides a method of A. For example:

  B.prototype.toString.superclass = A.prototype.toString;
  

• In the B() constructor, chain to your superclass constructor like this:

  super(this, arguments)
  

• And in a method like B.toString(), chain to A.toString() with exactly the same syntax.

This works because arguments.callee is the function that is currently running. Note that this chaining convention passes all the arguments of the subclass constructor or method to the superclass constructor or method, even if some of them are unused. So it won't work if a subclass constructor expects its arguments in a different order than the superclass constructor.

Note that the work of setting the superclass property on constructors and methods can be done by a suitable class definition utility method (such as example 9-10) in my book. This means that when writing your constructors and methods, you don't have to worry about linking the chain together; you can just call the super() method whenever you want to.

Anyone see any flaws in this approach? Comments? Suggestions? There is a simple test case defining a class hierarchy C extends B extends A below, but I haven't tried to test this extensively yet, and I''ve only run it in Firefox. Firefox is letting me use the function name "super" even though that is one of the identifiers that is unused but nominally reserved by ECMAScript v3. I choose super because Java uses that, but "chain" would probably be a safer choice.

Here is my test code:

// Magic constructor and method chaining function
function super(o, args) {
    if (args.callee.superclass)
	return args.callee.superclass.apply(o, args);
}

// Class A
function A(a) { this.a = a; }
A.prototype.toString = function() { return this.a + "" };

// Subclass B
function B(a, b) {
    super(this, arguments);
    this.b = b;
};
B.superclass = A;
B.prototype = new A();
B.prototype.toString = function() {
    return super(this, arguments) + this.b;
};
B.prototype.toString.superclass = A.prototype.toString;

// Sub-subclass C
function C(a,b,c) {
    super(this, arguments);
    this.c = c;
}
C.superclass = B;
C.prototype = new B();
C.prototype.toString = function() {
    return super(this, arguments) + this.c;
}
C.prototype.toString.superclass = B.prototype.toString;

// Create an instance of C and call toString() should display "123"
var c = new C(1, 2, 3)
alert(c.toString());

I was porting some JavaScript code that used the TrimPath template library. The templating system was too much for my needs, so I came up with the simple little class below. It simulates a Java-style output stream. Pass the id of a DOM element to the constructor to get started. Call the clear() method first, if necessary. Then call the write() method as many times as needed. Finally, always remember to call flush() to force your output to appear.

// A little class for outputting HTML into a DOM element
function DOMStream(id) {
    this.elt = document.getElementById(id);
    this.buffer = [];
}
DOMStream.prototype.clear = function() { this.elt.innerHTML = ""; };
DOMStream.prototype.write = function() {
    for(var i = 0; i < arguments.length; i++) this.buffer.push(arguments[i]);
};
DOMStream.prototype.flush = function() {
    this.elt.innerHTML += this.buffer.join("");
    this.buffer.length = 0;
};

If it is not obvious, an important point of this class it to avoid inefficient repeated string concatenation. This is why it uses an array as a buffer, and why the write() method accepts multiple arguments--so you don't have to concatenate them yourself.

Here's a sample usage:

var out = new DOMStream("placeholder");
out.write('<h2>', section_title, '</h2>');
outputTable(out, data);  // Function defined elsewhere
out.flush();

With this class defined, I ported the trimpath template to equivalent JavaScript functions, and ended up with a modest reduction in the number of lines of code (not counting the substantial reduction in lines of code I got by no longer using the templating library.)

The 5th edition of my book was not the only notable JavaScript book released this summer. I'd like to draw your attention to these two as well:

The authors of these two books, Christian Heilmann and Peter-Paul Koch, were kind enough to serve as technical reviewers on my book, and were also kind enough to send me copies of their own books.

I cannot actually review their books, however, since I have not had the time to read them. Nevertheless, the author's reputations speak well for the books. Both Christian and Peter-Paul are active web developers, members of the DOM Scripting Task Force (of the Web Standards Project) and advocates for unobtrusive scripting, accessibility, and other modern scripting practices.

Neither of these books aims to be "definitive" in the way mine does, but both are written by authors who have a lot of valuable experience with todays best practices. Since I haven't read the books myself, I can't tell you whether they share that experience effectively in these books. But I hold the authors in high esteem and want others to be aware of their work.

Musician Weird Al Yankovic has released a music video for his song "White and Nerdy". The cover of my JavaScript book appears in it. (I can't tell if its the fifth edition, though.)

You can find the video at the musician's MySpace site, and also by searching YouTube.

The JavaScript reference is 1 minute 14 seconds into the video.

At another blog I saw someone comment that they'd been looking for a list of what is new in the 5th edition of JavaScript: The Definitive Guide. I guess I haven't blogged about this yet. If you were looking at the book in a bookstore, you'd just turn to the Preface and read the section titled "What's New in the Fifth Edition". But if you're considering an online purchase, you can't do that. So, here's what I say about the new edition in the preface:

In yesterday's post, I promised to stop plugging my new book. But I've got a copy in my hands now, and I just can't stop myself. This post will contain only trivia, without links or exhortations to BUY IT NOW!

The first thing I noticed about 5th edition is that it is thinner than the 4th edition. Apparently it is printed on thinner (also noticeably whiter) paper than the 4th edition. Rest assured, however, that this new edition is longer (and better) than the old. The 5th edition is 10% longer (in page count). It has 92% more examples (125 examples instead of 65) and the new examples are 169% longer (lines of code) than the old ones.

But here's my favorite fact about the 5th edition: if you count the regular numbered pages, the roman-numeral preface pages, and the unnumbered filler pages at the back, the book is exactly 1024 pages long. One kilopage: I couldn't be prouder!

The 5th edition of my JavaScript book is finally shipping! (From amazon.com, anyway. Amazon.ca and amazon.co.uk don't seem to have it yet.) Interested?

1. Read a sample chapter.
2. Study the examples.
3. Order it from Amazon: see the ad in the left column.

I think this is the last time I'll plug my book on this blog!

O'Reilly has posted a PDF file of Chapter 21, JavaScript and XML, from my book

This chapter shows you how create, load, parse, serialize, transform and search XML documents using JavaScript. It includes a portable HTML templating system like IE's data island stuff.

Examples for this chapter, and all chapters of the book are also available.

I'm told that the book will be in bookstores by the end of the month. And you can pre-order it from Amazon today:

My JavaScript book has been sent to the printer! I'm told it should be in bookstores by the end of the month. You can always pre-order it, though:

Meanwhile, you can get your hands on the examples from the book now.

I just had the chance to review the copyright page of the 5th edition of my JavaScript book and noticed that its publication date (August, 2006) is exactly 10 years after the August, 1996 publication date of the first, "beta" edition of the book.

Thus inspired, I put together the following timeline of the JavaScript language, the client-side DOM, and of JavaScript: The Definitive Guide. I'm sure it is incomplete: additions in the comments are very welcome.

1996 March

JavaScript 1.0 released in Netscape 2.
document.write() and form manipulation are possible

1996 August

JavaScript: The Definitive Guide beta edition

1996 August

JavaScript 1.1 released in Netscape 3.
Adds array support to the core language.
Image rollovers become possible.

1996 August

IE 3 released: JScript 1.0/2.0

1997 January

JavaScript: The Definitive Guide second edition

1997 June

ECMAScript v1 standardized

1997 June

JavaScript 1.2 released in Netscape 4.
Adds switch and regexps to the language.
<layer> tag supports rudimentary DHTML

1997 October

IE 4 released: JScript 3.0
DOM based on document.all[] allows DHTML

1998 June

JavaScript: The Definitive Guide third edition

1998 June

ECMAScript v2 standardized (minor corrections only)

1998 October

JavaScript 1.3 released in Netscape 4.5.
Compliant with ECMAScript v1

1998 October

W3C publishes DOM Level 1 standard

1999 March

IE 5 released: JScript 5.0

1999 December

ECMAScript v3 standardized

2000 July

IE 5.5 released: JScript 5.5
partial support for W3C DOM

2000 November

JavaScript 1.5 released in Netscape 6.
Compliant with ECMAScript v3
Basic support for DOM, <layer> tag abandoned

2000 November

W3C publishes DOM Level 2 standard

2001 October

IE 6.0 released: JScript 5.5
substantial support for W3C DOM, except Events module

2002 January

JavaScript: The Definitive Guide fourth edition

2002 June

Mozilla 1.0 released

2004 November

Firefox 1.0 released

2005 February

Jesse James Garrett coins the term Ajax

2005 July

DOM Scripting Task Force forms,
publishes The JavaScript Manifesto

2006 August

JavaScript: The Definitive Guide fifth edition

Two final notes. First, Amazon is still saying that they'll be shipping copies of the book around August 1st. I can say for sure that the book won't be ready by the beginning of August. I don't actually know much about the printing process, but it seems reasonable to suppose that the book will be available later in August.

Second, I've lost my only copy of the 10-year old Beta edition of my book. If you have a copy, in reasonable condition, with the word "BETA" stamped across the rhino, and are would like to swap it for a brand-new copy (signed, if you want) of the 5th edition, send me an e-mail and we'll arrange a trade.

Update: comments closed. Comment spammers have discovered this entry.

Jitsu is a new web application framework. They call themselves "a next-generation user interface toolkit for building rich web applications"

Jitsu was developed by ATTAP, but has been open-sourced. The website is slick, and there is a well-written tutorial. Its a complete application framework, not something you can just add onto an application, so you really have to adopt it from the start. But its got a lot of nice features, and particularly strong bi-directional data-binding capabilities.

I haven't actually used it myself, but it looks very promising.

At long last, the 5th edition of JavaScript: The Definitive Guide is complete! Amazon is accepting pre-orders, and O'Reilly should be releasing a sample chapter soon.

I'm pleased with how this edition came out. In Part I, which covers the core language, the material on functions, closures, and classes has been strengthened, and there are new chapters on namespaces and on embedding JavaScript in Java 6 programs.

The biggest changes are in Part II of the book, which covers client-side JavaScript. Two new chapters on scripted HTTP and XML processing cover Ajax. Another new chapter on scripted client-side graphics covers the <canvas> tag, SVG, and alternatives. The chapter on cookies has been extended to discuss other client-side persistance mechanisms. And the chapter on scripting Java applets has been extended to also cover scripting Flash movies.

All these changes should bring the book up-to-date for "Web 2.0"

As far as I can tell, the DOM and HTML specifications are silient on the issue of when DOM elements become available to scripts embedded within the document. That is, if you have an element <div id="foo">, when can you call getElementById('foo') and be sure that you'll get a valid result?

To be safe, we typically wait for the onload() event. (Although the specifications don't explicitly guarantee that, either.) It appears to be common practice for browser vendors to make the DOM available while it is being parsed. That is, any script in the body of a document can access any elements that appear before it in the document.

Can anyone with more knowledge of the specifications, or with more experience writing production code comment on this issue?

This is, of course, related to the "onload problem" which Dean Edwards purports to have solved. In the comments to that post, Dean asserts that placing scripts at the very end of the body does not guarantee that those scripts will have access to the whole document. Dean's experience carries a lot of weight, but he does actually have code that demonstrates that this technique is unreliable.)

Comments are open.

[Update: this may be a false alarm. See how I resolved the problem.]

I'm still trying to find a workaround to the Flash persistence security hole that opens up when you script your flash movies from JavaScript.

What I want to try now is to code my ActionScript so that it compares the URL of the Flash movie of which it is a part (something like _root.movieClip._url) to the URL of the web page in which the movie is embedded. If they are not the same, then the actionscript knows that it is being hotlinked, and can refuse to return any persistent data.

My problem is that I don't know the ActionScript programming environment or API well, and I can't find a way to find the URL of the document in which the movie is embedded. It seems like there ought to be a way, but I don't know...

Anyone know how to do it?

[Update: this may be a false alarm. See how I resolved the problem.]

I've been looking for a way to work around the security hole in the flash persistence mechanism that I described in the previous post.

Since the hole occurs when one site hotlinks a Flash movie on another site, it seemed to me that the obvious solution would be to prevent hotlinking. So I set up a .htaccess file using the Apache mod_rewrite module and the HTTP_REFERER header. It worked: offsite links to the SWF file were prevented.

But this wasn't good enough. I didn't take caching into account. Suppose user has just visisted Site A, and that site uses a scriptable Flash movie to store persistent data locally. The user then visits Site B, which is the malicious one. Site B hotlnks the movie on Site A and scripts it to gain read access to the persistent data. My .htaccess file on the server was supposed to prevent this kind of hotlinking. But the server never gets involved: Site B doesn't need to ask the Site A server to send it the movie: the movie is already in the browser's cache, and Site B can read Site A's persistent data with no trouble.

Anyone have a solution to this problem? I'm ready to give up and declare that Flash-based persistence is only good as a cache for information that is already publically available on the network.

And as a meandering aside, let me say that there are some interesting uses of it there. I gather that XMLHttpRequest bypasses the browser cache, at least in certain browsers. Flash persistence could be used to create a persistent cache for XMLHttpRequest. This might be useful for a web site that had a bunch of pages that all did XSLT using the same stylesheet, which was retrieved with XMLHttpRequest.

Adam Vandenberg has an XMLHttpRequest caching solution but the cache only persists for the lifetime of the current page. Flash persistence might be an interesting addition to his caching code. (Of course, we'd have to confirm that this Flash security hole only gives read access to the cached data; otherwise any website could corrupt the Flash cache of any other.)

[Update: this may be a false alarm. See how I resolved the problem.]

Inspired by Brad Neuberg's AMASS project, I've been experimenting with using the Flash plugin as a client-side persistence mechanism, as an alternative to cookies. It allows the storage of much more data, and that data doesn't have to be uploaded to the server for every webpage that uses it. This seems important, and projects like AMASS seem like they have breakthrough potential.

However, I've discovered something I'm worried about. Persistence in Flash is done with the SharedObject class. By default persistent data is not shared among Flash movies: it is only accessible to the movie that stored it. SharedObject has a way to loosen this up, like the path attribute of a cookie. You can specify that all Flash movies in the same directory can share persistent data, for example. Or you can specify that all Flash movies from the same website can share data.

This may be fine when the data is locked inside the Flash movies. But when Flash movies are scriptable, we run into problems. Both AMASS and my own experiments with Flash-based persistence use a simple invisible movie to make the necessary calls to the SharedObject. Both script the Flash plugin with JavaScript so that we can persist client-side JavaScript data.

And this is where the problems arise. Flash governs access to the data based on the URL of the movie; it doesn't seem to care about the URL of the document within which the movie is embedded. Suppose, therefore, that Site A uses a scriptable Flash movie to persistently store some data from JavaScript. Site B can hotlink that scriptable movie right off of Site A, and the JavaScript code on Site B can read the data stored by the JavaScript code on Site A. (In my experiments, Site B cannot overwrite or delete the data stored by Site A, but it can read it.)

My testing was done using Firefox 1.0.6 using the Linux version of the Flash player 7.0 r25.

Update:I have now obtained the same results using Firefox 1.02 and Flash version 8 on a Windows machine.

I think this is a problem. I suspect that it is a big problem and that Macromedia needs to fix it. I'm only beginning to dabble in Flash and just learning about how to script Flash movies. I don't know if this security problem will affect movies that use SharedObject but do not explicitly open themselves up to scripting of that SharedObject. It could be that if you play a Flash-based game at Site C, site D could figure out a way to read your high score in that game.

Your thoughts are welcome in the comments, especially if you understand Flash and Flash scripting better than I do!

Sparklines are a new graphical concept championed by Edward Tufte, author of a number of very cool books that you have probably already heard about. Sparklines are an idea from his forthcoming book, and he explains them in a draft chapter from that forthcoming book at sparklines.org.

If you're using a browser that understands the data: URL, here's an example of a sparkline:

Server load: 16.

Note, that this particular sparkline is hard-coded in this blog entry. The demos below show how they are dynamically generated.

As part of my newfound obsession with client-side drawing techniques (I really think it could change the web) I've created a way to generate sparkline images using client-side Java and JavaScript. This is proof-of-concept stage, but if you're running Firefox and have the Java plugin installed, click on these links to try out the demos:

• spark.html: this demo uses LiveConnect to script Java directly. It does not use an applet at all. This means it will not work in IE. I don't know about other browsers' support for this kind of direct scripting of Java classes. This demo returns a PNG image encoded in a data: URL. This also means it won't work in IE, since IE does not support the data: scheme.
• spark2.html: this demo uses a SparklineGenerator applet to create the PNG image, which it returns as a javascript: URL. This is supposed to be for compatibility with IE, but it still doesn't work in that browser for me. It could probably be made to work somehow... This demo also moves toward an unobtrusive JavaScript technique in which sparklines are coded with a new <sparkline> tag, which gets replaced by the image after document load by JavaScript and the applet.

You'll need to understand both Java and JavaScript to make use of this stuff. But if it interests you, you can use my code for any purpose as long as you attribute it to me.

In my previous entry, I asked about the possiblity of using Flash as a vector graphics engine, for scripting by client-side JavaScript code. With help from Geoff Stearns and his blog Deconcept I've figured out how to do it.

I've figured out how to do it in Flash 8, at least. Flash 8 has a new ExternalInterface API that transparently makes ActionScript methods callable from JavaScript. (It is possible, though not nearly so simple to do similar things in earlier versions of Flash.) Here is my trivial ActionScript file, which I named Canvas.as:

import flash.external.ExternalInterface;

class Canvas {
    static function main() {
        Stage.scaleMode = "noScale";
        Stage.align = "TL";
        ExternalInterface.addCallback("beginFill", _root, _root.beginFill);
        ExternalInterface.addCallback("beginGradientFill", _root,
                                      _root.beginGradientFill);
        ExternalInterface.addCallback("clear", _root, _root.clear);
        ExternalInterface.addCallback("curveTo", _root, _root.curveTo);
        ExternalInterface.addCallback("endFill", _root, _root.endFill);
        ExternalInterface.addCallback("lineTo", _root, _root.lineTo);
        ExternalInterface.addCallback("lineStyle", _root, _root.lineStyle);
        ExternalInterface.addCallback("moveTo", _root, _root.moveTo);
    }
}

I compiled this code with the open-source mtasc ActionScript compiler like this:

mtasc -swf Canvas.swf -main -version 8 Canvas.as

This produced the file Canvas.swf which you can download by clicking the link.

The ActionScript code doesn't do anything except make the Flash drawing API accesible to JavaScript. If you embed this SWF file in your web page, it is simply blank, until you draw something to it yourself. Here is a demo:

<embed src="Canvas.swf" quality=high bgcolor=#FFFFFF width="400" height="200"
name="canvas" align="" type="application/x-shockwave-flash"
pluginspage="http://www.macromedia.com/go/getflashplayer">

<p><button onclick="draw()">Draw in the Canvas</button>
<script>
function draw() {
    document.canvas.beginFill(0xAAAAFF, 100);
    document.canvas.lineStyle(5, 0x0000FF, 100);
    document.canvas.moveTo (10, 10);
    document.canvas.lineTo (200, 100);
    document.canvas.lineTo (10, 190);
    document.canvas.lineTo (10, 10);
    document.canvas.endFill();
}
</script>

This demo uses the <embed> tag instead of the <object> tag, so it won't run in IE. But I have tested it in Firefox, and it works. But only if you have Flash 8 installed. And Flash 8 is not available for Linux yet. If you've got Flash 8 running in Firefox (or Safari?) give the demo a try!

Comments, suggestions, improvements, or whatever are very welcome. I plan to tweak Canvas.as a bit to expose methods for adding text, but I think that will be straightforward. I was disappointed to discover that the Flash drawing API is not nearly as robust as that of the <canvas> tag or of SVG of VML. In particular all curves must be done with quadratic beziers. Cubic beziers, circles, arcs and so on must all be approximated with a single curveTo() function. So it might be worth adding some basic shape-drawing primitives (circles, rounded rectangles, etc) to Canvas.as as well.

Feel free to use this code (and the swf file) for any purpose, with attribution.

I know very little about Flash, but it seems to me that it would make a good substitute for the <canvas> tag as well as SVG and VML. From a basic perusal of the Macromedia web site, it appears that it is possible for client-side JavaScript to communicate with the ActionScript interpreter in the Flash plugin. If so, JavaScript ought to be able to get ActionScript to draw lines, curves, and do other vector-graphics type things.

ActionScript has an eval() function. I would think that it would be possible to create a tiny SWF file that does nothing more than define a hidden TextField object and an onChanged event listener for that field. The event listener would take the new value of the TextField and pass it to eval(). Client-side JavaScript would then set the value of the TextField with the TSetProperty method that is exported by the Flash player. And voila! Client-side JavaScript can then make pass arbitrary ActionScript commands to the Flash Player. This gives us scriptable client-side vector graphics that work anywhere Flash is installed.

Can anyone tell me why this wouldn't work? Is anyone already doing it? If you make this work and release your code openly, you'll get a link and an attribution in the next edition of my JavaScript book!

I think 2D graphics are a lot of fun, and have always loved the Java2D API. How cool is it, then, that Safari includes and Firefox 1.5 will include a new <canvas> tag that exposes a drawing API very, very similar to the Java2D API?

The canvas tag has the potential to revolutionize a lot of web applications. But for this to happen, it needs to be ported to browsers other than Safari and Firefox 1.5. I'm sure it can be done for IE, somehow, but I don't want to tackle that right now.

Instead, my contribution is to begin implementing the canvas tag for Firefox 1.0 using a Java applet and Mozilla's XBL binding language. This is just at proof-of-concept stage, but it is usable enough to make simple demos work. The java applet and XBL files are here. The canvas.html file in that directory is a simple demo. It doesn't seem to work online, but if you copy the files to a local directory, it should work. (I suspect, but don't know, that XBL bindings have to be installed locally to work for security reasons.)

To use this, compile the .java file to a .class file. Then define a simple stylesheet that tells Firefox to use the canvas.xbl file to implement the canvas tag. This is what I use:

<style>
canvas {
    -moz-binding: url('canvas.xbl#canvas');
    display:block;
}
</style>

Anyone want to take over work on this? You need to know Java and JavaScript, and know more about XBL than I do. And if you know something about whatever IE uses instead of XBL, that would be a big help, too... Drop me a line if you're interested!

Here is a very simple XMLHttpRequest utility function:

/**
 * Use XMLHttpRequest to fetch the contents of the specified URL using
 * an HTTP GET request.  When the response arrives, pass it (as plain
 * text) to the specified callback function.
 */
HTTP.getText = function(url, callback) {
    var request = HTTP.newRequest();
    request.onreadystatechange = function() {
	if (request.readyState == 4) {
	    if (request.status == 200) callback(request.responseText);
	}
    }
    request.open("GET", url);
    request.send(null);
};

This function assumes the presence of a HTTP.newRequest() function to abstract away the differences in creating an XMLHttpRequest object in IE and other browsers. (You've seen a function like this in every Ajax utility library you've looked at...)

Since XMLHttpRequest uses ActiveX in IE, it is unavailable if the user has disabled ActiveX for security reasons. In this situation, we can try to fall back on other means of scripting HTTP. Both <iframe> and <script> tags have src attributes that can generate HTTP GET requests when dynamically set. Let's start there and see if we can use these tags to create an analog to the HTTP.getText() method shown above. (Full compatibility with XMLHttpRequest is not possible with these techniques, however.)

When a web application uses XMLHttpRequest to retreive content for display to the user, no page load occurs, and therefore, the browser's Back button cannot be used to step backwards through the user's browsing history within the application. Much has been made of this Back button problem in Ajax applications. But I'm not convinced that it is anything to worry about.

Maybe usability testers who have worked with Ajax applications can tell me otherwise, but it just doesn't seem that confusing to me. When I zoom a map in Google maps, for exmaple, I simply don't expect the Back button to unzoom it. If I've just clicked the "+" button to increase the magnification of the map, why would I expect a button named "Back" to decrease the magnification? I'm zooming in and out, not back and forward...

One small insight I've had about thi s is that the browser's Back and Forward buttons are not, and should not be, Undo and Redo buttons for web applications. I expect that all but the most inexperienced users have internalized this distinction. Or they will, unless the designers of Ajax applications insist on making Back work like an undo function within their application.

As a corollary, allow me to suggest that wizard-style interactions, with "Next" and "Previous" buttons are not well suited to Ajax applications. Next and Previous are uncomfortably close to Forward and Back, and I think that users of such an application will (and should) reasonably expect that the browser's Back button will perform the same action as the wizard's Previous button. So if you're going for a wizard-style user-interface, skip the Ajax and just do plain old form submissions and page loads so that browser history works okay...

Other than that, I don't think we need to worry about the Back button. But take this advice with a grain of salt. I've performed absolutely no testing with real-world users.

Microsoft invented the XMLHttpRequest, so they get to document it. Mozilla copies the Microsoft API and their documentation simply links to the Microsoft docs.

Unfortunately the Microsoft docs are lacking... (What is it about MSDN? Does it work better if you view it with IE? Trying to use it with Firefox on a Linux box is a uniformly frustrating experience. Hard to read, and even harder to navigate)

Here's what MSDN has to say about readyState values 2 and 3:

(2) LOADED: The send method has been called, but the status and headers are not yet available.
(3) INTERACTIVE: Some data has been received. Calling the responseBody and responseText properties at this state to obtain partial results will return an error, because status and response headers are not fully available.

The description of state 2 implies that the next state will be the one in which the status code and headers are avaialble. But then the documentation for state 3 just falls apart, saying that the body of the response is not available because the status and headers are not available.

This reads to me like a documentation mistake. In state 3 I would expect status and headers to be available, but I would not expect to be able to query the responseText property.

Has anyone figured out if it is okay to query the response headers in state 3? That is, is this a documentation error? Or does the documentation correctly describe an implementation error?

Update: In comments, Ian Hickson points out that the WhatWG draft spec for XMLHttpRequest addresses this precise question: in readyState 3, responseText should give us all text that has been downloaded so far, and getResponseHeader() and getResponseHeaders() should return any headers that have been downloaded so far. If we can assume that all headers will be in the first IP packet of the response, then those methods should work for readyState 3.

I would have sworn that there was a built in Array function for appending the contents of one array to another array.... But when I went to look it up, I found I was wrong.

Array.concat() concatenates the contents of two arays, but it creates and returns a new array rather than appending to the array on which it is invoked.

Array.push() and Array.splice() modify the array on which they are called, but append the array itself, rather than appending the contents of the array.

There isn't even an append() function in the Array Extras package coming in Firefox 1.5.

Here, therefore, is my solution to this problem:

// Append the elements of the array a to this array
Array.prototype.append = function(a) {
    Array.prototype.push.apply(this, a);
}

Java 6.0 ("Mustang") includes a JavaScript interpreter and a javax.script package for interacting with it. The code below is an example of how it works...

It occurs to me that it would be a useful convention for any script that creates and adds elements to the document to tag those elements by adding "synthetic" to the class attribute of the element

This way, other scripts that run later can identify elements that were not part of the original document. Some scripts will want to process syntehtic elements along with the originals, but some will want to skip those synthetic elements.

I don't have a compelling example use case to argue for this, but it just seems like as we start writing more and more useful unobtrusive scripts, we'll find that they interact with each other in unexpected ways. I haven't written any greasemonkey scripts myself, but I suspect that a convention like this would be helpful there, too...

If you, like me, use Technorati to find interesting blog entries tagged "javascript", then you've probably noticed in recent days that a site javascript-guide.info (I'm not going to give them a real link; you can type this URL in yourself) has had a lot of posts, but that they titles are kind of gibberish. Looking at the site, it appears to be a fake blog, with content that is some how automatically gleaned from elsewhere on the web. There are lots of ads on each page, so I suspect that this site exists to get Google AdSense clicks...

When I visited, I noticed a persistant "waiting for fueledstudios.com..." message in my browser's status line. So I used view source to find out what it was loading from that site. It turns out that the last think each page of this fake blog does is load a script from fueledstudios.com

This is where the JavaScript Decoding Challenge comes in. Take a look at the script. It is not heavily obfuscated, but whitespace has been stripped out. The name of the script indicates that it is tracking something. A glance at the script shows that it is doing something with the google adsense ads on the page. it also registers event handlers, uses setTimeout() to repeatedly do something, and communicates with its server by dynamically loading images that pass a lot of data in their URL.

This might all be perfectly benign. But I have the feeling that this site is doing something sleazy. It sure would be nice if we could tell Google that this site is violating their AdSense terms-of-service. That might cause the site to shut down and stop spamming Technorati with junk. So, this is the challenge: anyone want to sort out what this script actually does? I'm very curious, but can't spare the time...

Let me know if you figure anything out!

Vincenzo Alcamo wrote to me with another practical use for closures in JavaScript. Vincenzo is working on a JavaScript library he calls XMLLib, based on the Sarissa library. To prevent his library from cluttering the global namespace, Vinceno defines his symbols in his own XMLLib namespace, of course.

But he goes one step further. In Vincenzo's code, only publically exported methods and properties ever get stored in the XMLLib object. Internal, helper methods that are not intended for public use exist in a closure, where all the public methods can use them but where they are hidden from everyone else.

Schematically, the code looks like this:

// Create an empty namespace
XMLLIB = {}

(function() {  // anonymous function creates a closure for us
    // Thi s is a private method that stays hidden within the closure
    function private_utilty_method() { /* utility code here */ }

    // This method is exported into the XMLLib namespace, so it is a public method
    // But since it is defined here, it can call private methods like the one above.
    XMLLib.publicMethod = function() { /* ... */}
})();  // Close function definition and invoke it all at once

Contrast this approach to private symbols with that taken by JSAN. In that framework, a namespace may contain a mix of public and private symbols. The namespace also declares an array of symbols that should be exported by default, and an array of symbol that are allowed to be exported from the namespace. Any symbols excluded from these arrays are private, and the JSAN exporter functions will not copy them into the global namespace. That is, the JSAN framework does not have true privacy via closures, but defines a convention which, when followed, provides a measure of privacy.

Thanks for the tip, Vincenzo!

The typical way to create new classes in JavaScript is by subclassing another class to inherit its methods. It occurs to me that this is not the only way. Since functions are data values in JavaScript, it is also possible to extend a class simply by copying methods from another class into it. I call this "borrowing" (although I suspect that someone else has already named it something else--let me know):

The code at the end of this postdemonstrates. It creates a simple Rectangle class. Then another simple class called GenericToString. This second class isn't good for much on its own, but serves as a "mixin" that can be borrowed by other classes. The next bit of code in the example is a function that does the borrowing. This is followed by a new class Rectangle2, which does nothing of its own, just borrows methods from the first two classes.

Update: I should add here that this method borrowing technique can be compared to multiple inheritance. The Rectangle2 class inherits methods from Object and borrows methods from two other classes. If there was a way of writing abstract methods in JavaScript (perhaps just throw "this method is abstract!") then borrowing methods from a class that defined abstract methods would be a little like implementing an interface in Java.

I'm interested in reader's thoughts on this technique. Is anything like this used in other languages? Has anyone given this an official-sounding name?

The code is in the extended entry...

Unobtrusive JavaScript calls for JavaScript code to be removed from our HTML and placed in external .js files. At the same time, JavaScript frameworks and libraries are multiplying like crazy. More and more, we are going to be seeing web pages that load 2, 3, or more external .js files.

This means that we need to become disciplined about how we create those .js files, and adopt namespace conventions for these external modules of code so that two modules do not overwrite each others properties.

As far as I can tell, each JavaScript framework out there does this slightly differently. The most explicitly documented set of conventions I've seen are Casey West's rules for writing modules that work with his JSAN.use() function Casey's use() function is a perl-influenced function that loads (dynamically!) a .js module and then imports a set of symbols from the module into the global namespace. it is good stuff, but his module-structuring and namespace-creating rules are tied directly to his JSAN.use() code and I don't think they are general enough for people who want to write modules to be statically loaded with a <script> tag.. For example. a JSAN module defines the set of symbols that it can export, but the only way to import those symbols is via the use() function. There is not a separate import() function for importing symbols. without using use().

If you know of a well-articulated set of conventions for defining reusable modules of JS code, putting them in namespaces, and importing symbols, please add a comment to this post!

Here is my first cut at a list of requirements for a JavaScript module system:

1. A .js module must not define more than one symbol in the global namespace. If it does define a symbol, that symbol should probably be the same as, or derived from, its filename.
2. Any additional symbols defined by a module should be defined within a separate object, or "namespace".
3. We need a way to selectively import symbols from a namespace into the global namespace. This is for convenience only; the methods deifned by a module should be written to work from their own namespace or from the global namespace.
   1. We could standardize an import() function for doing this importing. Or just create the necessary conventions, so that different frameworks can create their own import() methods.
   2. Modules should probably be allowed (but not required) to declare their own list of exported symbols (as JSAN modules do). And if such a list is defined, the import facility should not allow non-exported symbolsto be imported.
4. For client-side programming, modules need a safe way to register code to be run in response to the onload event, without worrying that they will overwrite or be overwritten by the initialization code for some other module.
5. There should be a standard way for a module to declare its name and version number.
6. There should be a standard way for a module to register itself as loaded, and a standard way for a module to query the list of loaded modules (to verify that any modles it depends on are already present).

Please feel free to add your own ideas in the comments!

Like many others, I use the term "class" to informally describe the things we create in JavaScript using constructors and prototype objects. But JavaScript 2.0 is on the horizon, and since that version of the language has real classes, I want to start using a different term to describe JavaScript 1.x pseudo-classes.

Is anyone aware of a pre-existing convention that I can adopt? I've tried using "category" but I think it is too vague. My current feeling is that I should use the hypenated word "proto-type", although if I do this, I need to be careful not to confuse it with "the prototype object" that we use to create a proto-type.

If you have a thought or better suggestion, I'd love to hear about it. Just retaining our current usage of "class" is an option, too, of course.

Can you spot the bug in the following code?

// Compute the sum of the elements in array a
function sum(a) {
    // Bonus feature: if we're passed more than one argument,
    // then sum the arguments of the array instead.
    if (arguments.length > 1) a = arguments;
    var total = 0;
    for(var i = 0; i < a.length; i++) total += a[i];
    return total;
}

Hint: the function works correctly if you pass an array, but not if invoked with more than one argument.

Answer follows...

I found another useful example involving closures. Bob Ippolito's python-style iteration framework includes this nifty code:

arrayLikeIterator = function (arrayLike) {
    var i = 0;
    return {
        'next': function () {
            if (i >= arrayLike.length) {
                throw StopIteration;
            }
            return arrayLike[i++];
        }
    };
};

There is lots more to Bob's iteration framework at his blog. The iteration stuff is part of Bob's larger framework, MochiKit, which looks quite interesting.

For the record, I disagree with Bob about throwing an exception at the end of iteration. I tihnk he'd be better of returning a unique sentinel value (but probably not null). His StopIteration error object could work just fine as a sentinel, if he'd return it instead of throwing it.

Update: after exchanging a number of emails with Bob, I now understand that his iterators are python-style iterators, intended to be used as part of a larger framework, and that they are to be treated as opaque values and passed to other functions that do the work of iterating them. That is, the end user of the iterators never has to catch the exception that the iterator throws. Bob has convinced me when an iterator is processed by a chain of composed functions, the use of an exception to end the iterator is the most efficient way to go. So, I drop my objection to throwing an exception to end iteration, as long as you use Bob's iterators in the python-based style he intends.

I came up with the code below to demonstrate that two JavaScript closures can have the same scope chain. This gives them shared access to properties that are not accessible from anywhere else.

Here's the code:

//
// This function adds property accessor methods for a property with 
// the specified name to the object o.  The methods are named get<name>
// and set<name>.  If a predicate function is supplied, then the setter
// method uses it to test its argument for validity before storing it.
// If the predicate returns false, then the setter method throws an exception.
// 
// The unusual thing about this function is that the property value 
// that is manipulated by the getter and setter methods is not stored in
// the object o.  Instead, the value is stored only in a local variable
// in this function.  The getter and setter methods are also defined
// locally to this function, and therefore have access to this local variable.
// Note that the value is private to the two accessor methods, and it cannot
// be set or modified except through the setter.
// 
function makeProperty(o, name, predicate) {
    var value;  // This is the property value

    // The setter method simply returns the value
    o["get" + name] = function() { return value; }

    // The getter method stores the value or throws an exception if
    // the predicate rejects the value
    o["set" + name] = function(v) { 
        if (predicate && !predicate(v))
	    throw "set" + name + ": invalid value " + v;	
	else
            value = v;
    }
}

// The following code demonstrates the makeProperty() method 
var o = {};  // here is an empty object

// Add property accessor methods getName and setName()
// Ensure that only string values are allowed
makeProperty(o, "Name", function(x) { return typeof x == "string"; });

o.setName("Frank");  // Set the property value
print(o.getName());  // Get the property value
o.setName(0);        // Try to set a value of the wrong type

This example is simlar to Douglas Crockford's Private Members in JavaScript.

I don't think it is actually that useful in practice--I don't recommend that anyone start calling my makeProperty() function on their own objects! As far as I know, the most realistically useful application of JavaScript closures is Steve Yen's breakpoint() function.

I've been turning my attention from Java to JavaScript in recent days.

I discovered this morning that the Web Standards Project (or WaSP) has turned its attention to JavaScript, as well. They have recently announced the launch of the DOM Scripting Task Force to promote "unobtrusive" JavaScript programming using the W3C DOM. See the Manifesto and their Definitions. There is not much else at the task force's site yet. Here's an interesting example of unobtrusive JavaScript in action, however.

If WaSP can do for JavaScript programming what they have done for CSS, this is a great thing for the web!