davidflanagan.com > August 2005

Canvas tag in Java and XBL

I think 2D graphics are a lot of fun, and have always loved the Java2D API. How cool is it, then, that Safari includes and Firefox 1.5 will include a new <canvas> tag that exposes a drawing API very, very similar to the Java2D API?

The canvas tag has the potential to revolutionize a lot of web applications. But for this to happen, it needs to be ported to browsers other than Safari and Firefox 1.5. I'm sure it can be done for IE, somehow, but I don't want to tackle that right now.

Instead, my contribution is to begin implementing the canvas tag for Firefox 1.0 using a Java applet and Mozilla's XBL binding language. This is just at proof-of-concept stage, but it is usable enough to make simple demos work. The java applet and XBL files are here. The canvas.html file in that directory is a simple demo. It doesn't seem to work online, but if you copy the files to a local directory, it should work. (I suspect, but don't know, that XBL bindings have to be installed locally to work for security reasons.)

To use this, compile the .java file to a .class file. Then define a simple stylesheet that tells Firefox to use the canvas.xbl file to implement the canvas tag. This is what I use:

<style>
canvas {
    -moz-binding: url('canvas.xbl#canvas');
    display:block;
}
</style>

Anyone want to take over work on this? You need to know Java and JavaScript, and know more about XBL than I do. And if you know something about whatever IE uses instead of XBL, that would be a big help, too... Drop me a line if you're interested!

XMLHttpRequest with <script> and PHP

Here is a very simple XMLHttpRequest utility function:

/**
 * Use XMLHttpRequest to fetch the contents of the specified URL using
 * an HTTP GET request.  When the response arrives, pass it (as plain
 * text) to the specified callback function.
 */
HTTP.getText = function(url, callback) {
    var request = HTTP.newRequest();
    request.onreadystatechange = function() {
	if (request.readyState == 4) {
	    if (request.status == 200) callback(request.responseText);
	}
    }
    request.open("GET", url);
    request.send(null);
};

This function assumes the presence of a HTTP.newRequest() function to abstract away the differences in creating an XMLHttpRequest object in IE and other browsers. (You've seen a function like this in every Ajax utility library you've looked at...)

Since XMLHttpRequest uses ActiveX in IE, it is unavailable if the user has disabled ActiveX for security reasons. In this situation, we can try to fall back on other means of scripting HTTP. Both <iframe> and <script> tags have src attributes that can generate HTTP GET requests when dynamically set. Let's start there and see if we can use these tags to create an analog to the HTTP.getText() method shown above. (Full compatibility with XMLHttpRequest is not possible with these techniques, however.)

IFrames are commonly cited as the tag of choice for "dynamic scripting" without XMLHttpRequest. The basic idea is that you create a new iframe, set its onload property, and then set its src property to the URL that you want to fetch. The problem is that we want to fetch a plain text resource (such as a JavaScript file, which can be treated as plain text) but the <iframe> tag wants to display an HTML document. Plain text loaded into an iframe (in Firefox at least) ends up stuffed into a <pre> tag, and has its angle brackets and ampersands replaced with their corresponding HTML entities.

It should be possible to extract the contents of the <pre> tag and replace the entities with the characters they replaced. I decided to go a different route, however, and encode the plain text to be transferred in a CDATA section of an XML file. Here is a PHP script that does this:

<?php 
header("Content-Type: text/xml");
echo '<?xml version="1.0"?>';
echo '<data>';
echo '<![CDATA[';
readfile($_GET["url"]);
echo ']]>';
echo '</data>';
?>

With this script, it was pretty easy to write a getText() function that obtained the contents of a specified URL as plain text. In Firefox, anyway. Things got very complicated in IE, and I simply could not make it work on this platform. (Remember that the motivation is to find an alternative to XMLHttpRequest to use when ActiveX is disabled in IE).

Kae Verens has made this work but I'm not sure what he's doing differently. I don't think he's using an XML CDATA section, and I didn't study his server-side code well enough to figure out how he encodes the data for transfer to the client.

Anyway, after failing with the <iframe> approach, I decided to try using a <script> tag instead. Again, we use a server-side script to encode the contents of the URL we want to transfer. This time, we encode the contents as a (possibly very long) JavaScript string, and pass that string to a JavaScript function. One pleasant side effect of this approach is that there is no need for an onload or onreadystatechange event handler. When the script loads, it executes, and we've encoded the callback call into the script, so the callback just gets called.

Enough talk. Here is the server-side PHP script. It should be in a file name jsquoter.php.

<?php 
header("Content-Type: text/javascript");
// Get arguments from the URL
$func = $_GET["func"];       // The function to invoke in our js code
$filename = $_GET["url"];    // The file or URL to pass to the func
$lines = file($filename);    // Get the lines of the file
$text = implode("", $lines); // Concatenate into a string
// Escape quotes and newlines
$escaped = str_replace(array("'", "\"", "\n", "\r"), 
		       array("\\'", "\\\"", "\\n", "\\r"),
		       $text);
// Output everything as a single JavaScript function call
echo "$func('$escaped');"
?>

And here is the client-side function that uses the PHP script above. We've succeeded in duplicating the API of the HTTP.getText() function that we began with:

HTTP.getTextWithScript = function(url, callback) {
    // Create a new script element and add it to the document
    var script = document.createElement("script");
    document.body.appendChild(script);

    // Get a unique function name
    var funcname = "func" + HTTP.getTextWithScript.counter++;

    // Define a function with that name, using this function as a
    // convenient namespace.  The script generated on the server will
    // invoke this function
    HTTP.getTextWithScript[funcname] = function(text) {
	// Delete the script tag we created
	document.body.removeChild(script);
	// Delete this function
	delete HTTP.getTextWithScript[funcname];
	// And invoke the callback 
	callback(text);
    }

    // Encode the url we want to fetch, and the name of the function
    // as arguments to the jsquoter.php server-side script.  Set the src
    // property of the script tag to fetch the URL
    script.src = "jsquoter.php" + "?url=" + encodeURIComponent(url) +
                  "&func=" +
                  encodeURIComponent("HTTP.getTextWithScript." + funcname);
}

// We use this to generate unique function callback names in case there
// is more than one request pending at a time.
HTTP.getTextWithScript.counter = 0;

This code has been lightly tested in Firefox and IE6, (with PHP 4.1 on the server). It works for me! Feel free to use it yourself, and please post any suggestions or bug reports in the comments.

One final note: in some PHP configurations, the url argument we pass to jsquoter.php has to be a local file. In many configurations, however, it can be an arbitrary URL. In this case, the PHP script acts as a proxy server and defeats the "same-origin" restriction that prevents XMLHttpRequest from making requests to any server other than the one from which the script was loaded. This should be safe as long as your web application is in control of the URLs that are submitted to jsquoter.php. But don't let the user specify the URL to be loaded in this way, or you'll probably be opening up a security hole.

The Back Button is not an Undo Button

When a web application uses XMLHttpRequest to retreive content for display to the user, no page load occurs, and therefore, the browser's Back button cannot be used to step backwards through the user's browsing history within the application. Much has been made of this Back button problem in Ajax applications. But I'm not convinced that it is anything to worry about.

Maybe usability testers who have worked with Ajax applications can tell me otherwise, but it just doesn't seem that confusing to me. When I zoom a map in Google maps, for exmaple, I simply don't expect the Back button to unzoom it. If I've just clicked the "+" button to increase the magnification of the map, why would I expect a button named "Back" to decrease the magnification? I'm zooming in and out, not back and forward...

One small insight I've had about thi s is that the browser's Back and Forward buttons are not, and should not be, Undo and Redo buttons for web applications. I expect that all but the most inexperienced users have internalized this distinction. Or they will, unless the designers of Ajax applications insist on making Back work like an undo function within their application.

As a corollary, allow me to suggest that wizard-style interactions, with "Next" and "Previous" buttons are not well suited to Ajax applications. Next and Previous are uncomfortably close to Forward and Back, and I think that users of such an application will (and should) reasonably expect that the browser's Back button will perform the same action as the wizard's Previous button. So if you're going for a wizard-style user-interface, skip the Ajax and just do plain old form submissions and page loads so that browser history works okay...

Other than that, I don't think we need to worry about the Back button. But take this advice with a grain of salt. I've performed absolutely no testing with real-world users.

Google bombing google

I just had a strange comment spam attack. 9 copies of this comment

IP Address: 205.236.34.1
Name: search engine
Email Address: google@yahoo.nl
URL: http://www.google.com

Comments:

i come from best search engine http://www.google.com

It looks to me as if someone, masquerading as a Dutch Yahoo employee is trying to google bomb google so that the search "best search engine" links to google itself. Why do this?

Interestingly, all 9 copies of the comment had the same IP address, which is not what I usually see in comment spam attacks. So maybe the spammer is an amateur or has a very small network of zombie computers?

XMLHttpRequest.readyState == 3

Microsoft invented the XMLHttpRequest, so they get to document it. Mozilla copies the Microsoft API and their documentation simply links to the Microsoft docs.

Unfortunately the Microsoft docs are lacking... (What is it about MSDN? Does it work better if you view it with IE? Trying to use it with Firefox on a Linux box is a uniformly frustrating experience. Hard to read, and even harder to navigate)

Here's what MSDN has to say about readyState values 2 and 3:

(2) LOADED: The send method has been called, but the status and headers are not yet available.
(3) INTERACTIVE: Some data has been received. Calling the responseBody and responseText properties at this state to obtain partial results will return an error, because status and response headers are not fully available.

The description of state 2 implies that the next state will be the one in which the status code and headers are avaialble. But then the documentation for state 3 just falls apart, saying that the body of the response is not available because the status and headers are not available.

This reads to me like a documentation mistake. In state 3 I would expect status and headers to be available, but I would not expect to be able to query the responseText property.

Has anyone figured out if it is okay to query the response headers in state 3? That is, is this a documentation error? Or does the documentation correctly describe an implementation error?

Update: In comments, Ian Hickson points out that the WhatWG draft spec for XMLHttpRequest addresses this precise question: in readyState 3, responseText should give us all text that has been downloaded so far, and getResponseHeader() and getResponseHeaders() should return any headers that have been downloaded so far. If we can assume that all headers will be in the first IP packet of the response, then those methods should work for readyState 3.

Appending array contents to another array

I would have sworn that there was a built in Array function for appending the contents of one array to another array.... But when I went to look it up, I found I was wrong.

Array.concat() concatenates the contents of two arays, but it creates and returns a new array rather than appending to the array on which it is invoked.

Array.push() and Array.splice() modify the array on which they are called, but append the array itself, rather than appending the contents of the array.

There isn't even an append() function in the Array Extras package coming in Firefox 1.5.

Here, therefore, is my solution to this problem:

// Append the elements of the array a to this array
Array.prototype.append = function(a) {
    Array.prototype.push.apply(this, a);
}

Shrinking Java in a Nutshell

Matt Croydon writes at his blog you know your programming language is complicated when Java in a Nutshell has 1284 pages and weighs 3.2 pounds.

I know! And I worry about this. And with Java 6.0 in the works, it is only going to get bigger...

Unless some clever reader can think of a way to shrink it!

Comments are open...

Making CSS Counters Work in Deer Park

One of the very nice new features in Deer Park (the codename for Firefox 1.5 alpha) is that CSS counters are finally implemented!

Beware, though! If you try to use them following the examples in the current CSS 2.1 spec, they won't work! Apparently there is a new draft of CSS 2.1 coming out, and deer park implements counters that follow that draft.

All the examples out there on the net are based on the CSS spec. Since counters have never worked before, no one has written their own examples, just recycled things from the spec. They tend to look like this:

h1:before {
    content: counter(h1) ":"  /* display counter value */
    counter-increment: h1;  /* increment counter */
    counter-reset: h2 h3 h4;  /* reset other counters */
}

If you try this, all your h1 elements will be prefixed with the number 1. The counter-increment doesn't work if you do it this way. It has something obscure to do with the scope fo the counter-reset, but I didn't dig deep enough to try to understand what was going on.

To make it work, increment and reset your counters as part of the CSS styles for the tag itself, not as part of the :before pseudo-element. This code works, for example:

h1 {
    counter-increment: h1;
   counter-reset: h2 h3 h4;
}
h1:before {
    content: counter(h1) ": ";
}

Happy counting!

[If you want to know more about why this all is, try this bugzilla entry. Comment #109 is a good starting point.

Thanks to Philippe Wittenbergh for putting me on the right track with this post.]

Example: using JavaScript in Java

Java 6.0 ("Mustang") includes a JavaScript interpreter and a javax.script package for interacting with it. The code below is an example of how it works...

import javax.script.*;
import java.util.*;
import java.io.*;

/**
 * This class is like java.util.Properties, but allows property values to
 * be determined by evaluating JavaScript expressions.
 */
public class ScriptedDefaults {
    // Here is where we store name/value pairs of defaults
    Map<String,Object> defaults = new HashMap<String,Object>();

    // Accessors for getting and setting values in the map
    public Object get(String key) { return defaults.get(key); }
    public void put(String key, Object value) { defaults.put(key, value); }

    // Initialize the contents of the Map from a file of name:value pairs.
    // If a value is enclosed in curly braces, evaluate it as javscript.
    public void load(String filename) throws IOException, ScriptException {
        // Get a JavaScript interpreter
        ScriptEngineManager manager = new ScriptEngineManager();
        ScriptEngine engine = manager.getEngineByExtension("js");

        // Use our own name/value pairs as JavaScript variables
        Bindings bindings = new SimpleBindings(defaults);

        // Create a context for evaluating scripts in 
        ScriptContext context = new SimpleScriptContext();

        // Set those Bindings in the Context so that they are readable
        // by the scripts, but that variables defined by the scripts do
        // not get placed into our Map object.
        context.setBindings(bindings, ScriptContext.GLOBAL_SCOPE);

        BufferedReader in = new BufferedReader(new FileReader(filename));
        String line;
        while((line = in.readLine()) != null) {
            line = line.trim();  // strip leading and trailing space
            if (line.length() == 0) continue;    // skip blank lines
            if (line.charAt(0) == '#') continue; // skip comments
            
            int pos = line.indexOf(":");
            if (pos == -1)
                throw new IllegalArgumentException("syntax: " + line);
            
            String name = line.substring(0, pos).trim();
            String value = line.substring(pos+1).trim();
            char firstchar = value.charAt(0);
            int len = value.length();
            char lastchar = value.charAt(len-1);

            if (firstchar == '"' && lastchar == '"') {
                // Double-quoted quoted values are strings
                defaults.put(name, value.substring(1, len-1));
            }
            else if (Character.isDigit(firstchar)) {
                // If it begins with a number try to parse a number
                try {
                    double d = Double.parseDouble(value);
                    defaults.put(name, d);
                }
                catch(NumberFormatException e) {
                    // Oops.  Not a number.  Store as a string
                    defaults.put(name, value); 
                }
            }
            else if (value.equals("true"))         // handle boolean values
                defaults.put(name, Boolean.TRUE);
            else if (value.equals("false"))
                defaults.put(name, Boolean.FALSE);
            else if (value.equals("null"))
                defaults.put(name, null);
            else if (firstchar == '{' && lastchar == '}') {
                // If the value is in curly braces, evaluate as JavaScript code
                String script = value.substring(1, len-1);
                Object result = engine.eval(script, context);
                defaults.put(name, result);
            }
            else {
                // In the default case, just store the value as a string
                defaults.put(name, value);
            }
        }
    }

    public static void main(String[] args)
        throws IOException, ScriptException
    {
        ScriptedDefaults defaults = new ScriptedDefaults();
        defaults.load(args[0]);
        Set<Map.Entry<String,Object>> entryset = defaults.defaults.entrySet();
        for(Map.Entry<String,Object> entry : entryset) {
            System.out.printf("%s: %s%n", entry.getKey(), entry.getValue());
        }
    }
}

Synthetic tags and unobtrusive JavaScript

It occurs to me that it would be a useful convention for any script that creates and adds elements to the document to tag those elements by adding "synthetic" to the class attribute of the element

This way, other scripts that run later can identify elements that were not part of the original document. Some scripts will want to process syntehtic elements along with the originals, but some will want to skip those synthetic elements.

I don't have a compelling example use case to argue for this, but it just seems like as we start writing more and more useful unobtrusive scripts, we'll find that they interact with each other in unexpected ways. I haven't written any greasemonkey scripts myself, but I suspect that a convention like this would be helpful there, too...

Fake Blogs and JavaScript Decoding Challenge

If you, like me, use Technorati to find interesting blog entries tagged "javascript", then you've probably noticed in recent days that a site javascript-guide.info (I'm not going to give them a real link; you can type this URL in yourself) has had a lot of posts, but that they titles are kind of gibberish. Looking at the site, it appears to be a fake blog, with content that is some how automatically gleaned from elsewhere on the web. There are lots of ads on each page, so I suspect that this site exists to get Google AdSense clicks...

When I visited, I noticed a persistant "waiting for fueledstudios.com..." message in my browser's status line. So I used view source to find out what it was loading from that site. It turns out that the last think each page of this fake blog does is load a script from fueledstudios.com

This is where the JavaScript Decoding Challenge comes in. Take a look at the script. It is not heavily obfuscated, but whitespace has been stripped out. The name of the script indicates that it is tracking something. A glance at the script shows that it is doing something with the google adsense ads on the page. it also registers event handlers, uses setTimeout() to repeatedly do something, and communicates with its server by dynamically loading images that pass a lot of data in their URL.

This might all be perfectly benign. But I have the feeling that this site is doing something sleazy. It sure would be nice if we could tell Google that this site is violating their AdSense terms-of-service. That might cause the site to shut down and stop spamming Technorati with junk. So, this is the challenge: anyone want to sort out what this script actually does? I'm very curious, but can't spare the time...

Let me know if you figure anything out!