Update: Kyle Huey of Mozilla has updated this example to work with the latest version of the API. This version works with Firefox 10.
Update, March 17th: things have gotten a lot better since I first posted this, and I've make a number of edits below. Big thanks to Ben Turner and Jeremy Orlow who explained a lot in the comments!
I've been trying to create an IndexedDB example for the next edition of my JavaScript book. IndexedDB is a promising API< strike>, but it turns out to be just barely useable right now.
I'm using Firefox 4b10 RC and Chrome 10.0.642 11dev. Here are some things I've found out. Maybe they'll help someone!
- Both Mozilla and Chrome prefix the global indexedDB object. Use mozIndexedDB or webkitIndexedDB. Prior to FF4b9, apparently you had to use moz_indexedDB, with an underscore, but no longer.
- Chrome also puts webkit prefixes before the other related globals IDBTransaction and IDBKeyRange. Firefox does not (currently) prefix those.
I can't get Chrome to insert anything into a database. Even Google's tutorial at http://www.html5rocks.com/tutorials/indexeddb/todo/ isn't working. It complains that the DB is in read only mode and won't insert. Tested on Linux and MacOS. This is presumably a regression of some sort since the tutorial must have worked on Chrome at some point. It doesn't even work if you launch chrome with the magic --unlimited-quota-for-files which you need when using the filesystem API.- For large databases, like my demo, you have to start Chrome with --unlimited-quota-for-indexeddb
Chrome's error codes are off-by-one compared to the IndexedDB spec.They still differ from the spec, but Firefox now matches Chrome for most of the codes.The spec never defines what a "key path" is. The non-normative examples show it as a plain string. When creating an object store, FF requires an object like {keyPath:"property_name"} instead of just the "property_name" string. Google's examples show key paths like this, too, so they're apparently both out of sync together on this point.In comments, Ben and Jeremy explain that the spec was out of date.But it gets weirder. When creating an index on an object store, Firefox wants a plain string key path, and the index won't work if you specify an object.Ben explains this in comments.
- It is completely undocumented in the spec how you would express a key path for more deeply nested properties of the object being stored. An array of strings seems like the right thing to me, if anyone wants to take my advice. [Ben explains in comments that this issue has been deferred for a future revision of the spec.]
Finally, Firefox has diverged from the spec in one major way. There are lots of places where you make async requests in this API. The spec says that the result of your request will be in the request property of the event object that is passed to the onsuccess event handler. But Firefox has (relatively recently, I gather) moved the result value so that it is a property of the request object rather than the event object.. Instead of writing event.result, as the spec says, in Firefox you have to write request.result, or perhaps more elegantly: var result = event.result || event.target.result;Thhe spec was out of date, and Chrome has now followed Firefox. Now, you always get the result of an asynchronous operation from the result property fo the request object. The event object is no longer used.- Neither
Firefox(See Ben's comment below for directions on accessing the Firefox UI) nor Chrome have any UI yet for managing the databases created by this API. So if you mess up a database and want to delete it, you just have to find your profile directory and delete the appropriate file or directory there. This is the command I used on my Linux machine:rm -rf ~/.config/google-chrome/Default/IndexedDB/*localhost*.indexeddb ~/.mozilla/firefox/*.default/indexedDB/*localhost*
- I've just assumed that this API won't work right if you test it from a local file, so I've been running a simple node-based webserver. (That's why the database names shown above have "localhost" in their names).
My zipcode demo (written for the next edition of JavaScript: The Definitive Guide) loads a database of US postal codes and lets you search by postal code or by city name. It seems to work today in FF4b10RC1 and Chrome 11dev on Linux. I haven't tested Windows. or MacOS (The database includes latitude and longitude, so it would be sweet to hook it up to google maps!)
Update: Google's demo mentioned above can be made to work on FF4b10 by applying this patch.
Update 2: I've cleaned up my demo considerably and have updated the link above.
Update 3: Thanks to Guido Tapia who suggested in comments that I try the Chromium nightlies. This example sort of works with Chromium 11.0.652.0 (72925). The API works, so I think my example code is written correctly. But chrome's database engine can't handle all the zipcodes. It looks like it downloads one batch of a few thousand, then takes so long to process them that the next "onprogress" event gets the rest of the zipcode file. And this version of chrome can't insert 30000+ objects at once. It pegs my cpu and tries for 20 to 30 seconds then aborts the transaction. (I was surprised how slow Firefox was to insert all these zipcodes, but this version of Chrome is slower, and doesn't actually succeed...) This problem is fixed by adding the --unlimited-quota-for-indexeddb when you launch Chrome.
@__DavidFlanagan
Thanks for the write-up. I'm glad I'm not the only one trying to piece together the current state of IndexedDB.
By the way, the "Google's demo" link needs a trailing slash.
Gabriel,
You're welcome. I've fixed the broken link. Thanks for that. Good luck with IndexedDB.
This may be the reason why Microsoft decided against including any sort of IndexDB-thing into its next IE.
Did you try out their IndexDB-enabled highly experimental IE9-branch?
http://html5labs.interoperabilitybridges.com/
Schepp,
No, I didn't try the IE extension. I'd be interested to hear how it works, if anyone tries it...
I'm finishing my HTML5 book and IndexedDB is the only thing I can't get over haha. It's a nightmare at this moment.
David,
Download the latest Chromium build as it fixes alot of the IndexedDB bugs:
http://build.chromium.org/f/chromium/continuous/win/LATEST/
With this build I was able to get all my unit tests passing (in my project) after changing my use of objectrStore.clear and objectStore.getAll to use cursors instead. That was the only change needed. Error codes, read/write numbers, initialising object stores all remained the same.
Tnx
Guido
In fact have a look at this:
http://code.google.com/p/picnet-closure-repo/source/browse/trunk/src/picnet/data/IndexedDBRepository.js
Thats my indexed DB code that works on both webkit (nightly) and FF.
Thanks
Guido
Guido,
Thanks for suggesting I try the Chromium nightlies. The API works for me now using Chromium 11.0.652.0 (72925).
Unfortunately the DB engine can't handle having 40000 zipcode records thrown at it at once. It pegs my CPU for 20 or 30 seconds and then aborts the transaction after processing only a few thousand zipcodes.
Thanks Guido for the information. It was useful for me. I finally was able to finish the codes!
Just FYI, I found another inconsistency on the spec, but according to the developers of Chromium the spec needs to be updated. The word "remove" was replace by "delete" in the methods removeObjectStore() and remove().
Why are there some spam links at the bottom of your zipcode demo and some of your pages?
hi David-- fantastic info. can you post your patched version of the ToDo list? applying your patch did not work for me. many thanks!
Hi David,
Glad you're trying out IndexedDB! Sorry that the implementations vary a bit (from each other, as well as from the published spec), we're working on fixing it all up.
The spec is a little out of date with the changes that have been agreed upon in the w3c mailing list, though it should be updated soon. Let me see if I can't help clear up some of the confusing items you listed above:
5. Having an object store with a 'keyPath' is optional. In fact, it's one of two different options (the other being 'autoIncrement') that you can specify when making an object store. We may include additional options in future versions of the spec as well. Since the order in which these options are specified is unimportant and since there may be more in the future we've decided to have the spec require an 'options' object here. You can leave it out entirely or you can pass one or more of the options in this 'options' object.
6. For indexes the 'keyPath' is *not* optional, so it cannot be specified in an 'options' object and must instead be passed as the second parameter as a simple string. (There is currently only one option for index creation, 'unique', and that must be passed in an 'options' object if you want it.)
7. We decided to wait for the next version of IndexedDB before allowing nested properties to be specified as a key path. We have some other ideas about allowing compound and dynamic indexes that we are also going to explore, but for version 1 we decided simple key paths would be enough.
8. Here the spec is simply behind the times and we haven't had a chance to update it to reflect the rather large (as you noted) change in the request interface. This change will not make it into the next version of Chrome, as I understand it, but will be present in the following version.
9. Firefox, at least, has some UI for clearing IndexedDB databases. Clicking the favicon for a site that uses IndexedDB pops up a doorhanger, and in that doorhanger you can press the 'More information' button. The resulting dialog has a 'Permissions' tabs with a 'Offline Storage' entry that includes the amount of space currently being used by IndexedDB and gives the option to clear it. Firefox 5 will streamline all of our offline resources into a single location, but for 4 that's where it will live.
Also, I think you will find https://developer.mozilla.org/En/IndexedDB_Primer pretty useful. It explains a bit more about the API in a format that is more readable than the spec (in my opinion, at least!).
Hope that helps!
-Ben Turner
I'm one of the developers working on IndexedDB in Chromium. I'm sorry for all the pain being caused at the moment, but things should be pretty stable in the next couple weeks.
The spec right now is pretty horribly out of date. You can see all the bugs against it here: http://www.w3.org/Bugs/Public/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&component=Indexed+Database+API&longdesc_type=allwordssubstr&longdesc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED&emailtype1=substring&email1=&emailtype2=substring&email2=&bug_id_type=anyexact&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&known_name=IndexedDB&query_based_on=IndexedDB&field0-0-0=noop&type0-0-0=noop&value0-0-0= (Note that many of the bugs spawned mailing list discussions, so the full story is not always in the bug.)
With the exception of the "getAll" method (which we fundamentally disagree with and which likely won't make it into the spec), where FF4b10 is right now is pretty much where we intend to be in a couple weeks. So, to be honest, until then, coding against FF4b10 is probably your best bet.
The HTML5Rocks people know about the breakage and hope to fix it soon.
We're considering removing the webkit prefix before everything except "window.webkitIndexedDB". Do you guys think that'd be a positive change?
The problem with inserting into the database sounds like you're not specifying the transaction is READ_WRITE?
I'll look into the error code issue, but it shouldn't matter because you should be using the symbolic constants (i.e. webkitIDBDatabaseError.SOME_ERR_CODE) rather than hard coding numbers. :-)
We really do need to spec the key path. Right now Chrome accepts "any.dotted.notation[0].and[0][2].arrays" IIRC.
Thanks so much for the feedback and I'm sorry for the pain you've experience so far! In the future, feel free to ask questions in https://groups.google.com/a/chromium.org/group/chromium-html5 and/or file bugs at crbug.com (cc jorlow@chromium.org for IndexedDB ones).
Here are some examples on using IndexedDB on Firefox, Chrome and IE - http://nparashuram.com/ttd/IndexedDB/index.html
Ben, Jeremy:
Thanks for the detailed info. I've updated the example and now have it working in FF4RC and Chrome 11. Chrome does require the --unlimited-quota-for-indexeddb flag since my test database is kind of large.
Hi David,
Just wanted to give you a heads up that there has been a great indexedDB tutorial uploaded to YouTube. I tried it out in Firefox 4 and Chromium 13.x and it worked great. I had to lightly modify the JavaScript to get it working in Chromium (simple prefix stuff) but it worked.
Here is the URL explaining where to see the tutorial as well as the patch required to get it working with Chromium.
http://goo.gl/3jnaX
Thanks for the great work David,
Christopher
Thanks, i really appreciate your work. I ThumbsUp, Like and Buzz your article, keep it up :-)
http://backspacetab.com/2011/05/16/google-chrome-error-11/
Thanks, i really appreciate your work. I ThumbsUp, Like and Buzz your article, keep it up :-)
http://backspacetab.com/2011/05/16/google-chrome-error-11/
Please Chromium guys, just hire one person to work exclusively on documentation and testing of the IndexedDB api. Don't you have the resources? I am so frustrated trying to piece together working code from the Mozilla documentation which differs so much from the Chromium implementation. Arrrggh!
Hi David,
Thanks for updating your post! One thing I noticed in your zipcode demo is that the city name search no longer works in recent Firefox builds. The offending line seems to be this one:
var range = new IDBKeyRange.only(city);
Here 'only' is not a constructor and so the use of 'new' is inappropriate. The correct usage is:
var range = IDBKeyRange.only(city);
Thanks again!
-Ben