2007-01-14 16:34:36 - fjenett

Bye-bye <applet>. Hello <object>!

The object tag, xHTML-strict, loading-bars, applet preloading and appletobject.js

A quick summary since the text kinda grew long:

(1) Download these template(s) if you want to export your sketches from Processing to be embeded into valid xHTML 1. Versions for OpenGL applets are in there too.

(2) I found a method with fallbacks to improve the user experience with startup and loading times for applets on websites.

Enter here. 

Processing allows for generating web-ready applets via it's "export for web" feature. Currently this generates an applet-folder containing the source-files as .java and .pde, one or multiple jar-files (depending on your preferences settings), a loading-image (Processing icon, "loading.gif") and a (almost xHTML-1.0-Transitional) xHTML-file. This file is based upon two templates that come with Processing, applet.html and applet-opengl.html, which live inside Processings </path/to/app>/lib/export folder. One can alter the HTML being used for export by placing a template-file named "applet.html" inside your sketches root directory.

Following up on multiple requests on Processings discourse-forum for a standarts compliant, valid (xHTML) version of the template, i decided to spend some time making one.

The current situation is, that the <applet> tag as used by Processing is deprecated in xHTML-1.0-Strict. It will still work with most new browsers but should shift them into what's called the quirks-mode since the source-XML will not validate. This behavior might go away in the future and the exported applets might not be displayed anymore. As much as i doubt this will happen any time soon, it's not a bad idea to move on to be both as forward and backward compatible as possible.

I've based my approach on several sources, many of which do a lot of explaining on the change from <applet> to <object>, so i'm not going into much detail with that.

The main target of this little research is to have the xHTML object tag to embed an applet be:

Test setup

As testcases i made two simple sketches:

They are set up to do the following:

Each sketch was exported using one of two templates (see above), and these are the resulting files from Processing:

(*) with minor modifications to be almost xHTML-Transitional. The only thing that breaks it is the mayscript-attribute, which is not valid XHTML. Sadly FireFox will not recognize it if passed as param, therefore i left it in as was.

Results without OpenGL

All testing was done on a Mac PowerBook G4 1.67 Ghz, 1GB RAM.
Additional Systems were run with VirtualPC.
Systems: 


Jump down below "Notes" for a summary of the results.

----------------+---------------+---------------+-------+-------+--------------+
Browser         |fallback(*)    |               |js-    |param  |icon,         |
                |               |               |called |read   |loadingbar    |
                |               |               |               |              |
                |               |               |               |              |
                | O1    | A1    | O1    | A1    | O1            | O1    | A1   |
================+=======+=======+=======+=======+=======+=======+=======+======+
IE 3.0 (xp)     | X     | X     | -     | -     |               |       |      |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
IE 4.01 (xp)    | X(1,2)| X     | Xi    | -     | X     | X     | 1     |      |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
IE 5.01 (xp)    | X     | X     | Xi    | X     | X     | X     | 1     | 1    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
IE 5.23 (osx)   | X(20) | X     | Xo(9) | X     | -(19) | -(19) | 0     | 0    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
IE 5.5 (xp)     | X     | X(3)  | Xi    | -(3)  | X     | X     | 1     | 1    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
IE 6.0 (xp)     | X     | X     | Xi    | X     | X     | X     | 1     | 1    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
FF 1.5 (xp)     | X     | -(4)  | Xo    | X     | X     | X     | 1     | 1    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
FF 1.5 (osx)    | X     | -(8)  | Xo    | X     | X     | X     | (22)  | (22) |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
FF 2.x (osx)    | X     | -(8)  | Xo    | X     | X     | X     | (22)  | (22) |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
FF 2.0 (xp)     | X     | -(4)  | Xo    | X     | X     | X     | 1     | 1    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
NS 6.2.3 (xp)   | X     | X(5)  | -     | -(21) |       |       |       |      |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
NS 6.2 (osx)    | X(10) | X     | -(10) | -(10) |       |       |       |      |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
NS 7.2 (osx)    | X     | -(12) | Xo    | X     | (13)  | X     | 0     | 0    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
NS 7.2 (xp)     | X     | X(6)  | Xo    | X     | -     | X     | 1     | 1    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
NS 8.x (xp)     | skipped, uses IE / FF renderer                        |      |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
OP 8.54 (xp)    | X     | X(7)  | Xo    | X     | X     | X     | 0     | 0    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
OP 6 (osx)      | X     | -(25) | -(24) | X     | -     | -     |       | 0    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
OP 8.5 (osx)    | X     | X(15) | Xo    | X     | X(14) | X     | 0     | 0    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
OP 9.1 (xp)     | X     | X(7)  | Xo    | X     | X     | X     | 0     | 0    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
OP 9.1 (osx)    | -(25) | X(7)  | Xo    | X     | X(14) | X     | 0     | 0    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
Lynx (osx)      | X     | X     | text browser                                 |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
Sa 1.3.2 (osx)  | X     | X     | Xo    | X     | X     | X     | 1     | 1    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
Sa 2.x (osx)    | X(25) | X(26) | Xo    | X     | X     | X     | 1     | 1    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
appletviewer osx|  no html      | Xi    | X     |       | X     |       | 0    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
appletviewer lin|  no html      | Xi    | X     |       | X     |       | 0    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
KO 3.3 (lin)    | -(16) | X     | Xo    | X     | X     | X     | 0     | 0    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
EP 1.4.8 (lin)  | X     | -(17) | Xo    | X     | X     | X     | 1     | 1    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+
MO 5.0 (lin)    | X     | -(17) | Xo    | X     | X(23) | X     | 1     | 1    |
----------------+-------+-------+-------+-------+-------+-------+-------+------+

(*) Fallback means the fallback-text (inside applet or object) was 
used if Java was disabled via browser preferences / options.

X  = OK
Xi = OK, inner object tag used
Xo = OK, outer object tag used
-  = did not pass

0  = browsers own text, no icon, no loading-bar,
     sometimes status-bar has more info
1  = only icon was shown
2  = icon and loading-bar + text (as expected since Java 1.4.2)

IE = Internet Explorer
FF = Firefox
NS = Netscape
OP = Opera
Sa = Safari (AppleWebKit 418.9.1)

KO = Konqueror (khtml 3.3.2)
EP = Epiphany Web Browser (same as MO?)
MO = Mozilla

--------------------------------------------------------------------------------

(1) IE 4.01 (xp):
    Java not installed: shows two fields: missing image-field for outer tag,
    applet runs inside inner tag
    Java disabled:  ( gone away after installing Java 1.4.2 )

(2) IE 4.01 (xp):
    misses to ask about loading the plugin (as do all other IEs on win)

(3) IE 5.5 (xp):
    crashed every time after loading (might be due to multiple IEs / my setup)

(4) FF all (xp):
    No Java installed: misses fallback text but shows a:
    "Click here to download Plugin" (puzzle piece icon)
    Java disabled: nothing, no text, ...

(5) NS 6.2.3 (xp):
    Java not installed: misses fallback-text, shows "Click here to get the .."
    (puzzle piece icon) no link given to download Java
    Java disabled: works.
    
(6) NS 7.2 (xp):
    Java not installed: like (5), but link to sun works
    Java disabled: emtpy, no applet, no error, nothing.

(7) OP all (xp), 9.1 (osx): 
    Java not installed: message pops up "You need to install Java to view all
    content on this page. Do you want to download Java now? ( OK / No )"
    applet field is mini, misses size settings in tag
    Java disabled: just the mini-block saying something like "Java a"
    
(8) FF all (osx):
    Java disabled: misses fallback text. no indication that something's not
    displayed.

(9) IE 5.23 (osx):
	although it loads the applet the fallback-text is shown

(10) NS 6.2 (osx):
     java.lang.NullpointerException, not sure why ...

(12) NS 7.2 (osx):
     no fallback-text, no indication that something is missing

(13) NS 7.2 (osx):
     using lib "netscape.javascript" causes applet to fail:
     status says "Applet started" but field is just a white box
     
(14) OP all (osx):
     using processing lib "netscape.javascript" causes applet to fail:
     field is just a white box
     
(15) OP 8.5 (osx):
     like (7) but no popup, just mini box

(16) KO 3.3 (lin): 
     Java not installed: just a brown-grey box where the applet should be
     Java disabled: applet ran nonetheless ...

(17) EP 1.4.8 (lin), MO 5.0 (lin):
     Java not installed: (puzzle piece icon) with link, pops up alert
     saying that a plugin is missing
     Java disabled: blank block where applet should be

(19) IE 5.23 (osx):
     complains about script error, since it's not showing
     the alert-message i think this might happen when js is called from
     applet ... might be a wrong encoding in the js-string.
     
(20) IE 5.23 (osx):
     uses the standby-attribute text from object tag as fallback.

(21) NS 6.2.3 (xp):
     (puzzle piece icon) + missing plugin, clicking it goes to a missing
     page at netscape.com ...
     
(22) FF all (osx):
     seems to wait until the page is fully loaded (including applet)
     before displaying anything, bad with big applets (jars).
     no icon, no loading-bar, no text ...
     
(23) MO 5.0 (lin):
     javascript alert-window hung and was not closeable.

(24) OP 6 (osx):
     fallback text was displayed instead of applet

(25) OP 6, 9.1, Safari 2.0 (osx):
     Java disabled: applet ran anyway
     
(26) Sa (osx):
     Removing the Java-Plugins will sometimes not show fallback-text
     with applet-tag:
     Java enabled gives error-message, Java disabled gives empty block

--------------------------------------------------------------------------------

Notes:

- Many (most?) browsers seem to use internal caches for jars (and classes?).
  Caching was disabled in the plugin, nonetheless they reloaded the 
  applet almost instantly on a different page and tag.

- IE 5.2.3 osx :
  does not like the ";jpi-version=1.3.1" with object-tag as in:
  type="application/x-java-applet;jpi-version=1.3.1"
  
- Safari pre 2.x: there were reports saying that pre2.x versions did
  not read the params. could not confirm that ... might be due to
  a more recent applewebkit/418.9.1 on my machine (os-x 10.4.x)
  
- Safari 2.0.4, others?:
  using codebase=".." as attribute in the outer object tag is not working,
  make it a param just as with the inner-IE tag:
  <param name="codebase" value="--your-codebase--" />
  
- Safari 2.0.4, others?:
  calling javascript from applet when javascript is disabled is an 
  instant force-quit. not sure if this is happening in JSObject or via
  showDocument( 'javascript:..' );

--------------------------------------------------------------------------------
Again, bye-bye <applet>, hello <object>! 

As much as i dislike the object tag code-wise, it generates better results. It is by one case more compatible (compare 4th and 5th columns) and in regard to displaying the fallback-text it is far more reliable ( 2nd and 3rd column: 23-17 !).

Adding OpenGL

Changing the object tag to be OpenGL compatible was a snap and it turns out that it works as good as the old one. I've not done as extensive testing as applet-vs-object, but all major browsers supported it:

Still not the best user experience. 

After all this testing i figured (following another string of discussions) this might be a good chance to try to tackle some of the most annoying user-experience problems with applets: the long starting time the Java plugin (JVM) needs and the only-sometimes-working loading status display.

So, i added the following goals:  

Applets are not really user-friendly. Starting the Java-Plugin and with it the JVM takes time and once it's up you have to wait until all files (classes, jars) are downloaded for the applet to start. There are some things that we as developers can do to speed things up, but still a user with an old machine and a slow connection will look at a mostly ugly box until at least the very first class is loaded and running. This got a little better around Java 1.4.2, when the loading-image, message and bars were included, but they seems to have went away again with 1.5 (Java 5) leaving us with a funny sun.

There's only so much you can do on the Java side. A lightweight Java preloader would allow for an image or a message to be displayed, think splash screen. You could even try (java security?) to write your own ClassLoader which would theoretically give you access to the amount of bytes downloaded per file. But still the JVM must have started and the first file must be up and running.

Then i ran into Aaron Steeds (st33d) post about his appletobject.js. His idea was to use the same concept as the SWFObject for Flash, similar to what Karsten Schmidt (toxi) did here, and apply it to applets. Bravo!

Javascript to the rescue.

I've been playing with javascript based improvements to the applet loading status before. Back then i attempted to hide the applet-DOM-element until the applet was fully loaded and running. It worked sometimes, but not too many browsers liked it.

Aarons appletobject.js sparked the idea to go back and merge the two together (expanding both on the way). Here's what i have so far.

How it works ...

It's based on the Java to javascript connection called LiveConnect. After the AppletObject has been created and it's preload() function has been called it goes into a forked loop. For each jar that's needed for the applet to run there's a tiny applet injected to the page that loads a small class plus one jar to preload. That applet calls back (from Java to javascript) to the AppletObject once it's init() and again once it's start() functions were called by the Java-Plugin. When these get called the jar was successfully loaded into the Java cache. These callbacks are rerouted (after being handled internally) to user-defineable functions of the AppletObject (oninit(), onfail(error), onstep(percent), onload()). By overriding these you can for example add your own javascript magic to update a loading-bar on your page.

See sources for additional info: appletobject.js index.html

This method resolves both problems: the JVM is started before the applet is rendered to screen and all jar-files are preloaded. But it's not perfect; if your sketch consists of only one jar, then you should consider adding a spinning-icon instead of the loading-bar. Another problem is javascript and Java fighting for processor cycles, some browsers don't like that an might even crash. Let me know how it works for you and i'm happy to improve things if i can.