When Java landed on the world stage in the spring of 1995, it represented a paradigm shift for the software industry. It demonstrated a new way to administer and deliver software in the dawning network age. Java technology was well suited for networked environments because of its platform independence, built-in security mechanisms, and robustness. Java applets showed how software could be administered on a single server and downloaded to multiple clients via a Web browser. When a new version of a program was ready, only the server needed updating. Clients would automatically download the new version the next time they accessed the Web page.
The trouble with applets
Although implementing programs as applets embedded in Web pages greatly simplifies some aspects of software administration and delivery, it also places some constraints on the form and size of the programs. For example, developers are compelled to use the browser metaphor for the user interface of their programs. In this metaphor, the various functions of a program are divided among several applets, which are placed on different Web pages. Applets are downloaded when the user accesses the containing Web page. Although the browser metaphor can be a simple, easy-to-understand user interface for some programs, it can be cumbersome for others.
On the other hand, it is possible to write an entire program as a single applet that opens up a separate window from the browser, thus avoiding the browser metaphor. However, a different problem manifests itself as an applet grows larger, because applets usually are downloaded across the network each time they are used. When people want to use an applet, they go to the appropriate Web page in their browser, wait until the applet is downloaded across the network, and then use it. Once an applet exceeds a certain size, it becomes painful to use because it takes too long to download.
The size limit of applets hinges on the time it takes the applets to download. Various factors, such as network bandwidth, network traffic, and user patience, influence an applet's maximum practical size. Over time network performance can be expected to improve, thus bringing some relief to the applet size limitation. However, just as user expectations for software have increased to coincide with advances in available disk space, RAM, and processing power, so too will user expectations for network-delivered software likely increase as network bandwidth improves.
Popular browsers have also imposed one other restriction on applets in the interest of security: Applets downloaded across the network aren't allowed to read or write files on the local disk. This restriction may be relaxed in the future, by allowing the user to specify varying degrees of access to the local disk depending upon the source of the applet. Until that day comes, however, this security restriction is one of the barriers preventing applets from becoming a viable solution for many types of software problems.
A new channel for network-delivered Java
A few weeks ago a new technology was unveiled that addresses some of the problems with delivering software as applets. Marimba Inc., founded last February by four former members of the Java team at Sun, has released a beta version of a product called Castanet. Castanet enables the network delivery of bundles of software and content, which they call channels. In the first version of Castanet, a channel is either a Java program or a set of HTML pages. Clients use what Marimba calls a tuner, which allows users to subscribe to channels. The server uses what Marimba calls a transmitter to manage a channel. Here's a graphical depiction of Marimba's model and terminology:
A channel is, at its most fundamental level, a directory hierarchy filled with files. Castanet provides a way to distribute the channel's files and directory hierarchy from a server to clients. When you subscribe to a channel, you get a complete copy of the channel's files and directory hierarchy on your local disk.
Java programs (applets and applications) are composed of class files and, possibly, data files. With little or no tweaking to source code, Java programs can be delivered as Castanet channels. Castanet, therefore, allows Java programs to be delivered across networks without a browser. In this scheme, the Castanet tuner takes the place of the browser. This eliminates the requirement that network-delivered Java programs adopt the browser metaphor, because the program has jumped out of the browser.
The channel metaphor
The Castanet tuner uses the desktop metaphor rather than the browser metaphor. Channels look like regular applications on the client's desktop. Therefore, the developer has free reign in user-interface design. The channel metaphor, then, is one in which network-delivered applications, which are mirrored in entirety on the user's local disk, appear as desktop applications to the user. The main difference from the user's perspective between a Castanet channel and a traditional desktop application is that the channel automatically installs and evolves over time. In this regard channels behave less like traditional desktop applications and more like applets. The channel metaphor represents a new way of using the network, but looks and feels like the old way of using computers.
Java programs delivered as channels are free from the size constraint of applets. The larger an applet is, the longer it takes to download. Once an applet becomes too large, the program must be partitioned into several applets and spread across several Web pages. Java applets have this problem because they are delivered on demand. The browser does not attempt to retrieve the applet until the user requests the Web page that hosts the applet. At that point, the user is ready to go, but must wait for the applet to download. Java programs delivered as channels do not have this problem, because channels are not delivered on demand. They are, in effect, delivered before demand.
When you first subscribe to a channel, you must wait while the channel's entire directory hierarchy is downloaded to your local disk. This may take quite some time, depending on the size of the channel. However, after the initial download, only changes are delivered. Rather than downloading the channel in entirety each time, as is done with applets, channels are updated based on any files and directories that were added, deleted, or changed since the last download. Channels are not updated upon each use, as are applets, but rather on a regular frequency that is specified by the channel developer. You don't have to wait for a download when you want to use a channel. Because the channel is already on your local disk, you can start using it immediately.
Although channels break through the size barrier of applets, there is still a size limitation to Java channels gated by both disk space and download time. However, by abandoning the "download on demand" model of applets, a channel's maximum practical size is much greater than that of an applet.
At the frequency specified for each channel by the developer, the tuner sends a request to the appropriate transmitter for an update. The request sent by the tuner contains an "index" that identifies the version of the files and directories current on the local disk. No state is kept on the server that tells it anything about a client's current channel version. The transmitter determines whether any changes are necessary on the client side based upon the uploaded index, and replies with the changes required to make current the client's copy of the channel. Updates occur in units of files. If only one byte is changed in a file on the server, the entire file must be downloaded again to the client. This process of sending only the changes is called differential updating, a far more efficient use of bandwidth than the model used by applets, in which the entire applet is sent every time.
Marimba has applied for a patent for their solution to the problem of differential updating. Their solution uses a "pull" model, in which the clients periodically poll for updates, instead of a "push" model, in which the updates are broadcast from the server. The pull model allows for correct operation through firewalls, because the transaction is initiated at the client rather than the server. The pull model also works better with intermittently connected machines such as notebook computers.
Because servers do not keep track of any client's current version, the index sent by the tuner must contain enough information for the transmitter to determine the required changes on the client side, if any. Marimba designed a format for an index that indicates the version of each file and directory in the channel's hierarchy of files and directories. The index data structure is a tree structure where each node of the tree corresponds to one file or directory in the channel's hierarchy. Each node in the tree contains a 128-bit "fingerprint" that is generated by running the content of the file or directory through the MD5 algorithm. Regardless of a file's length, a 128-bit fingerprint is generated. The fingerprint is used as an indicator of the file's content, because the odds are low that two files, or two different versions of the same file, will have the same fingerprint. A fingerprint is generated for each directory as well, all the way up to the top-level directory in the channel.
When a transmitter receives an index, it compares the hierarchy of fingerprints from the client to the hierarchy of fingerprints representing the most current state of the channel. Comparison is aided by the fact that the directories, and not just the files, are fingerprinted. For example, if the fingerprint of the top-level directory matches, the transmitter can assume the client is already up to date, with no need to compare individual files. By comparing fingerprints, the transmitter can determine if any changes are needed on the client side, and if so, what changes are required.
The transmitter performs one other function to ease bandwidth requirements: It caches the most recent indexes. Each time a channel changes, a new index is generated that represents the new state of the channel. When a channel changes on the transmitter, the previous index is not thrown away. A certain number of previous indexes representing the most recent channel states is cached by the transmitter. When a client requests an update, it doesn't initially send its entire index. Instead, it just sends the 128-bit fingerprint for its top level directory. The transmitter compares this top-level fingerprint with those in its cache, and if a match is found, uses the cached index for its compare. If the transmitter does not find a match for the top-level fingerprint in its cache, the transmitter requests the complete index data structure from the client, which replies with the complete index. Marimba's mechanism for channel updating ensures that each client will always be at one of the states downloaded from the server, and never in an in-between state. Because most requests for updates from clients will take place when the client is in one of the most recent channel states, most update requests will not require the client to send a complete index, thus saving bandwidth.
Java channels have the same level of security as applets, with one exception. Java channels are assigned one directory on the local disk under which the channel can read and write files. Java channels effectively have access to their own persistent storage sandbox on the local disk. This addresses another problem that applets currently have in the popular browsers: safe access to persistent storage. Because each channel is assigned its own unique area on the local disk, one channel can't meddle with another channel's files, or with the user's non-channel files. It provides security while allowing persistent storage.
The channel model also ensures that a program can be used offline. With simple caching, such as is commonly employed by browsers, you don't get a piece of data in your cache unless you actually access that piece of data while connected to a network. For example, if you visit a site about camel grooming, which contains 20 Web pages packed full of useful information, but you only access 18 of them in a frenzy of information gathering, you'll only have 18 of those pages in your cache. If, later, you wish to review the information offline, you can call up the 18 pages in your cache, but won't be able to get at the two pages that were not cached.
A Castanet channel is downloaded to the client's disk in totality. Because one type of channel is a Web site, an entire Web site can be mirrored on a client's disk. If you subscribed to the camel-grooming Web site channel, you'd get all 20 Web pages on your disk. The same holds true for Java programs delivered as channels. They are always delivered as complete packages. Therefore, if you are offline, you may not have the latest version of the channel, but you will definitely have a complete version.