March of the Mac IDEs

Four Java development tools for Mac compared

1 2 3 4 Page 2
Page 2 of 4

One disadvantage of Roaster's project manager is that it does not give you a means to automatically "remove binaries." If you want to assure a clean build from scratch, you have to manually remove the .class files in the Finder. Both Café and Discover Java offer a convenient menu command to do this automatically.

Sun's JDK, of course, does not have a project manager, but if you keep your packages in places that make sense relative to the file you are compiling, or if you set the CLASSPATH variable appropriately (not so easy on the Mac JDK), the compiler will do a pretty good job of finding the files it needs and checking dependencies for you.

Class browser

All three of the IDEs provide a Smalltalk-style class browser, an approach to editing that harmonizes well with the object-oriented nature of Java. The differences in the browsers are largely trivial.

Roaster's is accessible from a nested submenu of the New... item in the File menu. This seems a less intuitive choice than the Window menu, which is where the other IDEs put it. In Metrowerks, although the browser is accessible from the Window menu, this item is grayed out unless you have checked the "Activate Browser" item in the Preferences dialog. This checkbox is unchecked by default, and checking it requires that you recompile your project before the browser item can be enabled in the menu. If you don't look carefully, you might get the mistaken impression that the class browser was an aspect of the CodeWarrior IDE that is not implemented for Java. All three IDEs allow editing of source code directly from the browser.

In addition to a class browser, both Roaster and Café also include a class tree, a feature not provided by CodeWarrior. Café's class tree is integrated into its class browser window, which is completely configurable. The Roaster class tree, also accessible from a nested submenu of the File menu, is a separate entity from the class browser, although it includes a browser-like section for class information. Roaster's class-browsing windows are generally less configurable than Café's, but Roaster does a much neater job of drawing the class tree itself, using non-intersecting perpendicular lines to connect the classes. It allows you to collapse the hierarchy at any node for less cluttered viewing.

Debugging tools

Debugging tools are an arena in which all three IDEs begin to smell of vaporware. All three companies claim that source-level debugging tools are just around the corner. Roaster's source-level debugger almost works in the current release (but not quite, especially on 680x0 Macs). The next version of Roaster, DR2.1 (to be released by the end of the summer), is expected to have a more fully-functional debugger.

Symantec has included a debugger in Café DR2, but it, too, lacks important functionality, such as the ability to remember breakpoints. It also crashes occasionally.

The Metrowerks debugger is not fully completed either. Instead of working from directly inside the IDE, you must launch a separate application, called MW Debug/Mac OS 1.5b2, which allows you to set breakpoints, and then run your applet or application with the Metrowerks Java interpreter. This is not at all intuitive, especially if you are used to CodeWarrior's normally smooth debugging process; indeed, the release notes claim that this is a temporary situation and that it will be fixed in future updates. Once you've got the debugger running -- not an easy task in its current incarnation, and with its current documentation -- it proves itself to be a well-designed, extremely useful tool, leveraging the quality of Metrowerks' existing Mac debugger.

The Mac version of the JDK does not come with any debugging tools at all, nor has Sun promised a Mac version of jdb, the command-line debugger that comes with Unix and Windows versions of the JDK.

Visual GUI-building tools

Symantec's Café Studio, which allows you to visually construct applet GUIs, is the only tool of its kind currently available for the Macintosh. Natural Intelligence has promised "visual tools," but the exact nature of these tools has yet to be revealed. No such tool is currently promised by Metrowerks, although it is likely that such a tool will be included by the time CodeWarrior 10 is released in September. (Still, this doesn't help you if you only own Discover Programming, which doesn't include major updates.)

Symantec concedes that Café Studio is a work in progress and that it is of pre-beta quality at its currently-shipping version, 1.0d2. It does not currently support all of the AWT widgets, the most noticeable omissions being Canvas and multiline TextArea widgets. Café Studio is provided in its current version in order to speed up the process of designing the simplest GUI components, and its output is Java code that you can edit, as long as you only type between the special comments that the Studio application places in the code.

From a user-interface point of view, some cosmetic changes in Café Studio would make it easier to use. For example, when you select a GUI item on the screen, it has only one handle for resizing, located in the lower right-hand corner. A better idea would be a handle at each corner, and on the sides as well, for flexible editing. This was apparently a shortcut for designing the product, since AWT widgets are placed via four variables: the ( x, y) coordinates of their upper-left corner, their width, and their height. Nonetheless, there is no need to translate this restriction to the user interface of a visual tool. Small things like this, and the fact that Studio only gives you a one-line text entry to define code for a widget's behavior in response to events, give Studio a clunky user interface. It would also be nice to see Symantec design some custom, composite widget classes for the final version of Café Studio, such as scrollable canvases, GIF buttons, or animations.

Studio has potential, but in and of itself it's not a reason to buy Café. Symantec deserves some credit for being the first to make such a tool available, but it's not really useful yet, except perhaps for the most rudimentary form-style interfaces. In the meantime, it seems likely that Café Studio is going to be discontinued in favor of a separately priced product called Visual Café. Although Symantec had initially promised that the Windows and Macintosh versions of Visual Café would be available simultaneously, lately the company has been noticeably silent with regard to this plan. The Windows version is promised by the end of July. If past performance is any indication, we can expect to wait a little longer for the Mac version.

Performance

Compilation speed

Sun's javac compiler is written in Java. Naturally, this means that its performance is going to be slower than a compiler that is executed at native speed. The JDK obviously uses javac. Metrowerks uses a version of javac hosted inside a plug-in for its modular architecture.

Both Symantec and Natural Intelligence, on the other hand, have written their own Mac-native Java compilers from scratch and give you the option of using either javac or the native compiler. In the case of Roaster, however, you really don't have a choice. Roaster's native compiler was described by the release notes in Developer Release 1 as "icky." You are warned that it tends to crash, and indeed, that's exactly what it does, even in Developer Release 2. The good news is that Roaster's version of the JVM is so efficiently implemented that using javac does not set you back all that much. Metrowerks also has a highly efficient virtual machine implementation, and in my tests it demonstrated nearly identical compile time to that of Roaster when using javac.

Café's native compiler represents the fastest method available of compiling Java on the Mac. On the other hand, if you use the javac option in Café, it runs significantly slower than any of the others. It's important to note that although Café's native compiler is the fastest one available, the current version cannot necessarily be counted on to produce correct Java bytecode. At least one reproducible bug, relating to bitwise operators (in particular &, |, and ^), has been discovered. Whenever these operators are used in an expression with the constant 1, Café's compiler generates erroneous bytecodes. Symantec has apparently developed a fix for this error but has not yet made it available to the public. For further updates regarding this issue, your best bet is to keep an eye on the comp.lang.java newsgroup.

The following table contains compile times, in minutes and seconds, for a fairly small sample applet I wrote (about 1,000 lines), consisting of nine classes in six files, five of which belonged to a package of their own:

Roaster JDK Café Metrowerks
Native Compiler crashed-0:41-
javac 0:571:141:340:59

Note that none of the tools give you an easy, built-in facility for measuring compile time. Since Roaster, Café, and CodeWarrior are all scriptable, I could have written AppleScripts to run and time the compiles, but this method would have excluded the JDK, which is not scriptable. Instead, I took the easy way out and used a stopwatch. All tests were done on my Power Mac 7100/66 with 24 megabytes of RAM. These are by no means exhaustive tests, and I have no doubt that others may find differing results using different benchmarks.

Run-time performance

Each of the tools provides its own implementation of the JVM. The JVM is the software entity that reads and executes platform-independent Java bytecodes. Each time it executes a bytecode instruction, it has to do extra work to translate that instruction into native code that can be run on the client platform. In DR2 of Café, Symantec has included a just-in-time (JIT) compiler (PowerPC only), which aims to significantly improve the run-time performance of Java by translating bytecode into native code as each bytecode file is loaded. Although the translation step delays execution of the code, it is only a one-time performance hit, typically outweighed by the benefit of running the code at native speed. Café currently sports the only JIT for the Mac. Natural Intelligence is working on a JIT for Roaster, but it is not yet available.

It's important to put performance of the virtual machine (VM) into perspective. Since Java applets are typically developed to run on someone else's physical machine, inside a Web browser, the speed of the virtual machine on the development platform is irrelevant to the end user. However, if you're compiling with javac, then VM performance is very important to you, because it affects your compile time. Also, if you're testing your applets on a very fast VM, you may be unaware of performance issues that arise when your applet runs on a slower VM, such as the one currently built into Netscape. So, although performance of the VM is not the most important factor in choosing an IDE, it's still worth knowing where each implementation falls in terms of performance.

Performance of the VM -- like that of any hardware CPU -- is highly benchmark-dependent, and to my knowledge, a full suite of benchmarks has not yet been developed for objectively evaluating the performance of implementations of the JVM. In an effort to obtain some sort of objective result, I added four lines of code to Sun's Fractal demo applet. The added code simply times the applet's run() method, and prints the result on the standard output. I then compiled the applet and ran it on all four VMs. I also ran it on Netscape Atlas Gold PR2 for the Macintosh. All tests were done on my Power Mac 7100/66 with 24 megabytes of RAM. Again, these tests are by no means exhaustive.

Time (milliseconds)

Roaster

3616

JDK

5573

Café

7990

Café (with JIT)

5853

Metrowerks

6121

Netscape Atlas PR2

10634

Quite surprisingly, both Roaster's virtual machine and that of Sun's JDK outperformed Café's just-in-time compiler on this particular test. Representatives from Symantec are fond of ridiculing Natural Intelligence's claim to the fastest virtual machine around, illustrating their point by racing Café's JIT against Roaster's VM -- a slanted comparison to begin with, especially given their choice of benchmarks: a group of sorting algorithms that each require only one class to be loaded. My measurements illustrate that the benefits of just-in-time compilation are dependent on the individual application in question.

Although this test does not disprove the usefulness of JIT compilation, it lends support to NI's claim that Roaster's virtual machine is the fastest implementation available on the Mac platform. Very likely, this is why Apple has chosen to license Roaster's VM along with its upcoming JIT (see "Future prospects," below), for inclusion in version 8 of MacOS.

Incidentally, my choice of the Fractal demo was not  a deliberate attempt to defeat the JIT. I chose this particular applet before Symantec released the JIT for Café. My reasons for choosing it were simply that it took a noticeably long time to run and that it had clear locations at which to insert timing code.

Note that Netscape's abysmal performance may be due to an inferior virtual machine implementation, or it may be due in part to the fact that Netscape's class loader, for security reasons, performs a verification step before running Java bytecodes, even if they are loaded from the local filesystem.

Stability

Those who experienced the first developer release of Roaster will be glad to find that this later release is much more stable. In DR2, Natural Intelligence has fixed virtually all of the crashing problems that plagued DR1. CodeWarrior is also quite stable.

1 2 3 4 Page 2
Page 2 of 4