Newsletter sign-up
View all newsletters

Enterprise Java Newsletter
Stay up to date on the latest tutorials and Java community news posted on JavaWorld

Sponsored Links

Optimize with a SATA RAID Storage Solution
Range of capacities as low as $1250 per TB. Ideal if you currently rely on servers/disks/JBODs

JavaWorld.com > Java Enterprise Edition >

Accelerate your Java apps!

Where does the time go? Find out with these speed benchmarks

By Mark Roulo, JavaWorld.com, 09/09/98

  • Print
  • Feedback
.
As a Java programmer, knowing the performance characteristics of different Java environments running on different operating systems is crucial. Having this information at hand can prepare you for potential bottlenecks, and it can save you from building bottlenecks into your apps accidentally. This article tests six different Java environments -- some with a just-in-time (JIT) compiler, some without -- running on four OSs and provides valuable benchmarks that can help you out with your Java development efforts.

The testing process

To understand performance characteristics and therefore where to expect bottlenecks, I ran benchmark tests on the following typical Java language constructs: method call, try/catch set-up, object creation, array creation, and array accessing. I didn't run tests of network I/O, disk I/O, or AWT performance -- the focus was purely on Java language performance. The tests were designed to avoid paging to disk.

Most of the tests required no garbage collection, so general system performance cannot be inferred by simply adding the results from the various tests together. I ran no general computation tests like "Tower of Hanoi" or "Sieve of Erastothenes." I omitted general computational tests because they're dedicated to showing relative speeds on differing platforms, and rarely show where bottlenecks are.

Target systems and environments

The benchmark tests I ran for this article were performed on a range of hardware systems and Java environments. The Java environments were:

Java Environments
Description JIT
Netscape Navigator 4.05 for Windows NT/95 Symantec Java! ByteCode Compiler Version 210.065
Netscape Navigator 4.05 for Power Macintosh Yes
Internet Explorer 4.0 for Windows NT/95 Yes
Symantec Visual Cafe PDE 2.1a for Windows NT/95 JDK 1.1.4 Symantec Java! ByteCode Compiler Version i300.009
Netscape Navigator 4.05 for SPARC No
Netscape Navigator 4.05 for Linux No


The hardware/OS platforms were:

Hardware/OS Systems
OS CPU (MegaHertz = MHz) RAM (megabytes = MB)
Windows NT SP3 Pentium Pro 200 MHz 128 MB
Macintosh 7.6.1 PowerPC 604e 180 MHz  
Solaris 2.5.1 UltraSPARC-1 167 MHz 128 MB
Red Hat Linux 5.1 Pentium-II 266 MHz 128 MB
Windows NT Dual Pentium Pro 180 MHz 32 MB


To compare the various systems, I converted the time it took to perform the various operations into clock cycles. Why? This conversion makes it possible to compare machines running CPUs at different speeds. In general, comparing different CPUs to each other in such a crude way can be dangerous, because the amount of work that can be done in a single clock cycle can vary a lot from CPU to CPU. The 80486, for example, averages about 2 clock cycles per instruction, while the Pentium executes closer to 1. Fortunately, the PowerPC 604e, UltraSPARC, Pentium Pro, and Pentium-II are roughly comparable. While cache behavior could be different between the various systems, this seems not to affect the performance much. All the tests ran without paging to disk.

Special resources
For information relating to the benchmark testing, I've provided the following links:

  • Print
  • Feedback

Resources
  • There isn't much information about HotSpot available (as of mid-July 1998), but one Web page with some data claims "Short lived object benchmark18 secs classic VM, 8 sec's on Jview, 6 secs malloc/free C, 2 secs HotSpot". If these numbers are representative, we should expect to see object creation time drop to about 3 times greater than C/C++ stack allocation, down from 10 to 12 times greater. http://java.sun.com/javaone/javaone98/keynotes/panel/bullets.htm
  • An interesting theoretical paper on object creation is "Garbage collection can be faster than stack allocation" by Andrew Appel. The paper is about object creation in Lisp, not Java, but still is relevant. It provides some ammunition for the position that in theory, Java object allocation on the heap can be faster than C/C++ object creation on the stack. http://www.cs.princeton.edu/fac/appel/papers/45.ps
  • A really good book on garbage collection is Garbage CollectionAlgorithms for Automatic Dynamic Memory Management by Richard Jones and Rafael Lins. I recommend this book to all serious Java programmers. http://www.cs.ukc.ac.uk/people/staff/rej/gcbook/gcbook.html
  • The article "Optimizing NET Compilers for Improved Java Performance" in the June 1997 issue of Computer compares some actual programs under various Java environments and also under C/C++. It provides a good starting point for answering the question, How fast is Java compared to C++? (as of mid-1997).
  • SPEC announced its JVM98 benchmark suite August 19 (just after we published this article) http://www.spec.org/osg/jvm98/
  • A "FAQs (Frequently Asked Questions) About the Multimedia Benchmarking Committee (MBC)" document notes in part"SPECjava will benchmark Java performance, which means essentially benchmarking the computing platform, as well as the Java virtual machine, the browser, and perhaps the just-in-time compiler." http://www.specbench.org/gpc/Dec97/mbc.static/mbcfaq~1.html
  • See JavaWorld's article "HotSpotA new breed of virtual machine" http://www.javaworld.com/jw-03-1998/jw-03-hotspot.html