Newsletter sign-up

View all newsletters

Sign up for our technology specific newsletters.

Enterprise Java

Email Address:

Which Java VM scales best?

A JavaWorld Exclusive! Results of new VolanoMark 2.0 server benchmark show how 12 virtual machines stack up

By John Neffenger, JavaWorld.com, 08/01/98

I find it ironic that the one company with a not-so-secret objective to "kill cross-platform Java" (according to an internal Microsoft document; see Resources below) also is the Java vendor that kept our 100% Pure Java company (Volano) in business for the past year and a half. Without the impressive performance of Microsoft's Java virtual machine, we would not have acquired the many VolanoChat customers who run our chat server on high-traffic Web sites. My article in the December issue of JavaWorld summarized our experience and reported the performance scores of four Java virtual machines using our Java server benchmark, VolanoMark 1.0.

While writing that article in November, I suggested to a Microsoft product manager that Sun might eventually trade places with Microsoft as the Java performance leader. "It'll never happen," was his quick reply. Although it might still be hard to argue with his response, most of the Java VM vendors have significantly narrowed Microsoft's lead.

Mostly in response to the December article, I have received more than 450 e-mail messages from 19 Java vendors trying to solve the technical problems they encountered while running VolanoMark and various Java server applications that the benchmark represents. Their efforts are now starting to reduce the great disparity that exists between Java virtual machines commonly used on the Internet today.

What is VolanoMark?

VolanoMark is a 100% Pure Java server benchmark characterized by long-lasting network connections and high thread counts. In this context, long-lasting means that the connections last several minutes or longer, rather than just a few seconds. The VolanoMark benchmark creates client connections in groups of 20 and measures how long it takes for the clients to take turns broadcasting their messages to the group. At the end of the test, it reports a score as the average number of messages transferred by the server per second. Its results have accurately predicted the real-world Java performance and scalability limits of our VolanoChat product line for almost two years now.

VolanoMark 2.0, which adds scalability measurements, is an update to the tool and is the first benchmark to be submitted to the SPEC Open System Group for use in its server-side Java benchmark suite. There are other Java benchmark programs under consideration for the SPEC suite that have different characteristics, such as the high connection-turnover rates of a Java Web server or the transaction-processing requirements of a Java database.

When trying to determine whether a product meets your performance and scalability requirements, the best approach is to write your own benchmark and obtain your own results. The next best approach is to run a benchmark program yourself on your own system. VolanoMark 1.0, for example, is available as a free download from Volano's Web site. (See the Resources section below.) In any case, when you read benchmark reports you should make sure you understand the characteristics of the test and the configuration of the system. Without information about the hardware configuration, the operating-system settings, and the Java virtual machine heap and stack options, benchmark scores are meaningless. Fortunately, SPEC will provide a set of well-defined rules for running its server-side Java benchmark suite when it becomes available early next year.

 

Improve efficiency with Visual Studio(r) 2010 Beta 2. - Microsoft

About Us | Advertise | Contact Us | Terms of Service/Privacy

Copyright, 2006-2008 Network World, Inc. All rights reserved.