Newsletter sign-up

View all newsletters

Sign up for our technology specific newsletters.

Enterprise Java

Email Address:

The need for speed, stability

Raw VM performance improves across the board in the latest VolanoMark 2.1.2 server-side benchmarks, but network scalability remains a thorny issue

By Scott Plamondon, JavaWorld.com, 10/01/99

Tower Technology's TowerJ 3.1.4 for Linux dominates 15 other VMs in both performance and scalability, according to the latest VolanoMark 2.1.2 benchmarks published on October 3 by Volano LLC.

Moreover, compared to previous VolanoMark results published on May 31 (see Resources for a link to this report), raw server-side VM performance on the benchmark improved for all vendors and platforms. This trend should continue as vendors compete to produce the top VM.

Even with the increased performance, however, the network scalability tests -- with two exceptions -- show disappointing results, with some VMs incapable of handling anything but the simplest concurrent connection levels.

VolanoMark explained

The VolanoMark 2.1.2 pure Java benchmark measures both raw server performance and server network scalability performance. Both tests pit 16 VMs on seven operating systems against each other on the same 200 MHz Intel Pentium Pro hardware, thus achieving a good apples-to-apples comparison.

For both tests, the VolanoMark benchmark creates client connections in groups of 20 and measures how long it takes for the clients to take turns broadcasting their messages to the group. At the end of the tests, it reports a score as the average number of messages transferred by the server per second.

The raw performance test runs with the server and all of the simulated clients on the same machine, communicating over a local loopback connection, running at a static 200 concurrent socket connections.

In contrast, the network scalability test moves the simulated clients onto another, more powerful machine and communicates with the server over a 10 Mbps Ethernet connection. The client side runs four times for each VM, simulating 1,000, 2,000, 3,000, and 4,000 concurrent socket connections. Only those VMs that successfully scale up to 4,000 connections pass the test.

In both the raw performance and network scalability tests, the higher the score, the better the result. (See Resources for more on the hardware and test methodology.)

Performance: Speedier than a fast bullet

VM bragging rights rest on speed, speed, speed, and in this test round, vendors do not disappoint. In terms of raw speed, the 16 tested VMs showed good results across all vendors and platforms -- a big plus for the Java community.

TowerJ above the rest

For raw performance, Tower Technology's TowerJ 3.1.4 for Linux finished first, with an average score of 2,309, as shown in Figure 1 below. TowerJ 3.1.4, however, is the only static compiler in the test group. As such, it takes class files and converts them into C source code, which it then compiles into a native executable program (with version 3, TowerJ can also dynamically load Java class files in their original byte code form). It's also the only contestant in the group not available for free. "TowerJ is a native compiler that costs thousands of dollars," says Volano CTO John Neffenger. "It's expected that they'd be on top."

Figure 1. VolanoMark 2.1.2 local performance test, measuring throughput in messages per second
Source: Volano LLC

Java Platform	Score
Tower TowerJ 3.1.4 Linux	2,309
IBM JDK 1.1.8 Windows NT	2,279
IBM JDK 1.1.8 OS/2	2,136
Microsoft VM 3229 Windows 2000	1,970
IBM JDK 1.1.8 Linux	1,770
Sun HotSpot 1.0.1 Windows NT	1,601
Sun JDK 1.3 Beta Windows NT	1,550
Sun JDK 1.2.2 Windows NT	1,485
Microsoft VM 3186 Windows NT	1,478
Sun JDK 1.2.1_04 Solaris	1,358
Novell JDK 1.1.7 NetWare	1,229
Sun JDK 1.2.1_03 Solaris	1,051
Sun JDK 1.2 Linux	915
Transvirtual Kaffe 1.0b4 Linux	389
Blackdown JDK 1.1.7 Linux	285
JDK 1.1.8 FreeBSD	173

Neffenger adds that, among the standard pure Java VMs, IBM's various entries closely follow TowerJ. Indeed, IBM VMs held an impressive three of the top five spots in the raw performance tests. Big Blue's JDK 1.1.8 for Windows finished second with a 2,279 score, just 30 points behind TowerJ -- a 1.30 percent difference. IBM also locked up third place, with the IBM 1.1.8 for OS/2 VM scoring 2,136.

 

Improve efficiency with Visual Studio(r) 2010 Beta 2. - Microsoft

About Us | Advertise | Contact Us | Terms of Service/Privacy

Copyright, 2006-2008 Network World, Inc. All rights reserved.