Newsletter sign-up

View all newsletters

Sign up for our Enterprise Java Newsletter

Enterprise Java

Which Java VM scales best?

A JavaWorld Exclusive! Results of new VolanoMark 2.0 server benchmark show how 12 virtual machines stack up

By John Neffenger, JavaWorld.com, 08/01/98

Page 3 of 6

Current Java VMs: Free and available now

Connection scalability of Java virtual machines that are freely available now. Bigger numbers are better. All JVMs ran the same test on the same Intel hardware platform except for Apple MRJ 2.0, which ran on a Power Mac G3.

These seven Java virtual machines are the ones we recommend to our customers for running the VolanoChat product live on production Web sites. The chart shows the VolanoMark 2.0 scores for seven Java VMs as the number of connections are increased. The number of connections are shown across the bottom, with the throughput in messages per second shown on the left. The scores are shown at up to 900 concurrent connections. Failed tests in the table are indicated by "---" (no score shown).

Microsoft SDK 2.02 still stands alone as the only fast and scalable Java virtual machine. Our customers with the highest Web site traffic currently have no other viable choice for a JVM. I had to restart the JavaSoft JDK 1.1.6 virtual machine five times between tests in order to reach 900 concurrent connections, and none of our large customers have succeeded in running our VolanoChat product with JavaSoft's JVM on Windows NT. IBM's recent updates to its operating system, TCP/IP stack, and JVM have given it a huge 50 percent performance improvement over the VolanoMark 2.0 tests I ran without the updates (not shown here), but I could not get more than 400 concurrent connections with IBM's JVM.

We had been waiting for more than a year for a stable Solaris Java VM and Sun finally delivered.

The SunSoft JDK 1.1.5 VM for Solaris may be a bit laggard in its performance, but its reliability is rock solid. We had been waiting for more than a year for a stable Solaris Java virtual machine with native threads and a just-in-time compiler, and Sun finally delivered on April 6, 1998. For me this was the biggest Java news of this year, but it went completely unnoticed (perhaps because its robust performance was not immediately apparent to the trade press that covered its release). After working with Java virtual machines on the server since early 1996, I had begun to lower my expectations of quality -- but the SunSoft JDK 1.1.5 sets a new standard for stability. No other JVM comes close. We ran our VolanoChat demonstration server on this JVM for more than 32 days non-stop -- that's roughly 40 messages per second, 24 hours per day for 32 days, without any down time and without even going over its initial 8-MB heap size! I think it would have run forever, but I had to bring it down in order to upgrade the VolanoChat server. (In fact, we haven't rebooted the Solaris 2.6 operating system since we first set it up on our Internet Web and chat server machine on November 5, 1997.)

	Number of connections
Java Virtual Machine	2	100	200	300	400	500	600	700	800	900
Microsoft SDK 2.02	1751	2525	2090	1767	1556	1365	1242	1126	1030	946
JavaSoft JDK 1.1.6	1420	2082	1997	1797	1578	1381	1235	1099	1011	951
IBM JDK 1.1.6	1235	2009	1843	1853	1603	---	---	---	---	---
SunSoft JDK 1.1.5	902	1221	1134	1051	888	725	322	178	145	102
Apple MRJ 2.0	119	1090	950	781	877	767	---	---	---	---
Linux JDK 1.1.6	836	688	429	---	---	---	---	---	---	---
FreeBSD JDK 1.1.5	516	443	382	---	---	---	---	---	---	---

Apple's Java virtual machine performed better than I had expected, although I had to restart the server between each test. The Linux and FreeBSD virtual machines do quite well considering that neither of them have a just-in-time compiler or native threads. For both Linux and FreeBSD, I was unable to get 300 concurrent connections or more, even after I increased their per-process file descriptor limits to more than 1000. In addition, the Linux and FreeBSD VMs have problems with the new Java 1.1 socket timeout feature, although I have been unable to isolate the cause. With the socket timeout feature enabled in the VolanoMark test, the Linux JVM runs extremely slowly and the FreeBSD JVM crashes with a core dump. Disabling the feature allowed them both to run the tests successfully.

 

About Us | Advertise | Contact Us | Terms of Service/Privacy

Copyright, 2006-2008 Network World, Inc. All rights reserved.