Newsletter sign-up

View all newsletters

Sign up for our technology specific newsletters.

Enterprise Java

Email Address:

A do-it-yourself framework for grid computing

Learn to build your own grid computing application using open source tools

By Anthony Karre, JavaWorld.com, 04/25/03

Internet computing is a term commonly used to describe the use of multiple networked computers to run dedicated computing applications. The most famous example is The Search for Extraterrestrial Intelligence (SETI) project, in which half a million home and corporate PC users voluntarily use their spare processing power to analyze radio telescope data for SETI.

The SETI project is typical of the Internet computing domain in that its application is both dedicated and vertically integrated. In other words, no generic use of the computing resources is or can be made, and all dataflow between the computing resource and the data server uses proprietary APIs. When an application upgrade is required, manual intervention is needed to download new executables, and typically no management layer exists for dynamically pushing new computing tasks or otherwise manipulating the available computing resources.

Transformation: Internet to grid computing

Driven by the success of the SETI project and others like it, researchers have been working to exploit the vast pool of computing resources connected to the Internet, but in a way that is secure, manageable, and extendable. In the last few years, various frameworks have been developed, with much emphasis on the management of shared resources. These frameworks typically provide the following features:

• Machine independence through the use of Java and Web services
• Security through Web service implementations like SOAP (Simple Object Access Protocol), with its built-in accommodation of HTTPS and user authentication
• Task abstraction, that is, the ability for a management system to submit essentially arbitrary computing jobs to grid-based resources
• Management of non-CPU resources, like file storage
• Dynamic management of all of the above given a real-time assessment of the grid's status

Some frameworks have successfully evolved to the point where their architects have made them available in toolkit form. One example of such a toolkit is the Globus Toolkit 3.0, available through the Globus Project. This toolkit has been steadily evolving and has heavily influenced the development of the Open Grid Services Architecture (OGSA). Other published examples include a compute framework developed by Sun Microsystems researchers based on the Jxta peer-to-peer communication API.

Though powerful, these frameworks require a high degree of knowledge capital to successfully deploy and exploit. This can represent a barrier to entry of sorts for individuals or groups who wish to experiment with grid computing. For new programmers, this is like trying to use Struts for your first Web application, rather than writing your own servlets from scratch.

A minimal grid computing framework

In this article, I show you a small do-it-yourself framework that makes booting up a grid computing environment easy. This framework can be quickly developed using the common open source applications Tomcat and Axis, and, while simple, it still meets the following minimal grid needs:

 

About Us | Advertise | Contact Us | Terms of Service/Privacy

Copyright, 2006-2008 Network World, Inc. All rights reserved.