A do-it-yourself framework for grid computing
Learn to build your own grid computing application using open source tools
By Anthony Karre, JavaWorld.com, 04/25/03
- Digg
- Reddit
- SlashDot
- Stumble
- del.icio.us
- Technorati
- dzone
Internet computing is a term commonly used to describe the use of multiple networked computers to run dedicated computing
applications. The most famous example is The Search for Extraterrestrial Intelligence (SETI) project, in which half a million home and corporate PC users voluntarily use their spare processing power to analyze radio
telescope data for SETI.
The SETI project is typical of the Internet computing domain in that its application is both dedicated and vertically integrated.
In other words, no generic use of the computing resources is or can be made, and all dataflow between the computing resource
and the data server uses proprietary APIs. When an application upgrade is required, manual intervention is needed to download
new executables, and typically no management layer exists for dynamically pushing new computing tasks or otherwise manipulating
the available computing resources.
Transformation: Internet to grid computing
Driven by the success of the SETI project and others like it, researchers have been working to exploit the vast pool of computing
resources connected to the Internet, but in a way that is secure, manageable, and extendable. In the last few years, various
frameworks have been developed, with much emphasis on the management of shared resources. These frameworks typically provide
the following features:
- Machine independence through the use of Java and Web services
- Security through Web service implementations like SOAP (Simple Object Access Protocol), with its built-in accommodation of HTTPS and user authentication
- Task abstraction, that is, the ability for a management system to submit essentially arbitrary computing jobs to grid-based
resources
- Management of non-CPU resources, like file storage
- Dynamic management of all of the above given a real-time assessment of the grid's status
Some frameworks have successfully evolved to the point where their architects have made them available in toolkit form. One
example of such a toolkit is the Globus Toolkit 3.0, available through the Globus Project. This toolkit has been steadily evolving and has heavily influenced the development of the Open Grid Services Architecture (OGSA). Other published examples include a compute framework developed by Sun Microsystems researchers based on the Jxta peer-to-peer communication API.
Though powerful, these frameworks require a high degree of knowledge capital to successfully deploy and exploit. This can
represent a barrier to entry of sorts for individuals or groups who wish to experiment with grid computing. For new programmers,
this is like trying to use Struts for your first Web application, rather than writing your own servlets from scratch.
A minimal grid computing framework
In this article, I show you a small do-it-yourself framework that makes booting up a grid computing environment easy. This
framework can be quickly developed using the common open source applications Tomcat and Axis, and, while simple, it still meets the following minimal grid needs:
- Digg
- Reddit
- SlashDot
- Stumble
- del.icio.us
- Technorati
- dzone
Resources
- Learn about the Search for Extraterrestrial Intelligence (SETI@home) project
http://setiathome.ssl.berkeley.edu/
- Global Grid Forum
http://www.ggf.org/
- Open Grid Services Architecture Working Group (OGSA WG)
http://www.ggf.org/ogsa-wg/
- The Globus Project
http://www.globus.org
- Sun Microsystems Research Paper"Framework for Peer-to-Peer Distributed Computing in a Heterogeneous, Decentralized Environment,"
Jerome Verbeke, Neelakanth Nadgir, Greg Ruetsch, Ilya Sharapov
http://wwws.sun.com/software/jxta/mdejxta-paper.pdf
- Project Jxta
http://wwws.sun.com/software/jxta
- Apache Tomcat homepage
http://jakarta.apache.org/tomcat/
- SOAP W3C (World Wide Web Consortium) specification
http://www.w3.org/TR/SOAP/
- The Apache Axis SOAP implementation
http://ws.apache.org/axis/
- "Create a Custom Java 1.2-Style ClassLoader," Ken McCrary (JavaWorld, March 2000)
http://www.javaworld.com/javaworld/jw-03-2000/jw-03-classload.html
- The Apache Struts Web application framework
http://jakarta.apache.org/struts/
- Apple Mac OS X Project Builder IDE
http://developer.apple.com/tools/projectbuilder/index.html
- More grid computing sources from IBM developerWorks
http://www-106.ibm.com/developerworks/grid/
- Learn more about Macintosh OS X development
http://developer.apple.com/macosx/
- More JavaWorld stories on Axis:
-
- For SOAP basics, read "Clean Up Your Wire Protocol with SOAP," Tarak Modi (JavaWorld)
-
- For an overview of grid computing, read "IBM's Grid Conversion," Robert McMillan (JavaWorld, September 2002)
http://www.javaworld.com/javaworld/jw-09-2002/jw-0906-grid.html
- Browse the Enterprise Java section of JavaWorld's Topical Index
http://www.javaworld.com/channel_content/jw-enterprise-index.shtml
- Browse the Java and Web Services section of JavaWorld's Topical Index
http://www.javaworld.com/channel_content/jw-webserv-index.shtml
- Talk more about grid computing in our Enterprise Java discussion
http://forums.devworld.com/webx?50@@.ee6b80a
- Sign up for JavaWorld's free weekly email Enterprise Java newsletter
http://www.javaworld.com/subscribe
- You'll find a wealth of IT-related articles from our sister publications at IDG.net