Newsletter sign-up
View all newsletters

Sign up for our technology specific newsletters.

Enterprise Java
Email Address:

Speed up batch file processing using generic programming and core reflection

Don't let batch file processing slow you down -- even if it is the computer that's doing all the repetitive monkey work

  • Digg
  • Reddit
  • SlashDot
  • Stumble
  • del.icio.us
  • Technorati
  • dzone
If you're at all like myself, you don't only use Java to write applications, applets, and JavaBeans. You also use Java to write everyday file and system utilities. You know, those myriad tools designed to make our lives as programmers easier. (Things like multiple-file search and replace, file analysis tools, file format massagers, and so on.) I've written several utilities that take a single file argument, and it often happens that I need to run such utilities on a list of files -- in other words, do some batch processing (typically on java or html files).

For example, say I've written a TAB-to-space file-converting utility (called Tab2Spc), and I need to apply this to all Java source files in a directory by creating a plain batch file like the following:

java Tab2Spc Angle.java
java Tab2Spc Border.java
java Tab2Spc Circle.java
 :             :
java Tab2Spc Value.java
java Tab2Spc World.java


Because Java executables only start running after the Java interpreter has loaded and initialized itself, you're looking at a multi-second delay before your utility can do any useful work. If that work itself only takes seconds (or less, as in the case of a simple TAB-to-space converter), using this utility in a batch script will waste an inordinate amount of time, by repeatedly loading and initializing the Java virtual machine (JVM).

What we really want is to have the JVM load and initialize just once per entire batch-processing session. The intuitive solution seems to be to create a ForEach Java utility that takes two parameters:

  1. The name of another Java program

  2. The name of a file containing filenames to execute the argument program on


Our previous batch file could then be transformed to the more simple ASCII file containing only the filenames to operate on:

Angle.java
Border.java
Circle.java
    :
Value.java
World.java


Starting the batch processing could then be achieved as follows:

java ForEach files.lst Tab2Spc


(Read this as: "For each file in files.lst, execute the utility Tab2Spc.")

Assuming that Tab2Spc had the standard application main() entry point,

public static void main(String[] args)


how would you invoke this main() from within the ForEach utility?

Because main() is a static method, you simply couldn't invoke it with the 1.0 release of Java! You could dynamically load the argument program easily enough with Class.forName()... and then instantiate an object of the program's class with Class.newInstance(), but those steps still wouldn't give you any way to invoke the main() method of the utility. Intuitively, we would like to do something like the following:

Class externalProgram = Class.forName("Tab2Spc");
Object instance = externalProgram.newInstance();
instance.main(someArgs);


That last line is where intuition and reality part company. Method main() isn't defined for objects of class Object, so we get the compile-time error:

Method main( String[] ) not found in class java.lang.Object.


Fine. More stubborn (but misplaced) Java programmer's intuition tells me I should use a cast to solve this problem, but a cast to what? The logical answer is a cast to an interface type of Application, where Application defines a single method main(), with the well-known signature. But here's where we encounter another obstacle: interfaces can't be used to define static methods. In the prehistoric days of pre-reflection Java, my less-than-perfect workaround solution was to create a very similar Application interface and have all my utilities implement this:

  • Digg
  • Reddit
  • SlashDot
  • Stumble
  • del.icio.us
  • Technorati
  • dzone
Comment
Login
Forgot your account info?
Add comment
Anonymous comments subject to approval. Register here for member benefits.
Have a JavaWorld account? Log in here. Register now for a free account.
Resources
  • ObjectSpace's JGL home page http://www.objectspace.com/jgl/
  • See JavaWorld's article on JGL "Need a good set of abstract data structures? ObjectSpace's JGL packs a punch!" http://www.javaworld.com/javaworld/jw-06-1997/jw-06-jgl.html
  • David R. Musser provides a page on generic programming. (Musser worked with Stepanov.) http://www.cs.rpi.edu/~musser/gp/
  • For an interview with Alexander Stepanov, see http://www.bml.ca/marine/stepanov.htm
  • The Java Tutorial section on Core Reflection is available at http://java.sun.com/docs/books/tutorial/reflect/index.html