Hello Camel: Automatic File Transfer

Apache Camel [1] is described on its main web page [2] (and in the Camel User Guide [3]) as "a versatile open-source integration framework based on known Enterprise Integration Patterns." The Camel framework is based on the book Enterprise Integration Patterns [4] and provides implementations of the patterns [5] described in that book [6]. I look at a "Hello World" type example of using Camel in this post.

The Camel web page and Users Guide also reference the StackOverflow [7] thread what exactly is Apache Camel? [8] that includes several good descriptions of Apache Camel. David Newcomb [9] has described Camel there:

Apache Camel is messaging technology glue with routing. It joins together messaging start and end points allowing the transference of messages from different sources to different destinations. For example: JMS->JSON, HTTP->JMS or funneling FTP->JMS, HTTP->JMS, JMS=>JSON.

In this post, I look at a simple use of Camel that doesn't require use of a JMS provider or even FTP or HTTP. Keeping the example simple makes it clearer how to use Camel. This example uses Camel to transfer files automatically from a specified directory to a different specified directory. Three cases will be demonstrated.

In the first case, files placed in the "input" directory are automatically copied to an "output" directory without affecting the source files. In the second case, the files placed in the "input" directory are automatically copied to an "output" directory and then the files in the "input" directory are stored in a special ".camel" subdirectory under the "input" directory. The third case removes the files from the "input" directory upon copying to the "output" directory (effectively a "move" operation). All three cases are implemented with almost identical code. The only difference between the three is in the single line specifying how Camel should handle the file transfers.

The next code listing shows the basic code needed to use Camel to automatically copy files placed in an input directory into a different output directory with Camel.

/**
* Simple executable function to demonstrate Camel file transfer.
*
* @param arguments Command line arguments; excepting duration in milliseconds
* as single argument.
*/
public static void main(final String[] arguments)
{
final long durationMs = extractDurationMsFromCommandLineArgs(arguments);
final CamelContext camelContext = new DefaultCamelContext();
try
{
camelContext.addRoutes(
new RouteBuilder()
{
@Override
public void configure() throws Exception
{
from("file:C:\\datafiles\\input?noop=true").to("file:C:\\datafiles\\output");
}
});
camelContext.start();
Thread.sleep(durationMs);
camelContext.stop();
}
catch (Exception camelException)
{
LOGGER.log(
Level.SEVERE,
"Exception trying to copy files - {0}",
camelException.toString());
}
}

The code above demonstrates minimal use of the Camel API and Camel's Java DSL support. A CamelContext [10] is defined with an instantiation of DefaultCamelContext [11] (line 10). Lines 13-21 add the Camel route to this instantiated context and line 22 starts the context with line 24 stopping the context. It's all pretty simple, but the most interesting part to me is the specification of the routing on line 19.

Because the instance implementing the RoutesBuilder [12] interface provided to the Camel Context only requires its abstract configure method to be overridden, it is an easy class to instantiate as an anonymous class inline with the call to CamelContext.addRoutes(RoutesBuilder) [13]. This is what I did in the code above and is what is done in many of the Camel examples that are available online.

Line 19 shows highly readable syntax describing "from" and "to" portions of routing. In this case, files placed in the input directory ("from") are to be copied to the output directory ("to"). The "file" protocol is used on both the "from" and "to" portions because the file system is where the "message" is coming from and going to. The "?noop=true" in the "from" call indicates that nothing should be changed about the files in the "input" directory (the processing should have "noop" effect on the source files).

As just mentioned, Line 19 in the code above instructs Camel to copy files already in or placed in the "input" directory to the specified "output" directory without impacting the files in the "input" directory. In some cases, I may want to "move" the files rather than "copying" them. In such cases, ?delete=true can be specified instead of ?noop=true when specifying the "from" endpoint. In other words, line 19 above could be replaced with this to have files removed from the "input" directory when placed in the "output" directory. If no parameter is designated on the input (neither ?noop=true nor ?delete=true), then an action that falls in-between those occurs: the files in the "input" directory are moved into a specially created new subdirectory under the "input" directory called .camel. The three cases are highlighted next.

Files Copied from datafiles\input to datafiles\output Without Impacting Original Files

from("file:C:\\datafiles\\input?noop=true").to("file:C:\\datafiles\\output");

Files Moved from datafiles\input to datafiles\output

from("file:C:\\datafiles\\input?delete=true").to("file:C:\\datafiles\\output");

Files Copied from datafiles\input to datafiles\output and Original Files Moved to .camel Subdirectory

from("file:C:\\datafiles\\input").to("file:C:\\datafiles\\output");

As a side note, the uses of fluent "from" and "to" are examples of Camel's Java DSL [14]. Camel implements this via implementation inheritance (methods like "from" and "to" are defined in the RouteBuilder [15] class) rather than through static imports [16] (an approach often used [17] for Java-based DSLs.)

Although it is common to pass anonymous instances of RouteBuilder to the Camel Context, this is not a requirement. There can be situations in which it is advantageous to have standalone classes that extend RouteBuilder and have instances of those extended classes passed to the Camel Context. I'll use this approach to demonstrate all three cases I previously described. The next code listing shows a class that extends RouteBuilder. In many cases, I would have a no-arguments constructor, but in this case I use the constructor to determine which type of file transfer should be supported by the Camel route.

The next code listing shows a named standalone class that handles all three cases shown above (copying, copying with archiving of input files, and moving). This single extension of RouteBuilder takes an enum in its constructor to determine how to configure the input endpoint.

package dustin.examples.camel;

import org.apache.camel.builder.RouteBuilder;

/**
* Camel-based Route Builder for transferring files.
*
* @author Dustin
*/
public class FileTransferRouteBuilder extends RouteBuilder
{
public enum FileTransferType
{
COPY_WITHOUT_IMPACTING_ORIGINALS("C"),
COPY_WITH_ARCHIVED_ORIGINALS("A"),
MOVE("M");

private final String letter;

FileTransferType(final String newLetter)
{
this.letter = newLetter;
}

public String getLetter()
{
return this.letter;
}

public static FileTransferType fromLetter(final String letter)
{
FileTransferType match = null;
for (final FileTransferType type : FileTransferType.values())
{
if (type.getLetter().equalsIgnoreCase(letter))
{
match = type;
break;
}
}
return match;
}
}

private final String fromEndPointString;
private final static String FROM_BASE = "file:C:\\datafiles\\input";
private final static String FROM_NOOP = FROM_BASE + "?noop=true";
private final static String FROM_MOVE = FROM_BASE + "?delete=true";

public FileTransferRouteBuilder(final FileTransferType newFileTransferType)
{
if (newFileTransferType != null)
{
switch (newFileTransferType)
{
case COPY_WITHOUT_IMPACTING_ORIGINALS :
this.fromEndPointString = FROM_NOOP;
break;
case COPY_WITH_ARCHIVED_ORIGINALS :
this.fromEndPointString = FROM_BASE;
break;
case MOVE :
this.fromEndPointString = FROM_MOVE;
break;
default :
this.fromEndPointString = FROM_NOOP;
}
}
else
{
fromEndPointString = FROM_NOOP;
}
}

@Override
public void configure() throws Exception
{
from(this.fromEndPointString).to("file:C:\\datafiles\\output");
}
}

This blog post has demonstrated use of Camel to easily route files from one directory to another. Camel supports numerous other transport mechanisms and data formats that are not shown here. Camel also supports the ability to transform the messages/data being routed, which is also not shown here. This post focused on what is likely to be as simplest possible example of how to apply Camel in a useful manner, but Camel supports far more than shown in this simple example.

Original posting available at http://marxsoftware.blogspot.com/ [18] (Inspired by Actual Events)