Java Tip 78: Recycle broken objects in resource pools

Here's how to improve performance and share objects between clients with the recycler idiom

Resource pools (otherwise known as object pools) are used to manage the sharing of objects between multiple clients. A client with access to a resource pool can avoid creating a new resource by simply asking the pool for one that has already been instantiated instead. Examples of object pooling can be found in many distributed systems: a number of Java server systems, including Enterprise JavaBeans servers, use this technique to speed client access and improve overall throughput.

To start out, let's review an interface to a generic resource pooling system that has been reduced to its essence. This interface is simply a generic description of a pool that can contain objects of any type. The object being pooled is referred to as the resource. Examples of resources might be database connections, socket connections, or available phone lines. Here's the interface:

public interface ResourcePool
{
   public Object getResource();
   public void releaseResource(Object resource);
}

When a resource, such as a database connection, is required, we do not create the object within the client code. Instead, we go to the resource pool and call the getResource() method, which returns one of the available objects within the pool. Normally, the objects in the pool have been created (instantiated) in advance. When we have finished with the object, we must return it to the ResourcePool. Note that, if all the users of a resource pool are not well-behaved code citizens, doing their job and returning the object to the pool, the benefits of the pooling model are broken, and we would be better off without it. So, as good clients of the pool, when we take an object out, we make a vow that we will put it back some time in the future.

It is quite possible that implementations of this interface will use a FactoryMethod to create the resource objects when the object pool is first created; this allows the pooling mechanism to be decoupled from the logic of creating the objects. The implementations could also use a lazy instantiation (as shown in Java Tip 67) to create a resource when all the currently constructed resources are in use.

For example, some resource pool implementations have a MAX and MIN capacity. When this type of pool is first instantiated, it creates the MIN number of resources, but as the number of requests exceed MIN, the pool creates more objects on demand by means of lazy instantiation until it reaches MAX.

The generic object pool can be configured at runtime to contain any type of resource. We may choose to do this by using factory objects in the concrete implementation of the resource pool. In this case, the factory object is responsible for creating the type of resource within the pool and may be hidden completely from the clients of the resource pool interface.

If you want to delve deeper into an implementation of a resource pool take a look at Thomas E. Davis's article, "Build your own object pools." For the moment, let's take a look at extending the interface to cope with objects that become broken while we are using them.

Dealing with objects that fail

When dealing with distributed systems, it is unwise to assume that objects will continue to operate seamlessly over long periods of time. Connections may time out; servers may crash or become unreachable on the network. Objects that make use of these resources may as a result also become broken. In fact, objects can fail in local systems, too. In either situation, if the broken object that the client has been using has come from a resource pool, things get a bit tricky.

On one hand, we have promised that, once we have taken an object from the pool, we will put it back. On the other, the next client will expect the object to be valid and useful when it takes it from the pool. If a broken object is put back into the pool, it will most likely cause problems for the next client that tries to use it.

What we really need is a mechanism that allows clients to return broken resources in a way that gracefully acknowledges that the resource is faulty. To do this we need to be able to call an explicit method on ResourcePool rather than returning the broken resource with releaseResource().

A real-world example

The scenario above is analogous to renting a car: you pick up your car from the rental agency expecting it to run properly; if it breaks down as soon as you drive out of the parking lot, however, you would expect the rental agency to replace it with one that works. No reputable car rental company would simply put the car back out on its rental lot without repairing it first. We'll come back to our car rental metaphor momentarily; first, let's review our resource pool interface in order to help solve our dilemma.

In our Java systems, we introduce a new method signature on the ResourcePool interface. This allows clients to signal to the pool that the resource they are replacing may be broken.

To keep the examples simple, we haven't specified any exceptions in the ResourcePool interface:

interface ResourcePool
{
   public Object getResource();
   public void releaseResource(Object resource);
   public void putBackBrokenResource(Object resource);
}

Here's how the client code may look. We'll keep to our metaphor of a car rental agency, assuming that we have a PoolFactory that is responsible for creating object pools, and that the pool manages HireCar (RentalCar) objects:

ResourcePool pool = PoolFactory.getPool();
int numTries=0;
boolean done=false;
while(!done)
{
   HireCar hireCar = null;
   try
   {
      hireCar = (HireCar)pool.getResource();
      hireCar.startEngine();
      //other operations
      done=true;
   }
   catch(CarBrokenException exp)
   {
      pool.putBackBrokenResource(hireCar);
      numTries++;
      if(numTries==MAX_TRIES)
         throw exp;
   }
   finally
   {
      pool.releaseResource(hireCar);
   }
}

For the purposes of this article, it is a little off-topic to consider a releaseResource() or getResource() method that does throw a Throwable, but this case brings some interesting issues to light. For more information on exceptions, see Resources.

With our new method in place, client code can put suspect objects back into the pool with a clear conscience. We leave it as the responsibility of the pool to deal with such objects. Now let's look at how we might implement a Car pool given the above interfaces.

The car pool

In the context of our car rental example, under normal circumstances the cars are taken in and out of the resource pool. Every time a car is reused from the pool, the cost of creating and destroying a new car is saved. This all seems fine, and is certainly ecologically friendly.

Now, let's say that, for some reason, a number of the cars start failing. Instead of reentering the pool via the releaseResource() method, the cars are put back via the putBackBrokenResource() method. The implementation of the latter method might look like this:

public synchronized void putBackBrokenResource(Object resource)
{
   // remove the resource from the local pool
   // (java.util.Collection) of resources
   pool_.remove(resource);
   // make a new car to replace the broken one in the pool
   pool_.add(factory_.make());
}

This works just fine if the object is not complex. If it is, however, we are in for a triple whammy!

The first problem is that this method will consume effort creating a new Car object from scratch, which we could be using more fruitfully in other parts of the system. The second is that we will have a very large unreferenced and unused object sitting around in memory waiting to be garbage-collected. This could have several undesirable effects -- unnecessary memory paging in the Java virtual machine, for example.

The third problem is that, in the case of complex objects and distributed objects, it is likely that finalizers or releaser strategies (see Resources) have been implemented. This in turn will increase effort at garbage collection time. In general, the larger and more complex the object is, the more there is to clean up.

Before we try to resolve this issue, let's take a step back and review some prior art in the form of creational design patterns.

Composed objects and builders

In the case of more complex objects, the factories we use within the resource pools may themselves become complex and the creation of objects very expensive. The Design Patterns catalog, linked to in Resources, lists a creational pattern for dealing with this type of problem.

The builder pattern describes how to break down the construction of nontrivial objects into the construction of a series of Builder objects. Each of these objects is responsible for building particular parts of the final composite object. The products of the Builders are required to create the object under construction. A Director object has overall responsibility for assembling all of these parts into the final composite object.

If we apply the builder pattern to our car rental example, what we may have are Builders that make wheels, chassis, engines, and other car parts, and a Directory object that will be responsible for assembling these parts into the final composite car.

Now, let's assume that our car pooling system uses a factory that implements the builder pattern. It is possible for us to reuse some of this pattern to implement a Recycler.

Using recyclers

A Recycler is an ecologically friendly version of the Director object that appears in the builder pattern just discussed. When a broken object is returned to the pool, it is the Recycler's job to examine the object and to try to replace only the part (or parts) of the object that are broken. This alleviates the cost of removing and creating new resource objects from scratch.

Once we incorporate the Recycler, the putBackBrokenResource() method looks like this:

public synchronized void putBackBrokenResource(Object resource)
{
   // try to fix the resource
   boolean fixed = recycler_.fix(resource);
   if ( fixed )
   {
      releaseResource(resource);
   }
   else
   {
      pool_.remove(resource) ;
      // make a new car to replace the broken one
      pool_.add(factory_.make());
   }
}

It is now the job of the Recycler to fix the problems with the resource object and signal whether the repair was successful or unsuccessful. So what happens in the Recycler's fix() method?

First the Recycler has to determine which part of the system is broken. This is achieved by testing each part and determining which element (or elements) fail. Once the broken parts are identified, the they can be replaced.

We will probably use the Builder objects that are already defined to create and replace the newly required parts. Here is an outline of a CarRecycler class:

public class CarRecycler implements Recycler
{
   public boolean fix(Object Resource)
   {
     Car car = (Car)resource;   // assume safe cast for simplicity
     boolean fixed = testAndReplaceParts(car);
     return fixed;
   }
   /**
   * Test the car and replace a part if broken.
   * Return true if broken part has been replaced,
   * false if no part has been replaced.
   */
   public boolean testAndReplaceParts(Car car)
   {
     boolean fixed    = testAndReplaceWheels(car);
     if (!fixed) fixed = testAndReplaceEngine(car);
     if (!fixed) fixed = testAndReplaceGearbox(car);
     // etc. etc.
      return fixed;
   }
}

You may have noticed that we assume that only one group of parts within the car assembly will fail at any time. While this may not be the case in actuality, it is a simplifying assumption that we have chosen to make in this example.

It also may be possible to gain some advantage by ordering the specific testAndReplace() methods based on how often the parts contained in a group appear to be failing.

There are many variations on this theme. As a simple example, if we test all of the parts and they all work, then we may choose to assume that it is the composing or containing object that is broken and simply replace that part using the parts from the old car. This requires a rewrite of the fix() method and some reworking of the pooling code.

In terms of real-world applications, we have found Recyclers useful in distributed systems designs in which a number of connections are attached to a Session object. In this particular context, the Session becomes unusable when a single connection fails, because its correct behavior is dependent on all of the connections being available. So, although the session is no longer usable, all but one of the connections are still valid. This scenario is a perfect example of a time when you would want to use a Recycler.

Conclusion

Remember that factories and Recyclers are encapsulated within the resource pool's implementation. This means that clients of the pool only see the simple resource pool interface that we have presented. In this way, the complexity of the recycling system is completely hidden from the client code that is using the pool.

Object recycling can be very useful in distributed systems and complex local systems, but only when the conditions are appropriate. Here are some guidelines to use when considering whether or not to use object recycling:

1 2 Page 1