High-availability mobile applications

Mobile databases and J2ME tools

Today's wireless networks are notoriously unreliable. Unavailable applications destroy the core value of mobile commerce, which is the promise of accessing information anytime, anywhere. In fact, the requirement of always-on connections is one of the major reasons why Wireless Application Protocol (WAP)-based thin-client mobile commerce failed to take off. Mobile commerce requires smart, mobile clients that can handle applications locally and continue to function even when the network is temporarily unavailable (i.e., offline operations). The "occasionally connected" mobile applications require advanced mobile data storage and management tools. Such tools are readily available on J2ME (Java 2 Platform, Micro Edition).

In this article, I first discuss general concepts and specifications of mobile databases. I use an example application to illustrate important designs and key components of J2ME mobile database applications. I also provide useful advice on how to choose the right mobile database in today's diverse market.

On-device data storage and management

The first requirement for offline operations is to store and manage application data on the device itself. The standard J2ME profiles provide only limited data storage and management capabilities. On low-end MIDP (Mobile Information Device Profile) phones, we have only linear record stores from the record management system (RMS); in the Foundation Profile (or PersonalJava), we have plain random files. The application developer must organize data using those generic facilities, which can prove tedious, inefficient, and error prone for large applications.

In the J2SE (Java 2 Platform, Standard Edition) and J2EE (Java 2 Platform, Enterprise Edition) worlds, the relational database represents the key technology for data management. Most Java developers are already familiar with common database access APIs such as JDBC (Java DataBase Connectivity). A lightweight relational database is therefore our natural choice for mobile data management.

In addition to supporting offline operation modes, on-device databases have other important benefits:

  • They can manage user preferences. Extreme personalization is touted as a major benefit of mobile commerce.
  • They can provide a performance cache to reduce network roundtrips, which could drastically improve application performance over today's slow and long latency wireless networks.

The JDBC optional package

The standard JDBC API has always been optionally available for high-end Java mobile devices. PersonalJava runtimes are often bundled with the optional JDBC 1.x API. PersonalJava's successor, the J2ME Foundation/Personal Profile (FP/PP), does not mandate JDBC. Instead, a rich subset of the JDBC 3.x API is made available to FP/PP devices through the J2ME JDBC optional package (JDBC OP). JSR (Java Specification Request) 169 is developing this optional package's specification; the final draft was proposed in November 2002. The JDBC OP supports most commonly used JDBC features except the following:

  • Connection pools
  • The ParameterMetaData interface
  • Setting parameters for stored procedures by name in the CallableStatement interface
  • SQL 99 types (the Struct, Array, Ref, SQLData, SQLInput, and SQLOutput interfaces)
  • Custom type mapping (the setTypeMap() and getTypeMap() methods)

Data synchronization

Another important aspect of the occasionally connected application paradigm is data synchronization. Standalone mobile databases are isolated pockets of data. Mobile applications are much more useful when we connect those islands with powerful backend servers. Connected mobile databases have the following advantages:

  • A mobile client often needs to access enterprise IT infrastructure. For example, a mobile sales application requires updated inventory data.
  • A backend application needs up-to-date and aggregated data to make intelligent business decisions or generate accurate reports. For example, a supply chain application needs to aggregate data from mobile sales people before deciding how much it should order from suppliers.
  • The back end enables information sharing among mobile peers.

As a result, smart clients should support not only offline management of on-device data, but also data synchronization with backend databases when the network is available. The added bonus of a synchronized solution is that the backend database can act as a content provisioning repository for simplifying mobile device management.

In theory, the application developer can handle the entire synchronization logic. However, developing an optimized, secure, and scalable synchronization solution requires much expertise. Mobile database products normally come with proprietary synchronization tools and APIs to take away the pain from developers.

What about SyncML?

A popular application-level synchronization protocol is SyncML. Besides generic client application platforms such as J2ME, mobile devices often come preinstalled with different sets of native smart applications. Examples include calendars, address books, and contact lists (personal information management, or PIM, applications) on smart phone or PDA devices. Those applications often synchronize data with desktop PCs or other central data repositories through proprietary protocols supported by vendor-supplied driver software. The incompatible protocols have created many problems for users. For example, synchronizing a Pocket PC device with a Mac or Linux desktop is difficult; if a person owns multiple mobile devices, he must install multiple drivers that could potentially conflict. Users demand a standard synchronization protocol that allows any device to synchronize with any backend application without proprietary driver applications.

SyncML is a standard XML data format that defines the syntax to describe simple PIM data such as vCard and vCalendar. SyncML can access database-powered enterprise backend information systems. For example, IBM WebSphere Everyplace Access provides SyncML clients access to Lotus Notes and Microsoft Exchange servers. Important enterprise databases such as IBM DB2 and Oracle9i Database have built-in SyncML support. Although SyncML is great for PIM-type applications, it is not powerful enough to synchronize generic relational databases; thus, this article does not focus on SyncML.

An example application

Now, through a simple example, we examine typical usage scenarios and key components of mobile database applications.

Mobile contact manager in action

The example is a mobile contact manager application provided by PointBase. ContactManager is included in PointBase 4.x distributions. For your convenience, I have included the source code in a download zip file in Resources. If you want to build and run the examples yourself, you still must download the appropriate jar files from PointBase.

The application itself is simple: it mainly duplicates features commonly found in advanced address book applications. For example, it allows the user to store contact name, address, and phone numbers with pictures; provides intuitive browsing and searching interfaces; and synchronizes with backend database servers. Figures 1 and 2 demonstrate the application in action in the standalone mode and synchronized mode, respectively. The screenshots are from a Pocket PC powered by Insignia's Jeode PersonalJava VM and a Mac OS X laptop powered by J2SE. The same byte-code application runs without modification on many platforms, demonstrating Java's power.

Figure 1. Standalone ContactManager in action on Pocket PC Jeode PersonalJava
Figure 2. Two synchronized ContactManager spokes in action on Mac OS X. Click on thumbnail to view full-size image.

The client-side application UI (user interface) is written with AWT (Abstract Window Toolkit), the only standard UI library supported on PersonalJava or J2ME/FP/PP devices. Behind those UI drivers, we have another code layer that provides access to a generic on-device JDBC database. The database access layer also provides logics to synchronize mobile databases with backend databases through PointBase's proprietary UniSync synchronization server. Now, we focus on the code in the data access layer, which is contained in a single class: DBManager.

On-device data access

Class DBManager is a singleton class that provides a single point of entry to the database from the application. The Singleton pattern avoids threading complexities for embedded databases. The code snippet below shows DBManager's constructor and initialization method. It connects to the database, loads the table schema, populates the table with sample data, and creates SQL statement templates (PreparedStatement) for later use. As we can see, everything here is standard JDBC. For enterprise Java developers, the following code should be easy to understand:

Listing 1. Connect to mobile database and initialize access objects

class DBManager {
  // DBManager is a singleton class.
  private static DBManager instance;
  private String driver;
  private String url;
  private String user;
  private String password;
  private boolean delay;
  private Connection connection;
  private Statement statement;
  private PreparedStatement insert;
  private PreparedStatement find;
  private PreparedStatement delete;
  private PreparedStatement update;
  private PreparedStatement all;
  static DBManager getInstance() {
    if (instance == null) {
      instance = new DBManager();
    return instance;
  private DBManager() {
    // Get parameters from runtime properties.
    // This allows us to switch to different JDBC databases
    // without changing the application code.
    Properties properties = ContactManager.getProperties();
    driver =
      properties.getProperty("driver", "com.pointbase.me.jdbc.jdbcDriver");
    url =
      properties.getProperty("url", "jdbc:pointbase:micro:pbdemo");
    user =
      properties.getProperty("user", "PBPUBLIC");
    password =
      properties.getProperty("password", "PBPUBLIC");
    delay =
  private void connect() {
    try {
      // Load the driver class.
      // If the database doesn't exist, create a new database.
      connection = DriverManager.getConnection(url, user, password);
      // Create template statement objects.
      statement = connection.createStatement();
      // If the database is newly created, load the schema.
      boolean newdb=initDatabase();
      // Load sample data for the new tables.
      if(newdb) {
    } catch (Exception e) {
  void disconnect() {
    try {
    } catch (Exception e) {
  // Create the table and load the schema.
  private boolean initDatabase() {
    try {
      String sql = "CREATE TABLE NameCard (ID INT PRIMARY KEY, "+
        "Name VARCHAR(254), Company VARCHAR(254), Title VARCHAR(254), "+
        "Address1 VARCHAR(254), Address2 VARCHAR(254), "+
        "Phone VARCHAR(254), Email VARCHAR(254), "+
        "Picture Binary(1000000))";
      // If the table already exists, this will throw an exception.
      // This means the database already exists.
      return true;
    } catch (SQLException e) {
      // Ignore the error - the table already exists, which is good
      // so we don't need to add demo data later on.
      return false;
  // Create statement templates.
  private void createStatement() {
    try {
      insert = connection.prepareStatement(
        "INSERT INTO NameCard (ID, Name, Company, Title, Address1, "+
        "Address2, Phone, Email, Picture) "+
        "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)");
      find = connection.prepareStatement(
        "SELECT * FROM NameCard WHERE (Name LIKE ?) "+
        "AND (Company LIKE ?) AND (Title LIKE ?) "+
        "AND ((Address1 LIKE ?) OR (Address2 LIKE ?)) "+
        "AND (Phone LIKE ?) AND (Email LIKE ?)");
      delete = connection.prepareStatement(
        "DELETE FROM NameCard WHERE ID = ?");
      update = connection.prepareStatement(
        "UPDATE NameCard SET ID=?, Name=?, Company=?, Title=?, "+
        "Address1=?, Address2=?, Phone=?, Email=?, Picture=? "+
        "WHERE ID = ?");
      all = connection.prepareStatement(
        "SELECT ID, Name, Company, Title, Address1, Address2, "+
        "Phone, Email FROM NameCard");
    } catch (SQLException e) {
  // Other methods.

Other methods in DBManager provide access to the database via simple JDBC API calls. The following code snippet demonstrates methods for searching and manipulating name-card records. These methods heavily use the SQL templates we defined before.

Listing 2. Data access methods

Vector findNameCardsByKeyword(String name, String company,
        String title, String address1, String address2,
        String phone, String email) {
  Vector NameCards = new Vector();
  String[] keywords = {name, company, title, address1, address2,
                       phone, email};
  try {
    for (int i = 0; i < keywords.length; i++) {
      String criteria = (keywords[i].equals("")) ? "%" :
                        "%" + keywords[i] + "%";
      find.setString(i + 1, criteria);
    ResultSet resultSet = find.executeQuery();
    while (resultSet.next()) {
      NameCard nameCard = new NameCard(resultSet.getInt(1),
            resultSet.getString(2), resultSet.getString(3),
            resultSet.getString(4), resultSet.getString(5),
            resultSet.getString(7), resultSet.getString(8));
      if (!delay)
  } catch (SQLException e) {
  return NameCards;
void addNameCard(NameCard nameCard) {
  try {
    insert.setInt(1, nameCard.getID());
    insert.setString(2, nameCard.getName());
    insert.setString(3, nameCard.getCompany());
    insert.setString(4, nameCard.getTitle());
    insert.setString(5, nameCard.getAddress1());
    insert.setString(6, nameCard.getAddress2());
    insert.setString(7, nameCard.getPhone());
    insert.setString(8, nameCard.getEmail());
    insert.setBytes(9, nameCard.getPicture().getBytes());
  } catch (SQLException e) {
void updateNameCard(NameCard nameCard) {
  try {
    update.setInt(1, nameCard.getID());
    update.setString(2, nameCard.getName());
    update.setString(3, nameCard.getCompany());
    update.setString(4, nameCard.getTitle());
    update.setString(5, nameCard.getAddress1());
    update.setString(6, nameCard.getAddress2());
    update.setString(7, nameCard.getPhone());
    update.setString(8, nameCard.getEmail());
    update.setBytes(9, nameCard.getPicture().getBytes());
    update.setInt(10, nameCard.getID());
  } catch (SQLException e) {
void deleteNameCard(NameCard nameCard) {
  try {
    delete.setInt(1, nameCard.getID());
  } catch (SQLException e) {
void loadPicture(NameCard nameCard) {
  try {
    ResultSet resultSet =
              "SELECT Picture FROM NameCard WHERE ID = " +
    Picture picture = new Picture();
  } catch (SQLException e) {
private int getNewID() {
  try {
    ResultSet resultSet = statement.executeQuery(
        "SELECT MAX(ID)+1 FROM NameCard");
    if (resultSet.next()) {
      return resultSet.getInt(1);
    } else {
      return 0;
  } catch (Exception e) {
  return 0;

Synchronize with backend databases

Class DBManager also allows the application developer to synchronize the mobile database with a backend database using PointBase's proprietary UniSync engine. Different vendors use different synchronization engines, but their concepts are similar. The synchronization process follows these steps:

  1. Create corresponding databases and tables on both the backend server and mobile devices.
  2. Create a hub on the synchronization server. The hub contains publications that specify the backend tables (or partial tables) available for synchronization (publish).
  3. Use the hub to create spokes. Spokes are objects on the synchronization server representing mobile devices. Each spoke has an ID. It can subscribe to the publications in the same hub through subscription objects. Using a spoke ID, the mobile device connects to the matching spoke and synchronizes to the subscribed backend tables.
  4. Start the synchronization server. This basically involves executing the com.pointbase.me.sync.Server class's main() method. The server class is available in the PointBase distribution package. There are several ways to run the server in different environments. Please refer to PointBase documentation for more details and example scripts. By default, the server listens at port 8124.
  5. Initiate the synchronization process using a spoke ID and spoke stub classes residing on the mobile devices.

Figure 3 illustrates the UniSync synchronization server architecture.

Figure 3. The architecture of UniSync server

Class ResetServer in Listing 3 demonstrates how to create hubs and spokes on the UniSync server:

Listing 3. Set up the synchronization server

String dsname;
String hubname="Hub";
Hub hub=manager.createHub(hubname);
Publication pub;
String pubname;
SpokeConfig spoke;
Subscription sub;
String subname="SubNameCard";
String tablename="NAMECARD";
String[] tables=new String[]{tablename};
// Publish the complete name-card table
// Create two spokes and subscribe to this publication
for(int i=1;i<=2;i++) {
  String name="Spoke"+i;
// Publish the name-card table without the picture column
SyncTable table=pub.getSyncTable(tablename);
table.dropSyncColumns(new String[]{"PICTURE"});
// Create two spokes and subscribe to this publication
for(int i=3;i<=4;i++) {
  String name="Spoke"+i;

The following code snippet from DBManager demonstrates how to obtain the spoke stub and process the synchronization on the device side. The comments embedded in the code illustrate the differences between the application's synchronized and standalone versions:

Listing 4. Data access via the synchronization server

// Import proprietary classes for sync
import com.pointbase.me.jdbc.*;
class DBManager {
  // In addition to JDBC connection variables,
  // we also need to define variables for sync
  // ... ...
  private Spoke spoke;
  private String spokename;
  private int spoke_id;
  private int spoke_range_start,spoke_range_end;
  final static int ROWS_PER_SPOKE=1<<16;
  private String syncurl;
  private String syncpassword;
  private DBManager() {
    // Get DB connection parameters
    // ... ...
    // Get sync parameters
    syncurl =
      properties.getProperty("syncurl", "http://localhost:8124");
    String spokeid =
      properties.getProperty("spokeid", "1");
    spokename =
      properties.getProperty("spoke", "Spoke"+spokeid);
    syncpassword =
      properties.getProperty("syncpassword", "pass"+spokeid);
    url =
  // The complete connect method using synchronization server
  private void connect() {
    try {
      System.out.println("Connecting to the database...");
      // If the database doesn't exist, create a new database
      connection = DriverManager.getConnection(url, user, password);
      statement = connection.createStatement();
      // Check sync metadata and create tables
      // Create prepared statements
    } catch (Exception e) {
  // The complete newID method using the sync server
  private int getNewID() {
    try {
      ResultSet rs = statement.executeQuery(
            "SELECT MAX(ID)+1 FROM NameCard WHERE "+
            "ID>="+spoke_range_start+" AND ID<"+spoke_range_end);
      int id=rs.getInt(1);
      if(rs.wasNull()) {
        return spoke_range_start;
      } else {
        return id;
    } catch (Exception e) {
    return 0;
  // Create table and load metadata from the sync hub
  void loadMeta() {
    try {
      SyncManager manager=SyncManager.getInstance(connection);
      if(spoke==null) {
            "Loading MetaData from url "+syncurl+
            " for spoke "+spokename+
            " using password "+syncpassword);
      spoke_id = spoke.getSpokeId();
      System.out.println("SpokeID is "+spoke_id);
      spoke_range_start = ROWS_PER_SPOKE * spoke_id;
      spoke_range_end =  spoke_range_start + ROWS_PER_SPOKE - 1;
    } catch (SyncException e) {
  // Synchronize spoke databases (mobile databases) with the hub
  // and backend databases
  void sync() {
    try {
    } catch (SyncException e) {
  // Other data access methods are the same as the non-synced version.

J2ME mobile database choices

Of course, PointBase is not the only choice for J2ME mobile database products. In the rest of this article, I compare and discuss several leading competitors. Before I do that, let me first overview the product landscape.

JDBC databases for high-end devices

For high-end devices that run PersonalJava or the Personal Profile, the database can be accessed via the JDBC APIs. The JDBC database is the easiest type of database to program. All vendors have at least one product in this category. The biggest challenge here is how to balance features with footprint. We want the smallest and fastest database that supports the exact set of features the application requires.

Lightweight databases for MIDP devices

CLDC (Connected Limited Device Configuration)/MIDP does not support the JDBC interface. Two common approaches support complex data management on MIDP devices:

  • Implement extremely lightweight relational databases and JDBC-like access APIs over RMS. This approach is resource expensive but proves the best in terms of developer productivity and feature support.
  • Directly extend the RMS class and implement simple row (de)serialization, data access, indexing/searching, and synchronization methods in the extended class. The extended class is a thin layer over RMS. It can mimic a relational table's behavior.

I discuss both approaches in the product review sections.

Synchronization servers

As you have seen, synchronization is a key feature for mobile databases. Most database vendors have their own proprietary synchronization servers. Those servers support many add-on features to optimize the synchronization process in the mobile environment. Important features of synchronization servers include the following:

  • Smart conflict resolution
  • Bandwidth reduction
  • End-to-end encryption
  • Performance tuning for backend engines
  • Asynchronous and scalable updates

In the next several sections, I examine and compare products from several leading vendors.

HSQL Database Engine

The open source HSQL Database Engine is based on Thomas Mueller's Hypersonic SQL Project. It is completely written in Java and is one of the most widely used embedded databases. It is included in many J2EE application servers. On mobile devices, HSQL runs on the PersonalJava and FP/PP platforms. HSQL is free for all. You can freely redistribute it with your applications, which proves handy for mobile applications.

HSQL provides a JDBC driver that supports 95 percent of the JDBC interface and all JDBC 1 data types. It supports transactions, foreign keys, and even Java stored procedures. Tables in HSQL can reside in memory or be persisted to disk files. HSQL has less than a 160-KB memory footprint. It also distributes a database management console for PersonalJava devices (tested on Sharp Zaurus).

However, HSQL lacks some of the advanced performance and security features commonly found in commercial mobile databases. More importantly, HSQL does not offer any synchronization solution. It also lacks a solution for MIDP devices.

PointBase Micro

PointBase is one of the leading vendors for pure Java embedded databases. The PointBase Micro database runs on both FP/PP/PersonalJava and MIDP platforms.

On the FP/PP/PersonalJava platforms, the PointBase Micro database supports most JDBC and SQL features with a footprint of merely 91 KB. It also supports advanced features such as database encryption. It lacks support for stored procedures and the CallableStatement interface. Since MIDP does not support JDBC, PointBase offers its own lightweight JDBC-like APIs for MIDP. PointBase Micro database's MIDP version provides a database browser console MIDlet.

PointBase Micro databases can easily synchronize with server-side PointBase Embedded databases and Oracle databases through the UniSync synchronization server.

Sybase iAnywhere Solutions

iAnywhere's SQL Anywhere Studio has a large market share in lightweight databases for laptop computers. It also has a strong presence in the mobile database market for pervasive devices. A key innovation behind Anywhere SQL Studio is its custom database generator. It lets the customer, not the database vendor, decide how to balance the database footprint against its supported features.

In the Anywhere SQL Studio, the user can specify the SQL statements she will call in the application. The studio will then generate a custom database with the user's choice of the underlying persistence mechanism and transactional features. The generated database is a pure Java class that comes with a set of APIs the user application can call. Data synchronization APIs using iAnywhere's MobiLink synchronization server are also built into the custom database.

In additional custom database generators, iAnywhere also offers large-footprint, general-purpose mobile databases with many security, optimization, and usability features, including support for Java stored procedures. Those databases run natively on popular mobile platforms to achieve better performance. Sybase iAnywhere SQL Studio does not yet support autogeneration of MIDP databases.

IBM DB2 Everyplace

IBM DB2 Everyplace is IBM's mobile database offering. For IBM customers, DB2 Everyplace integrates well with other IBM enterprise components (e.g., DB2 Universal Database and WebSphere MQ Everyplace) and IBM development tools (e.g., IBM WebSphere Studio Device Developer). DB2 Everyplace runs natively on many platforms including Palm OS, Symbian OS, Pocket PC, QNX Software Systems, and embedded Linux. DB2 Everyplace supports encrypted data fields and table storage optimization features.

On the MIDP platform, DB2 Everyplace has a product called FastRecordStore. It emulates an indexable and searchable relational table on top of MIDP RMS record stores.

DB2 Everyplace databases and FastRecordStores synchronize with backend databases via the IBM synchronization engine. DB2 Everyplace also comes with a tool called Mobile Application Builder that allows developers to visually build DB2 Everyplace applications.

Oracle9i Lite

Oracle9i Lite is Oracle's mobile database product. It runs on Palm OS, Pocket PC, Symbian OS, and Win32 platforms. The Win32 edition is intended to run on laptop computers and supports JDBC, multiuser mode, and Java stored procedures. Oracle9i Lite's Pocket PC and Symbian OS editions support JDBC. The Palm OS edition only supports Oracle's proprietary native OKAPI (Object Kernel API) and ODBC (Open Database Connectivity). The Oracle9i Lite suite includes a mobile development kit that automatically generates and packages mobile database applications from user custom requirements. Currently, it only generates native client applications.

The Oracle9i mobile database synchronizes with backend Oracle database servers through the Oracle Mobile Server. If we created an application using the Oracle9i Lite Mobile Development Kit, the Mobile Server automatically generates synchronization logic for the application. Oracle9i Lite supports synchronization over any TCP/IP-based network, including HTTP, CDPD (cellular digital packet data), and 802.11b Wireless LAN. We can also add new transport by using the Mobile Server Open Transport APIs. In addition, the Oracle mobile server supports asynchronous synchronization. During rush hours, each device just submits the synchronization content in a queue and leaves. Asynchronous operation is key to scalable solutions.

On the MIDP platform, Oracle provides an RMS-based product called SODA (Simple Object Database Access). SODA is in fact an object-oriented database for MIDP devices. It allows storage, and search and retrieval of JavaBean-like data objects.

Select the right mobile database

Selecting the right mobile database product is a complex business decision. Since mobile databases are embedded in the client application, you need to pay royalties for each product you distribute. It is important to negotiate a good contract to reduce the overall cost. Also, due to the proprietary nature of the synchronization solutions, each product has a certain degree of vendor lock-in. As a result, developers should not only consider the product's technical merits but also the vendor's overall reputation and stability. For example, if your mobile clients require both MIDP and Personal Profile databases, you should choose both products from the same vendor for a better bundle rate and a more manageable synchronization solution.

PointBase and Sybase iAnywhere offer excellent mobile database solutions. However, if your enterprise infrastructure is already primarily IBM or Oracle, it might be a better deal to select DB2 Everyplace or Oracle 9iLite mobile databases.

Mobile databases and synchronization solutions are important components in high-availability mobile applications. They enable the new and promising occasionally connected mobile application paradigm. After reading this article, you should be able to design and implement simple mobile database applications using JDBC and PointBase APIs. Since mobile databases run on small devices, you need to carefully balance features against performance and footprint when choosing commercial products.

Michael Yuan is a PhD candidate at the University of Texas, where he is a research associate at the Center for Research in Electronic Commerce and an open source Java developer. He is the author of the upcoming book Java Mobile Enterprise Application Development from Prentice Hall, to be released in fall 2003.

Learn more about this topic

Join the discussion
Be the first to comment on this article. Our Commenting Policies
See more