Using eXtremeDB Database Events and Time-To-Live Features in Java

As explained in the introduction page, eXtremeDB provides the capability to automatically delete obsolete database objects through the Time-To-Live feature, and to manage database object new, delete, update and checkpoint events. The Java APIs for these features are explained in the sections below.

Autoid

The autoid is a guaranteed unique value generated by the eXtremeDB runtime that can be used to quickly access a unique database object. Often an autoid reference field will be used to create relationships between classes. Please see the Class Relationships page for further details.

The autoid is defined in Java classes with the @Persistent annotation, for example:

 
    @Persistent(autoid=true)
    class Department
    {
        ...
        String name;
    }
     

In Java applications, the autoid value is generated when a new instance of the object is “stored”, i.e. when the transaction enclosing the object instantiation is committed. The value of the generated autoid is returned by the Connection method insert(). For example:

 
    Connection con = new Connection(db);
    con.startTransaction(Database.TransactionType.ReadWrite);
     
    Department dept = new Department();
    dept.code = DD[i].code;
    dept.name = DD[i].name;
    long autoid = con.insert(dept);
     
    con.commitTransaction();
     

The autoid of an object can be used for quick lookups. For example, suppose that class Employee contains an autoid field that references the employee's Department. The Employee class definition might look like the following:

 
    @Persistent
    class Employee
    {
        // Declare unique tree index by "name" field
        @Indexable(Type = Database.IndexType.BTree, Unique = true) 
        public String name;
 
        public long dept;
    }
 

With this definition, the Employee objects would store the appropriate Department autoid in the reference field dept, then the code to lookup the employee's Department might look like the following:

     
    Employee emp;
     
    // Find the Employee object of interest, then lookup its Department
     
    con.startTransaction(Database.TransactionType.ReadOnly);
    Cursor<Department> cursor = new Cursor<Department>(con);
    Department d = cursor.find(emp.dept);
    ...
     
    con.commitTransaction();
     

Time-To-Live

The Time-To-Live (TTL) mechanism facilitates automatic deletion of objects according to TTL policies. Two TTL policies are supported: maxCount and maxTime. The former sets an object count threshold, while the latter sets an object age threshold. Both policies can be set for a single class at the same time.

TTL policies are set using class annotations, just as for other eXtremeDB properties. (Please note that object age threshold in Java is always specified in microseconds.)

 
    @TTL(maxCount=10, maxTime=5000000)
    class A
    {
    // ...
    }
     

Note that the maxTime policy relies on the current system time. Changing the system clock will affect this policy. This can have important consequences in a distributed database as explained in the following section.

Side effects in a distributed environment

For distributed databases in a network (using eXtremeDB Cluster) it is important to note that clocks need to be carefully synchronized between machines participating in a cluster when the maxTime policy is used. Bear in mind the following:

1. In the cluster environment, the TTL clock is verified on the transaction initiator side only in the beginning of the commit phase 1. On the remote side the clock time is not verified and the transaction is applied regardless of the actual clock on that node. If the transaction is applied successfully on the remote nodes, the notifications are sent back and the initiator commits the transaction. The databases are kept consistent regardless of the clock on each node (i.e. their content is the same on every node as long as the transactions are committed).

2. It is possible that a node will have some data that violates the node’s TTL requirements — in the example above the clock on the remote node could be far ahead of the clock on the local node where the transaction had been initiated. By the remote node clock the record should’ve been removed, but it is going to be kept in the node’s database regardless.

3. It is also possible that the object would be removed from the node’s database even if the TTL on that node is not expired, because it has expired on a different node. For example, suppose that node1 clock is set to 1 pm and node2 clock is set to 2 pm. The transaction is initiated on the node1 and gets successfully committed to both nodes despite the fact that the TTL for the record expired on the node2. Shortly after the transaction is committed, node2 initiates another transaction and after verifying the TTL condition by its own clock, removes the object just inserted, naturally propagating the delete to the entire cluster. The record is thus short-lived (shorter than expected).

Event Interfaces

The Event attribute defines what events will trigger application notifications. How the application handles the events is determined at run-time by the event handlers. For Java applications only asynchronous events are possible. In asynchronous event handling, the application spawns a separate thread to handle each type of event. The event thread calls the Connection method waitEvent(). When the event occurs, eXtremeDB releases the thread. Upon releasing the thread, the runtime continues normal processing, so the handler thread runs in parallel with other threads, until it completes its processing and again calls waitEvent().

There is a small window of possibility for another instance of the event to occur before the event handler has completed its task and calls waitEvent() again to wait on the event (events are not queued). This window can be minimized if the handler delegates the processing of the event to yet another thread, allowing the handler thread to immediately wait on the event again. If this risk of an unhandled event cannot be tolerated, the application can maintain a separate table of unhandled events. Asynchronous events are activated after the transaction commits. If, within the scope of a single transaction, several objects are added, or deleted, or several fields are updated which have event handlers waiting, all the handlers will be activated simultaneously.

The individual threads to handle each event will have a ThreadProc like the following:

 
     
    private class EventThread extends Thread
    { 
        public Database db;
        public String event_name;
 
        public EventThread(Database db, String EventName)
        {
            this.db = db;
            this.event_name = EventName;
        }
         
        public void run()
        {
            Connection con = new Connection(db);
            try
            {
                while (!exit)
                {
                    con.con.waitEvent(event_name);
                }
            }
            catch (DatabaseError x)
            {
                if (x.errorCode >= 50)
                {
                    // Errors
                }
                else
                {
                    // Normal return codes including MCO_S_EVENT_RELEASED
                }
            }
            con.disconnect();
             
        }
    }
     

The main application thread will sleep for some milliseconds, then proceed with normal database processing. When terminating, the application will call the Connection method releaseAllEvents() and then stop the event handler threads. The event handlers will catch an exception with error code MCO_S_EVENT_RELEASED when the event is released. (See the SDK sample samples/java/events/asyncbasic for further implementation details.)