eXtremeDB Transaction Logging with Dynamic DDL

eXtremeSQL supports the SQL DDL operations to create, alter or drop tables and indexes. These operations obviously cause the database schema to be modified. Schema modifications are reflected in the log and are applied to the database transparently to the application.

Dynamic DDL is also supported when using Data Relay. Please refer to SDK sample “samples\native\tl\tlogiterpipe_ddl”.

(Note that when the database is restored from the log, further access is possible only through the SQL or UDA API as any previously generated API based on the pre-modified schema becomes invalid. An attempt to perform a transaction using the generated API will result in the MCO_E_SCHEMA_CHANGED error, indicating that the application must close the current connection.)

When the database is restored from the log, applications using the C/C++ API must:

1. ensure that there is sufficient memory for the new, dynamically created classes and indexes by assigning non-zero values to the max_classes, max_indexes and ddl_dict_size elements of the mco_db_params_t (or their SQL C++ API analogs).

2. pass an empty dictionary to the database runtime.

To illustrate this process consider two cases: with and without a database snapshot saved by mco_db_save(). If a snapshot of the database has been saved, then the following code snippet demonstrates how to restore the database from the log file:

 
    mco_db_h db;
    mco_db_params_t db_params;
    const char *db_name = "mydb";
    const char *tl_log_file = "mydb.log";
    mco_db_params_init ( &db_params );
    /* initialize the params with default values */
    ...
    db_params.max_classes = 100; // maximum number of classes possible in the database
    // memory space reserved for the dictionary
    db_params.max_indexes = 1000; // maximum number of indexes possible db_params.ddl_dict_size = 64 * 1024; 
     
    /* db_params.ddl_dict = 0; no DDL dictionary. This is required parameter. 
        But it is automatically zeroed by function mco_db_params_init */
    ...
    mco_db_load(stream_handle,
             input_stream_reader, 
            db_name,
            0 /* no static dictionary */,
            pdev->dev,
             pdev->n_dev, 
            &db_params );
             
    mco_db_connect(db_name, con);
    mco_translog_apply(con, tl_log_file, MCO_TRANSLOG_ALL_LABELS );
     

Alternatively, when no snapshot has been saved:

     
    mco_db_h db;
    mco_db_params_t db_params;
    const char *db_name = "mydb";
    const char *tl_log_file = "mydb.log";
    mco_db_params_init ( &db_params );
    /* initialize the params with default values */
    ...
    db_params.max_classes = 100; // maximum number of classes possible in the database
    // memory space reserved for the dictionary
    db_params.max_indexes = 1000; // maximum number of indexes possible db_params.ddl_dict_size = 64 * 1024; 
 
    /* db_params.ddl_dict = 0; no DDL dictionary. This is required parameter. 
        But it is automatically zeroed by function mco_db_params_init */
    ...
    /* define if autooid or native fields order are required */
    db_params.ddl_dict_flags = MCO_DICT_FLAGS_USE_AUTOOID |
                        MCO_DICT_FLAGS_PRESERVE_FIELDS_ORDER;
    ...
    mco_db_open_dev(stream_handle,
                 input_stream_reader, 
                db_name,
                0 /* no static dictionary */,
                pdev->dev, 
                pdev->n_dev,
                 &db_params );
                 
    mco_db_connect(db_name, con);
    mco_translog_apply(con, tl_log_file, MCO_TRANSLOG_ALL_LABELS );
     

When restoring a database from a log file, note that if the application was iterating a non-dynamic pipe (that was created without the MCO_TRANSLOG_DYNAMIC_PIPE flag), the "writer" thread receives error MCO_E_TL_PIPE_TERM indicating that the pipe was broken and the current transaction was not written into the pipe.

Java and C# applications set the max_classes, max_indexes and max_ddl_dict_size through the MaxClasses, MaxIndexes and MaxDictionarySize members of the Database::Parameters class. Python applications pass these values as arguments max_classes, max_indexes and maxDictionarySize to function open_database(). For example:

     
    db = exdb.open_database(dbname='opendb', dictionary=dict, is_disk=is_disk,
                    db_segment_size=128*1024*1024,
                    max_classes=100, max_indexes=1000,
                    maxDictionarySize=16*1024)
     

The following SDK samples are useful examples:

• tlogbasic_ddl - demonstrates maintaining a snapshot and log file using SQL C++ API
• xSQL/TL - demonstrates using the xSQL configuration file to implement the same functionality