Database Encryption in C

As explained in page Database Encryption, eXtremeDB provides the capability to encrypt in-memory and persistent databases with standard AES encryption or by a custom encryption implementation.

Data Encryption

The encryption of data in memory is controlled by the cipher_key element of the parameter structure (mco_db_params_t) passed into mco_db_open_dev(). If this parameter is not NULL, then pages are encrypted with the AES or custom encryption algorithm before being saved. Each page is encrypted individually.

It is also possible to enable encryption with the online backup feature. The encryption of backup files is enabled by specifying the cipher argument of functions mco_backup_create() and mco_backup_restore(). Note that this option has no relation to the database parameter mco_db_params_t::cipher_key.

Database snapshot files are also encrypted by functions mco_db_save(), and decrypted by mco_db_load() if field mco_db_params_t::cipher_key is specified. Note that the exact same cipher key must be specified for saving and loading an encrypted snapshot.

The choice of encryption algorithm is determined by the library linked with the application. The AES algorithm implementation is provided in library mcocryptaes.

For details regarding implementation of a custom encryption algorithm, please see page Custom Encryption C API.

In-Memory Database Encryption

Encryption of in-memory pages is enabled only when both the cipher_key is specified and flag MCO_DB_INMEMORY_PROTECTION is set in field mco_db_params_t.mode_mask. Internally, MCO_DB_INMEMORY_PROTECTION mode will cause the creation of a hash table of used database pages (pages being accessed by the application). The maximum number of pages accessed by one connection at a given time can be specified by the max_active_pages field of struct mco_db_params_t. The default value is 32. The number of bundles in this hash table (used to minimize the number of lock conflicts in case of concurrent access by multiple threads) can be specified using the page_hash_bundles field of mco_db_params_t. (This value should be set ideally to the maximum concurrency level; i.e. the number of CPU cores – the default value is 32).

The following code snippet demonstrates how to enable encryption with the C API:

 
    int main(int argc, char* argv[])
    {
        MCO_RET            rc;
        mco_db_h db = 0;
        mco_device_t       dev[1]; 
        mco_db_params_t    db_params;
        ...
         
        /* Initialize and customize the database parameters */
        mco_db_params_init ( &db_params );                  /* Initialize the params with default values */
        db_params.mem_page_size      = 128;    /* Set page size for in-memory storage */
        db_params.disk_page_size     = 0;  /* Set page size 0 for in-memory storage */
        db_params.db_max_connections = 1;                   /* Set total number of connections to the database */
        db_params.cipher_key         = "my_secure_key";     /* Set cipher key filename */
        db_params.mode_mask         = MCO_DB_INMEMORY_PROTECTION;     /* Set mode mask */
         
        rc = mco_db_open_dev(db_name, genericdb_get_dictionary(), dev, N_DEVICES, &db_params );
        if ( MCO_S_OK == rc ) 
        {
            /* Connect to the database, obtain a database handle */
            mco_db_connect(db_name, &db); /* No recovery connection data */
            ...
        }
     

Note that in-memory protection can be used only for pure in-memory databases (i.e. no disk manager is used). So if this mode is specified when the mcovtdsk library is used, the function mco_db_open_dev() or mco_db_load() will return error code MCO_E_ILLEGAL_PARAM.

Also note that encryption of database pages can reduce application speed by 2-10 times, especially for index operations. Using the MCO_DB_INCLUSIVE_BTREE option can help to reduce this penalty (see the Key-Value-Inclusive and Covering Indexes page for further details).

Compression and Encryption

If the mco_db_params_t::mode_mask flag MCO_DB_INMEMORY_COMPRESSION is set, then memory pages are being compressed with the ZIP algorithm (see page Database Compression for further details). If mco_db_params_t::cipher_key is set in addition to the mode_mask flag MCO_DB_INMEMORY_COMPRESSION then transient database pages are both zipped and encrypted. However note that this combination is not compatible with the disk manager, thus compression plus encryption is not applicable for hybrid databases (with both transient and persistent classes) .

Persistent Database Encryption

eXtremeDB provides the capability to protect the persistent database memory arena, including the memory allocated for storage and the cache (page pool), through the operating system’s VM page-protection mechanism. However, note that it is only possible on UNIX platforms and should be done only for debugging purposes.

As for in-memory database pages, the encryption of data on persistent storage is controlled by the cipher_key element of the parameter structure (mco_db_params_t) passed into mco_db_open_dev(). If this parameter is not NULL, then pages are encrypted with the AES or custom encryption algorithm before being saved to “disk” and decrypted when loaded from the “disk”. Each page is encrypted individually.

AES is a symmetric encryption algorithm. On the first pass, the data is encrypted, and on the second pass is decrypted back to its original form. Consequently in the encryption API there are not separate functions to encrypt and decrypt, but simply the one function encrypt().

The stub library mcocryptstub contains empty API implementations which simply halt processing with the error code MCO_E_UNSUPPORTED and is provided for developers to implement a custom encryption algorithm. (See section Custom Encryption Algorithm Implementation below for further details.)

Note that encryption of database pages can reduce application speed by 2-10 times, especially for index operations. Using the MCO_DB_INCLUSIVE_BTREE option can help to reduce this penalty (see the Key-Value-Inclusive and Covering Indexes page for further details).

 

Encryption of other files external to the database

Apart from protecting persistent and transient databases, encryption can be used to protect database snapshots made by functions mco_db_save() and mco_class_save(), to protect the TL transaction log (created by mco_translog_start()) and also to protect backup files created by function mco_backup_create().

The encryption of database snapshots and transaction log files is automatically enabled by specifying the cipher_key field just as for the encryption of disk and memory pages. Encryption of backup operations is however independent of database open parameters and is directly specified by the cipher argument passed to functions mco_backup_create() and mco_backup_restore().

 

Encryption Libraries

Please use the links below to view the AES encryption algorithm library and "stub" library (as well as other libraries) provided for your development platform:

Unix-Linux	Static libraries for Standard Unix or Linux 64bit or 32bit platforms
Windows	Static libraries for Windows 7 (or later) platforms
Integrity	Static libraries for Green Hills Integrity OS
VxWorks	Static libraries for Wind River VxWorks OS