xSQL Data Compression and Encryption

In-memory databases can be compressed to conserve memory and persistent database data and log files can be compressed to conserve disk space. Data can also be encrypted for security.

In-memory compression

For in-memory database compression the configuration file option db_params:mode_mask must be set with the inmemory_compression flag value. It is also possible to explicitly set the options page_hash_bundles, compression_level, compression_mask, and expected_compression_ratio. For example:

 
    db_params: 
    { 
        mode_mask: in_memory_compression,
        page_hash_bundles: 8,
        compression_level: 1,
        compression_mask: 3,
        expected_compression_ratio: 4
    }
     

Persistent database compression

Note that, as explained in Data Compression, disk file compression is supported only on Unix-Linux platforms.

To enable compression with eXtremeDB, the configuration file must specify which file to compress (or is compressed) and which special file system library to load to manage the compression. (The database files must use library mcofu98zip, while log files use library mcofu98ziplog.)

For example, to compress the persistent database file test.dbs, the configuration file would specify the flag value compressed for the file device and use library mcofu98zip with the following option settings:

 
    devices : [
        {assignment: database, size : 100m},
        {assignment: cache, size : 100m},
        {type:file, assignment : persistent, name : test.dbs, flags: compressed},
        {type:file, assignment : log, name : test.log},
    ],
     
    runtime_configuration:
    {
        fs_library: mcofu98zip,
    }
     

 

To compress the log file test.log, the configuration file would contain the following options:

 
    devices : [
        {assignment: database, size : 100m},
        {assignment: cache, size : 100m},
        {type:file, assignment : persistent, name : test.dbs},
        {type:file, assignment : log, name : test.log, flags: compressed },
    ],
     
    runtime_configuration:
    {
        fs_library: mcofu98ziplog,
    }
     

(Note that if both database and log files need to be compressed, xSQL must be run twice, once with mcofu98zip for the database file(s) and once with mcofu98ziplog for the log file - only one file system library, either mcofu98zip or mcofu98ziplog can be loaded at a time.)

Data Compression for IoT Communications

Data compression for IoT communications is enabled by setting the compression_level parameter of the iot_params configuration file section (which corresponds to the compression_level element of structure mco_iot_comm_paramst_t ).

Also, as explained in the Data Compression page is the formula used to determine the compression ratio. The xsql_iot_list() SQL function can be called to retrieve the statistics for a connection; fields rx_zratio and tx_zratio of returned Result Set (which correspond to elements sent_compression_ratio and recv_compression_ratio of structure mco_iot_connection_stat_t ) show the compression efficiency for incoming and outgoing communications.

 

Data Encryption

To encrypt a disk database, the configuration file section db_params must contain the cipher_key option:

 
    db_params : {
        cipher_key : "my_secret_password"
    }