Database Open Modes

The mode_mask element of the mco_db_params_t structure can be set with the following values:

MCO_DB_MODE_MVCC_AUTO_VACUUM

Causes automatic cleanup of stale versions (MVCC) when the database is opened. This mode can be used to delete obsolete versions of objects in the case of recovery after a disk-based database application crash.

This procedure can take a significant amount of time as it reads the database file to scan the database at startup. (Hence it is not the default mode.) If the disk-based database was gracefully shut down, then the next run should not use this option. But when the database is recovered after a crash, or even multiple crashes, its size can be reduced using MCO_DB_MODE_MVCC_AUTO_VACUUM mode. (See description of mco_disk_database_vacuum() )

MCO_DB_MODE_SMART_INDEX_INSERT

Causes the runtime to pre-sort transaction data based on the first index declared for the class prior to including them in indexes. This can optimize the index tree structure in the case of very large disk indexes in conjunction with large transactions. (The benefit may not be readily evident and depends to a large extent on the database schema. Hence this mode is not the default.)

MCO_DB_OPEN_EXISTING

Instructs the runtime not to initialize an all-in-memory database, but rather to treat the database memory device as an already initialized memory space. For example, an NVRAM based device could be switched off or lose power. Then, on the next startup, it has in fact already initialized and filled NV memory. So in MCO_DB_OPEN_EXISTING mode, the runtime simply opens that memory assuming that it was initialized by the previous run.

MCO_DB_USE_CRC_CHECK

Instructs the runtime to calculate and store the CRC for each database page.

MCO_DB_TRANSIENT

Forces all database classes to be “in-memory” (transient) even if the classes are declared persistent in the database schema. This may be useful to temporarily override the definition of persistent classes for testing purposes.

MCO_DB_LAZY_MEM_INITIALIZATION

By default, the runtime initializes all given memory: splits it into pages and links these pages into the page manager's list. But for very large databases (several gigabytes) it may take a significant amount of time. With this run-time option, memory is "formatted" on demand which delays the memory pages initialization procedure from the initialization stage to the moment of the first usage of each database page. So it speeds up database initialization at the expense of performance during data access. This option is not the default because it adds one extra check during page allocation; and by initializing memory in advance, physical memory is reserved for the database so that database access time is reduced.

MCO_DB_INMEMORY_PROTECTION

Enables the runtime to implement page level encryption. If flag MCO_DB_INMEMORY_PROTECTION and mco_db_params_t::cipher_key are flag then the memory pages of transient databases are encrypted by currently available symmetric encryption algorithm. However this combination is not compatible with the disk manager, thus it is not applicable for hybrid databases (i.e. with both transient and persistent classes) . (See Database Encryption).

MCO_DB_INCLUSIVE_BTREE

Allows the runtime to exploit CPU cache memory for data structures and algorithms which in many cases can improve performance dramatically over normal RAM access. This option can be especially effective when index B-Tree structures are very large. (See Key-Value-Inclusive and Covering Indexes)

MCO_DB_INMEMORY_COMPRESSION

Enables the runtime to conserve memory through data compression. If flag MCO_DB_INMEMORY_COMPRESSION and mco_db_params_t::cipher_key are set then transient database pages are both zipped and encrypted. However this combination is not compatible with the disk manager, thus it is not applicable for hybrid databases (i.e. with both transient and persistent classes) . (See Database Compression).

MCO_DB_SEPARATE_BITMAP

Allows the runtime to use separate memory allocation bitmap pages for aligned and unaligned memory objects. This can help avoid disk file fragmentation for persistent databases. Placing objects together (sequentially) can significantly speed-up a linear search. For in memory databases the locality of reference is not so critical (though sequential access is faster, especially with NUMA). But for eXtremeDB Persistent databases there are two ways to improve locality of references:

• By using allocation blocks: when a particular block (usually several megabytes) is used for allocation of objects of particular class. (See generated function classname_set_allocation_block_size().)
• By setting the MCO_DB_SEPARATE_BITMAP flag. In this case index pages and object pages will be allocated using different allocators. It allows better grouping of records and reduces fragmentation. But it can cause greater disk space consumption.

It is difficult to give a concrete recommendation when this option should be used or estimate its impact on performance and database size. So it should be considered an experimental option to validate empirically.

MCO_DB_MURSIW_DISK_COMMIT_OPTIMIZATION

When using the MURSIW transaction manager, this flag causes the runtime to save modified pages to the transaction log file and flush the log outside of a critical section, allowing read-only transactions to process. This may increase concurrency and reduce delays during transaction processing.

MCO_DB_BULK_WRITE_MODIFIED_PAGES

Causes the runtime to sort and write all non-pinned dirty pages, instead of writing just a single dirty page thrown away from the disk page pool when loading a new page from disk. This can reduce disk head movement with the possible drawback that a modified page may be written multiple times.

MCO_DB_DISABLE_PAGE_POOL_RESERVE

For Persistent databases, it is possible for the cache memory to be exhausted. For this reason a “reserved pool” of pages is added to the cache. The runtime automatically allocates this additional cache memory unless this flag is set to disable the reserve page pool. (See Cache Management)

MCO_DB_INDEX_PRELOAD

Causes the runtime to try to preload the next B-Tree page using system function fadvise(). This may increase the speed of sequential scans. (However fadvise() can cause a synchronous page read on Linux which is likely to decrease performance .)

MCO_DB_DISABLE_NESTED_TRANSACTIONS

Causes the runtime to disable nested transactions. This flag causes error MCO_ERR_NESTED_TRANS_TRAP to be thrown if mco_trans_start() is called before a commit or rollback.

MCO_DB_DISABLE_IMPLICIT_ROLLBACK

Causes the runtime to not implicitly rollback active transaction(s) on disconnect. Instead an error is reported. This can be used for debugging unintentional nesting of transactions.

MCO_DB_DISABLE_BTREE_REBALANCE_ON_DELETE

Causes the runtime to disable B-Tree rebalancing when deleting objects. Most frequently deletes are followed by inserts so rebalancing the B-Tree greatly decreases performance. But disabling this rebalancing can lead to a non-optimal tree (with a lot of almost empty pages).

MCO_DB_AUTO_ROLLBACK_FIRST_PHASE

If the commit of the first stage of a two-phase fails, then this flag causes the runtime to implicitly rollback the transaction, not waiting for the application to call mco_trans_rollback(). This is not the default behavior for two-phase commit because in some scenarios if the first phase of a transaction fails on some nodes in a network, other nodes may have to explicitly roll back their transactions.

MCO_DB_MVCC_COMPATIBILITY_MODE

Open a persistent database that was created while using the MVCC transaction manager with an application (or xSQL) using the MURSIW transaction manager. (It is also possible to instruct the xSQL to open the database in the transaction manager compatibility mode by setting the mode_mask in the configuration file section db_params:mode_mask).

MCO_DB_DISABLE_PAGE_POOL_RESERVE

By default, if the threshold for the minimal amount of available (not-pinned) pages is reached, the runtime will set the transaction error code, preventing commit of the current transaction. Setting this flag causes the runtime to not reserve space in the page pool, allowing it to use all of the page pool memory and thus make it possible to complete any active transaction.

MCO_DB_REDO_LOG_OPTIMIZATION

By default, when using the REDO log option, the transaction size is limited by the size of the page pool. This limitation can prevent applications from performing operations such as creating tables or indexes as the transaction size is effectively limited by the available memory. This flag can be set to enable the runtime, in many cases, to allow transactions larger than the pool page size. (See Setting Transaction Log Type and Parameters)

MCO_DB_DISABLE_HOT_UPDATES

Allows eliminating the update of indexes if index keys are modified by an update operation. "Hot updates" may increase search time because a search has to traverse the hot update chain to locate the proper object. If updates are relatively rare and the search speed is much more critical, then it can be useful to disable hot updates by setting this flag.

MCO_DB_SQL_AUTOCHECKPOINT

This causes all newly created database objects to be inserted into the defined indexes so that they can be accessed within the transaction without the need to explicitly call the checkpoint API.

MCO_DB_MODE_READ_ONLY

If the flag is set, a READ_WRITE or UPDATE transaction returns the MCO_E_ACCESS error code. Furthermore if the file device assigned to the persistent database (MCO_MEMORY_FILE, MCO_MEMORY_MULTIFILE or MCO_MEMORY_RAID) is detected to be read-only, the database runtime automatically adds the MCO_DB_MODE_READ_ONLY into the open database mode mask.

Note that, for persistent databases, the log file must remain writable. If the log device file is made read-only, or if the MCO_FILE_OPEN_READ_ONLY flag for the log device is set by the application, a fatal exception is raised.

Also, a database (either persistent or transient) can be loaded via the mco_db_load_*() API (or the corresponding Java, C# or Python APIs) from a previously saved image. In this case, the database is loaded and made read-only, so no further modifications to the database are allowed.

MCO_DB_USE_AIO

If the flag is set Asynchronous IO is used for persistent database writes. By default one AIO thread is started for each database and the default queue size is set to 10007. Sometimes it could be beneficial to change the number of AIO threads (for example if the database is comprised of multiple physical IO devices, or is resided on a RAID) and/or the queue size. However note that the number of I/O threads won't exceed the number of databases created in a process, regardless of the value of worker's threads passed by the application (the runtime does not allow more then one AIO thread to serve a database)

MCO_DB_INCREMENTAL_BACKUP

This flag controls the marking of database pages for incremental backup support. Without this flag enabled it is still possible to make backups, but snapshots only, as the database engine will not trace the modification of pages. It is possible to get the status and enable or disable the incremental backup processing at run time using mco_backup_incremental_processing_get() and mco_backup_incremental_processing_set()

MCO_DB_MVCC_TABLE_LEVEL_LOCKING

Directs the runtime to block the entire table’s space instead of a selective instances during database operations when using the MVCC transaction manager only. The influence of this option on database performance depends on the application’s database access pattern.

MCO_DB_DISABLE_SMART_ALLOC

This flag affects the allocation for objects. When set the eXtremeDB runtime allocates space for the average object size, calculated from statistics. This can be useful in some cases as it truncates before assignment, and reduces the number of full map rescans at object allocation.