Persistent Media I/O
As explained in the Persistent Database I/O page, there are two aspects of writing data to consider: performance and recovery in the case of system failure. To allow for recovery of the database in the case of a system failure, the eXtremeDB runtime uses a log file in addition to the database file(s) for persistent databases.
The following diagram illustrates the write process:
Non-buffered I/O is slow, so file I/O is usually buffered so that the database write operation does not write data directly to the persistent media but rather to the file system buffer. Then the file system will “flush” buffered data to the persistent media during a file system commit or
_sync().eXtremeDB provides three options to affect the performance of this write process: in addition to the Log File Type and the Transaction Commit Policy discussed in the Persistent Database I/O page, eXtremeDB provides the option of Asynchronous file I/O (AIO as described in the section below). AIO determines the manner in which the file system processes write requests. Also, eXtremeDB support for multifile and RAID devices is described in the Multi-File section below.
(Please see the Native Language APIs section for implementation details.)
Asynchronous file I/O
Asynchronous file I/O sends an I/O request to the kernel then the calling thread continues processing until the kernel signals to the thread that the I/O operation is complete. For large database transactions, AIO is a good way to optimize processing efficiency. However, for relatively fast I/O operations, the overhead of processing kernel I/O requests and kernel signals may make it less beneficial, particularly if many fast I/O operations need to be made.
The eXtremeDB asynchronous I/O mode implementation is optimized by grouping and reordering updates to make sure that multiple updates are synchronized with the media at once. Note that AIO stores disk operations in an in-memory queue and doesn't flush them on each commit. Thus, in case of a hardware failure that data will not be written to the disk and, after the recovery, the database could be missing some of the most recently committed (from the application's point of view) transactions. However, even if this occurs, the database will always remain in a consistent state.
AIO is presented primarily as a server-side feature. Servers don't normally allow applications to access the database directly. The access is mostly through a remote protocol. By default, AIO is disabled. But there are APIs to enable, start and stop AIO.
Note that the asynchronous I/O is not currently available for shared memory databases.
Multi-file databases
eXtremeDB supports three types of multi-file devices for persistent databases:
- multifile: which can be considered a virtual file consisting of multiple segments. When the first segment is full, we start filling of the second one and so on... For all segments except the last one size should be specified. The last segment can grow infinitely.
- RAID-0 (striping): blocks are scattered between RAID segments. For example in case of RAID-0 with two segments, the first block is written to the first segment, second - to the second, third - to the first and so on. RAID-0 can help to improve performance: for example each RAID device could be a separate disk (that should be controlled by a separate controller). But RAID-0 does not provide extra redundancy.
- RAID-1 (mirroring): data is copied to all RAID segments. So if the RAID consists of three segments, then the same data will be stored in all three segments (redundancy level 3). RAID-1 provides better reliability (in the case of a disk crash it is possible to use data on one of the other disks). It also improves read performance since read requests can be balanced between different disks.
File segments are defined in the memory device array passed to the database open call. For multifile implementations, segments can be added at runtime by means of the extend function. Multi-file segments cannot be removed.
Native Language APIs
Please use the links below to view the details for your development environment:
C Persistent Media I/O in C C++ Persistent Media I/O in C++ Java Persistent Media I/O in Java C# Persistent Media I/O in C# Python Persistent Media I/O in Python xSQL xSQL Persistent Media I/O