eXtremeDB is first and foremost an in-memory embedded database system (IMDS) ideally suited for embedded systems and other application domains that require extremely high performance / low latency, small footprint, compact storage, zero memory allocations, or any combination of these attributes. eXtremeDB provides superior performance by means of a very short code path, without sacrificing necessary DBMS features such as transactions that enforce the ACID properties, a data definition language, and support for SQL when needed. eXtremeDB also exhibits very small RAM, CPU and storage overhead. Designed for all-in-memory optimization, it boosts speed by eliminating file I/O, multiple data copies, and redundant logical processes, such as caching. When used with non-volatile (e.g. battery-backed) RAM, eXtremeDB delivers unsurpassed speed and persistence.
The eXtremeDB core provides data persistence with optimal performance through transaction logging, backup and database snapshot features. Or, by simply declaring database classes persistent, databases can be stored, entirely or partially, on persistent media.
Operations on “transient” (in-memory) database objects eliminate the complex and performance-draining tasks of cache and file management which has several beneficial side effects:
- Eliminating the complex logic associated with cache and file management reduces the code size (footprint) and execution path, and also eliminates multiple redundant copies of any given piece of data (i.e. a copy in the application, a copy in the database cache, a copy in the file system cache, and a copy in the file system itself).
- An all-in-memory database is optimized with different strategies than a disk-based database. The latter is concerned with minimizing file I/O operations, and will trade great use of memory and CPU instructions to avoid file I/O. An all-in-memory database doesn’t worry about disk I/O so is optimized to reduce CPU instructions and to maximize the amount of data that can be stored in a given amount of space. Consequently, eXtremeDB requires a fraction of the space of disk-based databases to store a given amount of data.
- eXtremeDB provides two libraries for all-in-memory databases: the optimized “Direct Pointer Arithmetic” library which accesses the memory locations of database records by simple pointer arithmetic (offering a 5% -15% performance advantage) or the “Offset” library which calculates record locations by first obtaining their offsets from the beginning address of the database then converting these to proper pointers (incurring a slight additional performance cost). Both libraries implement these calculations internally; in other words, the difference is transparent to application code. The advantages of the “Offset” library are A) that a saved binary image of an in-memory database can be loaded to a different starting address in memory, and, B) two (or more) applications that share an in-memory database in shared memory do not have to map the shared memory segment to the same address space in their local process (which is sometimes impossible). The advantage of the Direct Pointer Arithmetic library is better performance, and is ideal for in-memory databases within a single (possibly multi-threaded) process.
- Some embedded systems, such as flight safety related systems, are not permitted to use dynamic memory allocation because it can lead to memory leaks that could ultimately cause system failure. eXtremeDB is given a block of memory to use for the in-memory storage at start-up, so that no dynamic memory is allocated. An in-memory database can be increased in size, at the application’s discretion, but eXtremeDB itself will never allocate memory dynamically.
- To allow access to an all-in-memory database from multiple processes, an eXtremeDB database can be created in shared-memory so other applications can share the data directly.
eXtremeDB is available for a variety of embedded, real-time, desktop, and server operating systems. For operating systems with a single-process, multi-task architecture (for example, VxWorks 5.5 and INTEGRITY OS), eXtremeDB manages data in any dedicated region of memory and coordinates access by multiple tasks (threads). For multi-process architectures (for example, Sun Solaris, Linux, Windows, QNX Neutrino, etc.), eXtremeDB can manage a database in shared memory and coordinate access by multiple processes, each potentially with multiple threads.
eXtremeDB is a development tool that database-enables applications providing programming interfaces for a wide variety of programming languages and development environments, including:
- C/C++: C programmers on virtually all platforms will find the native C language API optimal, while C++ provides the means to encapsulate the database functionality in convenient application specific classes.
- Python: eXtremeDB provides a Python extension module "exdb.py" for the "core" C language APIs and a number of Python classes to facilitate application implementation using the popular Python language and interactive tools.
- Java Native Interface: a seamless integration of eXtremeDB with the Java programming language. Java programmers simply annotate Java class definitions with eXtremeDB attributes, and the database dictionary is implemented through the “reflection” capability of Java; no external schema language or compilation step is needed. The standard Java object oriented language is thus enhanced with optimal persistent data storage – whether all-in-memory, on-disk, or both.
- C# (.NET Framework): Windows developers will find the C# API ideal for incorporating high performance database functionality into .NET Framework applications. As with the Java API, the “persistent” attribute marks native C# classes for storage in an eXtremeDB database. Then eXtremeDB database functionality is implemented through classes like Database, Connection, and Cursor either by directly invoking their specialized methods or through LINQ queries. Also, the eXtremeDB .NET Framework LINQ provider allows the powerful variety of eXtremeDB indexes to significantly improve the performance of LINQ queries.
- eXtremeSQL: a high performance implementation of the industry standard SQL database language with C/C++ (proprietary and ODBC) and Java language interfaces (JDBC).
- All of the programming interfaces of eXtremeDB can be used within the same application and/or with the same database for complete programming flexibility.
Beyond the core eXtremeDB package, the following features extend the core functionality to meet specialized system requirements:
- eXtremeDB Transaction Logging: supports database persistence by transaction logging, a process that journals changes made to a database as the transactions are committed. With transaction logging enabled, the eXtremeDB Transaction Logging runtime appends database changes to a transaction log file so writes are always sequential, and never random. In the event of a hardware or software failure, the eXtremeDB runtime can recover the database from the log file. This edition also includes the “Data Relay” feature, an open replication scheme, that facilitates forwarding of eXtremeDB transactions to external systems (such as another middleware, or an enterprise DBMS).
- eXtremeDB High Availability: an optional module of eXtremeDB designed for applications that require the highest degree of reliability in case of system failures. The eXtremeDB High Availability libraries implement a time-cognizant two-phase commit (aka 2-safe or synchronous) protocol to guarantee that a master database instance and one or more replica database instances are perfectly synchronized, or a faster 1-safe/asynchronous protocol with lower consistency but much faster performance. It extends the core eXtremeDB programming interfaces with additional functions to allow replica databases to connect to and synchronize with the master, or for a replica to become the master in a “fail-over” scenario. Replica database instances are readable, facilitating load balancing of queries across all available database instances.
- eXtremeDB Cluster: an optional module of eXtremeDB for distributed database implementations. Applications running on different nodes in a network cluster all have the same copy of the database and all database modifications are automatically replicated across all nodes. Each node can perform
READ_WRITEandREAD_ONLYoperations, facilitating load-balancing and enabling cost-effective scalability through the exploitation of commodity hardware..- eXtremeSQL: an option of eXtremeDB that includes a high-performance implementation of the SQL database programming language. Applications can use standard SQL through a Windows ODBC driver, by linking to the eXtremeDB API library (bypassing the ODBC driver and driver manager), through JDBC, through an eXtremeDB proprietary C-language call level interface (CLI), or through a set of streamlined C++ classes. Built on the unsurpassed performance of eXtremeDB, and a SQL optimizer tuned to take advantage of the eXtremeDB runtime, eXtremeSQL delivers a fast interface for SQL programmers.
Product evolution
eXtremeDB has played a significant role in the success of thousands of real-time applications. Driven by requests from developers and enthusiastic customers, additional features have been added to extend the basic eXtremeDB core technology to address evolving user requirements, as can be seen from the list of versions and optional modules above. However, great care has been taken in each stage of the product evolution to make no compromises with our basic extreme performance goals. Each addition, be it eXtremeDB Transaction Logging, eXtremeDB High Availability, eXtremeDB Cluster, eXtremeSQL or the Java, C# and Python APIs, is provided in the form of separate libraries that can be linked into an application, as desired, to address specific application needs.
While developers who need the absolute best performance for mission critical applications are reassured to know that the underlying core runtime remains the fastest, most robust in the industry, they also appreciate knowing that less demanding applications can interface with the same eXtremeDB databases through SQL, for example, to generate reports or allow a flexible query interface.
As more diverse application data persistence features were requested, a number of features were added to extend functionality and developer control for all-in-memory as well as persistent databases including the following:
Choice of “optimistic” or “pessimistic” transaction management by simply linking the appropriate library:
- The Multi-Version Concurrency Control (MVCC) Transaction Manager, an optimistic (lockless) transaction manager which maximizes concurrency and scales excellently on multi-core systems, and allows choice of isolation levels when querying and updating the database
- The Multiple Readers Single Writer (MURSIW) Transaction Manager which is an extremely lightweight transaction manager that is ideal for in-memory databases on systems with relatively few concurrent tasks and/or with a high ratio of
READ_ONLYtoREAD_WRITEoperationsBinary Schema Evolution (BSE)
BSE provides the ability to load a pre-existing database into a newer database format and transform existing data objects automatically (e.g. new/dropped classes or fields, new/dropped indexes)
Transaction Logging
The Transaction Logging feature adds data persistence:
- logging can be turned on/off at runtime
- Data Relay: open replication to share data with external systems
Active Replication Fabric
eXtremeDB Active Replication Fabric provides automatic replication of data between server and edge devices.
XML Capability
The XML implementation provides additional data import / export capability.
Enhanced Database Security
- AES and CRC checking is provided for reliability when reading data from persistent storage
- Page-level Data Encryption for robust data security
NVRAM support
Support for database recovery from battery-backed memory.
Caching options
eXtremeDB provides performance optimization application-managed caching options, such as: