xSQL Data Loaders Sample CME

This sample demonstrates a data loader implemented with a user-defined function (UDF). The target database schema defines two classes, TradesSeq and TOBSeq which contain sequence fields. The CSV format input files contain trades and trade orders data in CME format. (The original data came from CME sample data.)

The UDF loads from the CSV files, composes trades and orders with legs, makes buffers to work with the sequence data more efficiently, and appends these buffers to the target database sequences.

The UDF is in a shared library that exports functions load_cme_trades() and load_cme_tob(). These functions are loaded intoxSQL by the following statements in the config file xsql.cfg:

 
    sql_statements: "
        create function load_trades(fname string, book string) returns integer as 'mcocme', 'load_cme_trades';
        create function load_tob(fname string, book string) returns integer as 'mcocme', 'load_cme_tob';",
     

The client for this sample is a python program loadfunc.py that uses remote SQL protocol. It scans the subdirectories under 2013 for trade and order files and executes SQL statements to load them.

To run the sample

First compile the UDF:

 
    cd modcme
    make
     

Then run command file server.bat to start xSQL as the server in one console window:

 
    server.bat
     

To start the client in a second console window, first run command file env.bat to export environment variables and then python to load the data:

     
    env.bat
    python loadfunc.py