Order Matching Simulator User Guide

When the market data is snapshot data, the order matching process is as follows:

1. Immediately match the limit order against the top N levels of the order book:

• Buy orders: When the order price ≥ Level 1 ask price, the order is matched sequentially against the levels of the ask side. The execution price is the price at the corresponding book level. The execution quantity is the minimum of: (ask level quantity * level fill ratio) and (order quantity - already filled quantity). If the order is not fully filled, it then enters the order resting phase (i.e., orders queued in the order book awaiting matching). The order is placed first in the queue at that price level, meaning there is no quantity ahead of it, and then enters the order matching phase.

• Sell orders: when the order price ≤ Level 1 bid price, the order is matched sequentially against the levels of the bid side. The execution price is the price at the corresponding book level. The execution quantity is the minimum of: (bid level quantity * level fill ratio) and (order quantity - already filled quantity). If the order is still not fully filled, it then enters the order resting phase. The order is placed first in queue at that price level, meaning there is no quantity ahead of it, and then enters the order matching phase.

2. Resting Orders

If an order’s price matches the levels on the same side of the order book, it enters the order resting phase. The quantity ahead of this order equals the total quantity at the same price level. The order then proceeds to the order matching phase.

3. Order Matching Stage (Unfilled or Partially Filled Portion)

• Order Matching Mode 1

  • If the fill ratio is set to 0, the process is the same as in Step 1. The order is matched against the corresponding opposite-side book levels based on the latest market data, with the execution price determined by the order’s limit price. Simulated order matching ends when the order is fully filled or at the end of the trading day, whichever occurs first. Any unfilled or partially filled order may be canceled upon receipt of a cancellation request.

  • If the fill ratio is greater than 0, matching is performed as follows:

    • Matching is performed based on the latest price and latest book from the snapshot market data:

      • When the latest price equals the order price, the system first deducts the quantity ahead of the order from the interval traded volume, and this portion is marked as the current traded volume. When the quantity ahead of the order falls to 0, the user's order begins to execute. The execution price is the order price, and the execution quantity is the smaller of the user's order quantity and the current traded volume multiplied by the fill ratio.

      • When the latest price is more favorable than the order price (for buy orders, latest price < order price; for sell orders, latest price > order price), the execution price is the order price, and the execution quantity is the smaller of the user's order quantity and the actual traded volume during the interval multiplied by the fill ratio.

      • Any remaining unfilled portion is then matched against opposite-side book levels by price, with the execution price set to the order price.

      • The matching continues until the order is fully filled or the market closes for the day, any unfilled or partially filled order may be canceled upon receipt of a cancellation request.

• Order Matching Mode 2

  Matching is performed based on the interval trade list and the latest book in the snapshot market data, following the price-time priority principle:

  • For a user's buy order, order matching is performed against the trade list. Trades in the list with prices less than or equal to the user's order price are eligible for matching. The system first deducts the quantity ahead of the user's order (that is, after subtracting the same-price market quantity ahead of the order, the remaining quantity following the user's order is marked as the current traded volume). When the current traded volume is greater than 0, the user's order can be filled. The execution price is the order price, and the execution quantity = min(user's remaining order quantity, current traded volume).

  • For a user's sell order, order matching is performed against the trade list. Trades in the list with prices greater than or equal to the user's order price are eligible for matching. The system first deducts the quantity ahead of the user's order (that is, after subtracting the same-price market quantity ahead of the order, the remaining quantity following the user's order is marked as the current traded volume). When the current traded volume is greater than 0, the user's order can be filled. The execution price is the order price, and the execution quantity = min(user's remaining order quantity, current traded volume).

  • Any remaining unfilled portion is then matched against opposite-side book levels by price, with the execution price set to the user's order price and the execution quantity = min(user's remaining order quantity, matched quantity * fill ratio).

  • If the order is not yet fully filled, update the queued order quantity at the same price as the user's order for orders ahead of the user's order in the queue:

  • Matching continues until the order is fully filled or the market closes for the day, any unfilled or partially filled order may also be canceled upon receipt of a cancellation request.

When the market data is tick-by-tick, the order matching process is as follows:

1. First, the limit order is matched immediately against the top N levels of the order book data (order matching may be performed according to the configured fill ratio).
2. Any unfilled or partially filled portion is then matched in accordance with the principles of price-time priority.
   1. For subsequent tick-by-tick data, user orders are matched against incoming orders at the opposite side of the book following price-time priority.
   2. An order that is unfilled or partially filled may also be canceled upon receipt of a cancellation request.
   3. Matching continues until the order is fully filled or the market closes for the day.

The SSE market order trading rules are as follows:

• Five Best Orders Immediate or Cancel (IOC): an order that is executed in sequence against the current five best prices on the opposite side. If any portion of the order remains unfilled after execution, the unfilled portion is canceled automatically.

• Five Best Orders Immediate to Limit: an order that is executed in sequence against the current five best prices on the opposite side. Any remaining unfilled portion is converted into a limit order at the latest traded price on the same side in the market order book; if none of the orders is executed, it is converted into a limit order at the best same-side price in the market data; if there is no same-side order, the order is canceled.

• Same-Side Best Price Order: an order whose quotation price will be the best price on the same side in the central order book when such order enters the SSE trading system. If there is no corresponding order on the opposite side, the order is canceled automatically.

• Opposite-Side Best Price Order: an order whose quotation price will be the best price on the opposite side in the central order book when such order enters the SSE trading system. If there is no corresponding order on the opposite side, the order is canceled automatically.

The SZSE market order trading rules are as follows:

• Fill or Kill (FOK): trades are executed at the opposite prices. If, at the time the user submits the order, the opposite-side market orders in the latest market-data order book can fully fill the order through sequential execution, the trades are executed sequentially; otherwise, the entire order is cancelled automatically.

• Five Best Orders Immediate or Cancel (IOC): executed in sequence against the current five best prices on the opposite side. Any unfilled portion is canceled automatically.

• Immediate or Cancel (IOC): an order that is executed in sequence against all the orders on the opposite side in the central order book at the time the order is routed into the Exchange trading system. Any unfilled portion is automatically canceled.

• Same-Side Best Price: an order whose quotation price is set at the best price on the same side in the central order book at the time the order is routed into the Exchange trading system.

• Opposite-side Best Price: an order whose quotation price is set at the best price on the opposite side in the central order book at the time the order is routed into the Exchange trading system.

Level 2 market snapshots with a 3-second frequency are used as the input for the order matching simulator. The fill ratio was set to 0.5 and the interval fill ratio to 0.1. For each stock, an order of 1,000 shares was placed every 3 seconds. The test measured execution time for 100, 500, and 2,000 stocks, respectively. The timing includes both data retrieval from the raw DFS table and computation time. The complete test script is provided in the appendix.

def strategy(mutable engine,msg,mutable contextdict,mutable logTB){ 
	/** Place an order for 1,000 shares per stock every 3 seconds*/
	try{
		temp=select last(DateTime) as time ,last(lastPx) as price from msg group by SecurityID
		codes=distinct(msg.SecurityID)
		prices=dict(temp.SecurityID,temp.price)
		for (icode in codes){
			MatchingEngineSimulator::insertMsg(engine,(icode,contextdict["now"],5,prices[icode],1000,1,1),2)
			}
		//log
		text="open_signal: "+string(contextdict["now"])+"  code: "+ 
		concat(temp.SecurityID,",")+" price:  "+string(concat(string(temp.price),","))
		logTB.append!(table(text as info))		
		}
	catch(ex){
		logTB.append!(table("error:"+string(contextdict["now"])+ex[1] as info))
	}
}
timer{
	for (idate in 2022.04.15..2022.04.15){
		contextdict["now"]=idate
		tb = select "XSHE" as symbolSource,SecurityID,DateTime,lastPrice,highestPrice,lowestPrice,deltas(TotalVolumeTrade)  as newTradeQty ,BidPrice,
		BidOrderQty,OfferPrice,OfferOrderQty from loadTable("dfs://注意
		date(DateTime)=idate and  DateTime.second() between 09:30:00:15:00:00
		context by SecurityID csort DateTime order by DateTime map  
		times=tb.DateTime
		nTimes=cumsum(groupby(count,times,times).values()[1])
		i=1
		for (i in 0..(size(nTimes)-1)){
			if(i==0){
				contextdict["now"]=times[nTimes[0]-1]
				msg=tb[0:nTimes[0]]
			}
			else{
				contextdict["now"]=times[nTimes[i]-1]
				msg=tb[nTimes[i-1]:nTimes[i]]
				}		
			if(size(msg)>0){
				savehqdataToMatchEngine(engine, msg,logTB)	
				strategy(engine,msg, contextdict, logTB)
				}
			}
		MatchingEngineSimulator::resetMatchEngine(engine)
	}
}

The detailed trade results are shown in the table below:

• Load the plugin.

loadPlugin("<path>/MatchEngineTest.txt")

This algorithmic trading plugin is named MatchEngineTest, and its txt file is located in the bin directory of the plugin source code.

• Plugin interface

MatchEngineTest::runTestTickData(messageTable, stockList, engineConfig)

messageTable contains the data produced after replaying market data (tick-by-tick orders and tick-by-tick trades).

stockList is the list of stocks for which the user places orders.

engineConfig is a dictionary with three configurable parameters. engineConfig[“engineName“] specifies the name of this test and is also used as the name when creating the order matching simulator. When creating multiple order matching simulators, the engine names must be unique. engineConfig[“market“] specifies the exchange to use: “XSHG” (Shanghai Stock Exchange) or “XSHE”. engineConfig[“hasSnapshotOutput“] has the same meaning as the “outputOrderBook” parameter in the order matching simulator and indicates whether to output the order book.

• Input and Output Description

The column names and corresponding types of the input table messageTable are as follows:

colName = `msgTime`msgType`msgBody`sourceType`seqNum
colType =  [TIMESTAMP, SYMBOL, BLOB, INT, INT]

msgType can take the values entrust, trade, and snapshot, which represent tick-by-tick orders, tick-by-tick trades, and snapshot data, respectively. Inside the plugin, the BLOB-format data in messageTable is parsed. In this plugin, the data source for algorithmic trading uses only tick-by-tick data. The schema of the tick-by-tick order and trade data is tickSchema, which is specified in the C++ code as follows:

TableSP tickSchema = Util::createTable({"symbol", "symbolSource", "TradeTime", "sourceType", "orderType", "price", "qty", "buyNo", "sellNo", "BSFlag", "seqNum"}, \
            {DT_SYMBOL, DT_SYMBOL, DT_TIMESTAMP, DT_INT, DT_INT, DT_DOUBLE, DT_INT, DT_INT, DT_INT, DT_INT, DT_INT}, 0, 0);

If the algorithmic trading logic needs to use snapshot data, the input data source must include snapshot data. The snapshot table schema is defined in the C++ code as follows:

TableSP snapshotSchema = Util::createTable({"SecurityID", "TradeTime", "LastPrice", "BidPrice", "OfferPrice", "BidOrderQty", "OfferOrderQty", "TotalValueTrade", "TotalVolumeTrade", "seqNum"}, \
            {DT_SYMBOL, DT_TIMESTAMP, DT_DOUBLE, DATA_TYPE(ARRAY_TYPE_BASE + DT_DOUBLE), DATA_TYPE(ARRAY_TYPE_BASE + DT_DOUBLE), DATA_TYPE(ARRAY_TYPE_BASE + DT_INT), \
            DATA_TYPE(ARRAY_TYPE_BASE + DT_INT), DT_DOUBLE, DT_INT, DT_INT}, 0, 0);

In the plugin implementation, the two result tables output by the order matching simulator, tradeOutputTable and snapshotOutputTable, are saved as shared tables. You can query these two tables to check the order matching simulation results.

// // Output results
vector<string> arg0 = {"tmp_tradeOutputTable"};
vector<ConstantSP> arg1 = {tradeOutputTable};
heap_->currentSession()->run(arg0, arg1);
runScript("share tmp_tradeOutputTable as "+ testName_ + "_tradeOutputTable");
arg0 = {"tmp_snapshotOutputTable"};
arg1 = {snapshotOutputTable};
heap_->currentSession()->run(arg0, arg1);
runScript("share tmp_snapshotOutputTable as "+ testName_ + "_snapshotOutputTable");

When algorithmic trading is executed in multiple threads by stock, each thread has its own tradeOutputTable and snapshotOutputTable. To check stock execution results, you need to aggregate the result tables from all threads before outputting them.

tradeResult = table(100:0, MatchEngineTest1_tradeOutputTable.schema().colDefs.name, MatchEngineTest1_tradeOutputTable.schema().colDefs.typeString)
for (i in 1..thread_num) {
    tradeResult.append!(objByName("MatchEngineTest"+i+"_tradeOutputTable"))
}
// Check the total executed volume of submitted user orders
select Symbol, sum(VolumeTraded) from tradeResult where OrderStatus != 3 and OrderStatus != 2 group by Symbol order by Symbol

This section describes the overall execution flow of the code, starting from the plugin entry point. The complete C++ code is provided in the appendix.

• Plugin interfaces

Each call to runTestTickData creates a new TickDataMatchEngineTest instance and invokes its runTestTickData method.

ConstantSP runTestTickData(Heap *heap, std::vector<ConstantSP> &arguments) {
    TableSP messageStream = arguments[0];
    ConstantSP stockList = arguments[1];
    DictionarySP engineConfig = arguments[2];
    TickDataMatchEngineTest test(heap, engineConfig);
    test.testTickData(messageStream, stockList);
    return new String("done");
}

The constructor of the TickDataMatchEngineTest class is defined as follows:

TickDataMatchEngineTest::TickDataMatchEngineTest(Heap * heap, ConstantSP engineConfig): heap_(heap) {
    string name = engine_config->get(new String("engineName"))->getString();
    market_ = engine_config->get(new String("market"))->getString();
    if (market_ == "XSHE") {
        marketOrderType_ = static_cast<int>(UserOrderType::XSHEOppositeBestPrice);
    }
    else {
        marketOrderType_ = static_cast<int>(UserOrderType::XSHGBestFivePricesWithLimit);
    }
    bool hasSnapshotOutput = engine_config->get(new String("hasSnapshotOutput"))->getBool();
    test_name_ = name;
    // Create the order matching simulator
    engine_ = createMatchEngine(name, market_, hasSnapshotOutput);
    ...
}

The testTickData function is defined as follows:

bool TickDataMatchEngineTest::testTickData(TableSP messageStream, ConstantSP stockList) {
    ...
    // MsgDeserializer is a parser class for BLOB-format data; tickDeserializer is used to parse tick-by-tick data
    MsgDeserializer tickDeserializer(tickSchema);
    ...
    ConstantSP nowTimestamp = messageCols[0]->get(0);
    ConstantSP lastTimestamp = nowTimestamp;
 for (int i = 0; i < messageStream->size(); ++i) {
        nowTimestamp = messageCols[0]->get(i);
        string data = messageCols[2]->getString(i);
        DataInputStream stream(data.c_str(), data.size());
        if (nowTimestamp->getLong() != lastTimestamp->getLong()) {
            context.timestamp = lastTimestamp;
            TableSP msgTable = tickDeserializer.getTable(); // Market data with the same timestamp is treated as one batch.
            saveQuatationToEngine(msgTable); // Send market data to the engine.
            handleStrategy(context, msgTable); // Execute the algorithmic strategy to generate user orders and send them to the engine.
            tickDeserializer.clear();
            lastTimestamp = nowTimestamp;
        }
        tickDeserializer.deserialize(&stream); // Parse this row and store it in the internal table of the parser class.
    }
    ...
}

• Insert market data and user orders into the engine.

The methods for saving market data and sending user orders are the same as those used in the DolphinDB script. Call the appendMsg method to pass data to the order matching simulator.

appendMsgFunc = heap_->currentSession()->getFunctionDef("appendMsg");
...
void TickDataMatchEngineTest::saveQuatationToEngine(ConstantSP data) {
    vector<ConstantSP> args = {engine_, data, new Int(1)};
    appendMsgFunc->call(heap_, args);
}
void TickDataMatchEngineTest::saveOrdersToEngine(ConstantSP data) {
    vector<ConstantSP> args = {engine_, data, new Int(2)};
    appendMsgFunc->call(heap_, args);
}

• Strategy execution

When placing orders, you must first build a user order table. Because an in-memory table is also stored in columnar format, you need to create `VectorSP` columns first and then build the table from those columns. For SZSE data, set market orders to 0 when placing orders; for SSE data, set market orders to 1. For the definition of market order types, see the plugin interface manual. After the table has been created, call the saveOrdersToEngine function to send the user orders to the engine.

void TickDataMatchEngineTest::handleStrategy(ContextDict& context, ConstantSP msgTable) {
    // Check whether it is time to trigger the strategy.
    ConstantSP timestampToTrigger = new Timestamp(tradeDate_->getInt() * (long long)86400000 + timeToTrigger_->getInt());
    if (!strategyFlag_ || timestampToTrigger->getLong() > context.timestamp->getLong()) {
        return;
    }
    // Get open orders.
    vector<ConstantSP> args = {engine_};
    TableSP openOrders = getOpenOrdersFunc->call(heap_, args);
    int openOrderSize = openOrders->size();
    if (openOrderSize > 0) {
        // Cancel orders.
        ConstantSP symbols = openOrders->getColumn(2);
        ConstantSP times = createVectorByValue(context.timestamp, openOrderSize);
        ConstantSP ordertypes = createVectorByValue(new Int(static_cast<int>(UserOrderType::CancelOrder)), openOrderSize);
        ConstantSP prices = createVectorByValue(new Double(0.0), openOrderSize);
        ConstantSP qtys = openOrders->getColumn(5);
        ConstantSP directions = createVectorByValue(new Int(1), openOrderSize);
        ConstantSP orderIDs = openOrders->getColumn(0);
        vector<ConstantSP> openOrderList = {symbols, times, ordertypes, prices, qtys, directions, orderIDs};
        TableSP ordertable = Util::createTable({"SecurityID", "time", "orderType", "price", "qty", "direction", "orderID"}, openOrderList);
        saveOrdersToEngine(ordertable);
    }
    // Place a market order for each stock at the best ask price.
    int stockSize = context.stockList->size();
    ConstantSP symbols = Util::createVector(DT_SYMBOL, stockSize);
    ConstantSP times = createVectorByValue(context.timestamp, stockSize);
    ConstantSP ordertypes = createVectorByValue(new Int(marketOrderType_), stockSize);
    ConstantSP prices = createVectorByValue(new Double(100.0), stockSize);
    ConstantSP qtys = createVectorByValue(new Int(2000), stockSize);
    ConstantSP directions = createVectorByValue(new Int(1), stockSize);
    ConstantSP orderIDs = createVectorByValue(new Int(1), stockSize);
    for (int i = 0; i < stockSize; ++i) {
        symbols->set(i, context.stockList->get(i));
    }
    vector<ConstantSP> limitOrderList = {symbols, times, ordertypes, prices, qtys, directions, orderIDs};
    TableSP ordertable = Util::createTable({"SecurityID", "time", "orderType", "price", "qty", "direction", "orderID"}, limitOrderList);
    saveOrdersToEngine(ordertable);
    // Update the next strategy trigger time.
    timeToTrigger_ = new Time(timeToTrigger_->getInt() + 60 * 1000);
    if (timeToTrigger_->getInt() > timeToEnd_->getInt()) {
        if (intervalIdx_ + 1 < int(timeIntervals_.size())) {
            ++intervalIdx_;
            timeToTrigger_ = timeIntervals_[intervalIdx_][0];
            timeToEnd_ = timeIntervals_[intervalIdx_][1];
        }
        else {
            // End of strategy.
            strategyFlag_ = false;
        }
    }
}

This test adopts a multi-threaded approach, where market data is distributed across threads based on the modulo of stock symbols. The overall test consists of two parts: market data replay and simulated order matching. We first use the replayDS and replay methods to read and replay market data from a DFS table, and then pass the replayed data to the algorithmic trading plugin.

1. Market Data Replay

• First, set the exchange and obtain the stock symbols to trade. When market is set to sz, symbols stores the stock codes of all securities in the SZSE trade table. When market is set to sh, symbols stores the stock codes of all securities in the SSE trade table.

trade_dfs = loadTable("dfs://TSDB_tradeAndentrust", "trade")
market = "sz"  // market = "sz" or "sh"
if (market == "sz") {
    market_name = "XSHE"
    symbols = exec SecurityID from trade_dfs where SecurityID like "0%" or SecurityID like "3%" group by SecurityID
}
else {
    market_name = "XSHG"
    symbols = exec SecurityID from trade_dfs where SecurityID like "6%" group by SecurityID
}

• Apply modulo partitioning to the stock codes to distribute the data across different threads.

def genSymbolListWithHash(symbolTotal, thread_num) {
    symbol_list = []
    for (i in 0:thread_num) {
        symbol_list.append!([]$STRING)
    }
    for (symb in symbolTotal) {
        idx = int(symb) % thread_num
        tmp_list = symbol_list[idx]
        tmp_list.append!(symb)
        symbol_list[idx] = tmp_list
    }
    return symbol_list
}
thread_num = 20 // Number of threads
symbol_list = genSymbolListWithHash(symbols, thread_num)

• The order and trade tables indicate tick-by-tick order and trade data, respectively. Using the replay function, these two tables are replayed into a single table, with the replayed data is sorted by the time column. In addition, when multiple records share the same timestamp, they can be sorted by specified fields. Because both tables are DFS tables, the replayDS and replay methods must be used together.

entrust_dfs = loadTable("dfs://TSDB_tradeAndentrust", "entrust")
trade_dfs = loadTable("dfs://TSDB_tradeAndentrust", "trade")
def replayBySymbol(market, marketName, symbolList, entrust, trade, i) {
    if (market == "sz") {
        ds1 = replayDS(sqlObj=<select SecurityID as symbol, marketName as symbolSource, TradeTime, 0 as sourceType, 
            iif(OrderType in ["50"], 2, iif(OrderType in ["49"], 1, 3)) as orderType, Price as price, OrderQty as qty, int(ApplSeqNum) as buyNo, int(ApplSeqNum) as sellNo,
            int(string(char(string(side)))) as BSFlag, int(SeqNo) as seqNum from entrust
            where Market = market and date(TradeTime)==2022.04.14 and SecurityID in symbolList>, dateColumn = "TradeTime", timeColumn = "TradeTime")
        ds2 = replayDS(sqlObj=<select SecurityID as symbol, marketName as symbolSource, TradeTime, 1 as sourceType, 
            iif(BidApplSeqNum==0|| OfferApplSeqNum==0,1,0) as orderType, TradePrice as price, int(tradeQty as qty), int(BidApplSeqNum) as buyNo, int(OfferApplSeqNum) as sellNo,
            0 as BSFlag, int(ApplSeqNum) as seqNum from trade
            where Market = market and date(TradeTime)==2022.04.14 and SecurityID in symbolList>, dateColumn = "TradeTime", timeColumn = "TradeTime")
    }
    else {
        ds1 = replayDS(sqlObj=<select SecurityID as symbol, marketName as symbolSource, TradeTime, 0 as sourceType, 
            iif(OrderType == "A", 2, 10) as orderType, Price as price, OrderQty as qty, int(ApplSeqNum) as buyNo, int(ApplSeqNum) as sellNo,
            iif(Side == "B", 1, 2) as BSFlag, int(SeqNo) as seqNum from entrust
            where Market = market and date(TradeTime)==2022.04.14 and SecurityID in symbolList>, dateColumn = "TradeTime", timeColumn = "TradeTime")
        ds2 = replayDS(sqlObj=<select SecurityID as symbol, marketName as symbolSource, TradeTime, 1 as sourceType, 
            0 as orderType, TradePrice as price, int(tradeQty as qty), int(BidApplSeqNum) as buyNo, int(OfferApplSeqNum) as sellNo,
            0 as BSFlag, int(TradeIndex) as seqNum from trade
            where Market = market and date(TradeTime)==2022.04.14 and SecurityID in symbolList>, dateColumn = "TradeTime", timeColumn = "TradeTime")
    }
    inputDict  = dict(["entrust", "trade"], [ds1, ds2])
    colName = `msgTime`msgType`msgBody`sourceType`seqNum
    colType =  [TIMESTAMP, SYMBOL, BLOB, INT, INT]
    messageTemp = table(100:0, colName, colType)
    share(messageTemp, "MatchEngineTest" + i)
 // For SZSE data, records with the same timestamp must be ordered with tick-by-tick orders first, followed by tick-by-tick trades."
 // The `sourceType` of tick-by-tick order records is `0`, and the `sourceType` of tick-by-tick trade records is `1`, so they can be sorted by the `sourceType` field.
    if (market == "sz") {
        replay(inputDict, "MatchEngineTest" + i,`TradeTime,`TradeTime,,,1,`sourceType`seqNum)
    }
 // For SSE data, records must be ordered with tick-by-tick trades first, followed by tick-by-tick orders.
    // For SSE data, `seqNum` is already strictly ordered, so the records can be sorted directly by `seqNum` here.
    else {
        replay(inputDict, "MatchEngineTest" + i,`TradeTime,`TradeTime,,,1,`seqNum)
    }
}
// Calculate the total time required for market data replay.
timer {
    job_list = [] // `job_list` stores the IDs of all submitted jobs.
    for (i in 1..thread_num) {
        job_list.append!(submitJob("TestJob" + i, "", replayBySymbol, market, market_name, symbol_list[i-1], entrust_dfs, trade_dfs, i))
    }
    // The `true` argument to `getJobReturn` indicates that the current thread blocks until the job completes and then returns.
    for (i in 0: thread_num) {
        getJobReturn(job_list[i], true)
    }
}
// The replay results can be stored as disk tables for subsequent testing.
db = database("<path>/" + market + "_messages_" + thread_num + "_part")
for (i in 1..thread_num) {
     saveTable(db, objByName("MatchEngineTest" + i), "MatchEngineTest" + i)
}

• After replaying the market data by stock code, we obtain thread_num shared in-memory tables. Their names are MatchEngineTest followed by a number. For convenience in subsequent testing, you can choose to save the replay results. Because CSV files do not support BLOB data, you can store these tables as disk tables.

1. Order Matching Simulation

• First, load the order matching simulator plugin and the algorithmic trading plugin.

loadPlugin("<path>/PluginMatchingEngineSimulator.txt")
loadPlugin("<path>/MatchEngineTest.txt")

• Prepare the data and configure the parameters. The number of threads here must match the number used for market data replay. Note that the algorithm testing plugin expects market data fields in the form `msgTime`msgType`msgBody`sourceType`seqNum . This is the result of replaying the tick-by-tick order and trade tables into a single table.

thread_num = 20
market = "sz"
market_name = "XSHE"
messagesList = [] // Store the market data table for each thread
symbolList = []  // Store the stock symbols for each thread
message_num = 0
for (i in 1..thread_num) {
    // Read the replayed market data table from the disk table and get the stock symbols in the table
    messages = select * from loadTable("<path>/" + market + "_messages_" + thread_num + "_part", "MatchEngineTest"+i)
    symbols = exec distinct left(msgBody, 6) from messages
    message_num += messages.size()
    messagesList.append!(messages)
    symbolList.append!(symbols)
}

• doMatchEngineTest indicates the task to be executed by a single thread. It mainly consists of setting the engine_config parameters to be passed in and calling the algorithm testing plugin interface. The plugin's internal execution process was described in the previous section. Note that the engine_config dictionary must be created inside the function. This is because engine_config is passed to the plugin interface by reference rather than as a copy. If engine_config is created outside the function, modifications in one thread may affect other threads during concurrent execution.

def doMatchEngineTest(messageData, symbolList, engineName, marketName, hasSnapshotOutput) {
    engine_config = dict(string, any)
    engine_config["engineName"] = engineName
    engine_config["market"] = marketName
    engine_config["hasSnapshotOutput"] = hasSnapshotOutput
    MatchEngineTest::runTestTickData(messageData, symbolList, engine_config)
}
ClearAllMatchEngine()
// Calculate the total time for order matching simulation
timer {
    test_job_list = [] // Store the IDs of all test jobs
    hasSnapshotOutput = false
    for (i in 1..thread_num) {
        messages = objByName("MatchEngineTest" + i)
        test_job_list.append!(submitJob("EngineTestJob" + i, "", doMatchEngineTest, 
                            messages, symbol_list[i-1], "MatchEngineTest" + i, 
                            market_name, hasSnapshotOutput))
    }
    // Wait for all jobs to finish before returning
    for (i in 0: thread_num) {
        getJobReturn(test_job_list[i], true)
    }
}

• After all multithreaded tasks have completed, merge each thread's algorithmic trading results (tradeOutputTable and snapshotOutputTable) and store the merged results in tradeResult and snapshotResult, respectively.

tradeResult = table(100:0, MatchEngineTest1_tradeOutputTable.schema().colDefs.name, MatchEngineTest1_tradeOutputTable.schema().colDefs.typeString)
snapshotResult = table(100:0, MatchEngineTest1_snapshotOutputTable.schema().colDefs.name, MatchEngineTest1_snapshotOutputTable.schema().colDefs.typeString)
for (i in 1..thread_num) {
    tradeResult.append!(objByName("MatchEngineTest"+i+"_tradeOutputTable"))
    snapshotResult.append!(objByName("MatchEngineTest"+i+"_snapshotOutputTable"))
}
// After merging the results into `tradeResult`, delete each thread's `tradeOutputTable` and `snapshotOutputTable`
for (i in 1..thread_num) {
    try {
        undef("MatchEngineTest" + i + "_tradeOutputTable", SHARED)
        undef("MatchEngineTest" + i + "_snapshotOutputTable", SHARED)
    } catch(ex) {print(ex)}
}

• Query the tradeResult table to check the total executed quantity for each stock.

select Symbol, sum(VolumeTraded) from tradeResult where OrderStatus !=2 and OrderStatus != 3 group by Symbol order by Symbol

The total time for market data replay includes both data loading and replay processing. The simulation time covers BLOB parsing and the computation performed by the matching engine.Both metrics are measured using timer .

The following sections show the order matching results for SZSE tick-by-tick data and SSE tick-by-tick data under different test configurations. When partitioning stock symbols, the test script uses a hash-based method: each symbol is assigned to a different thread based on the modulo result. Because the volume of market data varies by stock, execution times may differ across threads. To reduce overall test time, try to keep the amount of market data assigned to each thread roughly balanced.

The test results show that the runtime of the order matching simulation is not proportional to the number of threads. This is because simulated order matching is a compute-intensive task. When the thread count becomes too high, adding more threads introduces additional resource contention and scheduling overhead, which increases the overall runtime.