Performance and Feature Comparison of DolphinDB Backtesting Framework and Other Backtesting Products

Backtrader is a widely used open-source Python-based quantitative trading framework, mainly used for algorithmic trading and quantitative strategy backtesting. It uses an event-driven trading architecture and is suitable for backtesting medium- and low-frequency strategies, but does not support tick-level backtesting. Backtrader's main features include support for the margin system for derivatives such as futures, about 122 built-in indicators (such as SMA, EMA, MACD, Stochastic, RSI), and integration with the ta-lib library. For indicator computation, Backtrader pre-computes all strategy indicators in a vectorized manner during data preloading, thereby enabling batch computing. The native version does not support dynamic dividends and ex-rights adjustments for the Chinese A-share market.

DolphinDB Backtest not only supports medium- and low-frequency strategy backtesting, but also supports high-precision tick-by-tick backtesting. It supports the margin system and strategy backtesting for dividends and ex-rights in the Chinese A-share market, as well as margin financing and securities lending. In addition, DolphinDB Backtest supports computation of multiple indicators, covering the ta, mytt, alpha101, and Guotai Junan 191 indicator libraries, as well as the built-in m-series, tm-series, and cum-series functions for computing complex custom indicators, delivering more comprehensive backtesting.

This section describes a performance comparison based on the implementation of a stock CTA strategy that uses MACD and RSI indicators as entry signals.

Strategy Overview

The core logic is as follows: using minute-level data, compute changes in two indicators (MACD and RSI), and set thresholds for the indicators. Once a threshold is exceeded, buy and sell operations are triggered. The specific strategy logic is detailed below.

• Open a long position: When the MACD turns from negative to positive (the previous MACD was negative and the current MACD is positive), generate the first buy signal. If the RSI rises above 30 and exits the oversold area, the market is considered bullish, and the second buy signal is generated. When both buy signals are received at the same time, submit a buy order at the current closing price.
• Open a short position: When the MACD turns from positive to negative (the previous MACD was positive and the current MACD is negative), generate the first sell signal. If the RSI falls below 70 and exits the overbought area, the market is considered bearish, and the second sell signal is generated. When both sell signals are received at the same time, obtain the position quantity to ensure there is a sufficient position available to sell, and then submit a sell order at the current closing price.

Strategy Implementation in Backtrader

Below is the main strategy implementation code for Backtrader. The complete script can be found in the appendix:

 def next(self):
    if self.order:
        return
    try:
        if not self.position:
                if (self.macd[0] > 0 and self.macd[1] < 0 
                  and self.rsi[0] > 30 and self.rsi[1] < 30 ):  
                    self.order = self.buy()
        else:
                if (self.macd[0] < 0 and self.macd[1] > 0 
                  and self.rsi[0] < 70 and self.rsi[1] > 70 ):
                    self.order = self.sell()
    except:
        pass

Strategy Implementation in DolphinDB Backtest

Below is the main strategy implementation code for DolphinDB Backtest. The complete script can be found in the appendix:

def onBar(mutable context, msg, indicator){
   //...
    for (istock in msg.keys()){
        if(macd > 0 and prevMacd < 0 and rsi > 30 and prersi < 30 ){
            ... // Variable naming and passing
            lastPrice = msg[istock]["close"]
            Backtest::submitOrder(context["engine"], (istock,context["tradeTime"] 
              , 5, lastPrice, 1000, 1),"buy")
        }
        if(macd < 0 and prevMacd > 0 and rsi < 70 and prersi > 70 ){
            ... // Variable naming and passing
            openOrders = Backtest::getOpenOrders(context["engine"], istock, , "close")
			remainQty = sum(nullFill(getOpenQty(openOrders), 0))
            if(pos-remainQty > 0){
                Backtest::submitOrder(context["engine"], (istock,context["tradeTime"]
                   , 5, lastPrice, 1000, 2), "sell")
            }
        }
    }
}

Performance Comparison Test

Below, DolphinDB Backtest's performance is tested through a comparison with Backtrader.

The test uses 175,689 records from 3 stocks from 2021.01.04 to 2021.12.31 as backtesting samples, and compares the performance of DolphinDB Backtest and that of Backtrader:

The test results show that DolphinDB Backtest has a significant speed advantage in backtesting, mainly for the following reasons:

• Backtrader is completely Python-based in most cases, whereas DolphinDB Backtest is developed entirely in C++.
• DolphinDB Backtest is implemented within DolphinDB itself, effectively reducing the time cost associated with transferring data from the database to the client.
• DolphinDB Backtest supports engines for incremental indicator computation, such as the reactive state engine.
• DolphinDB Backtest introduces JIT compilation, reducing the time overhead of code interpretation.

MetaTrader4 is a foreign exchange trading platform that provides real-time currency pair quotes, chart analysis, automated trading support, and strategy backtesting. MetaTrader4 uses MQL4 as its backtesting programming language, which is a subset of C and is a compiled language, so it executes faster than interpreted languages such as Python. MetaTrader4 supports snapshot-level strategy backtesting, but does not support high-precision order matching. Its indicator computation is based on seven fixed periods: 1 minute, 5 minutes, 15 minutes, 30 minutes, 1 hour, 4 hours, and daily. It includes 20 common indicators, such as moving average (MA), relative strength index (RSI), Bollinger Bands, and MACD, and also supports custom indicators. However, developing custom indicators in MQL4 requires strong coding skills from strategy researchers.

DolphinDB Backtest not only supports high-precision tick-by-tick backtesting but also supports high-precision order matching, with order execution results fully taking factors such as market liquidity into account. DolphinDB supports comprehensive indicator computation, with a large number of built-in m-series, tm-series, and cum-series functions, making it more convenient to implement complex indicators.

This section, based on snapshot market data, compares the performance by implementing a forex CTA trend strategy that uses Bollinger Band breakouts and RSI as entry signals.

Strategy Overview

The strategy computes indicators on an hourly basis and performs real-time risk monitoring for take-profit and stop-loss. The specific strategy logic is detailed below.

• Open a long position: the current RSI is greater than 70, the Bollinger Bands are moving upward, and the price breaks above the upper band
• Open a short position: the current RSI is less than 30, the Bollinger Bands are moving downward, and the price breaks below the lower band

If there are pending long or short orders, snapshot market data must be used to determine whether to cancel orders or trigger take-profit or stop-loss.

Strategy Implementation in MetaTrader4

Below is the main strategy implementation code for MetaTrader4. The complete script can be found in the appendix:

void OnTick(){
    //...
   double rsi1 = iRSI(NULL, 0, 11, PRICE_CLOSE, 1);
   double bandsUpper1 = iBands(NULL, 0, 20, 2, 0, PRICE_CLOSE, MODE_UPPER, 1);
   double bandsLower1 = iBands(NULL, 0, 20, 2, 0, PRICE_CLOSE, MODE_LOWER, 1);
   if(buynum ==  0 && rsi1 > 70&& Close[1] > Open[1] && Ask > bandsUpper1){   
      OrderSend(Symbol(), OP_BUY, Lots, Ask, Slippage, Ask-slPoint * Point , 0, 
      "Order Buy",  MagicNum, 0, Blue);
   }
   // RSI is less than 30, upward pattern (needs to be increased), current price breaks below the lower band
   if(sellnum == 0 && rsi1 < 30&& Open[1] > Close[1] && Bid < bandsLower1){
      OrderSend(Symbol(), OP_SELL, Lots, Bid, Slippage, Bid + slPoint * Point , 0, 
      "Order Sell", MagicNum, 0, Red);  
   }
}

Strategy Implementation in DolphinDB Backtest

Below is the main strategy implementation code for DolphinDB Backtest. The complete script can be found in the appendix:

def onBar(mutable context, msg, indicator){ // Main strategy implementation functions
	//...
	if(rsi_ > 70. and askPrice0 > bhigh and close > open){
	        if(longpos <1){
				 //...
	            orderId=Backtest::submitOrder(context["engine"], 
				(istock,symbolSource,
	            context["barTime"], 5, round(askPrice0,5), 
	            askPrice0 - context["sl"] + context['Slippage'],
				askPrice0 + context["tp"] + context['Slippage'], 
	            2, 1, context['Slippage'], 0, 
				context["barTime"]+36000000), "openBuy", 5)
	        	return
    		}
    	}
	if(rsi_ < 30. and bidPrice0 < blow and close < open){
		if(shortpos <1){
			 // ...
		    orderId=Backtest::submitOrder(context["engine"], (istock, symbolSource,
		    context["barTime"], 5, round(bidPrice0,5), 
		    bidPrice0 + context["sl"] - context['Slippage'], 
		    bidPrice0 - context["tp"] - context['Slippage'], 2, 2, 
			context['Slippage'], 0, 
		    context["barTime"] + 36000000), "openSell", 5)
			return
		}
	}
}

Performance Comparison Test:

Below, DolphinDB Backtest's performance is tested through a comparison with MetaTrader4.

The test uses 143 million records of EUR/USD forex data from 2029.01.02 to 2023.12.31, covering two years and 1,872 executed orders. The following table lists the backtest execution times for MetaTrader4 and DolphinDB Backtest:

The results show that MetaTrader4 is faster than DolphinDB Backtest for minute-level forex backtesting because MetaTrader4 uses a C-like compiled language and enables a simpler fill-at-quote execution mode during trade simulation. After enabling JIT, DolphinDB Backtest's performance approaches that of MetaTrader4, even though it uses a more time-consuming high-precision order matching simulator.

VNPY is a quantitative trading framework that provides backtesting for CTA strategies. It supports only single-instrument CTA strategy backtesting and does not support the real-time margin system for derivatives such as futures. VNPY has about 40 built-in indicators, including the simple moving average (SMA), exponential moving average (EMA), MACD, Stochastic, and RSI. Its Indicator computation supports strategy implementation at the 1-minute, 1-hour, daily, and weekly frequency, but is limited to simple single-instrument CTA strategy backtesting.

DolphinDB Backtest is a general strategy backtesting plugin that supports multi-instrument, medium- and low-frequency, and high-precision tick-by-tick strategy backtesting. It supports the margin system, strategy backtesting for dividends and ex-rights adjustments in the A-share market, as well as margin trading and securities lending. DolphinDB Backtest also supports computation of multiple indicators, covering the ta, mytt, alpha101, and Guotai Junan 191 indicator libraries, as well as the built-in m-series, tm-series, and cum-series functions for computing complex custom indicators.

The futures market is one of the important markets supported by DolphinDB. This case study continues to use a minute-level futures CTA strategy as the test case to examine the performance differences between the two in indicator computation.

Strategy Overview

A futures CTA strategy is implemented based on technical analysis indicators combining ATR (Average True Range) and RSI (Relative Strength Index). The specific strategy logic is as follows:

• Open long position: RSI is greater than 70, and ATR is greater than its 10-minute average
• Open short position: RSI is less than 30, and ATR is greater than its 10-minute average

Stop-loss logic:

• Close long position: Assume holding a long position. The operation is triggered after the OHLC bar reaches its highest point and then retraces by 0.004.
• Close short position: Assume holding a short position. The operation is triggered after the OHLC bar reaches its lowest point and then rebounds by 0.004.

Strategy Implementation in VNPY

Below is the main strategy implementation code in VNPY. The complete script can be found in the appendix:

   def on_bar(self, bar: BarData):
        """
        Callback of new bar data update.
        """
        #...
        atr_array = am.atr(self.atr_length, array=True)
        self.atr_value = atr_array[-1]
        self.atr_ma = atr_array[-self.atr_ma_length:].mean()
        self.rsi_value = am.rsi(self.rsi_length)
        if self.pos == 0:
            self.intra_trade_high = bar.high_price
            self.intra_trade_low = bar.low_price
            if self.atr_value > self.atr_ma:
                if self.rsi_value > self.rsi_buy:
                    self.buy(bar.close_price + 5, self.fixed_size)
                elif self.rsi_value < self.rsi_sell:
                    self.short(bar.close_price - 5, self.fixed_size)
        if (self.pos > 0) &(bar.high_price > self.intra_trade_high):
             self.intra_trade_high = bar.high_price
        elif (self.pos > 0)  & 
          (bar.close_price < self.intra_trade_high*(1 - self.closeLine)):
            self.sell(bar.close_price, abs(self.pos), stop = True)
        if (self.pos < 0) &(bar.low_price < self.intra_trade_low):
             self.intra_trade_low = bar.low_price
        elif (self.pos < 0)  & 
          (bar.close_price > self.intra_trade_low*(1 + self.closeLine)):
            self.cover(bar.close_price, abs(self.pos), stop=True)

        #...

Strategy Implementation in DolphinDB Backtest

Below is the main strategy implementation code in DolphinDB. The complete script can be found in the appendix:

def onBar(mutable context, msg, indicator){
	keys =msg.keys()
	for(i in keys){
		... // 
		//Buy when there is no long position and the trend is bullish
		if(longPos < 1 and shortPos < 1 and atr>matr 
			and rsi > context["buySignalRSI"] and matr > 0){
			Backtest::submitOrder(context["engine"], (istock, symbolSource,
			context["tradeTime"], 5, price + 0.02, 0., 2, 1, 0), "buyOpen")
		        	highDict[istock] = price
		        	continue
		}
		//Sell when there is no short position and the trend is bearish
		if(longPos < 1 and shortPos < 1 and atr > matr 
			and rsi < context["sellSignalRSI"] and matr > 0){
			Backtest::submitOrder(context["engine"], ( istock,
				symbolSource,context["tradeTime"], 5, price-0.02, 0., 2, 2, 0), 
				"sellOpen")
		    lowDict[istock] = price
		    continue
		}
		if(longPos > 0 and price > highDict[istock]){//Update the highest price
		        	highDict[istock] = max(price, highDict[istock])
		} 
		else if(longPos > 0  
			and price <= highDict[istock]*(1 - context["closeLine"])){
		//Close position
			Backtest::submitOrder(context["engine"], (istock, symbolSource,
				context["tradeTime"], 5, price - 0.02, 0., 2, 3, 0), "sellClose")
		}
		if(shortPos > 0 and price < lowDict[istock]){//Update the lowest price
		        	lowDict[istock] = min(price, lowDict[istock])
		} 
		else if(shortPos > 0  
			and price >= lowDict[istock]*(1 + context["closeLine"])){
		//Close position
			Backtest::submitOrder(context["engine"], (istock, symbolSource, 
				context["tradeTime"], 5, price + 0.02, 0., 2, 4, 0), "buyClose")
		}		
	}
}

Below, DolphinDB Backtest's performance is tested through a comparison with VNPY.

Performance Comparison Test

VNPY is a Python-based open-source quantitative trading framework that supports various modes such as backtesting, simulated trading, and live trading. Using VNPY as the benchmark, a comparative test between DolphinDB Backtest and VNPY is conducted.

The test uses 20,000 records of minute-level market data from the main continuous futures contract 'AG2306' over one month as the backtesting sample to compare the performance of DolphinDB Backtest and VNPY:

Even for small sample sizes, DolphinDB Backtest still has a significant performance advantage because it can call a series of DolphinDB functions and complete backtesting within the DolphinDB environment, thereby reducing the complexity of the backtesting system. As VNPY only supports backtesting for a single instrument, only one instrument was backtested here to ensure consistency in the test conditions.

Kungfu Trader is a quantitative trading platform with a core based on C++. It supports real-time trading, simulated trading, and live trading, and provides strategy backtesting for stocks, futures, foreign exchange, and other asset classes based on Python. The platform supports high-precision order matching for tick-by-tick Level-2 trades in the Chinese A-share market and supports tick-by-tick strategy backtesting. Kungfu Trader allows native Python to be used to define indicator computation for high-frequency factors.

DolphinDB Backtest supports high-precision tick-by-tick strategy backtesting for SSE and SZSE Level-2 market data, as well as multiple financial instruments, including stocks, futures, foreign exchange, and interbank bonds. It is developed entirely in C++, and the scripting language used for backtesting also supports JIT compilation to improve strategy execution efficiency.

The core logic of the order book imbalance strategy is to generate order signals by analyzing the current quantities of buy and sell orders, which places high demands on a quantitative platform's real-time order update capability. Therefore, we selected a stock order book imbalance strategy as the benchmark for performance testing to evaluate the order update and snapshot-level data processing capabilities of DolphinDB Backtest and Kungfu Trader.

Strategy Overview

This strategy is a stock trading strategy that uses market depth and changes in trading volume as entry signals. Using real-time market data, it analyzes the ratio between buy-side and sell-side trading volumes and sets corresponding thresholds. Once a threshold is exceeded, it triggers buy and sell operations. In the market snapshot data, the buy-side and sell-side trading volumes, as well as the corresponding position status, are computed simultaneously. The specific strategy logic is as follows:

• Open a long position: When the buy-side trading volume is more than twice the sell-side trading volume, and the sell-side trading volume is greater than zero, generate a buy signal. Retrieve the current position size. If the position size does not exceed the maximum value, submit a buy order at the current best ask price.
• Open a short position: When the sell-side trading volume is more than twice the buy-side trading volume, and the buy-side trading volume is greater than zero, generate a sell signal. Retrieve the current position size, ensure there is sufficient position available to sell, and then submit a sell order at the current best bid price.

Strategy Implementation in Kungfu Trader

Below is the main strategy implementation code for Kungfu Trader. The complete script can be found in the appendix:

# Snapshot data callback
def on_quote(context, quote, location, dest):
    #...
    # Strategy logic
    ask_volume_sum = sum(x for x in quote.ask_volume)
    bid_volume_sum = sum(x for x in quote.bid_volume)
    if bid_volume_sum > 2 * ask_volume_sum and ask_volume_sum > 0:
        position = get_position(quote.instrument_id, context.book) 
        if not position or position.volume < MAX_POSITION:
            order_id = context.insert_order(quote.instrument_id, 
              quote.exchange_id, source, account, quote.ask_price[0], ORDER_VOLUME,
              PriceType.Limit, Side.Buy, Offset.Open)
    if ask_volume_sum > 2 * bid_volume_sum and bid_volume_sum > 0:
        position = get_position(quote.instrument_id, context.book) 
        if position and position.volume > ORDER_VOLUME:
            order_id = context.insert_order(quote.instrument_id, 
              quote.exchange_id, source, account, quote.bid_price[0], ORDER_VOLUME,
              PriceType.Limit, Side.Sell, Offset.Close)
            
                            

Strategy Implementation in DolphinDB Backtest

Below is the main strategy implementation code for DolphinDB Backtest. The complete script can be found in the appendix:

def onSnapshot(mutable context, msg, indicator){
	maxP = context["MAX_POSITION"]
	orderV = context["ORDER_VOLUME"]
	istock = msg.symbol
	ask_volume_sum = sum(msg.offerQty)
	bid_volume_sum = sum(msg.bidQty)
	if (ask_volume_sum > 0){
	    if (bid_volume_sum > 2*ask_volume_sum){
	        posL = Backtest::getPosition(context["engine"],istock).longPosition
	        if(posL < maxP){
	            buyPrice = msg.bidPrice[1]
	            Backtest::submitOrder(context["engine"], 
					(istock,context["tradeTime"], 5, buyPrice, orderV, 1),'oimOBUY')	
	        }
	    }
	}
	if (bid_volume_sum > 0){
	    if (2*bid_volume_sum < ask_volume_sum){
	        posH = Backtest::getPosition(context["engine"],istock).shortPosition[0]
	        if(posH >= orderV){
	            sellPrice = msg.offerPrice[1]
	            Backtest::submitOrder(context["engine"],  
					(istock,context["tradeTime"], 5, sellPrice, orderV, 2),'oimOSELL')      	
	        }
	    }
	} 
}

Performance Comparison Test

The performance tests of Kungfu Trader and DolphinDB Backtest are shown below.

Using 850,000 records of snapshot market data from eight stocks over one month as test data, we submitted the test through the Kungfu Windows client to the server to evaluate the operating efficiency of Kungfu Trader (only Python APIs), and compared it with the performance of DolphinDB Backtest, obtaining the following results:

Due to the improved data transfer speed brought by DolphinDB and the support of the C++ framework, DolphinDB Backtest runs significantly faster than Python APIs of Kungfu Quant.