Backtest

We begin with the example script configuration to demonstrate how to set the corresponding backtesting engine options.

userConfig = dict(STRING, ANY) 
userConfig["startDate"] = 2023.02.01 
userConfig["endDate"] = 2023.02.28 
userConfig["strategyGroup"] = "stock" // Strategy type: stock
userConfig["cash"] = 10000000
userConfig["commission"] = 0.0005
userConfig["tax"] = 0.001
userConfig["dataType"] = 1		 // Market data type: snapshot
userConfig["msgAsTable"] = false 
userConfig["frequency"] = 0 
// Configure global strategy variables
context = dict(STRING,ANY)
context["initPrice"] = dict(SYMBOL,ANY)
context["feeRatio"] = 0.00002
userConfig["context"]= context

The strategy type and market data type should be determined based on the backtesting strategy. In this example, the strategy type is stock and the market data type is snapshot ( userConfig["strategyGroup"] = "stock" and userConfig["dataType"] = 1 ).

The global strategy variable context is a dictionary of user-defined strategy variables, which can be defined according to the requirements of the strategy. In addition, there are several basic strategy configuration options, such as trading latency, commission rate, and initial capital, which can be configured as needed. In this example, the market data messages passed to the market data callback function are dictionaries. Starting with version 3.00.2, the engine can be created using the createBacktester function, with the JIT parameter used to enable or disable JIT optimization, which can significantly improve backtesting efficiency.

Next, define the backtesting strategy based on the required logic and create the backtesting engine. The following event functions are required to create an engine for this type of stock backtesting based on snapshot data.

def initialize(mutable context){
}
def beforeTrading(mutable context){
}
def onSnapshot(mutable context,msg, indicator){
}
def onOrder(mutable context,orders){
}
def onTrade(mutable context,trades){
}
def afterTrading(mutable context){
}
def finalize(mutable context){
}

Create a backtesting engine:

callbacks = dict(STRING, ANY)
callbacks["initialize"] = initialize
callbacks["beforeTrading"] = beforeTrading
callbacks["onSnapshot"] = onSnapshot
callbacks["onOrder"] = onOrder
callbacks["onTrade"] = onTrade
callbacks["afterTrading"] = afterTrading
callbacks["finalize"] = finalize
strategyName = "Backtest_test1"
try{Backtest::dropBacktestEngine(strategyName)}catch(ex){print ex}
engine = Backtest::createBacktester(strategyName, userConfig, callbacks, )
timer Backtest::appendQuotationMsg(engine, tb)

// Enable JIT optimization
strategyName = "Backtest_test2"
try{Backtest::dropBacktestEngine(strategyName)}catch(ex){print ex}
engine2 = Backtest::createBacktester(strategyName, userConfig, callbacks, true, )
timer Backtest::appendQuotationMsg(engine2, tb)

When creating the engine as described above, the callbacks parameter is a dictionary of strategy callback functions. The corresponding callback functions can be customized based on the strategy requirements. Since the data type in this example is snapshot data, only the onSnapshot market data callback function needs to be specified: callbacks["onSnapshot"] = onSnapshot.

Starting with version 3.00.2, the createBacktester function can be used to create a backtesting engine, with JIT optimization enabled or disabled as needed. The try statement deletes the backtesting engine with the specified name to prevent errors when creating a new engine. This command does not raise an error if the specified engine does not exist.

Data Replay:

Market data fields vary slightly across asset classes and frequencies for the same asset. The data passed to the backtesting engine must conform to the required data structure; otherwise, the backtest will fail. If the existing market data does not match the field names, field order, or data types required by the backtesting engine, the data can be transformed as needed during data import before being passed to the engine. The following describes the required data structure for stock backtesting based on snapshot market data:

colName = ["symbol", "symbolSource", "timestamp", "lastPrice", "upLimitPrice",
"downLimitPrice", "bidPrice", "bidQty",
"offerPrice", "offerQty", "prevClosePrice"]
colType = ["STRING", "STRING", "TIMESTAMP", "DOUBLE", "DOUBLE", "DOUBLE", "DOUBLE[]",
"LONG[]", "DOUBLE[]", "LONG[]", "DOUBLE"]
tb=table(1:0, colName, colType)
// If the data does not meet these requirements, you can convert it as follows:
// tb = select ContractID as symbol,Market as symbolSource,concatDateTime(Date, BarTime) as timestamp,LastPrice as lastPrice,
// UPLimitPrice as upLimitPrice,downLimitPrice as downLimitPrice, //......
// from t

Run the Backtest:

Use appendQuotationMsg to insert normalized market data into the backtesting engine and run the backtest:

Backtest::appendQuotationMsg(engine, tb)

After all market data has been written, append a data record with msgType set to “END”. The backtest then terminates and calculates the final results.

Finally, the backtest results can be retrieved through the engine interface as needed.

tradeDetails = Backtest::getTradeDetails(engine)		 	// Trade details
openOrders = Backtest::getOpenOrders(engine) 	// Query the current list of unfilled (incomplete) orders
dailyPosition = Backtest::getDailyPosition(engine)	// Daily positions

The examples above show only some of the available backtest results. For more information about the interfaces, see the preceding sections.

We begin with the example script configuration to demonstrate how to set the corresponding backtesting engine options.

userConfig = dict(STRING, ANY) 
userConfig["startDate"] = 2023.02.01 
userConfig["endDate"] = 2023.02.28 
userConfig["strategyGroup"] = "stock" // Strategy type: stock
userConfig["cash"] = 10000000
userConfig["commission"] = 0.0005
userConfig["tax"] = 0.001
userConfig["dataType"] = 6		 // Market data: tick-by-tick and snapshot (wide table)
userConfig["msgAsTable"] = false 
userConfig["frequency"] = 0 
userConfig["enableSubscriptionToTickQuotes"] = true 
userConfig["outputQueuePosition"] = 1 
// Configure global strategy variables
context = dict(STRING,ANY)
context["maxBidAskSpread"] = 0.03 
context["maxVolatility_1m"] = 0.05
userConfig["context"] = context

The strategy type and market data type should be determined based on the backtesting strategy. In this example, the strategy type is stock and the market data type is tick-by-tick and snapshot data (wide table), so userConfig["strategyGroup"] = "stock" and userConfig["dataType"] = 6 .

The global strategy variable context is a dictionary of user-defined strategy variables, which can be defined according to the requirements of the strategy. The strategy also includes basic configuration options such as trading latency, commission rates, and initial capital, which you can set according to your requirements. In this example, the market data messages passed to the market data callback function are dictionaries. Starting with version 3.00.2, the engine can be created using createBacktester, with the JIT parameter used to enable or disable JIT optimization, which can significantly improve backtesting efficiency.

Next, define the backtesting strategy based on the required logic and create the backtesting engine. The following event functions are required to create an engine for stock backtesting based on tick-by-tick and snapshot (wide-table) data.

def initialize(mutable context){
}
def beforeTrading(mutable context){
}
def onTick(mutable context, msg, indicator){
}
def onSnapshot(mutable context, msg, indicator){
}
def onOrder(mutable context, orders){
}
def onTrade(mutable context, trades){
}
def afterTrading(mutable context){
}
def finalize(mutable context){
}

Create a backtesting engine:

callbacks = dict(STRING, ANY)
callbacks["initialize"] = initialize
callbacks["beforeTrading"] = beforeTrading
callbacks["onTick"] = onTick
callbacks["onSnapshot"] = onSnapshot
callbacks["onOrder"] = onOrder
callbacks["onTrade"] = onTrade
callbacks["afterTrading"] = afterTrading
callbacks["finalize"] = finalize
strategyName = "Backtest_test1"
try{Backtest::dropBacktestEngine(strategyName)}catch(ex){print ex}
engine = Backtest::createBacktester(strategyName, userConfig, callbacks, )
timer Backtest::appendQuotationMsg(engine, tb)
// Enable JIT optimization
strategyName = "Backtest_test2"
try{Backtest::dropBacktestEngine(strategyName)}catch(ex){print ex}
engine2 = Backtest::createBacktester(strategyName, userConfig, callbacks, true, )
timer Backtest::appendQuotationMsg(engine2, tb)

When creating the engine as described above, the callbacks parameter is a dictionary of strategy callback functions. The corresponding callback functions can be customized based on the strategy requirements. In this example, since the market data combines tick-by-tick and snapshot (wide-table) data, both the onTick and onSnapshot callback functions must be specified: callbacks["onSnapshot"]=onSnapshot.

Starting with version 3.00.2, the createBacktester function can be used to create a backtesting engine, with JIT optimization enabled or disabled as needed. The try statement deletes the backtesting engine with the specified name to prevent errors when creating a new engine. This command does not raise an error if the specified engine does not exist.

Data Replay:

Market data fields vary slightly across asset classes and frequencies for the same asset. The data passed to the backtesting engine must conform to the required data structure; otherwise, the backtest will fail. If the existing market data does not match the field names, field order, or data types required by the backtesting engine, the data can be transformed as needed during data import before being passed to the engine. The following schema is required for stock backtesting based on tick-by-tick and snapshot (wide-table) market data:

colName = ["symbol", "symbolSource", "timestamp", "sourceType", "orderType", "price", "qty", "buyNo",
"sellNo", "direction", "channelNo", "seqNum", "lastPrice", "upLimitPrice",
"downLimitPrice", "totalBidQty", "totalOfferQty", "bidPrice", "bidQty",
"offerPrice", "offerQty", "prevClosePrice"]
colType=["SYMBOL", "STRING", "TIMESTAMP", "INT", "INT", "DOUBLE", "LONG", "LONG",
"LONG", "INT","INT", "LONG", "DOUBLE", "DOUBLE", "DOUBLE", "LONG",
"LONG", "DOUBLE[]", "LONG[]", "DOUBLE[]", "LONG[]", "DOUBLE"]
tb = table(1:0, colName, colType)
//If the data does not meet these requirements, you can convert it as follows:
//tb = select ContractID as symbol,Market as symbolSource,concatDateTime(Date,BarTime) as timestamp,sourceType as sourceType,Price as price,
//Qty as qty,...... from t

Run the Backtest:

Use appendQuotationMsg to insert normalized market data into the backtesting engine and run the backtest:

timer Backtest::appendQuotationMsg(engine, tb)

Finally, the backtest results can be retrieved through the engine interface as needed.

tradeDetails = Backtest::getTradeDetails(engine)
openOrders = Backtest::getOpenOrders(engine)
dailyPosition = Backtest::getDailyPosition(engine) //Daily positions
enableCash = Backtest::getAvailableCash(engine) //Available funds

We begin with the example script configuration to demonstrate how to set the corresponding backtesting engine options.

userConfig = dict(STRING, ANY) 
userConfig["startDate"] = 2023.02.01 
userConfig["endDate"] = 2023.02.28 
userConfig["strategyGroup"] = "futures" // Strategy type: futures
userConfig["cash"] = 10000000
userConfig["commission"] = 0.0005
userConfig["tax"] = 0.001
userConfig["dataType"] = 3		 // Market data interval: minute
userConfig["frequency"] = 0 
context = dict(STRING,ANY) // Global strategy variable context
context["buySignalRSI"] = 70.
context["sellSignalRSI"] = 30.
context["highPrice"] = dict(STRING, ANY)
userConfig["context"] = context

The strategy type and market data type should be determined based on the backtesting strategy. In this example, the strategy type is futures and the market data type is minute-level data, so userConfig["strategyGroup"] = "futures" and userConfig["dataType"] = 3 . The global strategy variable context is a dictionary of user-defined strategy variables, which can be defined according to the requirements of the strategy. In addition, there are several basic strategy configuration options, such as trading latency, commission rate, and initial capital, which can be configured as needed. In this example, the market data messages passed to the market data callback function are dictionaries. Starting with version 3.00.2, the engine can be created using the createBacktester function, with the JIT parameter used to enable or disable JIT optimization, which can significantly improve backtesting efficiency.

Next, define the backtesting strategy based on the required logic and create the backtesting engine. The following event functions are required to create an engine for this type of backtesting (minute-level futures backtesting).

def initialize(mutable context){
}
def beforeTrading(mutable context){
}
def onBar(mutable context, msg, indicator){
}
def onOrder(mutable context, orders){
}
def onTrade(mutable context, trades){
}
def afterTrading(mutable context){
}
def finalize(mutable context){
}

Create a backtesting engine:

callbacks = dict(STRING, ANY)
callbacks["initialize"] = initialize
callbacks["beforeTrading"] = beforeTrading
callbacks["onBar"] = onBar
callbacks["onOrder"] = onOrder
callbacks["onTrade"] = onTrade
callbacks["afterTrading"] = afterTrading
callbacks["finalize"] = finalize 
strategyName = "Backtest_test1"
try{Backtest::dropBacktestEngine(strategyName)}catch(ex){print ex}
engine = Backtest::createBacktester(strategyName, userConfig, callbacks, , securityReference)

// Enable JIT optimization
strategyName = "Backtest_test2"
try{Backtest::dropBacktestEngine(strategyName)}catch(ex){print ex}
engine2 = Backtest::createBacktester(strategyName, userConfig, callbacks, true, securityReference)

When creating the engine as described above, the callbacks parameter is a dictionary of strategy callback functions. The corresponding callback functions can be customized based on the strategy requirements. Since this example uses minute-level data, you only need to specify the onBar callback function: callbacks["onBar"]=onBar. Alternatively, the createBacktester function can be used to create a backtesting engine, with the JIT parameter used to enable or disable JIT optimization.

• The try statement deletes the backtesting engine with the specified name to prevent errors when creating a new engine. This command does not raise an error if the specified engine does not exist.

• Futures backtesting requires the contract reference table securityReference. For related information, see the "Contract Reference Table Description" section on the Futures page.

securityReference = table(symbol as symbol, take(100., size(symbol)) as multiplier, take(0.2, size(symbol)) as marginRatio,
				take(0.01, size(symbol)) as tradeUnit, take(0.02,size(symbol)) as priceUnit, 
				take(0.03, size(symbol)) as priceTick, take(1.5, size(symbol)) as commission, take(1, size(symbol)) as deliveryCommissionMode)

Data Replay:

Market data fields vary slightly across asset classes and frequencies for the same asset. The data passed to the backtesting engine must conform to the required data structure; otherwise, the backtest will fail. If the existing market data does not match the field names, field order, or data types required by the backtesting engine, the data can be transformed as needed during data import before being passed to the engine. The following describes the required data structure for minute-level futures backtesting:

colName = ["symbol", "symbolSource", "tradeTime", "tradingDay", "open", "low", "high", "close",
"volume", "amount", "upLimitPrice", "downLimitPrice", "prevClosePrice",
"settlementPrice", "prevSettlementPrice"]
colType = [SYMBOL, SYMBOL, TIMESTAMP, DATE, DOUBLE, DOUBLE, DOUBLE, DOUBLE, LONG, DOUBLE,
DOUBLE, DOUBLE, DOUBLE, DOUBLE, DOUBLE]
tb = table(1:0, colName, colType)
//If the data does not meet these requirements, you can convert it as follows:
//tb = select ContractID as symbol, Market as symbolSource, concatDateTime(Date, BarTime) as tradeTime, TradeDate as tradingDay, OpenPrice as open,
//LowPrice as low, HighPrice as high, ClosePrice as close, long(volume) as volume, Value*1.0 as amount, ULimitPrice as upLimitPrice,
//LLimitPrice as downLimitPrice,...... from t

Run the Backtest:

Insert the market data that meets the requirements into the backtesting engine through the appendQuotationMsg interface, and run the backtest:

timer Backtest::appendQuotationMsg(engine, tb)

Finally, the backtest results can be retrieved through the engine interface as needed.

res1 = select * from Backtest::getTradeDetails(engine) where orderStatus in [0,1]
availCash1 = Backtest::getAvailableCash(engine)  // Available funds
dailyPos1 = Backtest::getDailyPosition(engine)  // Positions held at the end of each trading day

We begin with the example script configuration to demonstrate how to set the corresponding backtesting engine options.

userConfig = dict(STRING, ANY)
userConfig["startDate"] = 2023.02.01 
userConfig["endDate"] = 2023.02.28 
userConfig["strategyGroup"] = "cryptocurrency" // Strategy type: cryptocurrency
userConfig["frequency"] = 0
cash = dict(STRING, DOUBLE)
cash["spot"] = 100000.
cash["futures"] = 100000.
cash["option"] = 100000.
userConfig["cash"] = cash
userConfig["dataType"] = 3 // Minute-level
userConfig["matchingMode"] = 1
userConfig["msgAsTable"] = false
// Funding rate table
// userConfig["fundingRate"] = select symbol, settlementTime,decimal128(lastFundingRate,8) as lastFundingRate from CryptoFundingRate
userConfig["fundingRate"] = table(1:0, [`symbol, `settlementTime, `lastFundingRate], [STRING, TIMESTAMP, DECIMAL128(8)])
// Configure global strategy variables
context = dict(STRING, ANY)
context["initPrice"] = dict(SYMBOL, ANY)
context["feeRatio"] = 0.00002
context["N"] = dict(SYMBOL, ANY)
userConfig["context"] = context

The strategy type and market data type should be determined based on the backtesting strategy. In this example, the strategy type is cryptocurrency and the market data type is minute-level data, so userConfig["strategyGroup"] = "cryptocurrency" and userConfig["dataType"]=3 . Unlike strategies for other asset classes, cryptocurrency strategies require separate initial capital to be set for spot, futures, and options accounts. These amounts should be stored in the userConfig["cash"] dictionary. In addition, the perpetual contract funding rate table must be specified using userConfig["fundingRate"].

The global strategy variable context is a dictionary of user-defined strategy variables, which can be defined according to the requirements of the strategy. In addition, there are several basic strategy configuration options, such as trading latency, commission rate, and initial capital, which can be configured as needed. In this example, the market data messages passed to the market data callback function are dictionaries.

Next, define the backtesting strategy based on the required logic and create the backtesting engine. The following event functions are required to create an engine for this type of backtesting (minute-level cryptocurrency backtesting).

def initialize(mutable context){
}
def beforeTrading(mutable context){
}
def onBar(mutable context,msg,indicator){
}
def onOrder(mutable context,orders){
}
def onTrade(mutable context,trades){
}
def finalize(mutable context){
}

Create a backtesting engine:

callbacks = dict(STRING, ANY)
callbacks["initialize"] = initialize
callbacks["beforeTrading"] = beforeTrading
callbacks["onBar"] = onBar
callbacks["onOrder"] = onOrder
callbacks["onTrade"] = onTrade
callbacks["finalize"] = finalize
strategyName = "Backtest_test1"
try{Backtest::dropBacktestEngine(strategyName)}catch(ex){print ex}
engine = Backtest::createBacktester(strategyName, userConfig, callbacks, , securityReference, 
initialize, beforeTrading, onBar, onOrder, onTrade, finalize)
go

When creating the engine as described above, the corresponding strategy callback functions can be customized based on the strategy requirements and specified when creating the engine. Since the data type in this example is minute-level data, only the onBar market data callback function needs to be specified. In addition, the createBacktestEngine interface can be used to create the backtesting engine. For the unused onSnapshot market data callback function, a space must be provided as a placeholder for the corresponding parameter.

• The try statement deletes the backtesting engine with the specified name to prevent errors when creating a new engine. This command does not raise an error if the specified engine does not exist.

• Cryptocurrency backtesting requires you to set securityReference.

securityReference=select last(contractType)  as contractType from tb group by symbol
update securityReference set optType=1
update securityReference set strikePrice=decimal128(0, 8)
update securityReference set contractSize=decimal128(100.,8) //......

Data Replay

Market data fields vary slightly across asset classes and frequencies for the same asset. The data passed to the backtesting engine must conform to the required data structure; otherwise, the backtest will fail. If the existing market data does not match the field names, field order, or data types required by the backtesting engine, the data can be transformed as needed during data import before being passed to the engine. The following describes the schema requirements for backtesting cryptocurrency minute-level market data:

colName = [`symbol, `symbolSource, `tradeTime, `tradingDay, `open, `low, `high, `close, `volume, `amount, `upLimitPrice,
 `downLimitPrice, `prevClosePrice, `settlementPrice, `prevSettlementPrice, `contractType]
colType = [SYMBOL, SYMBOL, TIMESTAMP, DATE, DECIMAL128(8), DECIMAL128(8), DECIMAL128(8), DECIMAL128(8), DECIMAL128(8),
DECIMAL128(8), DECIMAL128(8), DECIMAL128(8), DECIMAL128(8), DECIMAL128(8), DECIMAL128(8), INT]
tb=table(1:0, colName, colType)
// If the data does not meet these requirements, you can convert it as follows:
tb = select symbol+"_"+string(contractType) as symbol,symbolSource,tradeTime,tradingDay,decimal128(open,8) as open,decimal128(low,8) as low,
decimal128(high,8) as high,decimal128(close,8) as close,decimal128(volume,8) as volume,decimal128(amount,8) as amount,
decimal128(upLimitPrice,8) as upLimitPrice,decimal128(downLimitPrice,8) as downLimitPrice,decimal128(prevClosePrice,8) as prevClosePrice,
decimal128(settlementPrice,8) as settlementPrice,decimal128(prevSettlementPrice,8) as prevSettlementPrice,contractType from Crypto1minData

Finally, the backtest results can be retrieved through the engine interface as needed.

tradeDetails_spot = Backtest::getTradeDetails(engine, "spot") // Trade details
tradeDetails_futures = Backtest::getTradeDetails(engine, "futures")
Backtest::getDailyPosition(engine, "spot") // Daily positions

The examples above show some of the available backtest results. Some cryptocurrency APIs require you to specify the account type (spot, futures, or option).