Stock#

The backtesting plugin supports various market data types for stock assets, including: tick or tick + snapshot, snapshot, snapshot + tick trade, minute-level frequency, daily frequency, tick (wide table), and tick + snapshot (wide table).

Engine Configuration#

Key	Description	Remarks
“start_date”	start date	DATE type, required (e.g., “2020.01.01”)
“end_date”	end date	DATE type, required (e.g., “2020.01.01”)
“asset_type”	strategy group	must be “stock”
“cash”	initial cash flow	DOUBLE type, required
“commission”	commission	DOUBLE type, default 0.0
“tax”	stamp duty	DOUBLE type, default 0.0
“data_type”	the type of market data	INT type, required, which can be: 0: tick (orders + trades) + snapshot 1: snapshot 2: snapshot + trades 3: minute-level data 4: daily data 5: tick (orders and trades merged into a wide table) 6: tick + snapshot (orders, trades and snapshot merged into a wide table)
“frequency”	aggregation frequency for converting tick/snapshot data into bars	INT type, default 0. When data_type=0 or 5, frequency > 0 is required, and ticks are aggregated into snapshots at the specified frequency to trigger `on_snapshot`. When data_type=0, 1, or 2, if frequency > 0, snapshot data are aggregated into bar data at the specified frequency to trigger `on_bar`.
“msg_as_table”	format of market data	BOOL type, default false. false: dictionary true: table (only supported when creating engine via `createBacktestEngine`)
“matching_mode”	order matching mode	INT type, options: 1: daily data : match at close; minute-level data: orders will be matched with market data whose time is later than the order time 2: daily data : match at open; minute-level data: match when data time = order time, using the close price 3: Match at order price. Note: For `data_type=3` or `4`, default is mode 1. For high-frequency data (not 3 or 4), setting this parameter to either 1 or 2 will both result in using the simulated matching engine, which matches orders according to price priority and time priority rules.
“bench_mark”	benchmark instrument	STRING or SYMBOL type. Must end with `.XSHG` (SSE) or `.XSHE` (SZSE), e.g., `"000300.XSHG"`.
“latency”	order latency (ms)	INT type, the latency from order submission to execution.
“stock_dividend”	basic information table of dividend and ex-rights	TABLE type, field descriptions are given below
“enable_algo_order”	enable algorithmic orders	true: enable false: disable.
“enable_indicator_optimize”	enable indicator calculation optimization	true: enable false: disable
“is_backtest_mode”	backtesting mode	true (default): backtesting false: matching
“data_retention_window”	data retention policy for indicator optimization	STRING/INT type. Effective only when both enable_indicator_optimize = true and is_backtest_mode = true: ”ALL” (default): Retain all data. ”None”: Retain no data. Retain data by trading days, e.g., “20d” retains 20 trading days. Retain data by row count, e.g., “20” retains the latest 20 rows per symbol.
“add_time_column_in_indicator”	add time column to indicator subscription results	true: add time column false: omit time column
“context”	strategy context structure	A dictionary consisting of global strategy variables, for example: `context = {} context[“buySignalRSI”] = 30.0 config[“context”] = context`
“callback_for_snapshot”	Snapshot callback mode	Available only for `data_type=1`. INT type, default `0`. 0: trigger `on_snapshot` only 1: trigger both `on_snapshot` and `on_bar` 2: trigger `on_bar` only. If `frequency > 0`, must trigger `on_bar` (`callback_for_snapshot = 1 or 2`).
“enable_subscription_to_tick_quotes”	enable tick data subscription	Available for `data_type=0,5,6`. true: enable false (default): disable Required for `on_tick` when data_type is 0 or 5.
“output_queue_position”	output the order’s position in the market order book. If enabled, the following five indicators will be added to the trade details and pending order interfaces: Total quantity of resting orders better than the order price in the market. Total quantity of resting orders worse than the order price in the market. Total quantity of resting orders equal to the order price in the market. Total quantity of resting orders equal to the order price and placed earlier than the user’s order. Number of price levels better than the order price.	Available for `data_type=0,5,6` and when `enable_subscription_to_tick_quotes=true`. INT type: 0: disabled 1: include latest tick in order book 2: exclude latest tick (pre-matching position).
“prev_close_price”	previous closing price table	TABLE with columns `[symbol, tradeDate, prevClose]`. Required for GEM stocks on SZSE. Available for `data_type=0,5,6`.
“orderbook_matching_ratio”	the proportion of an order that gets filled	DOUBLE type, 0–1.0, default 1.0, available for `data_type=0,1,2,5,6`.
“matching_ratio”	matching ratio within price intervals	DOUBLE type, 0–1.0, default 1.0, available for `data_type=0,1,2,5,6`.
“set_last_day_position”	set base position	TABLE type, field descriptions are given below

The parameter set_last_day_position is used to configure the base position. Detailed field definitions for set_last_day_position are shown below:

Field Name	Type	Description
symbol	SYMBOL	stock symbol
longPosition	LONG	long position quantity
costPrice	DOUBLE	average holding price
closePrice	DOUBLE	previous closing price

Note:

The configuration parameters of the engine vary depending on data_type:

The callback_for_snapshot is only set when using snapshot-based market data.
The parameters enable_subscription_to_tick_quotes, output_queue_position, and prev_close_price are only supported when data_type = 0, 5, or 6.
The parameters orderbook_matching_ratio and matching_ratio are only available when data_type = 0, 1, 2, 5, or 6.

Basic Information Table of Dividend and Ex-rights#

The stock_dividend table contains basic information on stock dividends and splits including the following fields:

Field	Description
symbol	stock symbol
endDate	fiscal year for dividend
annDate	announcement date
recordDate	record date
exDate	ex-dividend and ex-rights date
payDate	payment date
divListDate	bonus share listing date
bonusRatio	bonus share ratio per share
capitalConversion	capitalization ratio per share
afterTaxCashDiv	cash dividend per share (after tax)
allotPrice	rights issue price
allotRatio	rights issue ratio per share

Margin Trading#

Engine Configuration#

In addition to the above base configuration, users can include additional parameters specific to margin trading and securities lending strategies.

The market data schema, callback functions and stock dividend information for margin trading strategies are consistent with those used for stocks.

In such cases, the asset_type should be set to "securityCreditAccount", and only snapshot, minute-level, or daily data types are supported.

Key	Description	Remarks
“asset_type”	strategy type	must be “securityCreditAccount”
“data_type”	market data type	INT type, required, which can be 1: snapshot 3: minute-level data 4: daily data
“lineOfCredit”	credit line	DOUBLE type, required.
“margin_trading_interest_rate”	margin trading interest rate	DOUBLE type, required, e.g. 0.15
“secu_lending_interest_rate”	securities lending interest rate	DOUBLE type, required, e.g. 0.15
“maintenance_margin”	maintenance margin ratios	Required. `DOUBLE` array, e.g. `[1.45, 1.3, 1.2]` representing warning, margin call, and minimum lines respectively.
“long_concentration”	net long position concentration	DOUBLE array, e.g. `[1.0, 0.85, 0.6]`. Controls buying concentration at each margin level. The last element represents the highest line, and the first is the base line.
“short_concentration”	net short position concentration.	DOUBLE array, e.g. `[1.0, 0.85, 0.6]`. Lower concentration implies better diversification.
“output_order_info”	whether to output risk control logs in order details table	true: output false: no output
“repay_without_margin_buy”	whether securities bought via margin can offset securities sold via lending	true: allowed false: not allowed
“set_last_day_position”	set base position	TABLE type, field descriptions are given below

The parameter set_last_day_position is used to configure the base position. Detailed field definitions for set_last_day_position are shown below:

Field	Type	Description
symbol	SYMBOL	security symbol
marginSecuPosition	LONG	collateral long position quantity
marginSecuAvgPrice	DOUBLE	average buy price of collateral
marginPosition	LONG	margin trading long position quantity
marginBuyValue	DOUBLE	margin buy value
secuLendingPosition	LONG	securities lending short position quantity
secuLendingSellValue	DOUBLE	securities lending sell value
closePrice	DOUBLE	closing price
conversionRatio	DOUBLE	margin conversion ratio
tradingMargin	DOUBLE	margin trading margin ratio
lendingMargin	DOUBLE	securities lending margin ratio

Security Reference Table#

Field	Type	Description
symbol	SYMBOL	security symbol
tradeDate	DATE	trading date
symbolSource	SYMBOL	exchange
securityRating	SYMBOL	security category
marginable	BOOL	whether the security can be used as collateral true: eligible false: ineligible
conversionRatio	DOUBLE	margin conversion ratio (affects available collateral balance).
tradingMargin	DOUBLE	margin trading margin ratio (affects available balance).
lendingMargin	DOUBLE	securities lending margin ratio (affects available balance).
eligibleForMarginTrading	BOOL	whether margin trading is allowed true: allowed false: not allowed
eligibleForLending	BOOL	whether securities lending is allowed true: allowed false: not allowed

Tick-by-Tick or Tick-by-Tick + Snapshot#

Market Data Structure Description#

When inserting market data into the engine through the appendQuotationMsg interface, the msg structure is defined as follows:

message_table = sf.streaming.table(
    types={
        "msgTime": "TIMESTAMP",
        "msgType": "SYMBOL",
        "msgBody": "BLOB",
        "symbol": "STRING",
        "channelNo": "INT",
        "seqNum": "LONG",
    },
    size=0,
    capacity=10000000
)

When tick-by-tick data is included, the field msgType can take four types: “entrust”, “trade”, “snapshot”, and “END”. When snapshot data with msgType="snapshot" is not used, the system can generate snapshot data at the specified frequency by aggregating tick-level entrust and trade data. Different tick data tables with various msgType values are processed by the replay function replayDS, and the replay output produces the table messageTable required for backtesting. The following describes the structures of different tick data tables.

Tick data table structures (entrust and trade):


symbol	SYMBOL	stock symbol, ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“XSHG” (Shanghai Stock Exchange) or “XSHE” (Shenzhen Stock Exchange)
timestamp	TIMESTAMP	timestamp of the record
sourceType	INT	0 for entrust data; 1 for trade data
orderType	INT	entrust: 1 Market; 2 Limit; 3 Best Bid/Offer; 10 Cancel (SSE only, where cancel orders are recorded in entrust) trade: 0 Executed trade; 1 Cancel (SZSE only, where cancel orders are recorded in trade)
price	DOUBLE	order price
qty	LONG	order quantity
buyNo	LONG	for trade data, refers to the original entrust; for entrust, the entrust order ID is filled
sellNo	LONG	for trade data, refers to the original entrust; for entrust, the entrust order ID is filled
direction	INT	1 (Buy) or 2 (Sell)
channelNo	INT	channel number
seqNum	LONG	tick sequence number

snapshot table structure:


symbol	SYMBOL	stock symbol, ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“XSHG” (Shanghai Stock Exchange) or “XSHE” (Shenzhen Stock Exchange)
timestamp	TIMESTAMP	timestamp
lastPrice	DOUBLE	last traded price
upLimitPrice	DOUBLE	upper price limit
downLimitPrice	DOUBLE	lower price limit
totalBidQty	LONG	total buy-side traded quantity within the interval
totalOfferQty	LONG	total sell-side traded quantity within the interval
bidPrice	DOUBLE[]	list of bid prices
bidQty	LONG[]	list of bid quantities
offerPrice	DOUBLE[]	list of ask prices
offerQty	LONG[]	list of ask quantities
signal	DOUBLE[]	indicator list
seqNum	LONG	tick sequence number
prevClosePrice	DOUBLE	previous closing price

When the market data replay for backtesting ends, a message with msgType="END" should be appended. Example:

message_table = sf.sql(
    "SELECT TOP 1 * FROM messageTable",
    vars={
        'messageTable': message_table
    }
)

sf.sql(
    "UPDATE messageTable SET symbol = `END",
    vars={'messageTable': message_table}
)

backtester.append_data(message_table)

Callback Functions#

Tick-by-tick callback function onTick: input parameter msg

When msg is a dictionary, each key represents a symbol, and the corresponding value contains tick-level market data and indicator results defined in initialize. Each tick object contains the following fields:


symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“XSHG” (Shanghai Stock Exchange) or “XSHE” (Shenzhen Stock Exchange)
timestamp	TIMESTAMP	timestamp
sourceType	INT	0 for entrust data; 1 for trade data
orderType	INT	entrust: 1 Market; 2 Limit; 3 Best Bid/Offer; 10 Cancel (SSE only, where cancel orders are recorded in entrust) trade: 0 Executed trade; 1 Cancel (SZSE only, where cancel orders are recorded in trade)
price	DOUBLE	order price
qty	LONG	order quantity
buyNo	LONG	for trade data, refers to the original entrust; for entrust, the entrust order ID is filled
sellNo	LONG	for trade data, refers to the original entrust; for entrust, the entrust order ID is filled
direction	INT	1 (Buy) or 2 (Sell)
channelNo	INT	channel number
seqNum	LONG	tick sequence number

Snapshot callback function onSnapshot: input parameter msg

When msg is a dictionary, each key represents a symbol, and the corresponding value contains snapshot-level market data and indicator results defined in initialize. Each snapshot object contains the following fields:


symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	SYMBOL	“XSHG” (Shanghai Stock Exchange) or “XSHE” (Shenzhen Stock Exchange)
timestamp	TIMESTAMP	timestamp
lastPrice	DOUBLE	last traded price
upLimitPrice	DOUBLE	upper limit price
downLimitPrice	DOUBLE	lower limit price
totalBidQty	LONG	total buy-side traded volume
totalOfferQty	LONG	total sell-side traded volume
bidPrice	DOUBLE[]	list of bid prices
bidQty	LONG[]	list of bid quantities
offerPrice	DOUBLE[]	list of ask prices
offerQty	LONG[]	list of ask quantities
signal	DOUBLE[]	indicator list

Snapshot#

Market Data Structure Description#

The market data table written through append_data has the following structure:

col_types = {
    "symbol": "SYMBOL",
    "symbolSource": "SYMBOL",
    "timestamp": "TIMESTAMP",
    "lastPrice": "DOUBLE",
    "upLimitPrice": "DOUBLE",
    "downLimitPrice": "DOUBLE",
    "totalBidQty": "LONG",
    "totalOfferQty": "LONG",
    "bidPrice": "DOUBLE[]",
    "bidQty": "LONG[]",
    "offerPrice": "DOUBLE[]",
    "offerQty": "LONG[]",
    "prevClosePrice": "DOUBLE"
}

message_table = sf.table(
    types=col_types,
    size=0,
    capacity=10000
)

Note:

The symbol must include the exchange suffix (“.XSHG”, “.XSHE”), e.g., 600000.XSHG, otherwise an error will occur.
The above is snapshot market data (frequency=0, callback_for_snapshot=0), i.e., input table structure for non-bar-synthesized snapshot data.
Field names must strictly match the table below. Except for the first column which must be symbol, other column order is not restricted. Additional columns of type INT, DOUBLE, STRING, or DOUBLE ARRAY VECTOR named signal are supported as extension fields.

Required fields for snapshot market data table:

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“XSHG” (Shanghai Stock Exchange) or “XSHE” (Shenzhen Stock Exchange)
timestamp	TIMESTAMP	timestamp
lastPrice	DOUBLE	last traded price
upLimitPrice	DOUBLE	upper price limit
downLimitPrice	DOUBLE	lower price limit
totalBidQty	LONG	total buy quantity within the interval
totalOfferQty	LONG	total sell quantity within the interval
bidPrice	DOUBLE[]	list of bid prices
bidQty	LONG[]	list of bid quantities
offerPrice	DOUBLE[]	list of ask prices
offerQty	LONG[]	list of ask quantities
signal	DOUBLE[]	indicator list (required only in JIT mode)
prevClosePrice	DOUBLE	previous close price (required only in JIT mode)

Snapshot table for bar-synthesized data (frequency>0, callback_for_snapshot=1 or 2) structure:

col_types = {
    "symbol": "STRING",
    "symbolSource": "STRING",
    "timestamp": "TIMESTAMP",
    "lastPrice": "DOUBLE",
    "upLimitPrice": "DOUBLE",
    "downLimitPrice": "DOUBLE",
    "totalBidQty": "LONG",
    "totalOfferQty": "LONG",
    "bidPrice": "DOUBLE[]",
    "bidQty": "LONG[]",
    "offerPrice": "DOUBLE[]",
    "offerQty": "LONG[]",
    "prevClosePrice": "DOUBLE",
    "open": "DOUBLE",
    "close": "DOUBLE",
    "low": "DOUBLE",
    "high": "DOUBLE",
    "volume": "LONG"
}

message_table = sf.table(
    types=col_types,
    size=0,
    capacity=10_000_000
)

Note:

When generating bar data (frequency>0 and callback_for_snapshot=1 or 2), five additional fields are required: “open”, “close”, “low”, “high”, “volume”.
Field names must strictly match the table below. Except for the first column which must be symbol, other column order is not restricted. Additional columns of type INT, DOUBLE, STRING, or DOUBLE ARRAY VECTOR named signal are supported as extension fields.

Required fields for snapshot table used in bar generation:

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“XSHG” (Shanghai Stock Exchange) or “XSHE” (Shenzhen Stock Exchange)
timestamp	TIMESTAMP	timestamp
lastPrice	DOUBLE	last traded price
upLimitPrice	DOUBLE	upper price limit
downLimitPrice	DOUBLE	lower price limit
totalBidQty	LONG	total buy quantity within the interval
totalOfferQty	LONG	total sell quantity within the interval
bidPrice	DOUBLE[]	list of bid prices
bidQty	LONG[]	list of bid quantities
offerPrice	DOUBLE[]	list of ask prices
offerQty	LONG[]	list of ask quantities
signal	DOUBLE[]	indicator list (required only in JIT mode)
prevClosePrice	DOUBLE	previous close price (required only in JIT mode)
open	DOUBLE	open price of generated bar
close	DOUBLE	close price of generated bar
low	DOUBLE	lowest price of generated bar
high	DOUBLE	highest price of generated bar
volume	LONG	volume of generated bar

At the end of a backtest replay, a message with symbol “END” should be sent:

message_table = sf.sql(
    "SELECT TOP 1 * FROM messageTable",
    vars={
        'messageTable': message_table
    }
)

sf.sql(
    "UPDATE messageTable SET symbol = `END",
    vars={'messageTable': message_table}
)

backtester.append_data(message_table)

Callback Functions#

Snapshot callback function on_snapshot: input parameter msg

When msg is a dictionary, each key represents a symbol, and the corresponding value contains snapshot data for that symbol. Each snapshot object contains the following fields:

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“.XSHG” (SSE) or “.XSHE” (SZSE)
timestamp	TIMESTAMP	timestamp
lastPrice	DOUBLE	last traded price
upLimitPrice	DOUBLE	upper limit price
downLimitPrice	DOUBLE	lower limit price
totalBidQty	LONG	total buy-side traded quantity
totalOfferQty	LONG	total sell-side traded quantity
bidPrice	DOUBLE[]	list of bid prices
bidQty	LONG[]	list of bid quantities
offerPrice	DOUBLE[]	list of ask prices
offerQty	LONG[]	list of ask quantities
signal	DOUBLE[]	other
open	DOUBLE	open price of generated bar (available when data_type=1 or 2 and callback_for_snapshot=1 or 2)
close	DOUBLE	close price of generated bar (available when data_type=1 or 2 and callback_for_snapshot=1 or 2)
low	DOUBLE	lowest price of generated bar (available when data_type=1 or 2 and callback_for_snapshot=1 or 2)
high	DOUBLE	highest price of generated bar (available when data_type=1 or 2 and callback_for_snapshot=1 or 2)
volume	LONG	volume of generated bar (available when data_type=1 or 2 and callback_for_snapshot=1 or 2)

Note:

When synthesizing bar data (frequency>0 and callback_for_snapshot=1 or 2), five additional fields must be included in the input msg: “open”, “close”, “low”, “high”, “volume”.
When callback_for_snapshot=1 or 2, the on_bar function is also called, with input msg structured as follows.

Bar callback function on_bar: input parameter msg

When msg is a dictionary, each key represents a symbol, and the corresponding value contains minute-frequency K-line data. Each K-line object contains the following fields:

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
tradeTime	TIMESTAMP	trading date
open	DOUBLE	open price
low	DOUBLE	lowest price
high	DOUBLE	highest price
close	DOUBLE	close price
volume	LONG	traded volume
amount	DOUBLE	traded amount
upLimitPrice	DOUBLE	upper limit price
downLimitPrice	DOUBLE	lower limit price
prevClosePrice	DOUBLE	previous close price
signal	DOUBLE[]	other

Snapshot + Tick Trade Details#

Market Data Structure Description#

Table structure for snapshot + tick trade details (frequency=0, callback_for_snapshot=0):

message_table = sf.table(
    types={
        "symbol": "STRING",
        "symbolSource": "STRING",
        "timestamp": "TIMESTAMP",
        "lastPrice": "DOUBLE",
        "upLimitPrice": "DOUBLE",
        "downLimitPrice": "DOUBLE",
        "totalBidQty": "LONG",
        "totalOfferQty": "LONG",
        "bidPrice": "DOUBLE[]",
        "bidQty": "LONG[]",
        "offerPrice": "DOUBLE[]",
        "offerQty": "LONG[]",
        "tradePrice": "DOUBLE[]",
        "tradeQty": "LONG[]",
        "prevClosePrice": "DOUBLE"
    },
    size=0,
    capacity=10000000
)

Note:

Symbol must include the exchange suffix (“.XSHG”, “.XSHE”), e.g., 600000.XSHG, otherwise an error will occur.
The above is snapshot + tick trade details data (frequency=0, callback_for_snapshot=0), i.e., input table structure for non-bar-synthesized data.

Field description (compared with snapshot data, adds tradePrice and tradeQty):

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“XSHG” (Shanghai Stock Exchange) or “XSHE” (Shenzhen Stock Exchange)
timestamp	TIMESTAMP	timestamp
lastPrice	DOUBLE	last traded price
upLimitPrice	DOUBLE	upper price limit
downLimitPrice	DOUBLE	lower price limit
totalBidQty	LONG	total buy quantity within the interval
totalOfferQty	LONG	total sell quantity within the interval
bidPrice	DOUBLE[]	list of bid prices
bidQty	LONG[]	list of bid quantities
offerPrice	DOUBLE[]	list of ask prices
offerQty	LONG[]	list of ask quantities
tradePrice	DOUBLE[]	list of executed trade prices
tradeQty	LONG[]	list of executed trade quantities
signal	DOUBLE[]	indicator list
prevClosePrice	DOUBLE	previous close price

Table structure for snapshot + tick trade details with bar synthesis (frequency>0, callback_for_snapshot=1 or 2):

message_table = sf.table(
    types={
        "symbol": "STRING",
        "symbolSource": "STRING",
        "timestamp": "TIMESTAMP",
        "lastPrice": "DOUBLE",
        "upLimitPrice": "DOUBLE",
        "downLimitPrice": "DOUBLE",
        "totalBidQty": "LONG",
        "totalOfferQty": "LONG",
        "bidPrice": "DOUBLE[]",
        "bidQty": "LONG[]",
        "offerPrice": "DOUBLE[]",
        "offerQty": "LONG[]",
        "tradePrice": "DOUBLE[]",
        "tradeQty": "LONG[]",
        "prevClosePrice": "DOUBLE",
        "open": "DOUBLE",
        "close": "DOUBLE",
        "low": "DOUBLE",
        "high": "DOUBLE",
        "volume": "LONG"
    },
    size=0,
    capacity=10000000
)

Note:

When synthesizing bar data (frequency>0 and callback_for_snapshot=1 or 2), five additional fields are required: “open”, “close”, “low”, “high”, “volume”.

Field description (adds tradePrice and tradeQty compared with snapshot data):

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“XSHG” (Shanghai Stock Exchange) or “XSHE” (Shenzhen Stock Exchange)
timestamp	TIMESTAMP	timestamp
lastPrice	DOUBLE	last traded price
upLimitPrice	DOUBLE	upper price limit
downLimitPrice	DOUBLE	lower price limit
totalBidQty	LONG	total buy quantity within the interval
totalOfferQty	LONG	total sell quantity within the interval
bidPrice	DOUBLE[]	list of bid prices
bidQty	LONG[]	list of bid quantities
offerPrice	DOUBLE[]	list of ask prices
offerQty	LONG[]	list of ask quantities
tradePrice	DOUBLE[]	list of executed trade prices
tradeQty	LONG[]	list of executed trade quantities
signal	DOUBLE[]	indicator list
prevClosePrice	DOUBLE	previous close price
open	DOUBLE	open price of generated bar
close	DOUBLE	close price of generated bar
low	DOUBLE	lowest price of generated bar
high	DOUBLE	highest price of generated bar
volume	LONG	volume of generated bar

At the end of a backtest replay, a message with symbol “END” should be sent:

message_table = sf.sql(
    "SELECT TOP 1 * FROM messageTable",
    vars={
        'messageTable': message_table
    }
)

sf.sql(
    "UPDATE messageTable SET symbol = `END",
    vars={'messageTable': message_table}
)

backtester.append_data(message_table)

Callback Functions#

Snapshot callback function on_snapshot: input parameter msg

When msg is a dictionary, each key represents a symbol, and the corresponding value contains snapshot + tick trade details for that symbol. Each snapshot object contains the following fields:

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“.XSHG” (SSE) or “.XSHE” (SZSE)
timestamp	TIMESTAMP	timestamp
lastPrice	DOUBLE	last traded price
upLimitPrice	DOUBLE	upper limit price
downLimitPrice	DOUBLE	lower limit price
totalBidQty	LONG	total buy-side traded quantity
totalOfferQty	LONG	total sell-side traded quantity
bidPrice	DOUBLE[]	list of bid prices
bidQty	LONG[]	list of bid quantities
offerPrice	DOUBLE[]	list of ask prices
offerQty	LONG[]	list of ask quantities
tradePrice	DOUBLE[]	list of executed trade prices
tradeQty	LONG[]	list of executed trade quantities
signal	DOUBLE[]	other
open	DOUBLE	open price of generated bar (available when data_type=1 or 2 and callback_for_snapshot=1 or 2)
close	DOUBLE	close price of generated bar (available when data_type=1 or 2 and callback_for_snapshot=1 or 2)
low	DOUBLE	lowest price of generated bar (available when data_type=1 or 2 and callback_for_snapshot=1 or 2)
high	DOUBLE	highest price of generated bar (available when data_type=1 or 2 and callback_for_snapshot=1 or 2)
volume	LONG	volume of generated bar (available when data_type=1 or 2 and callback_for_snapshot=1 or 2)

Note:

When synthesizing bar data (frequency>0 and callback_for_snapshot=1 or 2), five additional fields must be included in the input msg: “open”, “close”, “low”, “high”, “volume”.
When callback_for_snapshot=1 or 2, the on_bar function is also called, with input msg structured as follows.

Bar callback function on_bar: input parameter msg

When msg is a dictionary, each key represents a symbol, and the corresponding value contains minute-frequency K-line data. Each K-line object contains the following fields:

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
tradeTime	TIMESTAMP	trading date
open	DOUBLE	open price
low	DOUBLE	lowest price
high	DOUBLE	highest price
close	DOUBLE	close price
volume	LONG	traded volume
amount	DOUBLE	traded amount
upLimitPrice	DOUBLE	upper limit price
downLimitPrice	DOUBLE	lower limit price
prevClosePrice	DOUBLE	previous close price
signal	DOUBLE[]	other

Minute-level or Daily Frequency#

Market Data Structure Description#

Table structure for minute-level or daily frequency data written via append_data:

message_table = sf.streaming.table(
    types={
        "symbol": "SYMBOL",
        "tradeTime": "TIMESTAMP",
        "open": "DOUBLE",
        "low": "DOUBLE",
        "high": "DOUBLE",
        "close": "DOUBLE",
        "volume": "LONG",
        "amount": "DOUBLE",
        "upLimitPrice": "DOUBLE",
        "downLimitPrice": "DOUBLE",
        "prevClosePrice": "DOUBLE"
    },
    size=0,
    capacity=10000000
)

Note:

Field names must strictly match the table below. the first column must be symbol; other column order is flexible. Additionally, INT, DOUBLE, STRING columns or a column named signal of type DOUBLE ARRAY VECTOR are supported as extended fields.

Required fields for minute-level or daily frequency table:

Field	Type	Description
symbol	SYMBOL	stock symbol
tradeTime	TIMESTAMP	timestamp
open	DOUBLE	open price
low	DOUBLE	lowest price
high	DOUBLE	highest price
close	DOUBLE	close price
volume	LONG	traded volume
amount	DOUBLE	traded amount (required only in JIT mode)
upLimitPrice	DOUBLE	upper limit price
downLimitPrice	DOUBLE	lower limit price
prevClosePrice	DOUBLE	previous close price (required only in JIT mode)
signal	DOUBLE[]	other fields (required only in JIT mode)

At the end of a backtest replay, a message with symbol “END” should be sent:

message_table = sf.sql(
    "SELECT TOP 1 * FROM messageTable",
    vars={
        'messageTable': message_table
    }
)

sf.sql(
    "UPDATE messageTable SET msgType = `END",
    vars={'messageTable': message_table}
)

backtester.append_data(message_table)

Callback Function#

K-line callback function on_bar: input parameter msg

When msg is a dictionary, each key represents a symbol, and the corresponding value contains minute-frequency K-line data. Each K-line object contains the following fields:

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
tradeTime	TIMESTAMP	trading date
open	DOUBLE	open price
low	DOUBLE	lowest price
high	DOUBLE	highest price
close	DOUBLE	close price
volume	LONG	traded volume
amount	DOUBLE	traded amount
upLimitPrice	DOUBLE	upper limit price
downLimitPrice	DOUBLE	lower limit price
prevClosePrice	DOUBLE	previous close price
signal	DOUBLE[]	other

Tick-by-Tick (Wide Table)#

Market Data Structure Description#

Input table structure for backtesting: append_data

message_table = sf.table(
    types={
        "symbol": "SYMBOL",
        "symbolSource": "INT",
        "timestamp": "TIMESTAMP",
        "sourceType": "INT",
        "orderType": "INT",
        "price": "DOUBLE",
        "qty": "LONG",
        "buyNo": "LONG",
        "sellNo": "LONG",
        "direction": "INT",
        "channelNo": "INT",
        "seqNum": "LONG",
        "reserve1": "DOUBLE"
    },
    size=0,
    capacity=1000
)

Tick-by-tick (wide table) data structure:

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	INT	0 (SSE) or 1 (SZSE)
timestamp	TIMESTAMP	timestamp
sourceType	INT	0 for entrust order data; 1 for trade data
orderType	INT	entrust: 1 market; 2 limit; 3 best of own; 10 cancel (SSE only, cancel recorded in entrust); trade: 0 executed; 1 cancel (SZSE only, cancel recorded in trade)
price	DOUBLE	order price
qty	LONG	order quantity
buyNo	LONG	original trade reference; for entrust, filled with order number
sellNo	LONG	original trade reference; for entrust, filled with order number
direction	INT	1 (buy) or 2 (sell)
channelNo	INT	channel number
seqNum	LONG	tick sequence number
reserve1	DOUBLE	reserved field 1 (wide table)

Compared with standard tick data, wide table adds reserve1 as a reserved field.

At the end of a backtest replay, add a message with msgType = “END”:

message_table = sf.sql(
    "SELECT TOP 1 * FROM messageTable",
    vars={
        'messageTable': message_table
    }
)

sf.sql(
    "UPDATE messageTable SET msgType = `END",
    vars={'messageTable': message_table}
)

backtester.append_data(message_table)

Callback Function#

Tick-by-tick callback function on_tick: input parameter msg

When msg is a dictionary, each key represents a symbol, and the corresponding value contains tick data and any indicators defined in initialize. Each tick object contains the following fields:

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“.XSHG” (SSE) or “.XSHE” (SZSE)
timestamp	TIMESTAMP	timestamp
sourceType	INT	0 for entrust order data; 1 for trade data
orderType	INT	entrust: 1 market; 2 limit; 3 best of own; 10 cancel (SSE only, cancel recorded in entrust); trade: 0 executed; 1 cancel (SZSE only, cancel recorded in trade)
price	DOUBLE	order price
qty	LONG	order quantity
buyNo	LONG	original trade reference; for entrust, filled with order number
sellNo	LONG	original trade reference; for entrust, filled with order number
direction	INT	1 (buy) or 2 (sell)
channelNo	INT	channel number
seqNum	LONG	tick sequence number
reserve1	DOUBLE	reserved field 1 (wide table)

Snapshot callback function on_snapshot: input parameter msg

When msg is a dictionary, each key represents a symbol, and the corresponding value contains snapshot data and any indicators defined in initialize. Each snapshot object contains the following fields:

Field	Type	Description
symbol	SYMBOL	stock symbol ending with “.XSHG” for SSE or “.XSHE” for SZSE
symbolSource	STRING	“.XSHG” (SSE) or “.XSHE” (SZSE)
timestamp	TIMESTAMP	timestamp
sourceType	INT	0 for entrust order data; 1 for trade data
orderType	INT	entrust: 1 market; 2 limit; 3 best of own; 10 cancel (SSE only, cancel recorded in entrust); trade: 0 executed; 1 cancel (SZSE only, cancel recorded in trade)
price	DOUBLE	order price
qty	LONG	order quantity
buyNo	LONG	original trade reference; for entrust, filled with order number
sellNo	LONG	original trade reference; for entrust, filled with order number
direction	INT	1 (buy) or 2 (sell)
channelNo	INT	channel number
seqNum	LONG	tick sequence number
reserve1	DOUBLE	reserved field 1 (wide table)

Tick-By-Tick + Snapshot (Wide Table)#

Market Data Structure Description#

During backtesting, the input table messageTable has the following structure: append_data

message_table = sf.table(
    types={
        "symbol": "SYMBOL",
        "symbolSource": "STRING",
        "timestamp": "TIMESTAMP",
        "sourceType": "INT",
        "orderType": "INT",
        "price": "DOUBLE",
        "qty": "LONG",
        "buyNo": "LONG",
        "sellNo": "LONG",
        "direction": "INT",
        "channelNo": "INT",
        "seqNum": "LONG",
        "lastPrice": "DOUBLE",
        "upLimitPrice": "DOUBLE",
        "downLimitPrice": "DOUBLE",
        "totalBidQty": "LONG",
        "totalOfferQty": "LONG",
        "bidPrice": "DOUBLE[]",
        "bidQty": "LONG[]",
        "offerPrice": "DOUBLE[]",
        "offerQty": "LONG[]",
        "prevClosePrice": "DOUBLE"
    },
    size=0,
    capacity=1000
)

Tick-by-tick + snapshot (wide table) market data table structure:

Field	Type	Description
symbol	SYMBOL	stock code ending with “.XSHG” for shanghai or “.XSHE” for shenzhen
symbolSource	INT	0 (shanghai) or 1 (shenzhen)
timestamp	TIMESTAMP	timestamp
sourceType	INT	0 represents entrust; 1 represents trade; 2 represents snapshot data
orderType	INT	entrust: 1 market; 2 limit; 3 best for own; 10 cancel (shanghai only, recorded in entrust) trade: 0 trade; 1 cancel (shenzhen only, recorded in trade)
price	DOUBLE	order price
qty	LONG	order quantity
buyNo	LONG	original trade data; filled with entrust order number
sellNo	LONG	original trade data; filled with entrust order number
direction	INT	1 (buy) or 2 (sell)
channelNo	INT	channel number
seqNum	LONG	tick sequence number
lastPrice	DOUBLE	latest trade price
upLimitPrice	DOUBLE	upper limit price
downLimitPrice	DOUBLE	lower limit price
totalBidQty	LONG	total bid quantity
totalOfferQty	LONG	total offer quantity
bidPrice	DOUBLE[]	bid price list
bidQty	LONG[]	bid quantity list
offerPrice	DOUBLE[]	offer price list
offerQty	LONG[]	offer quantity list
prevClosePrice	DOUBLE	previous close price
reserve1	DOUBLE	reserved field 1 (wide table)
reserve2	DOUBLE	reserved field 2 (wide table)

At the end of backtesting, send a message with symbol “END”:

message_table = sf.sql(
    "SELECT TOP 1 * FROM messageTable",
    vars={
        'messageTable': message_table
    }
)

sf.sql(
    "UPDATE messageTable SET msgType = `END",
    vars={'messageTable': message_table}
)

backtester.append_data(message_table)

Callback Function Description#

Tick market callback function on_tick: input parameter msg.

Msg is a dictionary keyed by symbol. Each value is the tick data and calculation results defined in initialize. Each tick object contains the following fields:

Field	Type	Description
symbol	SYMBOL	stock code ending with “.XSHG” for shanghai or “.XSHE” for shenzhen
symbolSource	STRING	“.XSHG” (shanghai) or “.XSHE” (shenzhen)
timestamp	TIMESTAMP	timestamp
sourceType	INT	0 represents entrust; 1 represents trade; 2 represents snapshot
orderType	INT	entrust: 1 market; 2 limit; 3 best for own; 10 cancel (shanghai only, recorded in entrust) trade: 0 trade; 1 cancel (shenzhen only, recorded in trade)
price	DOUBLE	order price
qty	LONG	order quantity
buyNo	LONG	original trade data; filled with entrust order number
sellNo	LONG	original trade data; filled with entrust order number
direction	INT	1 (buy) or 2 (sell)
channelNo	INT	channel number
seqNum	LONG	tick sequence number
lastPrice	DOUBLE	latest trade price
upLimitPrice	DOUBLE	upper limit price
downLimitPrice	DOUBLE	lower limit price
totalBidQty	LONG	total bid quantity
totalOfferQty	LONG	total offer quantity
bidPrice	DOUBLE[]	bid price list
bidQty	LONG[]	bid quantity list
offerPrice	DOUBLE[]	offer price list
offerQty	LONG[]	offer quantity list
prevClosePrice	DOUBLE	previous close price
reserve1	DOUBLE	reserved field 1 (wide table)
reserve2	DOUBLE	reserved field 2 (wide table)

Snapshot Market Callback Function Description#

Snapshot market callback function on_snapshot: input parameter msgmmsg is a dictionary keyed by symbol. Each value is the snapshot data and calculation results defined in initialize. Each snapshot object contains the following fields:

Field	Type	Description
symbol	SYMBOL	stock code ending with “.XSHG” for shanghai or “.XSHE” for shenzhen
symbolSource	STRING	“.XSHG” (shanghai) or “.XSHE” (shenzhen)
timestamp	TIMESTAMP	timestamp
sourceType	INT	0 represents entrust; 1 represents trade; 2 represents snapshot
orderType	INT	entrust: 1 market; 2 limit; 3 best for own; 10 cancel (shanghai only, recorded in entrust) trade: 0 trade; 1 cancel (shenzhen only, recorded in trade)
price	DOUBLE	order price
qty	LONG	order quantity
buyNo	LONG	original trade data; filled with entrust order number
sellNo	LONG	original trade data; filled with entrust order number
direction	INT	1 (buy) or 2 (sell)
channelNo	INT	channel number
seqNum	LONG	tick sequence number
lastPrice	DOUBLE	latest trade price
upLimitPrice	DOUBLE	upper limit price
downLimitPrice	DOUBLE	lower limit price
totalBidQty	LONG	total bid quantity
totalOfferQty	LONG	total offer quantity
bidPrice	DOUBLE[]	bid price list
bidQty	LONG[]	bid quantity list
offerPrice	DOUBLE[]	offer price list
offerQty	LONG[]	offer quantity list
prevClosePrice	DOUBLE	previous close price
reserve1	DOUBLE	reserved field 1 (wide table)
reserve2	DOUBLE	reserved field 2 (wide table)