Maintenance and Visualization

DolphinDB provides different functions that allow you to monitor the CEP engine. You can retrieve information about the CEP engine and create customized read-only data views to monitor the data generated by the engine for easy consumption by external clients.

Additionally, DolphinDB's web interface offers a CEP Streaming Engine view, accessible through the Stream tab. This view provides a visual representation of the CEP engine's status, performance metrics, and output data.

Checking Status of CEP Engine

Checking Status of All CEP Engines

You can use getStreamEngineStat().CEPEngine to check the status of all CEP engines in the current session. It returns a table containing the following columns:

Column Name Description
name the name of the CEP engine
user the name of the user who creates the engine
status CEP engine status
lastErrorMessage the last error message
lastErrorTimestamp the timestamp of the last error message
useSystemTime whether useSystemTime is set to true
numOfSubEngine the number of sub-engines
queueDepth the queue depth
eventsReceived the number of receivedevents
eventsEmitted the number of events emitted to the output queue
eventsOnOutputQueue the number of events in the output queue

Checking Status of specific CEP Engine

You can use getCEPEngineStat to check the status of a specific CEP engine.

Syntax

getCEPEngineStat(engine)

Arguments

engine is the engine object returned by createCEPEngine.

Return Values

It returns a dictionary containing the following keys:

  • EngineStat: a dictionary with the info of engine status (same as the returned info as getStreamEngineStat().CEPEngine).

  • eventSchema: a table with the info of all events received.

Column Name Description
eventType the event type
eventField the field names (separated by comma) of the event type
fieldType data types (separated by comma) of eventField
fieldTypeId data type IDs of eventField
fieldFormId data form IDs of eventField (0: scalar; 1: vector; 2: pair; 3: matrix; 4: set; 5: dictionary; 6: table). Currently, only 0 and 1 will be returned.

  • subEngineStat: a table with the info of sub-engine status.

Column Name Description
subEngineName the name of the sub-engine
eventsOnInputQueue the number of events in the input queue
monitorNumber the number of monitors
listeners the number of listeners
timers the number of timers. When using addEventListener with any of the following parameters specified: at, wait, within, or exceedTime, a timer will be created.
eventsRouted the number of events routed to the front of the input queue
eventsSent the number of events sent to the end of the input queue
eventsReceived the number of receivedevents
eventsConsumed the number of matched events
lastEventTime the timestamp of the last event
lastErrorMessage the last error message
lastErrorTimestamp the timestamp of the last error message

  • dataViewEngines: a table with the info of data view engines' status.

Column Name Description
name the name of the data view engine
user the name of the user who creates the engine
status data view engine status
lastErrorMessage the last error message
lastErrorTimestamp the timestamp of the last error message
keyColumns key column(s) (separated by space) specified by parameter keyColumns.
outputTableName the output table name
useSystemTime whether useSystemTime is set to true
throttle the time interval between data writes to the outputTable. If throttle parameter is not specified, returns NULL.
numItems the row count
memoryUsed the memory (in bytes) occupied by data view engine

Visualizing with Data View

The CEP engine operates on an event-driven basis. As events are continuously fed into the engine, it generates and updates numerous intermediate variables within its internal computations. Users typically want to monitor the latest values of these variables and observe their trends over time. To facilitate this, DolphinDB provides a data view engine that enables the CEP engine to write the monitored values to the data view engine during its execution.

The data view engine is responsible for maintaining an up-to-date snapshot of each monitored variable and exporting the data to a target table, usually a stream table, which can be subscribed by other clients.

Creating a Data View Engine

The createDataViewEngine function allows you to create a data view engine that returns a keyed table with keyColumns as the key. The table maintains the latest record for each key.

Syntax

createDataViewEngine(name, outputTable, keyColumns, timeColumn, [useSystemTime=true], [throttle])

Arguments

name is a string indicating the name of the data view engine. It consists of letters, digits, or underscores (_) and must start with a letter.

outputTable is an in-memory table or a DFS table. If you want to display real-time data, or to graph trends of the data, it must be a stream table.

keyColumns is a STRING scalar or vector that specifies one or more columns in the outputTable as the key columns. For each unique value in the keyColumn, only the latest record is retained.

timeColumn is a STRING scalar which specifies the time column in the output table.

useSystemTime (optional) is a Boolean value indicating whether to use the system time as the time column for the output table.

  • If the input data does not contain a time column, useSystemTime should set to true, i.e., the time column of outputTable is the system time.

  • If the input data contain a time column, useSystemTime should set to false, i.e., the time column of outputTable uses the timestamp of each record.

throttle (optional) is of DURATION type, which specifies the time interval between data writes to the outputTable.

Obtaining Data from a Data View Engine

To get the handle of data view engine defined in a specific CEP engine, you can use getDataViewEngine.

Syntax

getDataViewEngine([CEPEngine], dataViewEngineName)

Arguments

CEPEngine (optional) is the handle of a CEP engine.

dataViewEngineName is the name of data view engine.

Examples

Example 1: With CEPEngine not specified, obtain data from data view engine "dv1" in the current CEP engine.
dvEngine= getDataViewEngine(,`dv1)
select * from dvEngine
Example 2: With CEPEngine specified, obtain data from data view engine "dv1" in CEP engine "cep1".
dvEngine= getDataViewEngine(`cep1,`dv1)
select * from dvEngine

To simply get the handle of a data view engine outside a CEP engine context, you can use the getStreamEngine function. For example,

select * from getStreamEngine(`dv1)

Inserting Data to Data View Engine

Use append!, tableInsert, insert into to insert data to a data view engine (with the handle returned by getDataViewEngine).

Updating Values of Data View

The data view engine maintains a keyed table with keyColumns (of createDataViewEngine) as the primary key. You can call updateDataViewItems to update values of a data view through keys.

Syntax

updateDataViewItems(engine, keys, valueNames, newValues)

Arguments

engine is the handle of a data view engine.

keys is a scalar, vector, or tuple, indicating the key(s) to be updated.

valueNames is a STRING scalar or vector, indicating the columns (from the outputTable of createDataViewEngine) to be updated.

newValues is a scalar, vector, or tuple, indicating the updated values.

Deleting Values from Data View

To remove values from a data view, calldeleteDataViewItems.

Syntax

deleteDataViewItems(engine, keys)

Arguments

engine is the handle of a data view engine.

keys is a scalar, vector, or tuple, indicating the key(s) to be removed.

Note:

The updateDataViewItems and deleteDataViewItems functions can be called within a CEP engine context or outside of it.

  • Within a CEP engine: The system will first attempt to locate a data view engine handle with specified engine within the current CEP engine context. If a matching engine is found, the corresponding update and delete operation will be performed. If no such engine, the system will then search for a data view engine with the same handle outside of the CEP engine context.

  • Outside a CEP engine: The system will search for and perform corresponding operation on a data view engine handle with specified engine only from the context outside of any CEP engine.