StreamBroadcastEngine#

class swordfish._swordfishcpp.StreamBroadcastEngine#

The stream broadcast engine distributes the same data stream to different target tables.

Use this engine when you need to process a single stream of data in multiple ways. For example, save one copy to disk while sending another copy to a computing engine for further processing.

StreamBroadcastEngine.create returns a Builder object, and then call submit to create an Engine object to which you can ingest the data for stream processing.

engine_type: EngineType#

The type of the streaming engine.

stat: StreamBroadcastEngineStat#

Descriptive statistics related to the streaming engine.

classmethod create(name, table_schema, outputs)#

Creates a new instance of a StreamBroadcastEngine.

Parameters:

  • name (str) – The name of the engine. It can have letters, numbers and “_” and must start with a letter.

  • table_schema (Union[Table, TypeDict]) – Specifies the column names and corresponding types of the input stream. If a Table is provided, its schema must match the schema of the subscribed stream table. Whether the table contains data or not doesn’t matter.

  • outputs (List[Table]) – A list of two or more tables. The schema of each table must match table_schema.

Returns:

A builder object to configure and create the StreamBroadcastEngine.

Return type:

StreamBroadcastEngineBuilder

Examples

>>> import swordfish as sf
>>> table_schema = {"id": "LONG", "name": "STRING"}
>>> output_table1 = sf.table(types=table_schema)
>>> output_table2 = sf.table(types=table_schema)
>>> my_engine = sf.engine.StreamBroadcastEngine.create(
...     "MainStreamEngine", table_schema, [output_table1, output_table2]
... ).submit()