input

The input plugin is developed based on libarchive and is used to decompress in-memory data or file content and return the decompressed text. The plugin can automatically recognize compression formats (currently supporting zstd and gzip) and decompress the data.

Installation

Version Requirements

DolphinDB Server: 2.00.14/3.00.2 and later, deployed on a Linux x86-64, Linux ABI system.

In the DolphinDB client, use the listRemotePlugins function to view plugins available for installation.
```
login("admin", "123456")
listRemotePlugins()
```
Use the installPlugin function to install the plugin.
```
installPlugin("input")
```
Use the loadPlugin function to load the plugin.
```
loadPlugin("input")
```

Method References

readMemory

Syntax

input::readMemory(data)

Details

Reads binary data (BLOB) from memory, automatically detects its compression format (supports zstd and gzip), and decompresses it.

Parameters

data: A scalar of type BLOB that specifies the data to be decompressed.

Returns

The decompressed text content, which is a scalar of type STRING.

readFile

Syntax

input::readFile(path)

Details

Reads a file from the specified path, automatically detects its compression format (supports zstd and gzip), and decompresses it.

Parameters

path: A scalar of type STRING that specifies the path of the file to be decompressed.

Returns

The decompressed text content, which is a scalar of type STRING.

blobToLong

Syntax

input::blobToLong(data)

Details

Converts binary data of fewer than 8 bytes into a LONG value.

Parameters

data: A scalar of type BLOB that specifies the data to be converted.

Returns

A scalar of type LONG.

setLogLevel

Syntax

input::setLogLevel(logLevel)

Details

Sets the minimum log level for the plugin. The system outputs plugin logs at the specified level and above. The default level is INFO. This function only sets the plugin log level and does not affect the log level of the DolphinDB server.

Parameters

logLevel: A string specifying the log level. The valid values, in ascending order of severity, are DEBUG, INFO, WARNING, and ERROR.

Returns

None.

Example

Set the minimum log level to WARNING. After this setting, the system outputs only plugin logs at the WARNING and ERROR levels.

input::setLogLevel(WARNING)

getLogLevel

Syntax

input::getLogLevel()

Details

Gets the current log level of the plugin.

Parameters

None.

Returns

A string indicating the current log level.

Example

input::getLogLevel() // Output: 'INFO'

Examples

Example 1: Read a Record Table From a Binary File

Assume that an external system periodically generates a binary file in which each record sequentially contains trade time, symbol, price, and quantity, all as fixed-length fields. The file can be read in DolphinDB as follows:

// Load the plugin
loadPlugin("input")

// Read and decompress the file from the specified path
path = "./trade.bin"
content = readFile(path)

// Convert to a table
table = parseJsonTable(content)

In the preceding example, trade.bin is a compressed JSON file. readFile reads the entire file and decompresses it into data of type STRING. The data can then be transformed using DolphinDB string parsing and processing functions.

Example 2: Read Data From Shared Memory

In some high-frequency scenarios, an external process writes the latest data to a shared memory region. readMemory reads data directly from the memory address, reducing the latency caused by disk I/O and data copying.

// Load the plugin
loadPlugin("input")

// Receive data from the message queue
// content = ...
content = readMemory(content)

// Convert to a table
table = parseJsonTable(content)

In the preceding example, the content is compressed data received through the message queue. readMemory decompresses it into data of type STRING. The data can then be transformed using DolphinDB string parsing and processing functions.