DBConnectionPool

The DolphinDB C++ API offers a connection pooling feature to streamline communication and avoid the overhead of repeatedly creating and tearing down connections. It allows you to establish a pool of connections during the initial startup phase and reuse these connections for subsequent operations. By leveraging connection pooling, you can execute scripts concurrently as well as asynchronously. This section provides a comprehensive guide on creating a connection pool, utilizing the pool to run scripts or functions, and retrieving the execution results.

Constructing a DBConnectionPool

The connection pooling feature in DolphinDB C++ API is implemented by the DBConnectionPool class. The constructor declaration for DBConnectionPool is as follows:

DBConnectionPool(const string& hostName, int port, int threadNum = 10, const string& userId = "", const string& password = "",
		         bool loadBalance = false, bool highAvailability = false, bool compress = false, bool reConnect = false, bool python = false);

Arguments

For Connection

  • host: The server address to connect to.
  • port: The port number of the server.
  • threadNum: The number of connections to create. Defaults to 10.
  • userId: The username for server login. Defaults to None.
  • password: The password for server login. Defaults to None.

Connect DBConnectionPool (with threadNum connections) to a DolphinDB server by specifying the domain/IP address and the port number. You can also log in to the server by specifying the user credentials.

For Load Balancing

  • loadBalance: Whether to enable load balancing when connecting to the DolphinDB server. Defaults to false. With loadBalance specified as true, DBConnectionPool will evenly distribute connections to all available nodes.

Note: loadBalance can only be set to true for a DolphinDB cluster.

For High Availability Mode

  • highAvailability: Whether to enable the high availability mode for the API. Defaults to false.

Note: If HA mode is enabled, all data nodes and compute nodes in the cluster are available for connection. In this case, loadBalance (if specified) does not take effect. For more information on HA mode parameter, refer to section Connecting to Server.

For Compression

  • compress: Whether to enable compression. Defaults to false.

Compressed communication can be a useful option when writing or querying large amounts of data. When compress is set to true, the data will be compressed before being transferred, which can reduce bandwidth usage. Note that it increases the computational complexity on the server and API client, which may impact performance.

For Reconnection

  • reConnect: Whether to reconnect if the API detects a connection exception when HA mode is disabled.

When HA mode is enabled, the system automatically reconnects when connection exception is detected, and it is not required to specify reConnect. Otherwise, set reConnect to true for auto reconnection.

Others

  • python: Reserved for potential future use, but as of now serves no functional purpose. Leaving it to default.

Examples

Instantiate a DBConnectionPool object with a capacity of 10 connections and establish a connection to the DolphinDB server running at 127.0.0.1:8848. The load balancing feature is disabled.

DBConnectionPool pool("127.0.0.1", 8848, 10, "admin", "123456", false);

Executing Scripts and Functions

The DBConnectionPool class provides two functions to run the scripts and functions. The function declarations are as follows:

void run(const string& script, int identity, int priority=4, int parallelism=2, int fetchSize=0, bool clearMemory = false);
void run(const string& functionName, const vector<ConstantSP>& args, int identity, int priority=4, int parallelism=2, int fetchSize=0, bool clearMemory = false);

Arguments

  • script: The script for execution.
  • functionName: The name of the function to be executed.
  • args: The arguments of the function.
  • priority: Specifies job priority. The default value is 4.
  • parallelism: Specifies the maximum number of threads to execute the tasks of a job on a data node at the same time. The default value is 2.
  • fetchSize: Specifies the size of a block (defaults to 0), which returns a BlockReader object. Use its read function to read data in blocks when querying tables of large size. Note that fetchSize cannot be smaller than 8192 (records).
  • clearMemory: Removes all variables created during that run call immediately after it completes. The default value is false. This helps prevent unnecessary variable buildup and reduces memory usage.
  • identity: The task IDs of submitted jobs. Avoid assigning the same ID to multiple tasks when executing tasks concurrently. If the same ID is used, the result of the task that completes later will overwrite the result of the task that completed earlier.

Examples

Example 1: Execute a script.

pool.run("1+1", 0);

Example 2: Execute function add.

vector<ConstantSP> args; 
ConstantSP x = Util::createVector(DT_DOUBLE, 3); 
double array_x[] = {1.5, 2.5, 7}; 
x->setDouble(0, 3, array_x);
args.push_back(x);
ConstantSP y = Util::createVector(DT_DOUBLE, 3); 
double array_y[] = {8.5, 7.5, 3}; 
y->setDouble(0, 3, array_y); 
args.push_back(y);
pool.run("add", args, 1);

Obtaining Results and Status of Jobs

The DBConnectionPool class provides two functions to get status and results of jobs. The function declarations are as follows:

bool isFinished(int identity);
ConstantSP getData(int identity);

Arguments

  • identity: The task IDs of submitted jobs.

Examples

int main(int argc, const char **argv)
{
    DBConnectionPool pool("127.0.0.1", 8848, 10, "admin", "123456", false);
    pool.run("1+1", 0);
    while(!pool.isFinished(0)){
         std::cout << "wating...\n";
    }
    std::cout << pool.getData(0)->getString() << std::endl;
}