SimpleDBConnectionPool

Python API provides the SimpleDBConnectionPool for managing and reusing connections. It provides methods for initializing and closing the pool, as well as acquiring and releasing connections.

To use this connection pool, set parameters for SimpleDBConnectionPoolConfig, and pass it as the configuration to the constructor to initialize the connection pool. Once created, users can obtain a connection with acquire, and release a connection with PoolEntry.release(). When a connection is returned to the pool, it becomes idle and can be utilized later.

Connection Setting

Note:

Starting from Python API version 3.0.4.0, all configuration classes are implemented based on pydantic.BaseModel. All interfaces that accept configuration options now allow parameters to be passed either as keyword arguments or in a dictionary.

ConnectionSetting

  • enable_ssl:bool, default False. Whether the session uses the SSL protocol for communication. To enable the SSL protocol, the configuration parameter enableHTTPS must be set to true on the server side. For details, see the DolphinDB Manual - Secure Communication.
  • enable_async:bool, default False. Whether the session enables asynchronous communication.

    The asynchronous mode allows communication with the server only through the Session.run method, with no values being returned. With asynchronous communication enabled in the Python API, you can write data to the server without waiting for a response before proceeding to the next task. However, since execution errors will not return any error information, it is not recommended to enable this.

  • compress:bool, default False. Whether the session enables compressed communication.

    Compressed communication can be a useful option when writing or querying large amounts of data. When compress is set to True, the data will be compressed before being transferred by the session, which can reduce bandwidth usage. Note that it increases the computational complexity on the server and API client, which may impact performance. If compression is enabled, protocol=PROTOCOL_DDB must be specified.

  • parser:str, default “dolphindb”. Used to specify the script parser. The value can be: "dolphindb", "python", "kdb". It is recommended to use this parameter to specify the parser type. It is supported only on DolphinDB server version 3.00.
  • protocol: protocol name, default PROTOCOL_DEFAULT (equivalent to PROTOCOL_DDB). The protocol for transferring data between the API client and the DolphinDB server.

    Currently, the protocols PROTOCOL_DDB, PROTOCOL_PICKLE, and PROTOCOL_ARROW are supported. Protocols determine the format of data returned after running DolphinDB scripts through Python API. For more information about the protocols, see Data Type Conversion.

  • show_output: bool, default True. Determines whether to print content passed to the print() function in the terminal after executing the script.
  • sql_std: optional, default DolphinDB. The SQL dialect used when running scripts. Possible values are DolphinDB, Oracle, and MySQL.

    Note:

    • Oracle and MySQL dialects are supported in DolphinDB server versions 1.30.22/2.00.10 and higher.
    • This parameter accepts string values such as "ddb", "mysql", or "oracle" (e.g., sql_std="ddb"). You can also use the SqlStd enum from dolphindb.settings (e.g., sql_std=SqlStd.DolphinDB).

ConnectionConfig

  • host: The address of the server to connect to.
  • port: The port number of the server.
  • userid: The username for server login.
  • password: The password for server login.

While creating the session, you can connect and log in to a DolphinDB server by specifying the server domain name/IP address, port number and user credentials.

  • startup: The script to execute the tasks when the connection is established. It can be used to load plugins and DFS tables, define and load stream tables, etc.
  • enable_high_availability: bool, default False. Whether to enable the high availability mode for the API.
  • high_availability_sites: optional. The IP address and port number of all available nodes, in the format of <ip>:<port>.

enable_high_availability and high_availability_sites configure the high availability (HA) of the DolphinDB Python API. In HA mode, when connecting to cluster nodes, the API will connect to the node with the lowest load. When you create multiple sessions using a single-threaded approach, the connections are established to ensure load balancing on all available nodes. However, if the sessions are created simultaneously using a multi-threaded approach, all sessions will connect to the same node at the same time. Since cluster information might not yet be synchronized, each session may query the cluster for the node with the minimum load and end up selecting the same node, which cannot guarantee load balancing across nodes.

  • keep_alive_time: int, default 60 (seconds). The duration between two keepalive transmissions to detect the TCP connection status.

    Set the parameter to release half-open TCP connections timely when the network is unstable.

  • reconnect: bool, default False. Whether to reconnect if the API detects a connection exception when high availability is disabled.
  • try_reconnect_nums: int, default None. Number of reconnection attempts for each connection. If not specified, reconnection attempts are not limited. If specified, limited attempts are applied in HA and non-HA modes. In HA mode, cycles through each node once per attempt, up to try_reconnect_nums cycles.
  • read_timeout: int, default None. The read timeout in seconds for TCP connections. If not set (None), the system will wait indefinitely. This corresponds to the SO_RCVTIMEO option for TCP connections.
  • write_timeout: int, default None. Specifies the write timeout in seconds for TCP connections. If not set (None), the system will wait indefinitely. This corresponds to the SO_SNDTIMEO option for TCP connections.

These timeout parameters allow for the configuration of read and write timeout for TCP connections. For example, if the read timeout is set to 5 seconds, an exception will be thrown when a script requiring 10 seconds to complete is executed.

SimpleDBConnectionConfig

This class inherits from ConnectionSetting and ConnectionConfig.

min_pool_size, max_pool_size
  • min_pool_size: int, default 5. Minimum number of connections in the pool, which can be used to initalize the connection pool.
  • max_pool_size: int, default 5. Maximum number of connections in the pool. The value of max_pool_size must be no smaller than that of min_pool_size.

idle_timeout

int, default 600000 (10 minutes). Maximum idle time (in ms) for connections before closure. Its minimum value is 10000 (10 seconds). Connections exceeding this time will be closed.

check_interval

int, default 60000 (1 minute). The execution interval (in ms) for the background cleanup thread. Its minimum value is 1000 (1 second).

Methods

acquire

def acquire(self, timeout: int = None) -> PoolEntry: ...

Details

Retrieves an available connection object (PoolEntry) from the connection pool. For more details, see PoolEntry.

Parameters

  • timeout: the maximum time (in ms) to wait for a connection. If this parameter is set and no connection becomes available within the specified time, an error will be raised. The default is None, which means to wait indefinitely until a connection is obtained.

Returns

An available connection object (PoolEntry).

release

def release(self, conn: PoolEntry) -> None: ...

Details

Releases a connection object to the connection pool. Alternatively, you can call PoolEntry.release() to achieve the same result.

Parameters

  • conn: A connection object of type PoolEntry.

close_idle

def close_idle(self) -> None: ...

Details

Cleans up idle connections in the connection pool until total_count (the current total number of connections) is reduced to min_pool_size. Expired connections are reclaimed first during cleanup. This operation is a manual cleanup and is not affected by the automatic reclaim policy (idle_timeout).

shutdown

def shutdown(self) -> None: ...

Details

Closes the connection pool and cleans up all associated connections.

Attributes

Name Type Read/Write Description
active_count int Read-only Number of active connections
idle_count int Read-only Number of idle connections
total_count int Read-only Total number of connections
is_shutdown bool Read-only Whether the connection pool has been manually closed

PoolEntry

A connection obtained from SimpleDBConnectionPool is wrapped as a PoolEntry object, which provides safe access and management of a single connection within the pool.

Methods

run

Executes a script or function. The usage is the same as with Session.

upload

Uploads variables. The usage is the same as with Session.

release

def release(self) -> None: ...

Releases the current connection object to the SimpleDBConnectionPool.

Note: PoolEntry supports the with statement, and the connection will be automatically returned to the pool when the with block ends. See the "Usage Example" section for details.

Attributes

Name Type Read/Write Description
is_idle bool Read-only Check whether the connection is currently idle

Usage Examples

  1. Construct a SimpleDBConnectionPool

    Method 1: Pass keyword arguments directly.

    pool = ddb.SimpleDBConnectionPool(host='localhost', port=8848, min_pool_size=2, max_pool_size=5)

    Method 2: Create a SimpleDBConnectionPoolConfig object and pass it to the constructor.

    config = ddb.SimpleDBConnectionPoolConfig(host='localhost', port=8848, min_pool_size=2, max_pool_size=5)
pool = ddb.SimpleDBConnectionPool(config=config)

    Method 3: Use a dictionary as the constructor argument.

    pool = ddb.SimpleDBConnectionPool(config={
    'host': 'localhost',
    'port': 8848,
    'min_pool_size': 2,
    'max_pool_size': 5,
})
  2. Acquire and Use a PoolEntry Connection Object

    Method 1: Use SimpleDBConnectionPool.acquire() to get a connection, and call SimpleDBConnectionPool.release(conn) to return it after use.

    conn = pool.acquire()
print(conn.run("2 + 3"))  # Output: 5
pool.release(conn)

    Method 2: Use SimpleDBConnectionPool.acquire() to get a connection, and call PoolEntry.release() to return it after use.

    conn = pool.acquire()
print(conn.run("5 * 6"))  # Output: 30
conn.release()

    Method 3: Use SimpleDBConnectionPool.acquire() to get a connection, and combine it with the with statement to automatically return the connection when the block ends.

    with pool.acquire() as conn:
    print(conn.run("10.0 / 2.0"))  # Output: 5.0

PoolEntry in a Connection Pool

The PoolEntry returned by SimpleDBConnectionPool.acquire() inherits from DBConnection. On an acquired PoolEntry, you can use exec(), call(), upload(), and block read interfaces described in "Executing Commands and Managing Connections". The method parameters are the same as those of DBConnection and use snake_case names.

The lifecycle of a PoolEntry connection is managed by the connection pool. Therefore, you should not establish, log in to, or close the connection directly outside the pool. Calling the following methods directly throws RuntimeError:

  • connect(): The connection is created and maintained internally by the connection pool.

  • login(): Login information is specified when the connection pool is created.

  • close(): A pooled connection cannot be closed directly outside the pool. After use, call release() to return the connection, or use a with statement to return it automatically when the code block exits.

After a connection is returned, it enters the idle state and cannot continue executing commands. At this point, calling run(), exec(), call(), or upload() throws RuntimeError. To continue using a connection, acquire one again through acquire().

When with pool.acquire() as conn: is used, release() is called automatically when the code block exits. If is_busy is True, returning the connection throws RuntimeError, including automatic return when exiting the with block. An unfinished block read is also considered busy. Before returning the connection, read all data or call skip_all() to discard the remaining data.

Attribute Type Description
is_idle bool Indicates whether the connection has been returned to the pool. If True, the object is idle and cannot continue executing commands.
is_busy bool Indicates whether a command is being executed or the connection is still in the block read state. If True, release() cannot be called to return the connection.

Other attributes such as is_alive, is_closed, and session_id are the same as those of DBConnection. See Connection Status and Attributes. For connection acquisition, return, pool size, and other connection pool behavior, see Methods.