SimpleDBConnectionPool

Python API provides the SimpleDBConnectionPool for managing and reusing connections. It provides methods for initializing and closing the pool, as well as acquiring and releasing connections.

To use this connection pool, set parameters for SimpleDBConnectionPoolConfig, and pass it as the configuration to the constructor to initialize the connection pool. Once created, users can obtain a connection with acquire, and release a connection with PoolEntry.release(). When a connection is returned to the pool, it becomes idle and can be utilized later.

Connection Setting

Note:

Starting from Python API version 3.0.4.0, all configuration classes are implemented based on pydantic.BaseModel. All interfaces that accept configuration options now allow parameters to be passed either as keyword arguments or in a dictionary.

ConnectionSetting

  • enable_ssl:bool, default False. Whether the session uses the SSL protocol for communication. To enable the SSL protocol, the configuration parameter enableHTTPS must be set to true on the server side. For details, see the DolphinDB Manual - Secure Communication.
  • enable_async:bool, default False. Whether the session enables asynchronous communication.

    The asynchronous mode allows communication with the server only through the Session.run method, with no values being returned. With asynchronous communication enabled in the Python API, you can write data to the server without waiting for a response before proceeding to the next task. However, since execution errors will not return any error information, it is not recommended to enable this.

  • compress:bool, default False. Whether the session enables compressed communication.

    Compressed communication can be a useful option when writing or querying large amounts of data. When compress is set to True, the data will be compressed before being transferred by the session, which can reduce bandwidth usage. Note that it increases the computational complexity on the server and API client, which may impact performance. If compression is enabled, protocol=PROTOCOL_DDB must be specified.

  • parser:str, default “dolphindb”. Used to specify the script parser. The value can be: "dolphindb", "python", "kdb". It is recommended to use this parameter to specify the parser type. It is supported only on DolphinDB server version 3.00.
  • protocol: protocol name, default PROTOCOL_DEFAULT (equivalent to PROTOCOL_DDB). The protocol for transferring data between the API client and the DolphinDB server.

    Currently, the protocols PROTOCOL_DDB, PROTOCOL_PICKLE, and PROTOCOL_ARROW are supported. Protocols determine the format of data returned after running DolphinDB scripts through Python API. For more information about the protocols, see Data Type Conversion.

  • show_output: bool, default True. Determines whether to print content passed to the print() function in the terminal after executing the script.
  • sql_std: optional, default DolphinDB. The SQL dialect used when running scripts. Possible values are DolphinDB, Oracle, and MySQL.

    Note:

    • Oracle and MySQL dialects are supported in DolphinDB server versions 1.30.22/2.00.10 and higher.
    • To use this parameter, import the SqlStd enumeration class from dolphindb.settings before specifying it, e.g., sql_Std=SqlStd.DolphinDB.

ConnectionConfig

  • host: The address of the server to connect to.
  • port: The port number of the server.
  • userid: The username for server login.
  • password: The password for server login.

While creating the session, you can connect and log in to a DolphinDB server by specifying the server domain name/IP address, port number and user credentials.

  • startup: The script to execute the tasks when the connection is established. It can be used to load plugins and DFS tables, define and load stream tables, etc.
  • enable_high_availability: bool, default False. Whether to enable the high availability mode for the API.
  • high_availability_sites: optional. The IP address and port number of all available nodes, in the format of <ip>:<port>.

enable_high_availability and high_availability_sites configure the high availability (HA) of the DolphinDB Python API. In HA mode, when connecting to cluster nodes, the API will connect to the node with the lowest load. When you create multiple sessions using a single-threaded approach, the connections are established to ensure load balancing on all available nodes. However, if the sessions are created simultaneously using a multi-threaded approach, all sessions will connect to the same node at the same time. Since cluster information might not yet be synchronized, each session may query the cluster for the node with the minimum load and end up selecting the same node, which cannot guarantee load balancing across nodes.

  • keep_alive_time: int, default 60 (seconds). The duration between two keepalive transmissions to detect the TCP connection status.

    Set the parameter to release half-open TCP connections timely when the network is unstable.

  • reconnect: bool, default False. Whether to reconnect if the API detects a connection exception when high availability is disabled.
  • try_reconnect_nums: int, default None. Number of reconnection attempts for each connection. If not specified, reconnection attempts are not limited. If specified, limited attempts are applied in HA and non-HA modes. In HA mode, cycles through each node once per attempt, up to try_reconnect_nums cycles.
  • read_timeout: int, default None. The read timeout in seconds for TCP connections. If not set (None), the system will wait indefinitely. This corresponds to the SO_RCVTIMEO option for TCP connections.
  • write_timeout: int, default None. Specifies the write timeout in seconds for TCP connections. If not set (None), the system will wait indefinitely. This corresponds to the SO_SNDTIMEO option for TCP connections.

These timeout parameters allow for the configuration of read and write timeout for TCP connections. For example, if the read timeout is set to 5 seconds, an exception will be thrown when a script requiring 10 seconds to complete is executed.

SimpleDBConnectionConfig

This class inherits from ConnectionSetting and ConnectionConfig.

min_pool_size, max_pool_size
  • min_pool_size: int, default 5. Minimum number of connections in the pool, which can be used to initalize the connection pool.
  • max_pool_size: int, default 5. Maximum number of connections in the pool. The value of max_pool_size must be no smaller than that of min_pool_size.

idle_timeout

int, default 600000 (10 minutes). Maximum idle time (in ms) for connections before closure. Its minimum value is 10000 (10 seconds). Connections exceeding this time will be closed.

check_interval

int, default 60000 (1 minute). The execution interval (in ms) for the background cleanup thread. Its minimum value is 1000 (1 second).

Methods

acquire

def acquire(self, timeout: int = None) -> PoolEntry: ...

Details

Retrieves an available connection object (PoolEntry) from the connection pool. For more details, see PoolEntry.

Parameters

  • timeout: the maximum time (in ms) to wait for a connection. If this parameter is set and no connection becomes available within the specified time, an error will be raised. The default is None, which means to wait indefinitely until a connection is obtained.

Returns

An available connection object (PoolEntry).

release

def release(self, conn: PoolEntry) -> None: ...

Details

Releases a connection object to the connection pool. Alternatively, you can call PoolEntry.release() to achieve the same result.

Parameters

  • conn: A connection object of type PoolEntry.

close_idle

def close_idle(self) -> None: ...

Details

Cleans up idle connections in the connection pool until total_count (the current total number of connections) is reduced to min_pool_size. Expired connections are reclaimed first during cleanup. This operation is a manual cleanup and is not affected by the automatic reclaim policy (idle_timeout).

shutdown

def shutdown(self) -> None: ...

Details

Closes the connection pool and cleans up all associated connections.

Attributes

Name Type Read/Write Description
active_count int Read-only Number of active connections
idle_count int Read-only Number of idle connections
total_count int Read-only Total number of connections
is_shutdown bool Read-only Whether the connection pool has been manually closed

PoolEntry

A connection obtained from SimpleDBConnectionPool is wrapped as a PoolEntry object, which provides safe access and management of a single connection within the pool.

Methods

run

Executes a script or function. The usage is the same as with Session.

upload

Uploads variables. The usage is the same as with Session.

release

def release(self) -> None: ...

Releases the current connection object to the SimpleDBConnectionPool.

Note: PoolEntry supports the with statement, and the connection will be automatically returned to the pool when the with block ends. See the "Usage Example" section for details.

Attributes

Name Type Read/Write Description
is_idle bool Read-only Check whether the connection is currently idle

Usage Examples

  1. Construct a SimpleDBConnectionPool

    Method 1: Pass keyword arguments directly.

    pool = ddb.SimpleDBConnectionPool(host='localhost', port=8848, min_pool_size=2, max_pool_size=5)

    Method 2: Create a SimpleDBConnectionPoolConfig object and pass it to the constructor.

    config = ddb.SimpleDBConnectionPoolConfig(host='localhost', port=8848, min_pool_size=2, max_pool_size=5)
pool = ddb.SimpleDBConnectionPool(config=config)

    Method 3: Use a dictionary as the constructor argument.

    pool = ddb.SimpleDBConnectionPool(config={
    'host': 'localhost',
    'port': 8848,
    'min_pool_size': 2,
    'max_pool_size': 5,
})
  2. Acquire and Use a PoolEntry Connection Object

    Method 1: Use SimpleDBConnectionPool.acquire() to get a connection, and call SimpleDBConnectionPool.release(conn) to return it after use.

    conn = pool.acquire()
print(conn.run("2 + 3"))  # Output: 5
pool.release(conn)

    Method 2: Use SimpleDBConnectionPool.acquire() to get a connection, and call PoolEntry.release() to return it after use.

    conn = pool.acquire()
print(conn.run("5 * 6"))  # Output: 30
conn.release()

    Method 3: Use SimpleDBConnectionPool.acquire() to get a connection, and combine it with the with statement to automatically return the connection when the block ends.

    with pool.acquire() as conn:
    print(conn.run("10.0 / 2.0"))  # Output: 5.0