3.00.3

Version: 3.00.3   Compatibility Level 2 with 3.00.2

For details, see Compatibility Changes.

Release Date: 2025-05-19

Available Downloads:

New Features

  • Added function histogram for computing the histogram of a dataset. (3.00.3.2)
  • Added function attributeTypes for retrieving all attributes and corresponding types of the class instance. (3.00.3.2)
  • Enhanced Configuration Access Security: (3.00.3.2)
    • Added configuration parameter enableConfigAccessControl to control whether permission checks are applied when accessing configuration parameters.
    • Added function getOauthClientSecret to allow administrator users to securely retrieve the value of the sensitive configuration oauthClientSecret.
    • Updated function getConfig/getConfigure to filter returned results based on the security level of each configuration item and the current user’s permissions when enableConfigAccessControl=true.
  • Added support for the HTTP protocol in Single Sign-On (SSO). (3.00.3.2)
  • Added function semiannualBegin and semiannualEnd for retrieving the first and last day of the semiannual period. (3.00.3.1)
  • Added support for three data centers across two regions architecture. (3.00.3.1)
  • Added functions getExecDir and getWorkDir to return the absolute paths of the DolphinDB executable directory and the working directory of the process. (3.00.3.1)
  • Added function createNearestJoinEngine and Orca API DStream::nearestJoinEngine to create a nearest-neighbor join engine for stream processing. (3.00.3.1)
  • Added Orca APIs: (3.00.3.1)
    • StreamGraph::setLocalConfigOnce to configure subscription settings between adjacent nodes in a stream graph.
    • StreamGraph::dropGraph to destroy the stream graph.
  • Added configuration parameters enableRawScriptLog，rawScriptLogRetentionTime，maxRawScriptLogSize to control raw script logging and set log retention and size limits. Added function getRawScriptLog to query raw script logs. (3.00.3.1)
  • Added function recursiveSplitText to recursively split the text based on separators. (3.00.3.1)
  • Added support for text vectorization. (3.00.3.1)
  • Launched ORCA, an enterprise-level real-time computing platform that enables easy development and deployment of latency-sensitive distributed stream processing tasks using declarative programming.
  • Added support for transparent data encryption (TDE).
  • Added support for data transmission encryption.
  • Enhanced login security.
  • Added support for data masking.
  • Added configuration parameter maxJoinTaskRetry to specify the maximum retry attempts for a single SQL JOIN task under memory pressure.
  • Introduced a set of new functions and configuration parameters to enhance login security, including:
    • Automatic account lockout after failed login attempts
    • Manual user lock/unlock, retrieval of user lock status
    • Customizable password complexity rules, password expiration notifications, and prevention of password reuse.
  • Added configuration parameter enableStructuredAuditLog to structured binary-format audit logs.
  • Added the functions scramClientFirst and scramClientFinal, and introduced the authMode parameter to the createUser function to support the SCRAM (Salted Challenge Response Authentication Mechanism).
  • Added function floatingRateBondDirtyPrice to calculate the dirty price per 100 face value for a floating rate bond.
  • Added function callableFixedRateBondDirtyPrice to calculate the dirty price of a fixed-rate bond with embedded options.
  • Added function convertibleFixedRateBondDirtyPrice to calculate the dirty price per 100 face value for a convertible bond.
  • Added function amortizingFixedRateBondDirtyPriceto calculate the dirty price per 100 face value for an amortizing fixed-rate bond.
  • Added function getDFSDatabasesByOwner to list databases created by the specified user.
  • Added function gmd5 and rowGmd5 to generate MD5 hashes.
  • Added function cubicHermiteSplineFit to performs cubic Hermite interpolation on the given vectors.
  • Added function rms to calculate the root mean square value of the input.
  • Added function boxcox to transform input data into an approximately normal distribution using the Box-Cox transformation method.
  • Added function arima to implement the Autoregressive Integrated Moving Average (ARIMA) model for time series analysis.
  • Added function runSQL to avoid SQL injection through dynamic parameter binding.
  • Added support for online modification of the enableNullSafeJoin configuration parameter through the setDynamicConfig function and added function getEnableNullSafeJoin to retrieve the dynamically modified value of enableNullSafeJoin.
  • Added SQL predicates notLike, notBetween, and notIn and their corresponding functions.
  • Added function createThresholdEngine to create a threshold engine for aggregate calculations triggered by cumulative value thresholds.
  • Added functions deleteMarketHoliday and listAllMarkets to manage trading calendars on the current node.
  • Added the built-in CEPMonitor class to the CEP engine and introduced new functions to retrieve monitor and listener instances.
  • Added function getLocalIOTDBStaticTable to retrive the static table of an IOTDB database stored on the local node.
  • Added functions and configuration parameters to enhance login security.
  • Added and optimized functions to support the SCRAM login protocol.
  • Added functions to facilitate multi-cluster management operations.
  • Added function kernelRidge which combines ridge regression regularization with the kernel trick to model complex nonlinear relationships in the data and outputs a regression prediction model.
  • Added support for creating connections and releasing resources manually via Linux terminal when Web connection fails. This feature is intended for emergency maintenance and is not recommended for routine operations.
  • Added configuration parametermaxQueryExecutionTime to specify the maximum execution time (in seconds) for a single query.
  • Added function pchipInterpolateFit to apply piecewise cubic Hermite interpolation to the given numeric vectors X and Y.
  • Added function setChunkLastUpdateTime to manually set the last update time of specified chunks.
  • Added function compose to create a composite function.
  • Added function temporalDeltas to calculate time differences between adjacent elements in a data structure.
  • Added function temporalDiff to calculate time differences between two data structures.
  • Added function deepCopy to create a deep copy of an object.
  • Added function bvls to perform Bounded-Variable Least Squares (BVLS) regression and return the estimated coefficients.
  • Added configuration parameter autoConversionToColumnarTuple to control whether a tuple column (of ANY type) with consistent-typed elements is automatically converted to a columnar tuple during in-memory table construction. To restore the legacy behavior, set this parameter to true.

Improvements

  • Improved performance of min/max queries on partition columns of TIMESTAMP type when partitioned by day with VALUE partitioning. (3.00.3.2)
  • Enhanced the startup process of the licenseServer client to support authentication and startup via the standby server when the primary server is unavailable. (3.00.3.2)
  • Enhanced syntax exception messages to include the line number. (3.00.3.2)
  • Enhanced the error message for DStream::sink when writing to keyed tables fails. (3.00.3.2)
  • The license function supports retrieving the module names Beluga, Backtest and MatchingEngineSimulator. (3.00.3.1)
  • Added support for NANOTIMESTAMP type in the eventTime column of dummyTable for createCryptoOrderBookEngine. (3.00.3.1)
  • Enhanced support for canceling jobs during SQL update operations. (3.00.3.1)
  • The value of configuration parameter subThrottle is now automatically adjusted to 1 if a value smaller than 1 is set. (3.00.3.1)
  • Improved the error message thrown when an alias is used for a GROUP BY field in the SELECT clause. (3.00.3.1)
  • Improved license server connection logic by adding multiple retry attempts when failing to connect to a site, and automatically switching to other available sites after repeated failures. (3.00.3.1)
  • Added subscription.sourceOffset in StreamGraph::setConfigMap to configure the offset for all stream tables created via StreamGraph::source in the stream graph. (3.00.3.1)
  • Added support for hyphens in usernames. (3.00.3.1)
  • Enhanced the functionality of the bondAccrInt, bondCashflow, bondConvexity, bondDirtyPrice, bondYield functions, and standardized their interface definitions. (3.00.3.1)
  • Added log output to the updateLicense function: when the update succeeds, it logs the new license expiration date and node limit; when it fails, it logs detailed error information. (3.00.3.1)
  • Added a parameter add for the function updateRule. (3.00.3.1)
  • Added parameter timeoutTrigger createWindowJoinEngine function to periodically check for unmatched or overdue data in the right table and trigger their computation. (3.00.3.1)
  • Optimized partition pruning to improve the performance of queries like select * from t where id in ids when ids is a table. (3.00.3.1)
  • Added support in subscribeTable for subscribing to shared stream tables in Orca stream graphs. (3.00.3.1)
  • Improved error messages for inconsistent table schemas when outputting data in Orca stream graphs. (3.00.3.1)
  • Enhanced storage-compute separation features:
    • Supported specifying multiple cache directories for a compute node.
    • The maximum memory cache size on compute nodes is now specified in GB instead of MB.
    • Supported automatic flushing of memory cache to disk after graceful shutdown.
    • Added functions to cache the specified data on compute groups, with task status monitoring supported.
    • Added function getComputeNodeCacheDetails to obtain detailed cache information on the current node.
    • Added new parameters database and table for function clearComputeNodeCache to clear the memory and disk cache of specified database/table.
    • Enabled compression of disk cache compute nodes.
  • Improved memory stability by providing functions and configuration options to prevent OOM (Out of Memory).
  • The PKEY engine's bloomfilter index now supports the following types: TEMPORAL, BINARY, and COMPLEX.
  • Added an error message when chunkGranularity="DATABASE" is specified during IOTDB engine creation.
  • Enhanced the stability of batch job execution under high-load and high-concurrency scenarios.
  • Optimized the logic for deserializing function views and scheduled job files during node startup. Previously, if a function contained a dictionary with function definitions as values, deserialization would fail, causing an unexpected shutdown. Now, it directly throws an error.
  • The getClusterDFSDatabases function now allows non-admin users to retrieve distributed databases in the cluster that they have permission to access.
  • Privileges QUERY_RESULT_MEM_LIMIT and TASK_GROUP_MEM_LIMIT can now be set for user groups.
  • The audit log now tracks all logins and logouts, permission changes, new connection creations, and configuration file updates.
  • The query logs (<HomeDir>/resource/hardware.log) now track the amount of data sent and received.
  • Enhanced certain parameters for FICC-related functions to support specifying STRING or DURATION type in addition to integers.
  • The schema function now provides detailed information, including dbUrl and tableName for DFS tables, and keyColumn for keyed tables.
  • The loadText function now supports parsing values of DECIMAL128 type.
  • The sortBy! function now supports an empty in-memory table as input.
  • The garch function now supports DECIMAL type.
  • When the aggrTopN function receives a typed empty vector as input, it now returns an empty vector instead of throwing an error.
  • The bfill/ffill function now supports filling null values in array vectors.
  • Added parameter index for the interpolate function to allow interpolation of X based on the index vector.
  • The tmTopN functions now accept trading calendars (market identifier) as DURATION scalars in their window parameters.
  • The iif function now supports tuples as inputs for the trueResult and falseResult parameters.
  • Added assembleRule parameter to higher-order functions, replacing the consistent parameter, to specify how the results of sub-tasks are merged into the final result.
  • The transpose function now supports converting nested dictionaries into a table and merging multiple dictionaries into a single table with array vector and tuple columns.
  • The ewmMean function now supports halfLife of DURATION type, along with a new parameter times to specify the timestamp of each observation.
  • Added support for offset=-3 in subscribe to subscribe from the earliest retained record in the current stream table, including both in-memory and persisted data.
  • Optimized TSDB engine performance for concurrent queries and improved the efficiency of DELETE FROM operations.
  • Added new parameter isInnerJoin to function createLookupJoinEngine to enable inner join operations.
  • Added parameter mergeLastWindow and mergeSession for function createDailyTimeSeriesEngine.
  • Added support for metric calculation on DECIMAL128 data in the reactive state engine.
  • Added parameter preCache for function enableTablePersistence to specify the number of records to load into memory from the persisted stream table at server startup.
  • The TRUE and FALSE keywords in SQL queries are now case-insensitive.
  • Parallel join operation is now supported when enableNullSafeJoin is set to false.
  • Enhanced the stability for concurrent SQL queries.
  • The UPDATE statement now supports modifying array vectors in in-memory tables and adding columns of the ANY type to in-memory tables.
  • Added support for join order optimization.
  • Added support for partition pruning when modifying data in partitioned DFS tables with the UPDATE/DELETE statement.
  • Dictionary keys can now include keywords, such as context.
  • Optimized memory cache statistics for symbol base.
  • After calling cancelJob for a replay job, subsequent code execution is now terminated with an error message.
  • Added a limit of 1024 for the number of conditions and the length of expressions in the WHERE clause. Queries exceeding this limit will now throw an error.
  • When using the LIMIT or TOP clause, the system will no longer check the number of partitions, preventing errors caused by an excessive number of partitions.
  • Optimized the query performance when using the LIMIT statement.
  • Optimized the temporalParse function behavior to return NULL instead of an out-of-range error.
  • Optimized the dynamic selection mechanism of the GROUP BY algorithm.
  • Optimized the partition pruning mechanism.
  • Improved the error messages of the tmbeta function.
  • Added support for the ANY type in stream tables and in-memory tables.
  • Improved error handling in distributed queries when comparing CHAR constants with STRING partitioning columns in the WHERE clause. The system now raises an error message.
  • Added the “updateTime” field to the backup log file.
  • Added the recording of operations on the persisted MVCC table to the log.
  • Improved the rand function by making the count parameter optional, allowing it to return a random scalar when count is not specified.
  • Added support for the dropColumns! function to delete non-partitioning columns in DFS tables.
  • Added support for BLOB type index columns in TextDB.
  • Added a new parameter chunkId to the triggerNodeReport function, allowing reporting of specified chunk information.
  • The accessType parameter in the grant function now supports MAX_PARTITION_NUM_PER_QUERY to limit the number of partitions that can be queried at one time.
  • Added support for using non-constant expressions as values in JSON objects.
  • Classes now support array vector member variables.
  • Optimized the performance of getClusterDFSDatabases and getClusterDFSTables when processing clusters with a large number of chunks (hundreds of thousands or more).
  • Added support for backup and restore of empty tables in an IOTDB database.
  • Optimized class-related features:
    • Support for declaring array vector member variables.
    • Support for keyword arguments in member functions.
    • Support for class member functions returning multiple values when used in the reactive state engine.

Issues Fixed

  • Fixed an issue when subscribing to a stream table with subscribeTable, if the function specified in the filter parameter produced an output schema inconsistent with the input stream table, the system neither outputs data nor reports an error. (3.00.3.2)
  • Fixed memory leaks caused by querying tables if a GROUP BY column in an aggregate computation was of BLOB type. This issue was introduced in version 2.00.11. (3.00.3.2)
  • Fixed server crashes where inserting an empty vector into a stream table with a timestamp column set by setStreamTableTimestamp. (3.00.3.2)
  • Fixed an issue where excessive logging after partition updates in the TSDB engine could cause log files to grow rapidly and fill up disk space. This issue was introduced in version 2.00.15. (3.00.3.2)
  • Fixed an issue where memory usage returned by getUserHardwareUsage could be incorrect (negative) due to erroneous automatic type conversion. This issue was introduced in version 2.00.15. (3.00.3.2)
  • Fixed an issue where SQL aggregation operations could occasionally cause the server to crash. (3.00.3.2)
  • Fixed an issue where submitting multiple tasks concurrently could occasionally trigger a scheduling anomaly, causing the system to become unresponsive. (3.00.3.2)
  • Fixed server crashes that could occur when an empty data source was passed to the mr function due to missing input type validation. (3.00.3.2)
  • Fixed server crashes that could occur when an empty data source was passed to the mr function due to missing input type validation. (3.00.3.2)
  • Fixed an issue where the callback function trigger behavior did not match expectations when using the at parameter in addEventListener. (3.00.3.2)
  • Fixed an issue where the system could hang when writing to TSDB Database if the number of symbols exceeded the limit. (3.00.3.2)
  • Fixed an issue where using left join in functions specified by userDefinedMetrics of createOrderBookSnapshotEngine could cause the server to crash. (3.00.3.1)
  • Fixed an issue where temporary directories were not properly deleted when rolling back uncommitted update/delete transactions. (3.00.3.1)
  • Fixed an issue where enabling enableSeqNo during client write operations could cause memory leaks. (3.00.3.1)
  • Fixed an issue where scheduled jobs and subscriptions could occasionally fail to execute under high system load. (3.00.3.1)
  • Fixed an issue where partition pruning was not correctly applied for comparison conditions using < or <= on VALUE-partitioned tables. (3.00.3.1)
  • Fixed server crashes caused by out-of-bounds row index generation when deleting records via join queries on TSDB tables with soft deletion enabled and the keepDuplicates parameter not set to ALL. (3.00.3.1)
  • Fixed an issue where deleting a catalog did not remove the Orca stream graphs it contained. (3.00.3.1)
  • Fixed an issue where closing an abnormal subscription connection on the publisher side left a lock unreleased when other active subscriptions existed, blocking new subscriptions. (3.00.3.1)
  • Fixed an issue where the results of getAuthenticatedUserTicket and generateUserTicket could contain newline characters. Both functions now always return a single-line string. (3.00.3.1)
  • Fixed an issue where the subexecutor configuration did not take effect. This issue was introduced in version 3.00.3. (3.00.3.1)
  • Fixed server crashes caused by dropping a local table (using drop table) that shares the same name with a variable shared from another session. (3.00.3.1)
  • Fixed server crashes caused by the concurrent execution of share and existsStreamTable on the same stream table. (3.00.3.1)
  • Fixed an issue where the connection to the publishing node in cross-node subscription scenarios did not use the user information from XDB for login. (3.00.3.1)
  • Fixed server crashes caused by revoking table-related permissions with objs specified as a database object. (3.00.3.1)
  • Fixed an issue where the CEP engine does not support functions such as isVoid. (3.00.3.1)
  • Fixed an issue where queries using user-defined aggregation functions combined with interval grouping returned empty results. (3.00.3.1)
  • Fixed an issue where null values in the results returned by eig().values could not be properly filled. (3.00.3.1)
  • Fixed an issue where calling weekEnd inside a for loop could unexpectedly modify the input date variable. (3.00.3.1)
  • Fixed an issue where the vectorAR function failed on certain valid inputs. The behavior is now aligned with the Python implementation. (3.00.3.1)
  • Fixed a node hang issue caused by stream table persistence blocking all worker threads. (3.00.3.1)
  • Fixed a missing reconnect issue caused by stale subscription state. (3.00.3.1)
  • Fixed an issue where fromStdJson failed to parse JSON strings containing empty objects with closing braces on a new line. (3.00.3.1)
  • Fixed an issue in the CEP engine where repeatedly adding and removing event listeners with the same condition could cause the server to crash. (3.00.3.1)
  • Fixed an issue where accelerated replay within a specified time range failed to run at the configured speed when replaying heterogeneous stream tables. (3.00.3.1)
  • Fixed an issue where tuples returned by user-defined functions were not properly marked as mutable references, causing ineffective modifications on the original object. (3.00.3.1)
  • Fixed an issue where concurrent subscribe and unsubscribe operations on the same topic caused inconsistent subscription state. (3.00.3.1)
  • Fixed an issue where adding data nodes online failed with the error: "The number of nodes [] exceeds the max number of nodes specified in the license file []". (3.00.3.1)
  • Fixed server crashes when querying a keyed or indexed table with multiple primary key columns, where the filter condition contained an empty SYMBOL vector.
  • Fixed an issue in high-availability clusters where an existing table handle could not access data from newly added partitions without obtaining a new handle (reloading table via loadTable).
  • Fixed the issue causing incorrect results in a function view after server restarts when the view included CROSS JOIN (ANSI SQL compliant) combined with other types of table joins.
  • Fixed an intermittent thread assignment issue where the system may not allocate all stream subscription tasks to threads when enabling VS Code debugging.
  • Fixedan issue where catalog information might be lost after performing checkpoint operations on ACL logs.
  • Fixed an error where SQL queries failed if they contained both a user-defined aggregate function returning multiple values and a GROUP BY clause with interval.
  • Fixed precision loss for the segmentby function when calculating data of DECIMAL128 type.
  • Fixed a data loss issue in N-to-N replay that occurred when the data being replayed included empty partitions.
  • Fixed a type parsing error that occurred when passing a partially applied user-defined function to the moving function in the reactive state engine.
  • Fixed an issue where the forceTriggerSessionEndTime parameter did not take effect when mergeSessionEnd was set to true in createDailyTimeSeriesEngine. This issue was introduced in versions 3.00.2.4.
  • Fixed an error when inserting array vectors into the cross-sectional engine if the triggeringPattern was set to "perRow".
  • Fixed a "unrecognized column name" error that occurred when executing a multi-table JOIN query, if some partitions were offline due to node failures in the cluster.
  • Fixed an issue caused by naming conflict between table names and built-in functions during table join operations.
  • Fixed an error that occurred during join operations on shared keyed tables.
  • Fixed an issue where frequent concurrent updates and queries on tables in a TSDB database could result in empty results.
  • Fixed an issue where memory usage statistics were inaccurate when a stream table contained array vector columns.
  • Fixed an issue where stream table permissions were not automatically revoked after executing operations such as undef, dropStreamTable.
  • Fixed server crashes during concurrent execution of SQL queries when multiple joins involved a subquery as one of the tables, or when the FROM clause contained a subquery.
  • Fixed an issue where calculating maximum (via max) of a STRING column (with the first row being null) by group could result in incorrect results.
  • Fixed server crashes caused by a certificate format mismatch during online updates.
  • Fixed server crashes caused by OpenSSL version mismatch between server and plugins.
  • Fixed potential server crashes when the stream dispatch engine writes data to an asynchronously persisted stream table with stacked threads.
  • Fixed server crashes caused by a data race between the network thread and the subscription thread during stream table subscription.
  • Fixed potential crashes in movingWindowData when processing large scale datasets.
  • Fixed a data race due to concurrent access to partition metadata before a node goes online or offline.
  • Fixed server crashes caused by cache allocation failure in the OLAP proxy storage engine of the storage-compute separation architecture.
  • Fixed server crashes caused by invalid parameters passed to sqlColAlias.
  • Fixed server crashes caused by OOM issues in createWindowJoinEngine when processing large scale data.
  • Fixed a parsing failure when the user-defined function was used as an operator within a module.
  • Fixed an issue where SQL queries with a WHERE clause containing an empty condition returned an empty table instead of an error.
  • Fixed an issue where agent was not automatically set for nodes added via addNode without a cluster restart.
  • Fixed incorrect prompt message shown after password expiration when enhancedSecurityVerification is enabled.
  • Fixed server crashes that occurred when queries with lead/lag analytic functions were executed in MySQL syntax mode.

Deprecated Features

  • Deprecated Python Parser.
  • Deprecated functions registerSnapshotEngine and unregisterSnapshotEngine.