2.00.16

• Added configuration parameters and a function related to user scripts. (2.00.16.6)
  • Added enableRawScriptLog to control whether to record users' raw scripts and their execution logs.
  • Added rawScriptLogRetentionTime to specify the retention period of raw script logs.
  • Added maxRawScriptLogSize to specify the maximum size of a single raw script log file.
  • Added getRawScriptLog for querying all raw script logs.
• Added support for encrypting dos files through function encryptModule. (2.00.16.5)
• Added support for module marketplace features (2.00.16.5):
  • Added function listRemoteModules for listing matched module names in the module marketplace.
  • Added function installModule for downloading specified module files to the node module directory.
  • Added function getLoadedModules for displaying version information of loaded modules.
• Added function semiannualBegin and semiannualEnd for retrieving the first and last day of the semiannual period. (2.00.16.3)
• Added function histogram for computing the histogram of a dataset. (2.00.16.3)
• Enhanced Configuration Access Security: (2.00.16.3)
  • Added configuration parameter enableConfigAccessControl to control whether permission checks are applied when accessing configuration parameters.
  • Added function getOauthClientSecret to allow administrator users to securely retrieve the value of the sensitive configuration oauthClientSecret.
  • Updated function getConfig/getConfigure to filter returned results based on the security level of each configuration item and the current user’s permissions when enableConfigAccessControl=true.
• Added support for the HTTP protocol in Single Sign-On (SSO). (2.00.16.3)
• Added functions setIPConnectionLimit and getIPConnectionLimit to set and get the connection limit for different IP addresses on the current node (Linux only). (2.00.16.2)
• Added functions getExecDir and getWorkDir to return the absolute paths of the DolphinDB executable directory and the working directory of the process. (2.00.16.2)
• Added support for three data centers across two regions architecture. (2.00.16.1)
• Added function createNearestJoinEngine to create a nearest-neighbor join engine for stream processing. (2.00.16.1)
• Added functions deleteMarketHoliday and listAllMarkets to manage trading calendars on the current node.
• Added function getLocalIOTDBStaticTable to retrive the static table of an IOTDB database stored on the local node.
• Added functions and configuration parameters to enhance login security.
• Added and optimized functions to support the SCRAM login protocol.
• Added function kernelRidge which combines ridge regression regularization with the kernel trick to model complex nonlinear relationships in the data and outputs a regression prediction model.
• Added support for creating connections and releasing resources manually via Linux terminal when Web connection fails. This feature is intended for emergency maintenance and is not recommended for routine operations.
• Added configuration parametermaxQueryExecutionTime to specify the maximum execution time (in seconds) for a single query.
• Added function pchipInterpolateFit to apply piecewise cubic Hermite interpolation to the given numeric vectors X and Y.
• Added function setChunkLastUpdateTime to manually set the last update time of specified chunks.
• Added function temporalDeltas to calculate time differences between adjacent elements in a data structure.
• Added function temporalDiff to calculate time differences between two data structures.
• Added function deepCopy to create a deep copy of an object.
• Added function bvls to perform Bounded-Variable Least Squares (BVLS) regression and return the estimated coefficients.

• Added configuration parameter autoConversionToColumnarTuple to control whether a tuple column (of ANY type) with consistent-typed elements is automatically converted to a columnar tuple during in-memory table construction. To restore the legacy behavior, set this parameter to true.

• Optimized table join performance in high-cardinality single-column join scenarios, enhancing the execution efficiency of equi joins, left joins, and full joins. (2.00.16.7)
• Optimized the data caching mechanism for compute groups in storage-compute separation scenarios to reduce and better control their memory usage. (2.00.16.7)
• Optimized the mechanism for background threads to update disk space in storage-compute separation scenarios. (2.00.16.7)
• Optimized memory usage of compute nodes during full-table preloading in the TSDB engine under storage-compute separation scenarios. (2.00.16.7)
• Optimized system stability and enhanced capabilities related to Raft and DFS. (2.00.16.7)
• Optimized the logic of function getComputeNodeCacheWarmupJobStatus for retrieving completed historical jobs, increasing the return limit from 10 to 4096. (2.00.16.7)
• Optimized the expired chunk version reclamation mechanism. (2.00.16.7)
• Optimized the shutdown logic after license server heartbeat failures, and added license server status information. (2.00.16.7)
  • Optimized the shutdown mechanism when license server responses are not received, extending the automatic shutdown delay from 2 to 14 days.
  • Added function getLicenseServerStatus() for retrieving the license server status.
• Changed the root job ID of stream processing to a random UUID. (2.00.16.6)
• Optimized the query performance of the TSDB engine when variables are used in the WHERE clause. (2.00.16.6)
• Added support for automatically converting strings passed in through the API to DOUBLE, FLOAT or DECIMAL based on the field type, and inserting them into the table. (2.00.16.5)
• Optimized the thread allocation algorithm for subscribeTable to prioritize idle threads when hash is not specified. (2.00.16.5)
• Optimized the subWorkers table returned by getStreamingStat so that workerId is returned even when no data is consumed. (2.00.16.5)
• Added support for displaying the node alias in the return values of getAclAuditLog and getAuditLog. (2.00.16.5)
• Optimized memory usage of the persistent stream table during reload. (2.00.16.5)
• Optimized the volume selection logic for writes by adding support for weighted write allocation based on each volume’s free space percentage. (2.00.16.5)
• Added Exitcode to the return value of the socp function. (2.00.16.4)
• Optimized the error message when using functions in the metrics of the createCrossSectionalEngine function. Detailed error information is now properly returned. (2.00.16.4)
• Optimized the SSO process: (2.00.16.4)

  • Fixed HTTP request errors caused by default ports in SSO requests.

  • Enhanced the oauthUserField configuration to support parsing usernames from nested JSON fields.

• Added support for modifying members of DolphinDB objects (tuples and dictionaries) through assignment statements or calls to mutable functions (e.g., append!, clear!). (2.00.16.4)
• Optimized partition pruning to improve the performance of queries like select * from t where id in ids when ids is a table. (2.00.16.3)
• Improved performance of min/max queries on partition columns of TIMESTAMP type when partitioned by day with VALUE partitioning. (2.00.16.3)
• Added support for NANOTIMESTAMP type in the eventTime column of dummyTable for createCryptoOrderBookEngine. (2.00.16.3)
• Enhanced the startup process of the licenseServer client to support authentication and startup via the standby server when the primary server is unavailable. (2.00.16.3)
• Enhanced the error message when creating tables via createIPCInMemoryTable. (2.00.16.3)
• Enhanced syntax exception messages to include the line number. (2.00.16.3)
• The license function supports retrieving the module name Beluga. (2.00.16.2)
• Added log output to the updateLicense function: when the update succeeds, it logs the new license expiration date and node limit; when it fails, it logs detailed error information. (2.00.16.2)
• Improved license server connection logic by adding multiple retry attempts when failing to connect to a site, and automatically switching to other available sites after repeated failures. (2.00.16.1)
• Improved the error message thrown when an alias is used for a GROUP BY field in the SELECT clause. (2.00.16.1)
• Added support for hyphens in usernames. (2.00.16.1)
• Enhanced the functionality of the bondAccrInt, bondCashflow, bondConvexity, bondDirtyPrice, bondYield functions, and standardized their interface definitions. (2.00.16.1)
• Added a parameter add for the function updateRule. (2.00.16.1)
• Enhanced the createWindowJoinEngine function (2.00.16.1):
  • Added parameter timeoutTrigger to periodically check for unmatched or overdue data in the right table and trigger their computation.
  • Added parameters cachedTableCapacity and keyPurgeFreqInSec to periodically clean up unnecessary data and optimize memory usage.
• Improved memory stability by providing functions and configuration options to prevent OOM (Out of Memory).
• Added an error message when chunkGranularity="DATABASE" is specified during IOTDB engine creation.
• The ewmMean function now supports halfLife of DURATION type, along with a new parameter times to specify the timestamp of each observation.
• Added support for metric calculation on DECIMAL128 data in the reactive state engine.
• Added support for join order optimization.
• Optimized memory cache statistics for symbol base.
• After calling cancelJob for a replay job, subsequent code execution is now terminated with an error message.
• Added a limit of 1024 for the number of conditions and the length of expressions in the WHERE clause. Queries exceeding this limit will now throw an error.
• When using the LIMIT or TOP clause, the system will no longer check the number of partitions, preventing errors caused by an excessive number of partitions.
• Optimized the query performance when using the LIMIT statement.
• Optimized the temporalParse function behavior to return NULL instead of an out-of-range error.
• Optimized the dynamic selection mechanism of the GROUP BY algorithm.
• Optimized the partition pruning mechanism.
• Improved the error messages of the tmbeta function.
• Improved the rand function by making the count parameter optional, allowing it to return a random scalar when count is not specified.
• Added support for the dropColumns! function to delete non-partitioning columns in DFS tables.
• Added a new parameter chunkId to the triggerNodeReport function, allowing reporting of specified chunk information.
• Added support for using non-constant expressions as values in JSON objects.
• Optimized the performance of getClusterDFSDatabases and getClusterDFSTables when processing clusters with a large number of chunks (hundreds of thousands or more).
• Added support for backup and restore of empty tables in an IOTDB database.

• Fixed an issue where the system could incorrectly parse "minus 1" as "negative 1" and report an error when there were spaces around the subtraction operator in the compution expression of an SQL statement. (2.00.16.7)
• Fixed an issue where null values could appear in the result when executing pivot by directly on distributed database tables. (2.00.16.7)
• Fixed an issue where table alias resolution was incorrect when the table was a complex expression in an SQL statement. (2.00.16.7)
• Fixed an issue where the OLAP proxy engine displayed an incorrect cache status in storage-compute separation scenarios after flushing cache. (2.00.16.7)
• Fixed an issue where the error message was inaccurate when inserting into selected columns using insert into table (column_list) with a select statement. In versions that do not support this feature, the message now indicates that this usage is not yet supported and recommends upgrading to the latest version. (2.00.16.7)
• Fixed a concurrency issue when querying cross-sectional streaming engine, preventing SQL query errors caused by data inconsistencies (e.g., mismatched row counts) during concurrent read and write operations. (2.00.16.7)
• Fix an issue where getStreamingStat still returned stale entries after other nodes canceled subscriptions on the current node. (2.00.16.7)
• Fixed an issue in the join engine where results were not correctly updated when calculating array vector fields. (2.00.16.7)
• Fixed the following issues in INTEGRAL type parsing: (2.00.16.7)
  • Fixed an issue where parsing results were not as expected when importing SHORT, INT, or LONG data in scientific notation through loadText.
  • Fixed an issue where the minus sign could be lost when a negative LONG value ended with trailing spaces.
• (2.00.16.7)
• Fixed the issue where cached handles were not invalidated during moveReplicas operations in a high-availability cluster, occasionally leading to inconsistent query results. (2.00.16.6)
• Fixed the issue where the totalRows field returned by getStreamTables overflowed. (2.00.16.6)
• Fixed the issue where getSessionMemoryStat reported incorrect OLAPTablet memory statistics under OOM scenarios, which prevented cache cleanup logic from functioning properly. (2.00.16.6)
• Fixed the issue where remote script execution occasionally reported errors after using an XDB connection. (2.00.16.6)
• Fixed the issue where an error occurred when a query hit the cache engine buffer in TSDB engine, caused by the contain method of a SYMBOL vector accepting only LITERAL values. (2.00.16.6)
• Fixed the issue where an error occurred when selecting in-memory table data using the IN predicate in a SELECT statement. This issue was introduced in version 2.00.16.3. (2.00.16.6)
• Fixed the issue where results from certain windows were not output when the window parameter of createWindowJoinEngine was set to 0:0 and the number of records within a group exceeded the garbageSize limit. (2.00.16.6)
• Fixed the issue where an error occurred when appending a VOID column (e.g., generated by SELECT null) to a SYMBOL column. (2.00.16.6)
• Fixed the issue where the controller process failed to exit after running the stopController.sh script in a high-availability cluster with lanCluster=true. (2.00.16.6)
• Fixed an issue where, after using warmupStreamEngine to preheat the time series engine, writing new data caused the engine to output results for all preheated data instead of only the new data. (2.00.16.5)
• Fixed an issue in the time-series engine where using last, lastNot, or functions that return multiple values with keyPurgeFreqInSec enabled __ could produce incorrect results. (2.00.16.5)
• Fixed a cluster hang issue caused by concurrent dropTable and write operations. (2.00.16.5)
• Fixed server crashes that could occur when a column of SYMBOL type was set as the sortKey in the TSDB engine. (2.00.16.5)
• Fixed occasional errors when the stream subscription contains data of type ANY. (2.00.16.5)
• Fixed an issue where users encounter unexpected permission errors when performing DDL operations in clusters with compute groups. (2.00.16.5)
• Fixed an issue where exporting CSV with the saveText function incorrectly handles quotes in fields with the SYMBOL type. (2.00.16.5)
• Fixed an issue where the result of linearTimeTrend is incorrect when the window is larger than 1024. (2.00.16.5)
• Fixed an issue where required column names could not be correctly parsed in partitioned table queries of the TSDB/IOT engine when select used nested ternary operators ?:. (2.00.16.5)
• Fixed an issue where the bar function could generate out-of-range values when processing time types such as TIME, causing query and update failures. (2.00.16.5)
• Fixed an issue where getUserTableAccessRecords would throw an error when querying ranges across two or more job files. (2.00.16.5)
• Fixed an issue where the cluster could not be accessed internally when the cluster IP was not added to the IP whitelist. (2.00.16.5)
• Fixed an issue where data to be inserted into the stream table after setting a time column using setStreamTableTimestamp would no longer be checked for the number of columns. (2.00.16.5)
• Fixed issues in high-availability clusters: (2.00.16.4)

  • Followers failed to shut down properly when the controllers was configured with lanCluster=0.

  • Concurrent getClusterPerf executions from multiple controllers exceeding the number of available worker threads could cause the system to become unresponsive.

• Fixed an issue whereenableTableShareAndCachePurge might occasionally fail for a stream table that had multiple local subscriptions. (2.00.16.4)
• Fixed an issue where executing an update ... context by ... statement on a shared memory table caused the error “A snapshot table doesn't support data update”. (2.00.16.4)
• Fixed an issue where using the moving function on DECIMAL data in the Reactive State Engine produced results inconsistent with those obtained in batch mode. (2.00.16.4)
• Fixed an issue where scheduled jobs and subscriptions could occasionally fail to execute under high system load. (2.00.16.3)
• Fixed an issue where the system could hang when writing to TSDB Database if the number of symbols exceeded the limit. (2.00.16.3)
• Fixed an issue when subscribing to a stream table with subscribeTable, if the function specified in the filter parameter produced an output schema inconsistent with the input stream table, the system neither outputs data nor reports an error. (2.00.16.3)
• Fixed memory leaks caused by querying tables if a GROUP BY column in an aggregate computation was of BLOB type. This issue was introduced in version 2.00.11. (2.00.16.3)
• Fixed server crashes where inserting an empty vector into a stream table with a timestamp column set by setStreamTableTimestamp. (2.00.16.3)
• Fixed an issue where excessive logging after partition updates in the TSDB engine could cause log files to grow rapidly and fill up disk space. This issue was introduced in version 2.00.15. (2.00.16.3)
• Fixed an issue where memory usage returned by getUserHardwareUsage could be incorrect (negative) due to erroneous automatic type conversion. This issue was introduced in version 2.00.15. (2.00.16.3)
• Fixed an issue where SQL aggregation operations could occasionally cause the server to crash. (2.00.16.3)
• Fixed an issue where submitting multiple tasks concurrently could occasionally trigger a scheduling anomaly, causing the system to become unresponsive. (2.00.16.3)
• Fixed server crashes that could occur when an empty data source was passed to the mr function due to missing input type validation. (2.00.16.3)
• Fixed server crashes that could occur when an empty data source was passed to the mr function due to missing input type validation. (2.00.16.3)
• Fixed an issue where temporary directories were not properly deleted when rolling back uncommitted update/delete transactions. (2.00.16.2)
• Fixed an issue where enabling enableSeqNo during client write operations could cause memory leaks. (2.00.16.2)
• Fixed an issue where partition pruning was not correctly applied for comparison conditions using < or <= on VALUE-partitioned tables. (2.00.16.2)
• Fixed server crashes caused by out-of-bounds row index generation when deleting records via join queries on TSDB tables with soft deletion enabled and the keepDuplicates parameter not set to ALL. (2.00.16.2)
• Fixed an issue where jobs could fail due to rename errors caused by leftover files during retries when using the AWS plugin with Amazon S3 for tiered storage. (2.00.16.2)
• Fixed an issue where a write to a distributed table could fail in high availability scenarios if a data node was restarted during an active write transaction. (2.00.16.2)
• Fixed an issue where closing an abnormal subscription connection on the publisher side left a lock unreleased when other active subscriptions existed, blocking new subscriptions. (2.00.16.2)
• Fixed an issue where the results of getAuthenticatedUserTicket and generateUserTicket could contain newline characters. Both functions now always return a single-line string. (2.00.16.2)
• Fixed server crashes caused by dropping a local table (using drop table) that shares the same name with a variable shared from another session. (2.00.16.2)
• Fixed an error where pivot by queries failed with "Unrecognized column name" in cross-partition execution due to incorrect column name mapping. (2.00.16.2)
• Fixed server crashes caused by the concurrent execution of share and existsStreamTable on the same stream table. (2.00.16.1)
• Fixed an issue where the connection to the publishing node in cross-node subscription scenarios did not use the user information from XDB for login. (2.00.16.1)
• Fixed server crashes caused by revoking table-related permissions with objs specified as a database object. (2.00.16.1)
• Fixed an issue where queries using user-defined aggregation functions combined with interval grouping returned empty results. (2.00.16.1)
• Fixed an issue where null values in the results returned by eig().values could not be properly filled. (2.00.16.1)
• Fixed an issue where calling weekEnd inside a for loop could unexpectedly modify the input date variable. (2.00.16.1)
• Fixed an issue where the vectorAR function failed on certain valid inputs. The behavior is now aligned with the Python implementation. (2.00.16.1)
• Fixed a node hang issue caused by stream table persistence blocking all worker threads. (2.00.16.1)
• Fixed a missing reconnect issue caused by stale subscription state. (2.00.16.1)
• Fixed an issue where fromStdJson failed to parse JSON strings containing empty objects with closing braces on a new line. (2.00.16.1)
• Fixed an issue where tuples returned by user-defined functions were not properly marked as mutable references, causing ineffective modifications on the original object. (2.00.16.1)
• Fixed an issue where concurrent subscribe and unsubscribe operations on the same topic caused inconsistent subscription state. (2.00.16.1)
• Fixed an issue where adding data nodes online failed with the error: "The number of nodes [] exceeds the max number of nodes specified in the license file []". (2.00.16.1)
• Fixedan issue where catalog information might be lost after performing checkpoint operations on ACL logs.
• Fixed an issue caused by naming conflict between table names and built-in functions during table join operations.
• Fixed an issue where memory usage statistics were inaccurate when a stream table contained array vector columns.
• Fixed an issue where stream table permissions were not automatically revoked after executing operations such as undef, dropStreamTable.
• Fixed server crashes caused by a data race between the network thread and the subscription thread during stream table subscription.
• Fixed potential crashes in movingWindowData when processing large scale datasets.
• Fixed a data race due to concurrent access to partition metadata before a node goes online or offline.
• Fixed server crashes caused by cache allocation failure in the OLAP proxy storage engine of the storage-compute separation architecture.
• Fixed server crashes caused by invalid parameters passed to sqlColAlias.
• Fixed a parsing failure when the user-defined function was used as an operator within a module.
• Fixed an issue where SQL queries with a WHERE clause containing an empty condition returned an empty table instead of an error.

Deprecated functions registerSnapshotEngine and unregisterSnapshotEngine.