2.00.13

Version: 2.00.13   Compatibility Level 2 with 2.00.12

For details, see Compatibility Changes.

Release Date: 2024-06-30

Available Downloads:

New Features

  • Added functions gmd5 and rowGmd5 to generate MD5 hashes. (2.00.13.10)

  • Added configuration parameter maxJoinTaskRetry to specify the maximum retry attempts for a single SQL JOIN task under memory pressure. (2.00.13.10)

  • Added functions getDBAccess and getTableAccess to check the user/group access to specific databases and tables. (2.00.13.3)

  • Added parameter minPeriods to functions mcount and mrank for specifying the minimum number of observations in a window. (2.00.13.2)

  • Added function hmac for implementing hash-based message authentication code. (2.00.13.2)

  • Added function vectorNorm for calculating the norm of a vector or matrix. (2.00.13.2)

  • Added configuration parameter queryLogRetentionTime for setting the retention time of query logs. (2.00.13.2)

  • Added configuration parameter resourceSamplingLogDir to set the directory for storing resource tracking logs.

  • Added support for the ternary operator ?:.

  • Added configuration parameter jobLogRetentionTime to set the retention time of query information stored in data node logs.

  • Added configuration parameter batchJobFileRetentionTime to set the retention time for batch and scheduled job results.

  • Added function remoteRunCompatible that operates in the same way as function remoteRun but skips version verification between local and remote databases.

  • Added parameters priority and parallelism to function scheduleJob to set the priority and parallelism for scheduled jobs.

  • Added parameter bom to function saveText to set whether to include a BOM header when exporting files.

  • Added new optimization functions:

    • fminLBFGSB: Minimizes a function using the L-BFGS-B algorithm.
    • fminNCG: Performs unconstrained minimization using the Newton Conjugate Gradient method.
    • fminBFGS: Minimizes a function using the BFGS algorithm.
    • fminSLSQP: Minimizes a function using the Sequential Least Squares Programming method.
    • brute: Minimizes a function within a given range using the brute-force method.
  • Added functions piecewiseLinFit and pwlfPredict to fit a piecewise linear regression function and make predictions based on the regression model.

  • Added functions gaussianKde and gaussianKdePredict to estimate the probability density of a random variable using the Gaussian kernel estimation and make predictions based on the regression model.

  • Added function vectorAR to analyze multivariate time series using a vector auto-regression model.

  • Added function eqPercent to check element-wise equality of two inputs within a specified tolerance level.

  • Added function removeNode to remove compute nodes from a cluster.

  • Enhanced function addNode for adding multiple nodes simultaneously.

  • Added support for Chimp compression algorithm.

  • Added configuration parameters TSDBSymbolBaseEvictTime and TSDBCachedSymbolBaseCapacity determine when to evict least recently used SYMBOL base entries based on time or capacity limits.

  • Added function clearAllTSDBSymbolBaseCache to clear all cached SYMBOL base entries that are absent from both the cache engine and ongoing transactions.

  • Added support for DELETE JOIN in the SQL DELETE statement and the sqlDelete function.

  • Added function setTableComment for adding comments to DFS tables. Added parameter comment to SQL CREATE statement.

  • Added configuration parameter memLimitOfAllTempResults to set the memory limit for all temporary partitioned tables generated during distributed queries. Added related functions setMemLimitOfAllTempResults and getMemLimitOfAllTempResults to dynamically set and get the configuration value.

  • Added configuration parameter enableInsertStatementForDFSTable to set whether the insert into statement is allowed for DFS tables.

  • insert into statement now supports inserting multiple rows.

Improvements

  • The value of configuration parameter subThrottle is now automatically adjusted to 1 if a value smaller than 1 is set. (2.00.13.12)

  • Improved the error message thrown when an alias is used for a GROUP BY field in the SELECT clause. (2.00.13.12)

  • Enhanced the DELETE statement performance. (2.00.13.12)

  • Added the recording of operations on the persisted MVCC table to the log. (2.00.13.12)

  • Improved error handling in distributed queries when comparing CHAR constants with STRING partitioning columns in the WHERE clause. The system now raises an error message. (2.00.13.12)

  • Enhanced the stability for concurrent SQL queries. (2.00.13.11)

  • Optimized the query performance for VALUE-partitioned TSDB databases when filtering on partitioning columns. (2.00.13.11)

  • Added the “updateTime” field to the backup log file. (2.00.13.11)

  • Enhanced the stability of batch job execution under high-load and high-concurrency scenarios. (2.00.13.10)

  • Optimized TSDB engine performance for concurrent queries and queries after dimensionality reduction for the sort keys. (2.00.13.10)

  • Optimized memory usage of the createTimeSeriesEngine function when the following conditions are met (2.00.13.8):
    • windowSize is set as a scalar.
    • windowSize and step have the same value.
    • acceptedDelay > 0.
    • useSystemTime = false.
    • keyColumn is not empty.
  • Optimized partition creation performance for databases containing millions of existing partitions. (2.00.13.7)

  • Enhanced the log messages for asynchronous cluster replication. (2.00.13.7)

  • Enhanced the time-series streaming engine: When useSystemTime=false, acceptedDelay can now exceed windowSize. (2.00.13.6)

  • Improved memory efficiency when processing speed is slow. (2.00.13.6)

  • Privileges QUERY_RESULT_MEM_LIMIT and TASK_GROUP_MEM_LIMIT can now be set for user groups. (2.00.13.5)

  • Replaced the memory manager from tcmalloc_minimal with tcmalloc and added functions startHeapSample, dumpHeapSample, and stopHeapSample for heap sampling. (2.00.13.5)

  • stringFormat can now take a LONG integer input when the format parameteris set to %X. (2.00.13.5)

  • Added parameter depthFirst to function flatten for calculating nested elements through depth-first algorithm. (2.00.13.5)

  • When the X parameter of functions accumulate and reduce is a non-negative integer, a unary function, or a null value, and the func parameter is a multivariate function with only one required argument, func is interpreted as a unary function. (2.00.13.5)

  • The parameter n of state functions percentChange and deltas of the reactive state engine can be specified as a positive integer. (2.00.13.4)

  • Enhanced loadPlugin to support paid plugins. (2.00.13.3)

  • Optimized loading performance of persisted stream tables. (2.00.13.3)

  • Enhanced data type validation for partitioning columns when creating tables in a database with a time-based partitioning scheme. (2.00.13.3)

  • Enhanced the security of the login system by implementing IP-specific lockouts. Multiple failed login attempts occur from a particular IP address will temporarily block further login attempts only from that specific IP. (2.00.13.2)

  • For TSDB databases configured to retain all data (keepDuplicated set to ALL), the upsert! function no longer requires specifying the keyColNames parameter for all sort columns. (2.00.13.2)

  • The backup function now captures and stores table owner information, which enables the restore function to restore the corresponding owner information. (2.00.13.2)

  • Enhanced access control and other features to improve system security. (2.00.13.1)

  • In metaprogramming, you can now specify column alias using a macro.

  • The registerSnapshotEngine function now allows the snapshot engine to be registered on all data nodes for a DFS table, with a new parameter dedupColumnName added for this purpose.

  • Built-in functions with function patterns can serve as function operators.

  • The snapshot parameter of the restore function now defaults to true instead of false.

  • In asynchronous replication, slave clusters no longer require password configuration through the clusterReplicationExecutionPassword parameter. User credentials are now verified using a more secure method.

  • Enhanced TSDB redo log replay performance.

  • The clearAllCache function has been extended to include cache specific to the TSDB storage engine.

  • The version function now provides more comprehensive version details.

  • interval function enhancements:

    • The duration parameter can be specified using trading calendar identifiers.
    • Non-partitioned table queries now support user-defined aggregate functions when interval's duration and step parameters are inconsistent.
  • The flatten function can be applied on nested tuples.

  • The precision of logisticRegression now matches that of its sklearn equivalent.

  • Improved precision for functions: std, stdp, var, varp, skew, and kurtosis.

  • The func parameter of the moving higher-order function now supports functions returning tuples.

  • A new fill parameter has been added to the addMetrics function to specify the null filling method.

  • The parseJsonTable function now accepts tables containing BLOB columns.

  • The isDuplicated function now supports BLOB columns.

  • The fmin function parameters have been adjusted.

  • The rowAlign function supports columnar tuples.

  • Output of the rolling function can now have row labels.

  • TSDB databases now support compression of null columns during insertion.

  • The createSchema function now restricts adding a database to a single catalog.

  • Reactive state streaming engine enhancements:

    • prev can be applied on array vector columns.

    • Added support for 35 topN stateful functions.

    • keyColumn is now an optional parameter.

    • User-defined functions (UDFs) that return multiple values can now be used as the rvalue of assignment statements within user-defined stateful functions.

    • Improved error messages.

  • The rightTable parameter of createLookupJoinEngine can now be specified using metacode.

  • The metrics parameter of the following streaming engines now accepts constants or vector of constants: createReactiveStateEngine, createAsofJoinEngine, createLookupJoinEngine, createEquiJoinEngine, createLeftSemiJoinEngine.

  • New parameters cachePurgeTimeColumn, cachePurgeInterval and cacheRetentionTime have been added to enableTablePersistence and enableTableShareAndPersistence functions. These parameters specify how data in memory is cleaned for persisted stream tables.

  • Updated the acceptedDelay parameter in createTimeSeriesEngine and createDailyTimeSeriesEngine to allow specifying the delay time for a window to close and trigger.

  • New parameters outputHandler and msgAsTable, which allow passing calculation results to a unary function, have been added to the following streaming engines: createTimeSeriesEngine, createDailyTimeSeriesEngine, createReactiveStateEngine, createDualOwnershipReactiveStateEngine, createNarrowReactiveStateEngine.

  • INNER JOIN and LEFT JOIN support generating runtime filter using the IN predicate.

  • Enhanced query performance when filtering on partitioning columns in combination with the LIMIT clause.

  • Analytic functions with OVER(PARTITION BY xxx) can now be used within CASE WHEN statements.

  • Optimized higher-order functions accumulate and reduce in JIT.

  • JIT now supports handling indexed matrices.

Issues Fixed

  • Fixed an issue where a write to a distributed table could fail in high availability scenarios if a data node was restarted during an active write transaction. (2.00.13.13)

  • Fixed server crashes caused by out-of-bounds row index generation when deleting records via join queries on TSDB tables with soft deletion enabled and the keepDuplicates parameter not set to ALL. (2.00.13.13)

  • Fixed an issue where enabling enableSeqNo during client write operations could cause memory leaks. (2.00.13.13)

  • Fixed an issue where closing an abnormal subscription connection on the publisher side left a lock unreleased when other active subscriptions existed, blocking new subscriptions. (2.00.13.12)

  • Fixed server crashes caused by the concurrent execution of share and existsStreamTable on the same stream table. (2.00.13.12)

  • Fixed an issue where the connection to the publishing node in cross-node subscription scenarios did not use the user information from XDB for login. (2.00.13.12)

  • Fixed server crashes caused by revoking table-related permissions with objs specified as a database object. (2.00.13.12)

  • Fixed an issue where queries using user-defined aggregation functions combined with interval grouping returned empty results. (2.00.13.12)

  • Fixed an issue where null values in the results returned by eig().values could not be properly filled. (2.00.13.12)

  • Fixed an issue where the vectorAR function failed on certain valid inputs. The behavior is now aligned with the Python implementation. (2.00.13.12)

  • Fixed an error where pivot by queries failed with "Unrecognized column name" in cross-partition execution due to incorrect column name mapping. (2.00.13.12)

  • Fixed a missing reconnect issue caused by stale subscription state. (2.00.13.12)

  • Fixed memory leaks caused by transactions with excessively large transaction IDs (greater than 2³¹ – 1) that could not be properly reclaimed. (2.00.13.12)

  • Fixed occasional server crashes when executing SQL queries containing the DISTINCT statement. (2.00.13.12)

  • Fixed the issue where the “lastUpdated” field in the controller metadata could be null after restoring from a backup that was created immediately after a restart without any data updates. (2.00.13.12)

  • Fixed the issue where the server entered an infinite loop when executing decimalFormat(0, '0.00E0'). (2.00.13.12)

  • Fixed occasional hangs on the web client when executing code if the response data size is between 4050 and 4060 bytes. (2.00.13.12)

  • Fixed incorrect JSON serialization of DECIMAL32 values in the toStdJson function. (2.00.13.12)

  • Fixed server crashes when the dummyTable specified in createRuleEngine contained a SYMBOL column. (2.00.13.12)

  • Fixed server crashes caused by using UPDATE/DELETE with WHERE EXISTS. Now it raises an error instead. (2.00.13.12)

  • Fixed occasional server crashes when the X and Y of the corr function were tables with different numbers of columns. (2.00.13.12)

  • Fixed misleading error messages returned when table joins encountered more than 2,097,152 unique values in a SYMBOL column. (2.00.13.12)

  • Fixed server crashes caused by concurrent disk reads in the TSDB engine due to missing filter conditions in pivot by after partition pruning. (2.00.13.12)

  • Fixed server crashes caused by concurrent writes to an in-memory keyed table. (2.00.13.12)

  • Fixed a memory leak when calling a user-defined recursive function. (2.00.13.12)

  • Fixed an issue where concurrent subscribe and unsubscribe operations on the same topic caused inconsistent subscription state. (2.00.13.12)

  • In the time-series engine, the nunique, ifirstNot, and ilastNot functions specificed in the metrics argument now support expressions and nested functions as arguments. (2.00.13.12)

  • Fixed server crashes occurred when usingrowGmd5on large datasets. (2.00.13.11)

  • Fixed server crashes that occurred when queries with lead/lag analytic functions were executed in MySQL syntax mode. (2.00.13.11)

  • Fixed an error when deleting column(s) from a keyed table if any column name contains uppercase letters. (2.00.13.11)

  • Fixed server crashes caused by appending a partitioned DFS table to another. The system now reports an error for such case. (2.00.13.11)

  • Fixed server crashes occurring when connecting with a higher version server and running commands not supported by the current server. (2.00.13.10)

  • Fixed a data loss issue in N-to-N replay that occurred when the data being replayed included empty partitions. (2.00.13.10)

  • Fixed occasional server crashes during streaming subscription. (2.00.13.10)

  • Fixed an error where SQL queries failed if they contained both a user-defined aggregate function returning multiple values and a GROUP BY clause with interval. (2.00.13.10)

  • Fixed a "unrecognized column name" error that occurred when executing a multi-table JOIN query, if some partitions were offline due to node failures in the cluster. (2.00.13.10)

  • Fixed the issue causing incorrect results in a function view after server restarts when the view included CROSS JOIN (ANSI SQL compliant) combined with other types of table joins. (2.00.13.10)

  • Fixed an issue in high-availability clusters where an existing table handle could not access data from newly added partitions without obtaining a new handle (reloading table via loadTable). (2.00.13.10)

  • Fixed server crashes when querying a keyed or indexed table with multiple primary key columns, where the filter condition contained an empty SYMBOL vector. (2.00.13.10)

  • Fixed server crashes when incorrect parameters were passed to the makeKey function within a WHERE clause. (2.00.13.10)

  • Fixed an issue in heterogeneous replay where data sources (generated by replayDS with timeRepartitionSchema specified) were replayed in a disordered manner. (2.00.13.10)

  • Fixed potential server crashes when using user-defined optimization functions. (2.00.13.10)

  • Fixed unexpected query results due to errors in domain recovery during restart. (2.00.13.10)

  • Fixed slow response of function login when nodes in a high-availability cluster were under heavy load. (2.00.13.10)

  • Fixed incorrect results of createTimeSeriesEngine when the metrics parameter contained a user-defined aggregate function with multiple return values, and parallelism was greater than 1. (2.00.13.8)

  • Fixed occasional deduplication failures in high-concurrency scenarios when the keepDuplicates parameter was set to FIRST for a TSDB engine. (2.00.13.5)

  • Fixed invalid online configuration modifications in a high availability cluster after a leader switch. (2.00.13.5)

  • Fixed potential server crashes when concurrently executing tasks generated by sqlDS through submitJob. (2.00.13.5)

  • Fixed occasional server crashes caused by concurrent logins when enhancedSecurityVerification=true was configured. This issue was introduced in version 2.00.11. (2.00.13.5)

  • Fixed slow publishing of streaming data when only a small amount of data remained after stream tables were filtered. (2.00.13.5)

  • Fixed automatic interpolation in the results of createTimeSeriesEngine and createDailyTimeSeriesEngine when the value of acceptedDelay was equal to windowSize. (2.00.13.5)

  • Fixed an issue where daily time-series engines filled time column with "invalid date" after receiving cross-date data in different groups. (2.00.13.5)

  • Fixed precision loss of DECIMALs when executing the segmentby function. (2.00.13.5)

  • Fixed a server startup failure when function views included insert into statements. (2.00.13.5)

  • Fixed a potential metadata loss issue when using renameTable to rename a table that has been deleted by dropTable and the renaming operation occurred concurrently with a checkpoint operation. (2.00.13.5)

  • Fixed occasional freezing issues in the Windows JIT version. (2.00.13.5)

  • Fixed occasional failures in stream subscription due to data corruption. (2.00.13.4)

  • Modified the function flatten to be compatible with previous versions. (2.00.13.4)

  • Fixed potential server crashes caused by resource tracking. (2.00.13.3)

  • Fixed server crashes occurring when array vector elements exceeded the length limit during table joins. (2.00.13.3)

  • Fixed state cleanup issues during transaction anomalies when operating on MVCC tables. (2.00.13.3)

  • Fixed a bug where the specified outputHandler of createReactiveStateEngine failed to trigger if keyColumn was not specified. (2.00.13.3)

  • Fixed data loss caused by concurrent moveHotDataToColdVolume operations on the same partition. (2.00.13.3)

  • Fixed an issue where the batch job logs failed to record the job start times when disk space was fully occupied during job initiation. (2.00.13.3)

  • Fixed potential metadata loss in HA clusters due to checkpoint failures during a controller restart. (2.00.13.3)

  • Fixed an issue where rollingPanel failed to group data based on groupingCol. (2.00.13.3)

  • Fixed server crashes that occurred when executing background jobs containing lambda expressions, submitted via submitJob or scheduleJob, if the session that originally submitted the job had closed. (2.00.13.3)

  • Fixed incorrect results of SQL queries where an argument of an aggregate function was a scalar or a vector of incompatible length with the input table columns, specifically when using the vectorized grouping algorithm. (2.00.13.3)

  • Modified the function flatten to be compatible with previous versions. (2.00.13.3)

  • Fixed an error of “The result of distributed call is not a table” caused by EXEC distributed queries with positive LIMIT/TOP and ORDER BY clauses specifying a partitioning column in descending order. (2.00.13.3)

  • Fixed an issue of createDailyTimeSeriesEngine where data with timestamps smaller than sessionEnd was received after forceTriggerSessionEnd. (2.00.13.3)

  • Fixed a system failure to skip cluster replication tasks with corrupted data, which resulted in endless retry attempts. (2.00.13.3)

  • Fixed an issue in distributed query optimization using ORDER BY and LIMIT/TOP. (2.00.13.3)

  • Fixed an error of parseJsonTable when parsing JSON objects containing \\\". (2.00.13.3)

  • Fixed an issue where interactive jobs submitted through Web interface or VS Code extension by users with lower-than-default priority (or parallelism) still used default values. (2.00.13.2)

  • Fixed incorrect start date in the audit log for APPEND operations. (2.00.13.2)

  • Fixed an issue where uploading an encrypted module could cause variables to be lost in the current session. (2.00.13.2)

  • Fixed an error of the backup function on a compo-partitioned database if the filter condition matched the first-level partition and some second-level sub-partitions were empty. (2.00.13.2)

  • Fixed an error when executing the writeLogLevel function in a scheduled job after a server restart. (2.00.13.2)

  • Fixed a server crash when querying an array vector column with distinct. (2.00.13.2)

  • Fixed an issue where queries failed during ej operation if the table name was not specified in the where condition. (2.00.13.2)

  • Fixed system crashes after executing the createWindowJoinEngine function with invalid parameters. (2.00.13.2)

  • Fixed an error where using the move function may lead to incorrect calculation results if the input data was an intermediate result. (2.00.13.2)

  • Fixed occasional crashes when executing the mpercentile function multiple times. (2.00.13.2)

  • Fixed an error in JIT when using the rank function due to type inference issues. (2.00.13.2)

  • Fixed an error in JIT when using the isort function due to compatibility issues. (2.00.13.2)

  • Fixed parsing errors preventing addition of function views for no-return functions in module/plugin.

  • Fixed ARM server crashes caused by concurrent openBLAS thread operations.

  • Fixed incorrect results returned by the qclp function.

  • Fixed server crashes when the SQL statements within the metacode specified in the obj parameter of the saveText function were incorrectly enclosed with quotes ('').

  • Fixed server crashes when using at together with higher-order functions eachRight/eachLeft.

  • Fixed server crashes when loading excessively large files with the loadText function.

  • Fixed an issue where the dict function were not recognized in class inheritance.

  • Fixed the getUserHardwareUsage function returning unexpected results when provided with dates.

  • Fixed an issue where the "executionSet" field returned by the getSlaveReplicationQueueStatus function was occasionally randomized.

  • Fixed an issue where strings were incorrectly recognized as DATE values and failed to load with the loadText function.

  • Fixed an error when appending array vectors generated by the fixedLengthArrayVector function to a keyed table.

  • Fixed server crashes when the aggs parameter of wj was specified as count(head()).

  • Fixed SQL parsing issue when a statement included user-defined variables in the SELECT clause alongside a GROUP BY clause.

  • Fixed an issue in pj where, with left and right tables sorted by matching columns, only the first matched row from the right table was selected for joining, causing incomplete results.

  • Fixed SQL optimization failure caused by case mismatches between join columns and table columns.

  • Fixed an issue with partition pruning when the IN clause in a WHERE condition was followed by an in-memory table.