3.00.6

Changes Made to Match Industry Practices

  • Support subtype expressions for the ANY type and columnar tuple. The outputs of typestr, type, schema, and related functions have changed:

    • Use ANY[<BasicType>] to represent subtypes of ANY, such as ANY[INT];

    • The return value of type is represented by adding 128 to the type value of the basic type.

      a = array(ANY[DECIMAL32(2)], 0, 10)
isColumnarTuple(a)
typestr(a)
// Previous versions return 'ANY VECTOR'
// New version returns 'ANY[DECIMAL32(2)] VECTOR'

type(a)
// Previous versions return int(25)
// New version returns int(165)

b = table(10:0, `a`b`c, [INT, ANY[DECIMAL32(2)], DECIMAL64(4)])
schema(b).colDefs
/* Previous versions return
name typeString   typeInt extra comment
---- ------------ ------- ----- -------
a    INT          4
b    ANY          25
c    DECIMAL64(4) 38      4
*/
/* New version returns
name typeString        typeInt extra comment
---- ----------------- ------- ----- -------
a    INT               4
b    ANY[DECIMAL32(2)] 165     2
c    DECIMAL64(4)      38      4
*/

  • INT columns in in-memory tables no longer support writing STRING data:

    • Previous versions attempt to extract digits from the string and write the converted value;

    • The new version reports an error.

      share streamTable(10000:0, `time`sym`price`qty`id, [TIMESTAMP, SYMBOL, INT, INT, DOUBLE]) as trades
n = 500
timev = temporalAdd(2012.01.01T11:24:56.245, take([1, 3, 5, 7], n).sort(), "s")
symv = take(`AA`BB`CC, n)
pricev = rand("A" + string(1..10), n)
qtyv = rand(0..50 join int(), n)
id = rand(1..100 join int(), n) * 0.01
insert into trades values(timev, symv, pricev, qtyv, id)

// Previous versions can write the data by removing the letters in pricev and converting the remaining part to numbers.
// New version reports an error: Failed to append data to column 'price' with error: Failed to convert string to int: A2

  • Deprecated legacy changelog-related interfaces, including changelogStreamTable, getStreamTableChangelog, peekAppend, StreamGraph::changelogSource, and DStream::changelogSink.

  • The return format of getStatelessMetrics has changed. The outputMetricKey and triggerOn columns in the returned table now retain double quotation marks:

    • Previous versions return values such as b:value1 and a:f1;

    • The new version returns values such as "b" : "value1" and "a" : "f1".

  • Date fields of INSTRUMENT and MKTDATA types, such as referenceDate, settlement, and termDates, now only allow the DATE type:

    • Previous versions allow STRING or DATE values;

    • The new version no longer supports STRING values.

  • When the PKEY primary-key engine creates a database, only chunkGranularity = "TABLE" is supported; chunkGranularity = "DATABASE" is no longer supported.

  • The JIT distribution and support scope have changed:

    • JIT is now dynamically linked through libjit.so. The Linux ABI=1 version enables JIT by default; use isJIT() to confirm whether it has been loaded successfully.

    • The standalone JIT installation package has been removed, and JIT is no longer compatible with Windows.

    • JIT usages supported in previous versions but not currently supported in the new version include: recursive functions, RSE streaming engine-related functionality, expressions in the form table(xxx as id), symbol vector, SQL, multi-statement assignment, accessing values of temporary variables such as a()[2], metaprogramming, module, class vector, int128-based types (including INT128, IPADDR, UUID, COMPLEX, and POINT), changing a variable's type after its type has been determined, higher-order function symbols, and some dictionary key/value types.

System Impacts Caused by Bug Fixes

  • The return form of complex when the input is a matrix has changed:

    • Previous versions return a vector;

    • The new version returns a matrix.

      x = matrix([0.75, 0.125, 0.0, 0.125],
           [-0.25, 0.125, 0.0, 0.125],
           [-0.0, 0.0517767, -0.75, 0.3017767],
           [-0.0, -0.3017767, 0.75, -0.0517767])
typestr(complex(x, 0))

// Previous versions return 'FAST COMPLEX VECTOR'
// New version returns 'FAST COMPLEX MATRIX'

  • When loadText converts strings that do not match numeric formats to numeric types according to a specified schema, the result has changed:

    • Previous versions extract and convert the numeric prefix of the string;

    • The new version returns null values.

      t = table(["2013.06.13  13:30:10.008",
           "2012.06.13 13:30:10",
           "2012.06.13 13:30:10.008",
           "13:30:10.008007006",
           "2012.06.13 13:30:10.008007006"] as str)
saveText(t, "compat_numeric.csv")
schema = table(`str as name, `INT as type)
t1 = loadText("compat_numeric.csv", ",", schema)
print(t1)

/* Previous versions return
str
----
2013
2012
2012
13
2012
*/
// New version returns only null values

  • In 3.00.6, the build environment of the OpenBLAS library has changed, so the fallback architecture may differ from earlier versions. If the OPENBLAS_CORETYPE environment variable is not set, OpenBLAS selects an instruction set based on the machine architecture. As a result, functions that rely on OpenBLAS may produce numerical differences across machines or compared with earlier versions, such as differences in precision, eigenvector order, or sign. Affected calculations include pca, varma, vectorAR, derivative, arima, bvls, olsEx, piecewiseLinFit, and schur. To keep results consistent with the previous environment, set OPENBLAS_CORETYPE=Haswell before startup.

Upgrade Notes

  • When upgrading from 2.00.16 or 2.00.17 to 3.00.6, if the earlier version enabled enhancedSecurityVerification and executed lockUser, unLockUser, createUser, resetPwd, changePwd, or setIPConnectionLimit, ACL log compatibility must be handled first: configure useOldAclLog=1 to start the new version, wait until all controller nodes complete the automatic checkpoint (the completion log is Checkpoint completed, ACL fixed.), then change useOldAclLog to 0 or remove the configuration and restart all nodes in the cluster.

  • For backup files generated by backupSettings in 2.00.16 or 2.00.17, when restoring them in the new version with restoreSettings, specify the Server version that generated the backup file through serverVersion, for example, serverVersion="200.16" or serverVersion="200.17".

  • If streaming high availability was enabled in a 2.00.18 or 3.00.5 environment, rollback to an earlier version is not supported after upgrading to 3.00.6 because the serialization format for streaming high availability has changed.

  • After upgrading to 3.00.6, if users are created, deleted, or modified, or if aclCheckPoint or takeMasterCheckpoint is triggered, ACL logs or checkpoint data that cannot be recognized by earlier versions may be written, and rollback to an earlier version is not supported.