Standalone Mode

In standalone mode, the configuration file is dolphindb.cfg. Configuration parameters can be specified either in dolphindb.cfg, or in the command line that starts DolphinDB.

Thread Types

DolphinDB provides the following types of threads:

  • worker: threads that handle regular interactive tasks.

    Since version 1.30.22/2.00.10, DolphinDB has introduced a multilevel task assignment algorithm which divides the jobs and tasks (broken down from jobs) into different levels. The jobs and tasks are handled by the workers at the same levels. The levels of workers range from 0 to 5.

    A job submitted to the node starts at level 0 and is handled by a level 0 worker. Based on the number of partitions involved in the job, the level 0 worker breaks it into an equivalent number of tasks. Local tasks are executed concurrently by the level 0 or level 1 workers, while remote tasks are sent to remote nodes by remote executors. Tasks sent to remote nodes are marked as level 1 and handled by level 1 workers there. If a task is further divided into subtasks, the remote subtasks are marked one level lower than their parent and handled by workers at the same level on the remote nodes.

    With different levels of workers handling different levels of jobs and tasks, deadlock issues caused by circular dependencies across nodes can be effectively prevented.

    The number of workers determines the concurrent computing capability of a system.

  • remote executor: an independent thread that sends tasks to remote computing nodes. The remote executor is failure tolerant. If one machine fails, the remote executor will send the task to another machine with the data necessary for the task.

  • batch job worker: threads that handle batch jobs. Batch jobs are submitted with function submitJob and are usually time-consuming jobs. A batch job worker will be destroyed automatically after being idle for over 60 seconds.

  • dynamic worker: if there are jobs in the queue of regular interactive jobs and no jobs are completed in the previous second, the system will create dynamic workers to process the jobs in the queue. A dynamic worker will automatically be destroyed after being idle for 60 seconds. Dynamic workers make the system more responsive and can solve deadlocks among multiple nodes.

  • web worker: threads that handle HTTP requests. DolphinDB provides a web-based cluster management interface to interact with DolphinDB nodes.

  • urgent worker: threads that only handle time-sensitive system-level jobs such as logins (login) and job cancellations (cancelJob/cancelConsoleJob). The configuration parameter urgentWorkerNum specifies the maximum number of urgent workers. The default is 1.

Parameter Description
WorkerNum The size of worker pool for regular interactive jobs. The default value is the number of CPU cores.
remoteExecutors=1 The number of remote executors. The default is 1. It is recommended to set it to the number of cluster nodes - 1.
maxBatchJobWorker=4 The maximum number of batch job workers. The default value is the value of workerNum.
maxDynamicWorker=4 The maximum number of dynamic workers. The default value is the value of workerNum.
WebWorkerNum=1 The size of the web worker pool to process HTTP requests. The default value is 1.
urgentWorkerNum=1 The maximum number of urgent workers. The default value is 1.
maxCachedBatchJobNum=2048 The maximum number of batch jobs in the batch job queue. The default value is 2048.
jobParallelismHardLimit=false A Boolean value indicating whether to set the limit on the maximum number of workers a task can use. The default value is false. If it is set to true, the maximum number of workers a task can use cannot exceed the specified parallelism.
defaultJobParallelism=2 A positive integer in range (0,64] indicating the default parallelism of the background task submitted by users. The default value is 2. Note that if the value of this parameter falls outside the specified range, the server fails to start.
enableMultiThreadMerge A Boolean value indicating whether the results of subqueries assigned to each node can be merged using multiple threads. The default value is false. When set to true, for multi-threaded merging to be effectively performed, the subquery result must contain at least two columns and have more than 50 million entries. This configuration parameter can provide a significant performance boost for queries involving a large number of partitions.

Since version 2.00.11, a new configuration parameter compactWorkerNumPerVolume is added to tune the size of worker pool handling level file compaction, which provides control over resource usage, compaction speeds, and load balancing across volumes.

Parameter Description
compactWorkerNumPerVolume The size of worker pool for the level file compaction on each volume. The default value is 1.

Memory Management

Parameter Description
OLAPCacheEngineSize=0 Alias:chunkCacheEngineMemSize The capacity of cache engine in units of GB. After cache engine is enabled, data is not written to disk until data in cache exceeds 30% of chunkCacheEngineMemSize. The default value is 0 indicating the cache engine is not enabled. To enable the cache engine, we must set chunkCacheEngineMemSize > 0 and dataSync = 1.
maxMemSize=0 The maximum memory (in units of GB) allocated to DolphinDB. If set to 0, it means no limits on memory usage.
memoryReleaseRate=5 The rate at which unused memory is released to the operating system. It is a floating point number between 0 and 10. The default value is 5. memoryReleaseRate = 0 means that unused memory will not be released actively; memoryReleaseRate = 10 means that memory will be released at the fastest speed.
warningMemSize When memory usage exceeds warningMemSize (in units of GB), the system will automatically clean up the cache of some databases to avoid OOM exceptions. The default value is 75% of MaxMemSize.
reservedMemSize A positive number specified in units of GB. When the available memory in DolphinDB is less than reservedMemSize, DolphinDB only allocates a memory block of limited size (as specifiedby maxBlockSizeForReservedMemory) for each memory request. reservedMemSize is provided to restrict the memory allocation for each memory request to improve the likelihood that there is enough memory for critical operations that use a small amount of memory (error reporting, rollbacks, etc.). For example, when data writes fail due to insufficient memory, the transactions can be rolled back to guarantee data consistency. If the parameter is not specified, the system sets reservedMemSize = 5% * maxMemSize and reservedMemSize must be between 64MB and 1GB.
maxBlockSizeForReservedMemory The maximum size (in units of KB) of the memory block that DolphinDB allocates for each memory request when its available memory is less than reservedMemSize. The default value is 64.It is not recommended to set it too high as exceptions or crashes may occur if there isn't enough memory left for critical database operations.
TSDBCacheEngineSize A positive number indicating the capacity (in GB) of the TSDB cache engine. The default value is 1. The memory used by the cache engine may reach twice the set value as an extra memoryblock will be allocated to cache the incoming data while data in the original memory is being flushed to disk. If the flush process is not fast enough, the newly allocated memory may also reach TSDBCacheEngineSize and thus block the writer thread. If the parameter is set too small, data in the cache engine may be flushed to disk too frequently, thus affecting the write performance; If set too high, a large volume of cached data is not flushed to disk until it reaches TSDBCacheEngineSize (or after 10 minutes). If power failure or shutdownoccurs in such cases, numerous redo logs are to be replayed when the system is restarting, causing a slow startup.
TSDBLevelFileIndexCacheSize The upper limit of the size (in GB) of level file metadata of the TSDB engine. It is of floating type. The default value is 5% of the maximum memory for DolphinDB (set by maxMemSize),and the minimum value is 0.1 (GB).
TSDBCacheEngineCompression Specifies whether to compress the data in TSDB cacheEngine. The default value is true, which means to compress the data.

In prior versions, decimal numbers are parsed as DOUBLE floating point numbers by default. As of version 2.00.10.4, the parseDecimalAsFloatingNumber configuration parameter can specify the system default behavior for handling decimal numbers during parsing.

Parameter Description
parseDecimalAsFloatingNumber =true Whether decimal numbers are parsed as DOUBLE or DECIMAL64 type. True (default): parse decimals to DOUBLE floating point numbers. False: parse decimals to DECIMAL64 fixed point numbers.
regularArrayMemoryLimit=2048 The limit on the memory size (in units of MB) of a regular array. Its value must be a power of 2. The default value is 2048 (MB). The actual available memory is min(regularArrayMemoryLimit, maxMemSize/2).
appendTupleAsAWhole Specifies how a tuple is appended (append!) or joined (join!) to another tuple. True (default): append or join the tuple as an embedded tuple element. False: append or join each element of the tuple independently.

Hard Disk

To mitigate I/O bottlenecks, a data node can be configured with multiple volumes to improve the speed of parallel read and write.

Log files include metadata log file, redo log, query log, system log and batch job log. They have different requirements for disk performance. Generally, metadata log and redo log should be stored on SSDs while other logs can be stored on HDDs.

Parameter Description
batchJobDir The folder for batch job logs and results. The default value is <HomeDir>/batchJobs.
chunkMetaDir The folder for the metadata of data chunks on each data node. The default value is <HomeDir>/storage/CHUNK_METADATA. In a cluster, each data node should be configured with different chunkMetaDir.
dataSync=0 If dataSync=1, DolphinDB generates redo log when the database is being updated. It can avoid data loss in the event of a system crash or power outage. The default value of dataSync is 0.
jobLogFile=jobLog.csv The path and name of the job log file that contains descriptive information of all the queries that have been executed for each node. It must be a csv file. The default folder for thejob log file is the log folder. The default name of the job log file is nodeAlias_job.log.
enableDFSQueryLog=false

Different from parameter jobLogFile that only records executed queries, enableDFSQueryLog records all completed and ongoing queries and subqueries on DFS tables. The default value is false.

To set it to true, parameter perfMonitoring must be enabled. When set to true, a query log named nodeAlias_query.log will be generated in the directory where nodeAlias_job.log is located. The query log contains the following fields: node, userId, sessionId, rootId, type, level, time, database, table, and jobDesc. Subqueries on DFS tables return a query log with fields "database" and "table" left empty.
logFile=DolphinDBlog The path and name of the log file. It displays the server configuration specifications, warnings and error messages.
logLevel=INFO The log file only keeps log information equal to higher than the specified level. From the lowest to the highest level, the possible values are DEBUG, INFO, WARNING andERROR. The default value is INFO.
maxLogSize=1024 The system will archive the server log after the log reaches the specified size limit (in MB). The default value is 1024 and the minimum value is 100. The archived log filename adds a prefix to the original log file name. The prefix is in the format of \date\seq, e.g. 20181109000. seq has 3 digits and starts with 000
redoLogDir The directory of the redo log. The default value is /log/redoLog.
TSDBRedoLogDir The directory of the redo log of TSDB engine. The default value is <HomeDir>/log/TSDBRedo. TSDBMeta (including level file metadata) is stored in <HomeDir>/log, and transaction data is stored in <HomeDir>/storage/CHUNKS. To configure the parameter: Use the relative paths, i.e., do not start with '/'; Include <ALIAS> in the path: /home/xxx/<ALIAS>/redolog; Configure for each node separately: node1.TSDBRedoLogDir=/home/xxx/node1/redolog, node2.TSDBRedoLogDir=/home/xxx/node2/redolog.
redoLogPurgeInterval=30 Clear the redo log of transactions whose data have been persisted at intervals specified by redoLogPurgeInterval in terms of seconds. The default value is 30.
redoLogPurgeLimit=4000 Clear the redo log of transactions whose data have been persisted if the size of the redo log exceeds redoLogPurgeLimit in terms of MB. The default value is 4000.
volumes=/hdd/hdd1/volumes, /hdd/hdd2/volumes, /hdd/hdd3/volumes, /hdd/hdd4/volumes The folder where data files are saved in the distributed file system on a data node. The default value is <HomeDir>/<nodeAlias>/storage.
diskIOConcurrencyLevel=4 The number of threads for reading and writing to disk. The default value is 1. diskIOConcurrencyLevel = 0 means the threads handling the relevant tasks will read and write to the disk. When diskIOConcurrencyLevel > 0, the system will create the specified number of threads to read and write to the disk concurrently. For HDDs, it is recommended to set diskIOConcurrencyLevel to the number of volumes configured on the node (specified by the configuration parameter volumes). For SSDs, it is recommended to set diskIOConcurrencyLevel = 0
marketHolidayDir Specifies the directory to the file of market holidays. Either absolute or relative directory can be specified. The system searches the relative directory in the following order: home directory of the node, the working directory of the node, and the directory with the DolphinDB executable. The default directory is <HomeDir> /marketHoliday. The file must be a single-column CSV file with DATE data. Based on the CSV files under marketHolidayDir, a trading calendar is created with the file name as its identifier. Note:
  • Weekends are recognized as holidays by default, therefore, only weekday holidays need to be filled in the file.
  • It is recommended to name the file in Market Identifier Code, e.g., "XNYS.csv".
enableDfsRecoverRedo Enable recovery redo log.
recoverLogDir The directory of the recovery redo log. It must be stored on a separate disk (preferably SSD) from the data files.
enableAuditLog Whether to enable logging DDL operations to a separate log file. The default value is false. When set to true, an audit log will be generated in the directory <logFile>/<ALIAS>_audit.log. The maximum size of an audit log file is 128M, beyond which another file prefixed with a timestamp (in second) will be automatically generated for continuous storage of DDL operations.
auditLogRetentionTime A floating-point number indicating the amount of time (in days) to keep an audit log before deleting it. For example, 0.5 means 12 hours. The default value is 0, indicating that audit log files are not deleted.

IMOLTP

Parameter Description
enableIMOLTPEngine A Boolean value that determines whether to enable the IMOLTP engine. The default value is false.
enableIMOLTPRedo A Boolean value that determines whether to enable redo log mechanism for IMOLTP databases. The default value is true.
IMOLTPRedoFilePath A string that specifies the path where the redo log files will be stored. If a relative path is provided, the redo log files will be located under the <homeDir>/IMOLTP/ directory. The default value is <homeDir>/IMOLTP/im_oltp.redo. Note that the path must exist and be unique.
IMOLTPSyncOnTxnCommit This parameter only takes effect when enableIMOLTPRedo = true. It is a Boolean value that determines whether a transaction should be committed after the data is safely persisted to redo log files on disk. The default value is false, which means the transaction is committed after redo log is written to system buffer instead of disk. In cases of power outages or operating system crashes, data cannot be restored if it is false.
enableIMOLTPCheckpoint A Boolean value that determines whether to enable checkpoint mechanism for IMOLTP databases. The default value is true.
IMOLTPCheckpointFilePath A string that specifies the path where the checkpoint files will be stored. If a relative path is provided, the checkpoint files will be located under the <homeDir>/IMOLTP/ directory. The default value is <homeDir>/IMOLTP/im_oltp.ckp. Note that the path must exist and be unique.
IMOLTPCheckpointThreshold A LONG integer that sets the threshold for the redo logs (in MiB) to trigger a checkpoint. The default value is 100.
IMOLTPCheckpointInterval A LONG integer that sets the interval (in seconds) to trigger a checkpoint. The default value is 60.

User Login and Access Control

Parameter Description
thirdPartyAuthenticator Specifies a function view (which takes the same arguments as login) for third-party authentication. Users can connect to a third-party system through DolphinDB plugins such as HttpClient. Once specified, DolphinDB server authenticates the user login through the third-party system.
enhancedSecurityVerification=false

A Boolean value indicating whether to enable enhanced security verification, which includes password complexity verification and password retry limit. Set to false (default) to disable this function and true to enable. When it is enabled:

  • The password created or modified must meet the following conditions:

    • 8-20 characters

    • at least 1 capital letter

    • at least 1 special character, including !"#$%&'()*+,-./:;<=>?@[]^_`{|}~

  • If a password is entered wrongly 5 times in a minute, the user's account will be blocked for 10 minutes.
thirdPartyCreateUserCallback Specifies one or multiple function views (split with comma) to execute when the createUser has been used. The callbacks will be invoked in the order they are specified, from left to right. Note: Exceptions during the callback will be caught and logged, but only the last one will be thrown after all callbacks have finished executing.
thirdPartyDeleteUserCallback Specifies one or multiple function views (split with comma) to execute when the deleteUser has been used. The callbacks will be invoked in the order they are specified, from left to right. Note: Exceptions during the callback will be caught and logged, but only the last one will be thrown after all callbacks have finished executing.
strictPermissionMode A Boolean value specifying whether to enable strict permission mode on the current node. The default value is false. When set to true, operations such as disk read/write are only allowed for administrators. Related functions include: saveTextFile, saveAsNpy, backup, restore, restoreDB, restoreTable, backupDB, backupTable, migrate, file, files, writeObject, readObject, loadPlugin, close, fflush, mkdir, rmdir, rm, writeLog, run, runScript, test, saveTable, savePartition, saveDualPartition, saveDatabase, saveText, loadText, loadModule, saveModule.
enableShellFunction A Boolean value indicating whether an administrator can call the shell function. False (default) means the shell function cannot be called by any user. True means the permission to use the shell function is granted for the administrator.

Network

Parameter Description
enableHTTPS=false Whether to enable HTTPS protocol. The default value is false.
localSite =localhost:8848:local8848 Host address, port number and alias of the local node, separated by ":". In standalone model, the default value is localhost:8848:local8848.
maxConnections=64 The maximum number of connections (from GUI, API, other nodes, etc) to the local node.
maxConnectionPerSite=2 The maximum number of connections from the local node to remote nodes.
tcpNoDelay=1 Whether to enable the TCP_NODELAY socket option. The default value is false.

DolphinDB provides the configuration parameter tcpUserTimeout to set the maximum hold time for data that has not been acknowledged or not been sent due to TCP Zero Window. When the specified timeout is exceeded, the TCP connection will be actively closed.

Parameter Description
tcpUserTimeout=300000 Set the socket option TCP_USER_TIMEOUT. The default value is 300000 (in ms).

License Server

From DolphinDB 1.30.21/2.00.9 onwards, license server is supported for resource allocation. License server is only available with license of type 3. You may contact our technical support for more information.

Parameter Description
licenseServerSite Specifies the LAN information of the node where the license server is located. It is specified in the format of <host:port:alias>, where port must be the same as that specified in the license file. The license server is a standalone server that assigns hardware resources to each node (except the agent) in a cluster. If licenseServerSite is configured with the same value as localSite, license server runs on the current node; Otherwise the current node requests resources from the license server. If licenseServerSite is not configured, the current node uses the local license file.
bindCores Specifies the CPU cores (by core ID) for the node. To specify multiple CPU cores, you can separate the core IDs with a comma (","); To specify consecutive cores, you can use a hyphen ("-") to join the starting and ending IDs. For example, to bind CPU cores numbered 1 to 4, it can be 1, 2, 3, 4 or 1-4. This parameter only takes effect on the node that requests resources from the license server.
  • If configured, the process runs on the specified cores.
  • If not configured, the process runs on fixed cores according to the license file.
  • Otherwise, random CPU cores are used.

Streaming

The required configuration parameters for a publisher node are maxPubConnections and persistenceDir.

Parameter Description
maxMsgNumPerBlock=1024 The maximum number of records in a message block. The default value is 1024.
maxPersistenceQueueDepth= 10000000 The maximum depth (number of records) of a message queue to persist a stream table to disk. The default value is 10,000,000.
maxPubQueueDepthPerSite= 10000000 The maximum depth (number of records) of a message queue on the publisher node. The default value is 10,000,000.
maxPubConnections=0 The maximum number of subscriber nodes that the publisher node can connect to. The default value is 0. For the node to server as a publisher, we must set maxPubConnections>0.
persistenceDir= /home/DolphinDB/Data/Persistence The directory where shared streaming tables are persisted to. To enable persistence, persistenceDir must be specified. In a cluster, each data node should be configured with different persistenceDir.
persistenceWorkerNum=1 The number of workers responsible for persisting streaming tables to disk in asynchronous mode. The default value is 0.
Parameter Description
MaxSubConnections=64 The maximum number of publishers that the subscriber node can connect to. The default value is 64.
MaxSubQueueDepth=10000000 The maximum depth (number of records) of a message queue on the subscriber node.
persistOffsetDir= /home/DolphinDB/streamlog The directory to save the offset of the last subscribed message that has been processed. If persistOffsetDir is not specified, it will be saved under the directory specified by persistenceDir. If persistenceDir is not specified, it will be saved in the streamLog folder under the home directory.
subExecutorPooling=true A Boolean value indicating whether streaming executors use pooling mode. The default value is false. If it is set to 0, it means the thread can conduct message parsing and can also process messages.
subExecutors=2 The number of message processing threads in the subscriber node. Only when subscription is enabled is this parameter relevant. The default value is 1. If it is set to 0, it means the thread can conductmessage parsing and can also process messages.
subPort=8000 The port number that the subscription thread is listening on. Before version 2.00.9, this parameter must be specified to enable the node(s) to serve as subscriber(s). Since version 2.00.9, this parameter can be unspecified.
subThrottle=1000 A non-negative integer in milliseconds, indicating the interval at which the system checks whether throttle in function subscribeTable has been reached. The default value is 1000. If the interval specified by the parameter throttle in subscribeTable is less than the configuration parameter subThrottle, the parameter handler in subscribeTable will be triggered to process messages at an interval of subThrottle. To set throttle is less than 1 second, you need to modify the configuration parameter subThrottle first. For example, to set throttle =0.001 (second), please set subThrottle =1 first. Note: This parameter is only valid if the parameter batchSize is specified in function subscribeTable.
localSubscriberNum=1 Sets the number of threads used to distribute the messages from the publish queue if the publisher and subscriber are on the same node. The default value is 1. If a number greater than 1 is specified, the system will allocate the threads to distribute messages in parallel to the processing threads.

System Management

Parameter Description
config=dolphindb.cfg The configuration file. The default value is dolphindb.cfg. It can only be specified in command line.
console=1 A Boolean value indicating whether to start a DolphinDB console. The default value is 0. It can only be specified in command line.
home=/home/DolphinDB/server The DolphinDB home directory where the configuration files, the license file, the log file and other related files are located. It can only be specified in command line.
logFile=dolphindb.log The log file. The default value is dolphindb.log. It can only be specified in command line.
maxPartitionNumPerQuery=65536 The maximum number of partitions that a single query can search. The default value is 65536.
checkExistingPartitionNumForQuery Whether to use the number of existing partitions obtained from the controller to check if the number of partitions for a single query exceeds the maximum (configured by maxPartitionNumPerQuery). False (default) means that the number of partitions set in partitioning scheme is used.
mode=single single: standalone mode; datanode: data node; controller: controller node; agent: agent node.
moduleDir=modules The directory for the module files. The default value is the relative directory modules that needs to be created by the user. The system searches the relative directory modules in the following order: home directory of the node, the working directory of the node, and the directory with the DolphinDB executable.
newValuePartitionPolicy=skip Specifies how new data outside of existing VALUE partitions are handled in databases. It can be "add", "skip", or "fail".
  • skip (default): appends data that belongs to existing VALUE partitions and ignores out-of-scope data.
  • add: Creates new VALUE partitions to add all data.
  • fail: Throws exception if out-of-scope data is included. None of the data will be written. In most cases, add is recommended.
allowMissingPartitions=true Specifies the behavior when incoming data contains new partition values that do not match any existing partitions. true (default) means to write the data that belongs to existing partitions and ignore the out-of-scope data. false means not to write any incoming data to the database and throw an exception. Note: For VALUE partitions where newValuePartitionPolicy is set to "add" or "fail", data will be handled based on the corresponding rules. In those cases, allowMissingPartitions does not take effect.
perfMonitoring=1 Whether to enable performance monitoring. The default value is false for the standalone mode and true for the cluster mode.
pluginDir=plugins The directory for the plugin files. The default value is the relative directory plugins. The system searches the relative directory plugins in the following order: home directory of the node, the working directoryof the node, and the directory with the DolphinDB executable.
preloadModules=plugins::mysql, system::log::fileLog The modules or plugins that are loaded after the system starts. Use commas to separate multiple modules and plugins.
init=dolphindb.dos This file is executed when the system starts. The default file is <HomeDir> /dolphindb.dos. It usually contains definitions of system-level functions that are visible to all users and cannot be overwritten.
startup=startup.dos This file is executed after the system starts. The default file is <HomeDir> /startup.dos. It can be used to load plugins, load tables and share them, define and load stream tables, etc.
postStart=postStart.dos This file is executed after scheduled jobs are initialized to load scheduled jobs. The default file is <HomeDir> /postStart.dos.
run=dailyJobs.dos This file is executed after the startup script (startup.dos) is executed. The default folder is DolphinDB home directory. It can be used to schedule jobs. After this file is executed, the system terminates.
tzdb=C:DolphinDBservertzdb The directory of the time zone database. The default value is homeservertzdb.
webRoot=/home/DolphinDB/server/web The directory of the web server. The default value is <HomeDir> /web.
webLoginRequired=false Whether a user must log in to use the web-based cluster manager. The default value is false.
useHardLink=true Whether to use the hardlink feature of the file system. Set to true to use the system hardlink feature; Set to false to not use the hardlink feature. The default value is true.
maxFileHandles Sets the maximum number of file descriptors open in a DolphinDB process.
enableConcurrentDimensionalTableWrite =false Whether to allow conducting concurrent write or update/delete on dimension tables. The default value is false, indicating concurrent write and update/delete is disabled for dimension tables.
removeSpecialCharInColumnName=false Whether to allow column names to contain special characters or to start without a letter. The default value is false, which can be used to remove the special characters in the column names generated by pivot by. Specify the value as true to be compatible with version 1.30.14 and before.
enableChunkGranularityConfig=false Specify the chunk granularity to determine the level of the lock of a DolphinDB transaction. When writing to a chunk, the transaction locks it to prevent other transactions from writing to it. Before version 1.30.16/2.00.4, the chunk granularity is set at the database level, i.e., each partition in the database is a chunk. In this case, concurrent writes to different tablets in the same partition are not allowed. This configuration parameter was released with version 1.30.16/2.00.4. The default value is false, which indicates the chunk granularity is at the table level, i.e., each tablet of a partition is a chunk. Concurrent writes to different tables in the same partition are thus allowed. If it is set to true, you can specify the chunk granularity with the parameter chunkGranularity of function database.
TSDBAsyncSortingWorkerNum=1 A non-negative integer indicating the number of threads for asynchronous sorting in the TSDB cache engine. The default value is 1. Data written to the TSDB cache engine are sorted by sortColumns. The tasks of writing and sorting data can be processed either synchronously or asynchronously and the asynchronous mode optimizes writing performance. Set it to 0 to enable the synchronous mode.
logRetentionTime The amount of time to keep a log file before deleting it. The default value is 30 days (of floating-point data type). For example, 0.5 means 12 hours. If set to 0, log files are not deleted.
traceLogDir The SQL tracing log path. The default value is <HomeDir>/traces.
oldChunkVersionRetentionTime= 60 Sets the retention time (in minutes) for the old chunk versions. The upper limit is 240. When SQL update/upsert/delete is executed, the system generates a new replica of the chunk (named by "table_name_tid") to perform the update and delete operations. Each old chunk will be retained in the system for the specified retention time, and a maximum of 5 old chunks can be kept at the same time.
enableLocalDatabase=true A Boolean value indicating whether to enable the creation of a local database. The default value is true. Once configured, it takes effect for all nodes in the cluster.

Each persisted MVCC table has a log file. Operations (including adding, deleting and modifying) on an MVCC table are first written to the log. When the number of operations reaches mvccCheckpointThreshold, a log checkpoint is initiated, where the operations are performed on the table and the log is cleared.

The number of operations on an MVCC table are counted as below:

  • For adds (append!, tableInsert, insert into) and updates (update): the number of applied rows multiplied by the number of applied columns;

  • For deletes (delete): the number of applied rows.

Note that when you load an MVCC table with loadMvccTable, the log file needs to be replayed. If the amount of data in the log is too large, it may take a long time to replay, or even result in an OOM event.
Parameter Description
mvccCheckpointThreshold =5000000 Sets the threshold for the number of operations on an MVCC table to trigger a checkpoint. The value can be set in [100,000, 2^31-1], and the default value is 5,000,000.

Core Dumps

Parameter Description
enableCoreDump=true [Linux Only] Whether to check and enable core dumps when starting DolphinDB. The default value is true. true means that DolphinDB will check if core dumps are enabled during startup. If core dumps are disabled, it will try to enable them, provided the system's hard limit allows core dumps of a sufficient size. If core dumps are already enabled, no changes will be made. False means that DolphinDB will start up without checking or modifying the system's current core dump enablement state.
disableCoreDumpOnShutdown =false [Linux Only] Whether to disable core dumps during a graceful shutdown. The default value is false, meaning core dumps will not be disabled during a graceful shutdown. This parameter only takes effect for the local node.

Compatibility

Parameter Description
enableNullSafeJoin Whether to match NULL values in join columns when performing JOIN operations (including left join, left semi join, right join, equi join and full join). The default behavior, when it is set to false, is to disallow joining on NULLs. This follows the ANSI SQL semantics, which is the same as the behavior in version 2.00.10 and later.
movingIndexedObjectByIndexedLabel

A Boolean value to specify whether operations of moving functions on indexed matrices/vectors are performed column-wise or row-wise. True (default) means performing by columns, false means by rows.

The m-functions affected by this configuration parameter include: move, mcovar, mcorr, mbeta, mwavg, mwsum, mpercentile, mrank, mcount, mfirst, mlast, mavg, mmed, mprod, msum, msum2, mstd, mvar, mstdp, mvarp, mskew, mkurtosis, mmin, and mmax.
ignoreSpecialCharacterInPartitionId =true

A Boolean value to specify whether to ignore ":" and "." in partitioning columns when creating partition directories for value-partitioned tables with partitioning columns of STRING or SYMBOL type. The default value is true, which matches the behavior before version 2.00.11/1.30.23.

For example, by default, keys like ".a:bc." and "abc" of the partitioning column will map to the same directory path since ":" and "." are ignored. So data written to partitions ".a:bc." and "abc" will both end up being stored under "abc". If set to false, "." and ":" will be included as part of the path, which means data written to the ".a:bc." and "abc" partitions will be separated into different paths.
keepTupleInRowFunction Specify whether row-based vector functions or function byRow whose func is a vector function return a columnar tuple if the input argument is a columnar tuple. If it is set to true (default value), a columnar tuple is returned; if it is set to false, an array vector is returned.
logicOrIgnoreNull=true Specifies whether function or ignores NULL values. When set to true (default):
  • If the other operand is non-zero, it returns true.
  • If the other operand is zero, it returns false.
  • If the other operand is NULL, it returns NULL.
When set to false, it always returns NULL regardless of the value of the other operand.
nullAsMinValueForComparison=true Whether a NULL value is treated as the minimum value in data comparison. The default value is true. If it is set to false, the result of comparison involving NULL value is NULL.

The parameter decimalRoundingMode is introduced in 2.00.12 to set the rounding mode for the following conversion scenarios:

  • Parsing data of STRING/SYMBOL type to DECIMAL type (e.g., loading files with loadText).

  • Converting floating-point numbers to DECIMAL type.

  • Converting DECIMAL value to Integral type.

  • Converting high-precision DECIMAL value to low-precision DECIMAL value.

Note: As of 2.00.10, the rounding mode for the first case was changed from truncation to rounding. For other scenarios, truncation was used.

Parameter Description
decimalRoundingMode Specifies the rounding mode for DECIMAL type. The default is trunc, meaning the decimal part is truncated. Set it to round to round the decimal part.

DolphinDB Terminal

DolphinDB terminal is a command line interactive tool for connecting to a remote DolphinDB server to execute commands. It is supported in 1.10.4 and above.

Start the DolphinDB terminal in the Linux system with the following command:

rlwrap -r ./dolphindb -remoteHost 192.168.1.135 -remotePort 8848

Start the DolphinDB terminal in Windows with the following command:

dolphindb.exe -remoteHost 192.168.1.135 -remotePort 8848

After starting up successfully, the following is displayed:

DolphinDB Terminal 1.10.4 (Build:2020.04.03). Copyright (c) 2011~2020 DolphinDB, Inc.

To quit the terminal:

quit

The following are the parameters related to the DolphinDB terminal. These parameters can only be used in the command line.

Parameter Description
remoteHost IP address of the remote DolphinDB server.
remotePort Port number of the remote DolphinDB server.
stdoutLog Where to output the system log. The default value is 0 indicating file dolphindb.log; 1 means stdout; 2 means both stdout and file dolphindb.log.
uid User name of the remote DolphinDB server.
pwd Password of the remote DolphinDB server.
run The local DolphinDB script file that is sent to the remote DolphinDB server for execution when starting up the terminal. After the execution is completed, the terminal will automatically exit.

If an exception occurs when running a local script with -run on a remote server, the exception return value is 255.

Resource Monitoring

DolphinDB provides the feature to track resource usage at the user level on data nodes, including CPU and memory usage, DFS SQL queries, number of rows read, and data volumes accessed.

Parameter Default Description
resourceSamplingInterval -1 An integer (in seconds) specifying the sampling interval of resource monitoring. The default value is -1, indicating resource tracking is disabled. To enable resource sampling, a positive integer must be set.
resourceSamplingMaxLogSize 1024 A positive integer (in MB) specifying the maximum size threshold of the resource sampling log. The default value is 1024. Log rotation is applied to split the log once it reaches the specified size. The generated file name is prefixed with its generation timestamp. For example, the file 20231101162302_access.log is generated at 2023.11.01T16:23:02.
resourceSamplingLogRetentionTime -1 An integer (in days) specifying the maximum retention period for the resource sampling logs. The default value -1 means logs never expire.