Add Partitions

DolphinDB supports adding new partitions to VALUE or RANGE partitioned databases, and to COMPO partitioned databases with VALUE or RANGE domains.

Add VALUE Partitions

The following examples demonstrate partition addition using a COMPO-partitioned sample database that uses both RANGE and VALUE partitioning. This database has 5 date partitions spanning from 2024.08.07 to 2024.08.11.

n=1000000
ID=rand(50..59, n)
dates=2024.08.07..2024.08.11
date=rand(dates, n)
x=rand(10.0, n)
t=table(ID, date, x);

dbID=database(, RANGE, 50 100);
dbDate = database(, VALUE, 2024.08.07..2024.08.11)
db = database("dfs://compoDB", COMPO, [dbID, dbDate]);
pt = db.createPartitionedTable(t, `pt, `ID`date)
pt.tableInsert(t);

To accommodate data outside the current VALUE partitioning scheme (both before or after the current value range), there are two options:

  1. Set the newValuePartitionPolicy parameter to add in the configuration file (cluster.cfg for cluster mode, dolphindb.cfg for standalone mode). This setting affects all databases and allows automatic creation of new partitions for out-of-scope data. Alternative values are:
    1. skip (default): Only saves records that match existing partitions
    2. fail: Throws an exception if any record is out of scope
  2. Use function addValuePartitions. If newValuePartitionPolicy=skip or fail, use this function to manually extend the partitioning scheme of a particular database. For example:
    schema(db).partitionSchema[1]
//Original second-level (VALUE) partitioning scheme: [2024.08.07,2024.08.08,2024.08.09,2024.08.10,2024.08.11]

//Add new partitions before and after the value range
addValuePartitions(db,2024.08.06,1);
addValuePartitions(db,2024.08.12,1);

schemeschema(db).partitionSchema[1]
//Extended VALUE partitioning scheme: [2024.08.06,2024.08.07,2024.08.08,2024.08.09,2024.08.10,2024.08.11,2024.08.12]

Add RANGE Partitions

For a partitioned database with RANGE domain, new partitions cannot be automatically generated with a configuration parameter. Data that falls outside existing partitions is discarded by default. You can Use function addRangePartitions to extend the existing partitioning scheme. Note that the new partition can only be added after the last existing partition. For example:

schema(db).partitionSchema[0]
//Original first-level (VALUE) partitioning scheme: [50,100]

addRangePartitions(db,100 150 200 250,0);
schema(db).partitionSchema[0]
//Extended RANGE partitioning scheme:[50,100,150,200,250]

addRangePartitions(db,0 50,0);
// Raises an error