Drop Partitions

Only VALUE partitions can be deleted from partitioning scheme. To achieve this, set the deleteSchema parameter to true in dropPartition. Once deleted, these partitions must be recreated before any new data can be appended to them. The database must meet these requirements:

• The first level partitioning scheme is VALUE.
• The database contains exactly one table.

The following example demonstrates dropping VALUE partitions in a COMPO-partitioned database where the first level uses VALUE partitioning and the second uses RANGE partitioning:

// Generate sample data
drop database if exists "dfs://compoDB"
n=1000000
ID=rand(150, n)
dates=2024.08.07..2024.08.11
date=rand(dates, n)
x=rand(10.0, n)
t=table(ID, date, x)
dbDate = database(, VALUE, 2024.08.07..2024.08.11)
dbID = database(, RANGE, 0 50 100 150)
db = database("dfs://compoDB", COMPO, [dbDate, dbID])
pt = db.createPartitionedTable(t, `pt, `date`ID)
pt.append!(t);

//Delete 2024.08.07 partitions completely from database scheme 
dropPartition(db, 2024.08.07, tableName=`pt, deleteSchema=true);

To remove partition data while preserving partitioning scheme, set the deleteSchema parameter to false in dropPartition.

The following script creates a COMPO-partitioned database with VALUE (first level) and RANGE (second level) partitioning. It will be used in later examples:

drop database if exists "dfs://compoDB"
n=1000000
ID=rand(150, n)
dates=2024.08.07..2024.08.11
date=rand(dates, n)
x=rand(10.0, n)
t=table(ID, date, x)
dbDate = database(, VALUE, 2024.08.07..2024.08.11)
dbID = database(, RANGE, 0 50 100 150)
db = database("dfs://compoDB", COMPO, [dbDate, dbID])
pt = db.createPartitionedTable(t, `pt, `date`ID)
pt.append!(t);

Example 1: Delete one partition.

Use either of the following ways to delete the partition "/20240807/0_50".

(1) Specify the partition path.

dropPartition(dbHandle=db, partitionPaths="/20240807/0_50", tableName=`pt);

(2) partitionPaths can also be a filtering condition.

dropPartition(dbHandle=db, partitionPaths=[2024.08.07, 0], tableName=`pt);

For COMPO partitions, filtering condition partitionPaths is a tuple with each element corresponding to a partition level (starting from the first). Here 0 means the partition of [0, 50). We can choose any number from 0 to 49 to represent this partition.

Example 2: Delete a first level partition.

Use either of the following ways to delete the first level partition of 2024.08.08.

(1) Specify the paths of all partitions under 2024.08.08.

partitions=["/20240808/0_50","/20240808/50_100","/20240808/100_150"]
dropPartition(dbHandle=db, partitionPaths=partitions, tableName=`pt);

(2) Specify the filtering condition.

dropPartition(dbHandle=db, partitionPaths=(2024.08.08,), tableName=`pt)

Example 3: Delete a second level partition.

Use either of the following ways to delete the second level partition of [0,50).

(1) Specify the paths to all [0,50) partitions.

partitions=["/20240807/0_50","/20240808/0_50","/20240809/0_50","/20240810/0_50","/20240811/0_50"]
dropPartition(dbHandle=db, partitionPaths=partitions, tableName=`pt);

(2) Specify the filtering condition.

dropPartition(dbHandle=db, partitionPaths=[,[0]], tableName=`pt);

The filtering condition specified by partitionPaths does not apply to first level partitions as its first element is unspecified.

Example 4: Delete multiple same level partitions.

To delete the second level partitions of [0,50) and [100,150):

dropPartition(dbHandle=db, partitionPaths=[,[0,100]], tableName=`pt);

The [0,100] vector represents partitions [0,50) and [100,150).

Delete partitions "/20240807/0_50","/20240808/0_50","/20240807/100_150","/20240808/100_150":

dropPartition(dbHandle=db, partitionPaths=[[2024.08.07,2024.08.08],[0,100]], tableName=`pt);