Moving Functions (m-functions)

DolphinDB provides moving functions (m-functions) for sliding window aggregation. A window of a specified size slides over each element in the given data set to perform a calculation on the windowed values. The result has the same size as the given data.

Introduction

  • Higher-order function moving:

    moving(func, funcArgs, window, [minPeriods])

The m-functions are optimized for their specialized use cases and therefore have a better performance than the higher-order function moving().

  • Syntax templates for the m-functions:

    mfunc(X, window,[minPeriods])
    mfunc(X, Y, window,[minPeriods])

    Parameters:

    X (Y) is a vector/matrix/table.

    window is an integer no smaller than 2 or a scalar of DURATION type indicating the size of the sliding window. Note: The window size is capped at 102400 when m-functions are used in the streaming engines.

    minPeriods (optional) is a positive integer indicating the minimum number of observations in a window required to be not NULL (otherwise the result is NULL).

List of Functions

Based on the value passed by window, m-functions can be divided into two categories:

(1) When window is a positive integer (or of integral type) or a DURATION value:

(2) When window is a positve interger:

mimax, mimin, mmad, mcovar, mbeta

Windowing Logic

The m-functions operate over backward windows. That is, each element in the given data corresponds to a window (with the size of window) where it is included as the last value.

  1. When X is a regular (non-indexed) vector, matrix or table, the size of the sliding window is based on element counts.

    Note: For tables, only columns of Boolean and numeric types participate in the calculation.

    The m-functions (excluding mrank and mcount) provide the minPeriods parameter for specifying the minimum number of observations in a window.

    Specifically,

    • If minPeriods is not specified, return NULL for the first (window-1) windows.

    • If minPeriods is specified, return NULL for the first (minPeriods-1) windows.

    The following example illustrates how the window slides:

    X = 2 1 3 7 6 5 4 9 8 10
    msum(X, 3);
    // output
    [ , , 6, 11, 16, 18, 15, 18, 21, 27]
  2. When X is an indexed series or an indexed matrix, the size of the sliding window is based on time. If window is a positive integer, it takes the same time unit as the index of X.

    The following example illustrates how the window slides:

    T = [2022.01.01, 2022.01.02, 2022.01.03, 2022.01.06, 2022.01.07, 2022.01.08, 2022.01.10, 2022.01.11]
    X = 1..8
    X1 = indexedSeries(T, X)
    msum(X1, window=3d);
    label col0
    2022.01.01 1
    2022.01.02 3
    2022.01.03 6
    2022.01.06 4
    2022.01.07 9
    2022.01.08 15
    2022.01.10 13
    2022.01.11 15