pnodeRun

pnodeRun(function,[nodes],[addNodeAlias=true])

function is the local function to call. It must not be quoted. It can be a function with no parameters by definition, or a partial application that wraps the orginal function and its parameters to a function with no parameters. It can be a built-in function or a user-defined function.

nodes is aliases of nodes. It is an optional parameter. If it is not specified, the system will call the function on all live data nodes and compute nodes in the cluster.

addNodeAlias specifies whether to add aliases of nodes to results. It is an optional parameter. The default value is true. You can set it to false if the result from each node already contains the node alias.

Call a local function on all data nodes and compute nodes in a cluster in parallel and then merge the results.

Ex. 1 Call function getChunksMeta without specifying parameters

pnodeRun(getChunksMeta,,false);

Ex. 2 In the following example, the function sum and arguments 1..10 are wrapped into a partial application sum{1..10}.

pnodeRun(sum{1..10}, `nodeA`nodeB);

Ex. 3 pnodeRun is a very convenient tool for cluster management. For example, in a cluster of 4 nodes: "DFS_NODE1", "DFS_NODE2", "DFS_NODE3", and "DFS_NODE4", run the following script on each of the node:

def jobDemo(n){
    s = 0
    for (x in 1 : n) {
        s += sum(sin rand(1.0, 100000000)-0.5)
        print("iteration " + x + " " + s)
    }
    return s
};

submitJob("jobDemo1","job demo", jobDemo, 10);
submitJob("jobDemo2","job demo", jobDemo, 10);
submitJob("jobDemo3","job demo", jobDemo, 10);

To check the status of the most recent 2 completed batch jobs on each of the 4 nodes in the cluster:

pnodeRun(getRecentJobs{2});

pnodeRun(getRecentJobs{2}, `DFS_NODE3`DFS_NODE4);